Lesson 12: Monitoring Nginx with Access and Error Logs
Monitoring your Nginx web server is crucial for maintaining performance, security, and reliability. In this lesson, you'll learn how to leverage Nginx's built-in logging capabilities to gain valuable insights into your server's operation.
Learning Goals:
- Understand Nginx access and error log formats
- Configure custom log formats and destinations
- Analyze log data for troubleshooting and insights
- Implement log rotation and management
- Use tools for log monitoring and analysis
Understanding Nginx Logs
Nginx generates two primary types of logs that are essential for monitoring:
- Access Logs: Record all client requests to your server
- Error Logs: Capture errors and warnings during server operation
By default, these logs are located in /var/log/nginx/ on most Linux distributions.
Access Log Configuration
Default Access Log
The access log captures detailed information about each request, including client IP, timestamp, request method, status code, and more.
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
}
Custom Log Formats
You can create custom log formats to capture specific information relevant to your monitoring needs:
http {
# Extended format with response time
log_format extended '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" '
'rt=$request_time uct="$upstream_connect_time" '
'uht="$upstream_header_time" urt="$upstream_response_time"';
# Minimal format for high-traffic sites
log_format minimal '$remote_addr - $request - $status - $body_bytes_sent';
access_log /var/log/nginx/access.log extended;
}
Use custom log formats to include timing metrics like $request_time and $upstream_response_time for performance monitoring.
Error Log Configuration
Error logs help you identify and troubleshoot server issues:
error_log /var/log/nginx/error.log warn;
The log level can be set to:
debug: Detailed debug informationinfo: Informational messagesnotice: Normal but significant conditionswarn: Warning conditionserror: Error conditionscrit: Critical conditions
Conditional Logging
You can conditionally log requests based on specific criteria:
server {
listen 80;
server_name example.com;
# Don't log health checks
location /health {
access_log off;
return 200 "healthy\n";
}
# Log API requests separately
location /api/ {
access_log /var/log/nginx/api.log extended;
# Your API configuration
}
# Log 4xx and 5xx errors to a separate file
location / {
access_log /var/log/nginx/access.log extended;
access_log /var/log/nginx/errors.log extended if=$status_error;
}
map $status $status_error {
~^[45] 1;
default 0;
}
}
Log Analysis and Monitoring
Basic Command Line Analysis
Use common Unix tools to analyze your logs:
# Count requests by status code
tail -1000 /var/log/nginx/access.log | awk '{print $9}' | sort | uniq -c | sort -rn
# Top 10 IP addresses
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -10
# Requests per hour
awk '{print $4}' /var/log/nginx/access.log | cut -d: -f1,2 | sort | uniq -c
# Find slow requests (assuming extended format)
grep "rt=[1-9]" /var/log/nginx/access.log | awk -F'rt=' '{print $2}' | cut -d' ' -f1 | sort -n
Real-time Monitoring
Monitor logs in real-time for immediate insights:
# Tail access logs with grep filtering
tail -f /var/log/nginx/access.log | grep -v "ELB-HealthChecker"
# Watch for errors in real-time
tail -f /var/log/nginx/error.log
# Monitor slow requests
tail -f /var/log/nginx/access.log | awk -F'rt=' '$2+0 > 1 {print}'
Log Rotation and Management
Proper log management prevents disk space issues and maintains performance:
/var/log/nginx/*.log {
daily
missingok
rotate 52
compress
delaycompress
notifempty
create 0640 www-data adm
sharedscripts
postrotate
if [ -f /var/run/nginx.pid ]; then
kill -USR1 `cat /var/run/nginx.pid`
fi
endscript
}
Always test log rotation configurations to ensure they don't interrupt your Nginx service. The USR1 signal tells Nginx to reopen log files.
Advanced Log Configuration
Structured Logging with JSON
JSON format makes log parsing easier for modern monitoring tools:
http {
log_format json_combined escape=json
'{'
'"time_local":"$time_local",'
'"remote_addr":"$remote_addr",'
'"remote_user":"$remote_user",'
'"request":"$request",'
'"status": "$status",'
'"body_bytes_sent":"$body_bytes_sent",'
'"request_time":"$request_time",'
'"http_referer":"$http_referer",'
'"http_user_agent":"$http_user_agent",'
'"http_x_forwarded_for":"$http_x_forwarded_for"'
'}';
access_log /var/log/nginx/access.json json_combined;
}
Buffered Logging for Performance
For high-traffic sites, consider buffered logging:
http {
access_log /var/log/nginx/access.log extended buffer=32k flush=1m;
}
Common Pitfalls
- Disk space exhaustion: Not implementing log rotation can quickly fill your disk
- Performance impact: Excessive logging on high-traffic sites can degrade performance
- Missing context: Not including enough information in log formats makes troubleshooting difficult
- Security exposure: Logging sensitive data like passwords or tokens
- Ignoring errors: Not regularly monitoring error logs for emerging issues
- Buffer overflow: Setting buffer sizes too small for high-volume logging
- Permission issues: Nginx process lacking write permissions to log directories
Summary
Effective Nginx monitoring through access and error logs provides crucial visibility into your web server's health and performance. By configuring appropriate log formats, implementing proper log rotation, and regularly analyzing log data, you can proactively identify issues, optimize performance, and maintain a reliable web service. Remember to balance the detail level in your logs with performance considerations and security requirements.
Quiz
Nginx Logging and Monitoring
What is the primary purpose of Nginx access logs?