Monitor Log Files with Bash Script

Keeping track of log files is a critical task for system administrators and developers to monitor application behavior, detect anomalies, and debug issues. Automating this process reduces manual intervention and ensures real-time awareness. This blog post walks you through a Bash script that monitors a log file, detects new entries, and emails those entries to a designated address when run as a cron job.

The Script link

Below is the script that performs the log monitoring task. Here’s an overview of how it works:

1. Read the last n lines from the log file. You can configure how many lines to monitor by adjusting the tail_lines variable (default is 300).

2. Compare with previously saved logs. The script uses a file (last_logs.txt) to store logs from the last check.

3. Identify new logs. It compares the current logs to the previously saved logs and isolates any new entries.

4. Send the new entries via email. Using the mail command, the script emails the new entries to the specified recipient.

5. Update last_logs.txt. After processing, it saves the current logs for the next comparison.

Here’s the complete script:

  #!/bin/bash

# File paths
log_file_path="/var/log/file.log"
last_logs_file_path="last_logs.txt"

# Email details
# Here you can specify the sender
sender_email="[email protected]"

# Here you can specify the receiver
receiver_email="[email protected]"

# Here you can specify the email subject
subject="Log File Update"

# Number of last lines to check
# Here you can specify the number of lines that should be monitored at max
tail_lines=300

# Function to get the last n lines of a file
tail_file() {
    tail -n "$1" "$2"
}

# Function to read the previous lines from the last_logs.txt file
read_previous_lines() {
    if [[ -f "$last_logs_file_path" ]]; then
        cat "$last_logs_file_path"
    else
        echo ""
    fi
}

# Function to write the current lines to the last_logs.txt file
write_current_lines() {
    echo "$1" > "$last_logs_file_path"
}

# Function to send the email
send_email() {
    body="$1"
    echo -e "$body" | mail -s "$subject" -r "$sender_email" "$receiver_email"
}

# Main function to monitor the log file
monitor_log() {
    previous_lines=$(read_previous_lines)
    current_lines=$(tail_file "$tail_lines" "$log_file_path")
    email_body="\n"

    # Compare current lines to previous lines and find new lines
    new_lines=""
    while IFS= read -r line; do
        if ! grep -Fxq "$line" <<< "$previous_lines"; then
            new_lines+="$line"$'\n'
        fi
    done <<< "$current_lines"

    if [[ -n "$new_lines" ]]; then
        email_body+="$new_lines"
        write_current_lines "$current_lines"  # Update the last_logs.txt with the new content
    else
        email_body+="No changes.\n"
    fi

    # Send the email with the results
    send_email "$email_body"
}

# Start monitoring the log file (this will run once)
monitor_log
  

Deploying the Script with Cron link

To automate the script, we use a cron job to run it periodically. Here’s how to set it up:

1. Edit the Cron Table Open the crontab editor using:

  crontab -e
  

2. Schedule the Script Add the following line to run the script every hour:

  0 * * * * /path/to/your/script.sh
  

Replace /path/to/your/script.sh with the full path to the script. 3. Save and Exit

Configuration Options link

• Log File Path: Change the value of log_file_path to monitor a different log file.

• Email Settings:

1. Set the sender’s email (sender_email) and receiver’s email (receiver_email).
2. Customize the email subject using the subject variable.

• Number of Lines to Monitor: Adjust the tail_lines variable to specify how many lines from the log file should be checked each time.

Requirements link

• Ensure the mail command is installed and configured to send emails. On most Linux distributions, you can install it using:

  sudo apt-get install mailutils   # Debian/Ubuntu
sudo yum install mailx           # RedHat/CentOS
  

• Grant the script execution permissions:

  chmod +x /path/to/your/script.sh
  

Advantages of This Script link

• Customizable Monitoring: Tailor the number of lines and frequency to suit your log size and needs.
• Automated Notifications: Get instant updates on new log entries via email.

Conclusion link

This script is a great starting point for lightweight log monitoring and alerting. It’s especially useful for scenarios where sophisticated monitoring tools might be overkill or unavailable. Feel free to modify and expand its functionality based on your requirements!