4.6.4 Log rotation

Table of Contents

Why Log Rotation Matters

Applications and system services continuously append to their log files. Without rotation:

Log files grow indefinitely
Disks can fill, causing outages
Old logs become hard to manage
Performance can degrade when tools operate on huge files

Log rotation solves this by:

Automatically rotating (renaming) active log files
Compressing older logs
Deleting or archiving very old logs
Optionally running scripts before/after rotation

Most Linux systems implement this with logrotate, but some services also have their own built‑in mechanisms.

Basic Log Rotation Concepts

Key ideas you see across implementations:

Rotation trigger

By size (e.g. rotate when file exceeds 100M)
By time (daily, weekly, monthly)
Or both, depending on configuration

Retention policy

How many old logs to keep (rotate 4 → keep 4 old files)
How long to keep them (e.g. 30 days in some tools)
Whether to compress them

Naming scheme

Suffix numbers: app.log.1, app.log.2.gz, …
Date-based names: app.log-2025-12-12.gz
Occasionally a custom pattern

Post‑rotation behavior

Truncate vs rename
Signal or notify the application so it starts logging to the new file
Run maintenance scripts (e.g. reloading services, syncing logs)

logrotate Overview

logrotate is the standard log rotation utility on many distributions (especially those using traditional /var/log files).

It is typically:

Installed by default
Invoked automatically by cron or systemd timers
Configured via:

A main configuration file (e.g. /etc/logrotate.conf)
Per‑package / per‑service configs in /etc/logrotate.d/

logrotate runs periodically (often daily), evaluates its rules, and decides which log files need rotation at that time.

logrotate Configuration Structure

A typical setup:

Global configuration: /etc/logrotate.conf

This defines:

Default rotation period
Compression policy
Whether to use date extension, etc.
Inclusion of /etc/logrotate.d/

Service‑specific configs: /etc/logrotate.d/<service>

Packages place their own files here, defining:

Paths to their log files
Rotation schedule
Ownership/mode of newly created logs
Pre/post scripts

Example global file snippet:

# /etc/logrotate.conf
weekly
rotate 4
create
compress
include /etc/logrotate.d

This means:

Rotate logs weekly by default
Keep 4 old rotated copies
Create a new empty log after rotation
Compress old logs
Load additional rules from /etc/logrotate.d/

Basic logrotate Options

Within a logrotate block, you configure policies for specific log files:

/var/log/myapp/*.log {
    daily
    rotate 7
    missingok
    compress
    delaycompress
    notifempty
    create 0640 myapp myapp
}

Key directives:

Rotation frequency

daily, weekly, monthly, yearly
size 100M (rotate when size exceeds 100 MB)
You can combine time + size triggers (behavior depends on version)

Retention and compression

rotate N – keep N rotated logs, then delete oldest
compress – compress old logs (usually gzip)
nocompress – disable compression
delaycompress – start compression from the second old file, allowing one uncompressed recent backup
compresscmd, uncompresscmd – change compression tool (e.g. bzip2, xz)
compressoptions – extra options (e.g. -9)

Handling empty/missing files

missingok – do not complain if log file is missing
nomissingok – error if log is missing
notifempty – skip rotation if file is empty
ifempty – rotate even if file is empty

Creating new logs

create mode owner group – after rotation, create a new log file with given permissions and ownership
nocreate – do not create a new file (often when the application creates it itself)

File naming

dateext – append a date to rotated logs (e.g. log-2025-12-12.gz)
dateformat -%Y%m%d – customize date extension (must start with -)
extension .log – preserve / change extension for rotated logs

Rotation method

copytruncate – copy the current log to a new file and truncate the original
nocopytruncate – use the rename method (default)
sharedscripts – run postrotate/prerotate only once for all matched files
olddir /path – move rotated logs to another directory
noolddir – keep rotated logs in same dir

How logrotate Performs Rotation

Two main strategies:

Rename + create (default)

Current log file is renamed (e.g. app.log → app.log.1)
New empty app.log is created
The application must reopen the file (via signal or restart) to continue logging

Pros:

Safer from data‑loss perspective
Works well with apps that handle SIGHUP or reopen logs on rotation

Cons:

If the app keeps the file descriptor open and doesn’t reopen, it will keep writing to the rotated file

copytruncate

Current log is copied to a new file
Original is truncated to size 0

Pros:

Works even when app can’t be signaled or told to reopen logs
The app keeps logging to the same filename

Cons:

Small race window: entries written between copy and truncate can be lost or duplicated
More I/O (copying can be expensive for large files)

Choose based on how the specific application behaves and your tolerance for potential minor gaps.

Pre‑ and Post‑Rotation Scripts

Some services must be notified when logs rotate. logrotate allows custom scripts:

/var/log/nginx/*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 nginx adm
    sharedscripts
    postrotate
        [ -s /run/nginx.pid ] && kill -USR1 "$(cat /run/nginx.pid)"
    endscript
}

Script blocks:

prerotate ... endscript

Runs before rotation
Can be used to flush buffers or perform checks

postrotate ... endscript

Runs after rotation
Often used to signal or reload services

firstaction ... endscript / lastaction ... endscript

Run only once per configuration block, even if multiple logs are rotated

sharedscripts matters when multiple files match the pattern:

Without sharedscripts, the script runs once per file
With sharedscripts, it runs only once for the entire set

Managing Multiple Versions and Compression

A typical “balanced” rotation policy:

/var/log/app/app.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0640 appuser appgroup
}

Behavior:

Rotate daily
Keep 14 days’ worth of archives
Compress everything except yesterday’s rotated log
Skip empty/missing files
Ensure proper permissions

Date‑based naming might be preferable when you:

Ship logs to external systems
Need to correlate logs across multiple hosts by date
Want to avoid renumbering (.1, .2) changes

Example:

/var/log/app/app.log {
    daily
    rotate 30
    dateext
    dateformat -%Y-%m-%d
    compress
    missingok
    notifempty
}

Resulting files:

app.log
app.log-2025-12-12.gz
app.log-2025-12-11.gz
…

Testing and Forcing logrotate

When you modify logrotate configs, you can:

Check configuration syntax:

  sudo logrotate -d /etc/logrotate.conf

-d (debug) shows what would happen without making changes.

Force a rotation (ignoring time/size):

  sudo logrotate -f /etc/logrotate.conf

Run on a specific config file:

  sudo logrotate -f /etc/logrotate.d/myapp

These commands help validate behavior before waiting for the scheduled run.

Integration with cron and systemd

Common ways logrotate is scheduled:

cron (traditional approach)

Typical entry: /etc/cron.daily/logrotate which simply runs:

  /usr/sbin/logrotate /etc/logrotate.conf

systemd timer

Some distributions use a timer unit:

logrotate.timer – defines when to run (e.g. daily)
logrotate.service – runs the command

You can inspect the last run via systemctl status logrotate.service or check logs (e.g. via journalctl if systemd logging is used).

Application‑Specific Log Rotation

Some services implement their own rotation logic and may not rely entirely on logrotate. Common patterns:

Built‑in rotation
Web servers, databases, and some daemons can:

Write logs to date‑based files
Automatically roll over at midnight or on size thresholds
Keep a set number of historical logs

Signal‑based rotation
Some daemons reopen their log files when they receive a specific signal (e.g. SIGHUP, SIGUSR1). Logrotate’s postrotate block is used to send that signal.

When an application has strong internal log rotation, you might:

Disable logrotate for those logs, or
Use logrotate only to handle compression and long‑term retention if the app doesn’t support that itself.

Always check the service’s own documentation before designing your rotation strategy.

Common Pitfalls and How to Avoid Them

Application continues writing to rotated file

Cause: app doesn’t reopen log after rename
Remedies:

Use copytruncate, or
Ensure postrotate sends the correct signal or reload command

Too aggressive deletion

Cause: small rotate value, frequent rotation
Remedy: increase retention or reduce frequency, or archive to long‑term storage

Disorganized naming

Cause: mixing numeric and date‑based rotations, or enabling dateext after the fact
Remedy: standardize naming strategy, especially for logs that are collected centrally

Performance hit on large logs

Cause: compressing huge files frequently
Remedies:

Rotate more often with smaller files
Use less CPU‑heavy compression (gzip with lower level)
Schedule compression at off‑peak times (via delayed compression and external jobs)

Permissions issues after rotation

Cause: missing or incorrect create directive
Remedy: explicitly set create mode user group so apps can write to new files

Designing a Log Rotation Policy

When creating or tuning rotation:

Assess log volume

How many MB/GB per day per log?
Peak vs off‑peak patterns

Determine retention needs

Compliance requirements (e.g. 90 days, 1 year)
Operational needs (debugging, forensics)

Decide on naming and compression

Numeric vs date extensions
Compression format and level

Coordinate with log shipping

If logs are sent to a central system, ensure rotation doesn’t break shipping agents (e.g. configure them to follow renamed files or inode changes)

Test under load

Force rotation during busy periods
Check for missing or duplicated entries
Confirm applications continue logging properly

A well‑designed rotation policy balances disk usage, performance, and the ability to investigate issues long after they occurred.

Comments

Please login to add a comment.

Don't have an account? Register now!