Using logrotate with the “O” stuff

Red Hat ships their Enterprise Linux product with logrotate preconfigured to handle log rotation for many common services like the Apache HTTP Server. But logrotate can be used to manage other log files as well — like those generated by Oracle products.

Recently I decided it would be a good idea to separately log separate virtual hosts in an Oracle HTTP Server (OHS) deployment. After scouring the Internets, Oracle’s product forums and customer-only support knowledge base, all I could find were suggestions on how to use Apache’s rotatelogs (or the newer OHS-centric odl_rotatelogs) for this.

While rotatelogs would probably work just fine, I was loathe to clutter httpd.conf with repeated copies of the necessary command string. Another consideration was maintenance: complicated configurations tend to be the first thing that breaks down over time. As an administrator I always prefer standardizing on as few different methods of doing things as possible. Whether Oracle chooses to accept it or not, most things in the universe don’t come with an “O” prefixed to their name.

Since logrotate was already in place as Red Hat’s standard way of rotating logs, I decided it would be worth spending some time getting to understand it.

As it turns out getting logrotate configured to do the job wasn’t that hard, although there was trial and error involved (mostly because I failed to thoroughly read the man pages the first, second and third times).

The logrotate command, of course, gets its configuration from logrotate.conf, which on most Linux systems is /etc/logrotate.conf. Here’s a sample from my Red Hat desktop system:

# rotate log files weekly
weekly
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create
# uncomment this if you want your log files compressed
#compress
# RPM packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own wtmp -- we'll rotate them here
/var/log/wtmp {
    monthly
    minsize 1M
    create 0664 root utmp
    rotate 1
}
/var/log/btmp {
    missingok
    monthly
    minsize 1M
    create 0600 root utmp
    rotate 1
}

Red Hat’s custom configuration of things places separate configuration files for each application under /etc/logrotate.d. Here’s mine:

[root@myhost myuser]# ls /etc/logrotate.d
acpid   httpd   ppp     samba           snmpd   tomcat5     wpa_supplicant
conman  ldap    psacct  sa-update       squid   tux         yum
cups    mgetty  rpm     setroubleshoot  syslog  vsftpd.log

If you view the one for the web server (httpd), you’ll see this:

/var/log/httpd/*log {
    missingok
    notifempty
    sharedscripts
    postrotate
        /sbin/service httpd reload > /dev/null 2>/dev/null || true
    endscript
}

Here’s the one for samba:

/var/log/samba/*.log {
    notifempty
    missingok
    sharedscripts
    copytruncate
    postrotate
        /bin/kill -HUP `cat /var/run/smbd.pid /var/run/nmbd.pid /var/run/winbindd.pid 2> /dev/null` 2> /dev/null || true
    endscript
}

And here’s the one for tomcat5:

/var/log/tomcat5/catalina.out {
    copytruncate
    weekly
    rotate 52
    compress
    missingok
}

As you can see there are quite a few options available. Invoking any in a custom file for a service will override any default set up in logrotate.conf (e.g. compression, rotation and deletion increment, file size, etc).

Because I was dealing with a daemon not easily controlled from the shell due to the action of Oracle’s opmnd daemon (not to mention their customizations to httpd itself), I needed something that could roll the logs without stopping and starting the service. The key to that turned out to be the copytruncate method used by logrotate to truncate the log file in place after creating a copy (the normal action would be to copy the log and then delete the original, letting the recycled service like httpd create a new one). Although some logging information might be lost in the brief moment it took to copy the log, this was to me as acceptable as recycling httpd, if not more so.

Putting it all together, here’s what I did for OHS:

/u01/logs/httpd/*log {
   missingok
   notifempty
   sharedscripts
   copytruncate
}

*The sharedscripts command causes logrotate to only perform its pre and post rotate tasks once, and to create a new log file after copying.

References

logrotate man mage

The Ultimate Logrotate Command Tutorial

Deployment Guide, sec. 19.2.1, “Configuring logrotate”, Red Hat Enterprise Linux 6.

RHBA-2011:0816-2, 2011-07-21, “Logrotate Bug Fix Update”, Red Hat Enterprise Linux 5.