Apache HTTP Server Version 1.3

Module mod_log_config

This module provides for logging of the requests made to the server, using the Common Log Format or a user-specified format.

Status: Base
Source File: mod_log_config.c
Module Identifier: config_log_module
Compatibility: Was an extension module prior to Apache 1.2.

Summary

This module provides for flexible logging of client requests. Logs are written in a customizable format, and may be written directly to a file, or to an external program. Conditional logging is provided so that individual requests may be included or excluded from the logs based on characteristics of the request.

Three directives are provided by this module: TransferLog to create a log file, LogFormat to set a custom format, and CustomLog to define a log file and format in one step. The TransferLog and CustomLog directives can be used multiple times in each server to cause each request to be logged to multiple files.

Log File Formats

Unless told otherwise with LogFormat, the log files created by TransferLog will be in standard "Common Log Format" (CLF). The contents of each line in a CLF file are explained below. Alternatively, the log file can be customized (and if multiple log files are used, each can have a different format). Custom formats are set with LogFormat and CustomLog.

Common Log Format

The Common Log Format (CLF) file contains a separate line for each request. A line is composed of several tokens separated by spaces:

host ident authuser date request status bytes

If a token does not have a value then it is represented by a hyphen (-). The meanings and values of these tokens are as follows:

host

The fully-qualified domain name of the client, or its IP number if the name is not available.

ident

If IdentityCheck is enabled and the client machine runs identd, then this is the identity information reported by the client.

authuser

If the request was for an password protected document, then this is the userid used in the request.

date

The date and time of the request, in the following format:

date = [day/month/year:hour:minute:second zone] day = 2*digit month = 3*letter year = 4*digit hour = 2*digit minute = 2*digit second = 2*digit zone = (`+' | `-') 4*digit

request

The request line from the client, enclosed in double quotes (").

status

The three digit status code returned to the client.

bytes

The number of bytes in the object returned to the client, not including any headers.

Custom Log Formats

The format argument to the LogFormat and CustomLog directives is a string. This string is logged to the log file for each request. It can contain literal characters copied into the log files and the c-type control characters "\n" and "\t" to represent new-lines and tabs. Literal quotes and back-slashes should be escaped with back-slashes.

The characteristics of the request itself are logged by placing "%" directives in the format string, which are replaced in the log file by the values as follows:

%...a:          Remote IP-address
%...A:          Local IP-address
%...B:          Bytes sent, excluding HTTP headers.
%...b:          Bytes sent, excluding HTTP headers. In CLF format
		i.e. a '-' rather than a 0 when no bytes are sent.
%...c:          Connection status when response is completed.
                'X' = connection aborted before the response completed.
                '+' = connection may be kept alive after the response is sent.
                '-' = connection will be closed after the response is sent.
%...{FOOBAR}e:  The contents of the environment variable FOOBAR
%...f:          Filename
%...h:          Remote host
%...H		The request protocol
%...{Foobar}i:  The contents of Foobar: header line(s) in the request
                sent to the server.
%...l:          Remote logname (from identd, if supplied)
%...m		The request method
%...{Foobar}n:  The contents of note "Foobar" from another module.
%...{Foobar}o:  The contents of Foobar: header line(s) in the reply.
%...p:          The canonical Port of the server serving the request
%...P:          The process ID of the child that serviced the request.
%...q		The query string (prepended with a ? if a query string exists,
		otherwise an empty string)
%...r:          First line of request
%...s:          Status.  For requests that got internally redirected, this is
                the status of the *original* request --- %...>s for the last.
%...t:          Time, in common log format time format (standard english format)
%...{format}t:  The time, in the form given by format, which should
                be in strftime(3) format. (potentially localized)
%...T:          The time taken to serve the request, in seconds.
%...u:          Remote user (from auth; may be bogus if return status (%s) is 401)
%...U:          The URL path requested.
%...v:          The canonical ServerName of the server serving the request.
%...V:          The server name according to the UseCanonicalName setting.

The "..." can be nothing at all (e.g., "%h %u %r %s %b"), or it can indicate conditions for inclusion of the item (which will cause it to be replaced with "-" if the condition is not met). The forms of condition are a list of HTTP status codes, which may or may not be preceded by "!". Thus, "%400,501{User-agent}i" logs User-agent: on 400 errors and 501 errors (Bad Request, Not Implemented) only; "%!200,304,302{Referer}i" logs Referer: on all requests which did not return some sort of normal status.

Note that there is no escaping performed on the strings from %...r, %...i and %...o. This is mainly to comply with the requirements of the Common Log Format. This implies that clients can insert control characters into the log, so care should be taken when dealing with raw log files.

Some commonly used log format strings are:

Common Log Format (CLF): "%h %l %u %t \"%r\" %>s %b"
Common Log Format with Virtual Host: "%v %h %l %u %t \"%r\" %>s %b"
NCSA extended/combined log format: "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""
Referer log format: "%{Referer}i -> %U"
Agent (Browser) log format: "%{User-agent}i"

Note that the canonical ServerName and Port of the server serving the request are used for %v and %p respectively. This happens regardless of the UseCanonicalName setting because otherwise log analysis programs would have to duplicate the entire vhost matching algorithm in order to decide what host really served the request.

Using Multiple Log Files

The TransferLog and CustomLog directives can be given more than once to log requests to multiple log files. Unless the conditional form of CustomLog is used, each request will be logged to all the log files defined by either of these directives.

Use with Virtual Hosts

If a <VirtualHost> section does not contain any TransferLog or CustomLog directives, the logs defined for the main server will be used. If it does contain one or more of these directives, requests serviced by this virtual host will only be logged in the log files defined within its definition, not in any of the main server's log files. See the examples below.

Security Considerations

See the security tips document for details on why your security could be compromised if the directory where logfiles are stored is writable by anyone other than the user that starts the server.

Resetting the Log Files

The access log file typically grows 1MB or more for each 10,000 requests. It will probably be necessary to move or delete the log file on a regular basis. This cannot be done while the server is still running, because Apache will continue writing to the old log file. Instead, the server must be restarted after the log file is moved or deleted so that it will open a new log.

A typical scenario is:

   mv access_log access_log.old
   apachectl graceful
   # wait for all requests to the old server to complete
   # before doing anything with access_log.old

Alternatively, log files can be rotated automatically be writing them through a pipe to a program designed for that purpose such as rotatelogs.

Compatibility notes

This module is based on mod_log_config distributed with previous Apache releases, now updated to handle multiple logs. There is now no need to re-configure Apache to use configuration log formats.
The module also implements the CookieLog directive, used to log user-tracking information created by mod_usertrack. The use of CookieLog is deprecated, and a CustomLog should be defined to log user-tracking information instead.
As of Apache 1.3.5, this module allows conditional logging based upon the setting of environment variables. That is, you can control whether a request should be logged or not based upon whether an arbitrary environment variable is defined or not. This is configurable on a per-logfile basis.
Beginning with Apache 1.3.5, the mod_log_config module has also subsumed the RefererIgnore functionality from mod_log_referer. The effect of RefererIgnore can be achieved by combinations of SetEnvIf directives and conditional CustomLog definitions.

CookieLog directive

Syntax: CookieLog filename
Context: server config, virtual host
Module: mod_cookies
Compatibility: Only available in Apache 1.2 and above

The CookieLog directive sets the filename for logging of cookies. The filename is relative to the ServerRoot. This directive is included only for compatibility with mod_cookies, and is deprecated.

CustomLog directive

Syntax: CustomLog file|pipe format|nickname [env=[!]environment-variable]
Context: server config, virtual host
Status: Base
Compatibility: Nickname only available in Apache 1.3 or later. Conditional logging available in 1.3.5 or later.
Module: mod_log_config

The CustomLog directive is used to log requests to the server. A log format is specified, and the logging can optionally be made conditional on request characteristics using environment variables.

The first argument, which specifies the location to which the logs will be written, can take on one of the following two types of values:

file: A filename, relative to the ServerRoot.
pipe: The pipe character "|", followed by the path to a program to receive the log information on its standard input. Security: if a program is used, then it will be run under the user who started httpd. This will be root if the server was started by root; be sure that the program is secure.

The second argument specifies what will be written to the log file. It can specify either a nickname defined by a previous LogFormat directive, or it can be an explicit format string as described in the log formats section.

For example, the following two sets of directives have exactly the same effect:

     # CustomLog with format nickname
     LogFormat "%h %l %u %t \"%r\" %>s %b" common
     CustomLog logs/access_log common

     # CustomLog with explicit format string
     CustomLog logs/access_log "%h %l %u %t \"%r\" %>s %b"

The third argument is optional and allows the decision on whether or not to log a particular request to be based on the presence or absence of a particular variable in the server environment. If the specified environment variable is set for the request (or is not set, in the case of a 'env=!name' clause), then the request will be logged.

Environment variables can be set on a per-request basis using the mod_setenvif and/or mod_rewrite modules. For example, if you don't want to record requests for all GIF images on your server in a separate logfile but not your main log, you can use:

    SetEnvIf Request_URI \.gif$ gif-image
    CustomLog gif-requests.log common env=gif-image
    CustomLog nongif-requests.log common env=!gif-image

LogFormat directive

Syntax: LogFormat format|nickname [nickname]
Default: LogFormat "%h %l %u %t \"%r\" %>s %b"
Context: server config, virtual host
Status: Base
Compatibility: Nickname only available in Apache 1.3 or later
Module: mod_log_config

This directive specifies the format of the access log file.

The LogFormat directive can take one of two forms. In the first form, where only one argument is specified, this directive sets the log format which will be used by logs specified in subsequent TransferLog directives. The single argument can specify an explicit format as discussed in custom log formats section above. Alternatively, it can use a nickname to refer to a log format defined in a previous LogFormat directive as described below.

The second form of the LogFormat directive associates an explicit format with a nickname. This nickname can then be used in subsequent LogFormat or CustomLog directives rather than repeating the entire format string. A LogFormat directive which defines a nickname does nothing else -- that is, it only defines the nickname, it doesn't actually apply the format and make it the default. Therefore, it will not affect subsequent TransferLog directives.

TransferLog directive

Syntax: TransferLog file|pipe
Default: none
Context: server config, virtual host
Status: Base
Module: mod_log_config

This directive has exactly the same arguments and effect as the CustomLog directive, with the exception that it does not allow the log format to be specified explicitly or for conditional logging of requests. Instead, the log format is determined by the most recently specified specified LogFormat directive (that does not define a nickname). Common Log Format is used if no other format has been specified.

Example:

   LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""
   TransferLog logs/access_log

Apache HTTP Server Version 1.3