| A Logging System for Python |
Home
Download Copyright & License Recent Changes API Documentation |
| "Oh, I'm a lumberjack and I'm okay..." (Monty Python, The Lumberjack Song) |
Table of Contents
Abstract
Motivation
Influences
A Simple Example
Control Flow
Levels
Loggers
Handlers
Formatters
Filters
Configuration
The GUI Configurator
Case Scenarios
Thread Safety
On-The-Fly Reconfiguration
Module-Level Convenience Functions
Performance
Implementation Status
Acknowledgements
Still To Do
Download and Installation
Change History
Copyright and License
Abstract
There was a need for a standard logging system in Python, as comprehensively documented
in PEP 282 and enthusiastically
endorsed by the BDFL in the Parade
of the PEPs. By a happy coincidence, the package described here was already in
development and fairly close in intent and design to the description in the aforementioned
PEP, borrowing as it did heavily from JSR-47 (now JDK 1.4's java.util.logging package) and
log4j. This page describes it
in more detail. As I have tweaked the package to meet comments on PEP 282, I have structured
this page in the same way as the original PEP. This package is now part of Python 2.3, but if
you have an earlier version of Python, you can download the package from here and use it
with Python versions between 1.5.2 and 2.2.x.
Motivation
The Python community has been incredibly helpful to me, a relative newcomer to the
language. Python and its community has certainly saved me much time and effort, and it
seems appropriate to give something back to the community by offering up this package for
people to try. Any feedback will be
gratefully accepted.
Influences
This package owes its greatest debt to Apache log4j. Due notice was also taken
of log4j's comprehensive critique (no longer online) of JSR47. This package bears a close
resemblance to log4j, but is not a close translation. I have attempted to be more minimalist
(and hopefully more Pythonic) in my approach. You be the judge!
A Simple Example
Using the package doesn't get much simpler. It is packaged as a standard Python package
called (unsurprisingly) logging. You just need to import logging
and you're ready to go. Minimal example:
# --- app.py --------------------------------------------------------------------
import logging
logging.warn("Hello")
logging.error("Still here...")
logging.warn("Goodbye")
When you run app.py, the results are:
WARNING:root:Hello ERROR:root:Still here... WARNING:root:Goodbye
Don't worry about the format of the output - it's all configurable. Here's a slightly
more involved example; if you've just looked at PEP 282 you will probably get a feeling of
dej� vu. (This is intentional.)
# --- mymodule.py --------------------------------------------------------------------
import logging
log = logging.getLogger("MyModule")
def doIt():
log.debug("doin' stuff")
#do stuff...but suppose an error occurs?
raise TypeError, "bogus type error for testing"
# --- myapp.py -----------------------------------------------------------------------
import logging, mymodule
logging.basicConfig()
log = logging.getLogger("MyApp")
log.setLevel(logging.DEBUG) #set verbosity to show all messages of severity >= DEBUG
log.info("Starting my app")
try:
mymodule.doIt()
except Exception, e:
log.exception("There was a problem.")
log.info("Ending my app")
When you run myapp.py, the results are:
INFO:MyApp:Starting my app ERROR:MyApp:There was a problem. Traceback (most recent call last): File "myapp.py", line 9, in ? mymodule.doIt() File "mymodule.py", line 7, in doIt raise TypeError, "Bogus type error for testing" TypeError: Bogus type error for testing INFO:MyApp:Ending my app
But don't worry - the above output is not hardcoded into the package. It's just an
example of what you can do with very little work. As you can see, exceptions are handled
as one would expect.
Control Flow
The package pretty much matches the PEP regarding control flow. The user of the package
makes logging calls on instances of Logger, which are organized into a
hierarchy based on a "dotted name" namespace. This hierarchy is embodied in an
encapsulated singleton Manager instance (which can be ignored by users of the
package, for most purposes). Based on the type of logging call and the logging
configuration (see below), the call may be passed through a set of Filter
instances to decide whether it should be dropped. If not, then the logger consults a set
of Handler instances which are associated with it, and asks each handler
instance to "handle" the logging event. By default, the system moves up the
namespace hierarchy and invokes handlers on all loggers at or above the level of the
logger on which the logging call was made. (You can override this by setting a logger's
"propagate" attribute to 0 - no traversal up the hierarchy is made from such a
logger. But I'm getting ahead of myself...)
Handlers are passed LogRecord instances which (should) contain all the
information we're interested in logging. Handlers, too, can invoke filters to determine
whether a record should be dropped. If not, then the handler takes a handler-specific
action to actually log the record to a file, the console or whatever.
Levels
The following levels are implemented by default:
DEBUG INFO WARNING ERROR CRITICAL
The CRITICAL level replaces the earlier FATAL level. You can
use either (for now), but CRITICAL is preferred since FATAL
implies that the application is about to terminate. This is not true for many systems
which use logging - for example, a Web server application which encounters a CRITICAL
condition (e.g. running out of resources) will still try to keep going as best it can.
FATAL (and the corresponding fatal() methods) may be removed
in future versions of the package. Currently, CRITICAL is synonymous with FATAL
and critical() methods are synonymous with fatal().
Exceptions logged via exception() use the ERROR level for
logging. If it is desired to log exception information with arbitrary logging levels, this
can be done by passing a keyword argument exc_info with a true value to the
logging methods (see the API documentation for more details).
The levels are not deeply hardcoded into the package - the number of levels, their
numeric values and their textual representation are all configurable. The above levels
represent the experience of the log4j community and so are provided as the default levels
for users who do not have very specific requirements in this area.
The example script log_test4.py shows the use of bespoke logging levels
(as well as filtering by level at logger and handler, as well as use of filter classes).