Loggers, Handlers, and Formatters
Overview
Python's `logging` module separates concerns into three parts: Loggers (the interface you call), Handlers (where the logs go), and Formatters (what the logs look like). import logging logger = logging.getLogger('app') handler = logging.StreamHandler() # destination: console handler.setFormatter(logging.Formatter('%(levelname)s: %(message)s')) logger.addHandler(handler) logger.error('disk full') # -> ERROR: disk full When `logger.error('msg')` is called, the Logger packages it into a `LogRecord`, then passes it to every attached Handler. Each Handler uses its own Formatter to turn the record into text, and sends it to its destination (console, file, network). Think of it like a newsroom: the Logger is the reporter writing the story, the Formatter is the editor formatting it for print or web, and the Handlers are the delivery trucks taking it to different audiences.
It allows a single application to send different log levels to different destinations (e.g., debug logs to a file, errors to standard error and an alerting service) without changing the logging calls in the business logic.
Where used: FastAPI, Celery, Production Web Apps
Why learn this
- Route error logs to a file while keeping info logs in the console
- Configure complex third-party library logging behavior
- Format logs specifically for log aggregation systems
Code walkthrough
import logging
import sys
logger = logging.getLogger('walkthrough')
logger.setLevel(logging.DEBUG)
console = logging.StreamHandler(sys.stdout)
console.setLevel(logging.INFO)
console.setFormatter(logging.Formatter('CONSOLE: %(message)s'))
logger.addHandler(console)
logger.debug('Debug msg')
logger.info('Info msg')Focus: console.setLevel(logging.INFO)
Aha moment
import logging
import sys
root = logging.getLogger()
root.addHandler(logging.StreamHandler(sys.stdout))
child = logging.getLogger('my.child')
child.addHandler(logging.StreamHandler(sys.stdout))
child.warning('Watch this space')Prediction: How many times will 'Watch this space' print?
Common guess: Once, because the child logger only has one handler.
It prints twice. By default, loggers propagate their records up the hierarchy. The record is handled by the child's handler, then propagates up and is handled again by the root's handler.
Common mistakes
- Duplicate logs: A child logger propagates its records to the root logger by default. Fix: either set `logger.propagate = False` on the child, or avoid adding handlers that are also present on the root logger.
- Formatter attached to Logger: Formatters are attached to Handlers, not Loggers. Attaching one to a Logger doesn't work because different destinations need different formats.
Glossary
- LogRecord
- An internal object in Python that packages all the details about a single logging event, such as the message, time, and severity, e.g. the object created when you call `logger.error('msg')`.
- propagate
- The automatic passing of a log message from a specific logger up to the main, root logger to be processed, e.g. a `logging.getLogger('myapp')` message propagates to `logging.getLogger()` by default.
Recall questions
- What is the specific responsibility of a Handler in Python's logging system?
- Why are Formatters attached to Handlers instead of Loggers?
Understanding checks
What is printed to stdout?
PREFIX: System start
The Logger generates the LogRecord, passes it to the StreamHandler, which applies the Formatter's template string before writing to stdout.
Why does this code fail with an AttributeError?
The code attempts to call `logger.setFormatter()`, but formatters must be attached to handlers (`handler.setFormatter()`), not loggers.
Formatters belong to Handlers because a single Logger might have multiple Handlers, each needing a different format.
Practice tasks
Route errors differently
Given this working code that logs everything to the console, modify it so that `logging.ERROR` messages (and above) ALSO use `error_handler`, which is configured to format errors with 'ALERT: %(message)s'. Keep the standard console handler for INFO and above.
Continue learning
Previous: Logging levels and basicConfig
Previous: Class Creation and Instantiation
Next: Structured Logging with JSON