Closure Mechanics

Overview

Closures in Python are implemented using `cell` objects. When a nested function references a variable from its enclosing scope, Python creates a `cell` to store that variable. The inner function's `__closure__` attribute is a tuple of these cell objects. Each `cell` acts as an indirect pointer to the value. This provides the mental model: variables are labels, objects are boxes, and a closure cell is a special box that holds another label. A closure holds a live reference to the enclosing variable, not a snapshot, so it sees later mutations. Because closures hold live references, creating them in a loop can lead to the late-binding trap: funcs = [lambda: i for i in range(3)] print([f() for f in funcs]) # [2, 2, 2] All closures capture the same `i` variable, which equals `2` when the loop finishes. This late-binding behavior is a common trap in loop-generated callbacks.

Understanding how closures hold references rather than values explains why late-binding closures inside loops behave unexpectedly. It also clarifies how the `nonlocal` keyword can mutate state across function calls.

Where used: FastAPI dependency injection, Django signals, Decorator state

Why learn this

• Debugging unexpected behaviour in loop-generated callbacks
• Writing advanced stateful decorators

Code walkthrough

def rate_limiter(limit):
  calls = 0
  def check_limit():
    nonlocal calls
    calls += 1
    return calls <= limit
  return check_limit

limiter = rate_limiter(5)
limiter()
limiter()
print(limiter.__closure__[0].cell_contents)

Focus: `print(limiter.__closure__[0].cell_contents)` shows the `calls` cell holding the updated value of `2`.

Aha moment

handlers = []
for endpoint in ['/users', '/posts', '/comments']:
  handlers.append(lambda: f'Handling {endpoint}')

print([h() for h in handlers])

Prediction: What does this print?

Common guess: `['Handling /users', 'Handling /posts', 'Handling /comments']`

Closures capture the variable itself via a `cell`, not the value at definition time. When the lambdas run, the loop has finished and `endpoint` is `'/comments'`, so they all return that. This is called late binding.

Common mistakes

• Late binding in loops: Creating closures in a loop capturing the loop variable results in all closures pointing to the same final value. The fix is to use default arguments, like `def handler(e=endpoint):`, to capture the value eagerly.

Glossary

cell objects

Internal structures Python uses to store variables that are needed by a nested function, e.g. `func.__closure__[0].cell_contents`.

pointer

A reference that points to the location of a value in memory, rather than holding the value itself, e.g. `ref = my_list`.

Recall questions

• How does Python internally store the variables captured by a closure?
• Does a closure capture a variable's value at definition time or a reference to the variable itself?

Understanding checks

This code aims to print `[0, 1, 2]` but prints `[2, 2, 2]`. What is the bug?

The closures capture a reference (a `cell`) to the loop variable `i`, not its value at definition time. When the handlers run, the loop has ended and `i` is `2`.

Closures capture variables by reference, known as late-binding. This is a common bug when creating closures inside loops.

When a nested function captures an outer variable, how does Python store that variable?

Python creates a `cell` object to store the variable. The inner function's `__closure__` attribute contains a tuple of these cells, acting as indirect pointers to the current value.

This mental model of cells as boxes holding labels explains why closures track the most recent value of a variable rather than a static snapshot.

Practice tasks

Modify the Late-Binding Fix

The code currently captures the loop variable correctly using a default argument. Modify it so that the handler functions take an optional parameter `prefix`, returning an f-string like `'{prefix}: Handling {e}'`. Update the `print` statement to pass the prefix `'Route'` to each handler.

Challenge

Fix the Late Binding Callback Loop

The following code tries to create a list of route handlers for different endpoints, but they all return `'Handling /comments'` due to late binding: handlers = [] for endpoint in ['/users', '/posts', '/comments']: handlers.append(lambda: f'Handling {endpoint}') print([h() for h in handlers]) Fix the code so each handler captures the correct endpoint using a default argument.

Continue learning

Previous: Closures

Previous: LEGB rule: variable scope in Python

Return to Python Roadmap