Nested comprehensions
Overview
A nested comprehension allows you to put multiple `for` clauses in a single comprehension, or place a comprehension completely inside another one. Think of it like reading a book: you iterate through the chapters (outer loop), and then through the pages within each chapter (inner loop). When combining multiple `for` clauses, the order reads left-to-right exactly as the loops would be nested top-to-bottom in standard code. matrix = [[1, 2], [3, 4]] flat = [x for row in matrix for x in row] grid = [[x for x in row] for row in matrix] The first example flattens the structure because the multiple `for` clauses are in the same comprehension. The second example retains the two-dimensional structure because one comprehension is nested entirely inside the result expression of the other. # Trap: Getting the loop order backwards matrix = [[1, 2], [3, 4]] flat = [x for x in row for row in matrix] # Raises NameError The loop order in a flattened comprehension is evaluated strictly from left to right. This edge case raises a `NameError` when the inner loop tries to reference a variable that has not been bound yet by the outer loop. # Trap: Confusing filter placement matrix = [[1, 2], [3, 4, 5]] evens = [x for row in matrix if len(row) > 2 for x in row if x % 2 == 0] You can attach `if` conditions to any of the `for` clauses. The condition filters the variables immediately available to its left, meaning the outer `if len(row) > 2` evaluates before the inner loop even runs.
Nested data structures like `list`s of `list`s are common (e.g. database rows containing JSON arrays, paginated API endpoints returning pages of results). Nested comprehensions provide a concise way to flatten those structures or generate combinatorial cross-products without verbose `for` loop boilerplate.
Where used: Flattening nested `list`s into a single flat `list`, Generating combinations or Cartesian products, Initialising 2D matrices
Why learn this
- Flattening a `list` of pages returned by an API into a single `list` of items
- Understanding the execution order of multiple `for` clauses
- Knowing when a nested comprehension becomes too complex and should be rewritten as standard `for` loops
Code walkthrough
pages = [['item1', 'item2'], ['item3']]
# Equivalent to:
# for page in pages:
# for item in page:
flat_items = [item for page in pages for item in page]
print(flat_items)Focus: The outer loop `for page in pages` appears first on the left, followed immediately by the inner loop `for item in page`, successfully flattening the nested `list`.
Common mistakes
- Getting the loop order backwards: When flattening, developers often write `[x for x in sublist for sublist in matrix]`, which raises a `NameError`. The correct order matches nested `for` loops exactly. The outer loop (`for sublist in matrix`) comes first, then the inner loop (`for x in sublist`).
- Ignoring readability: Nesting three or more `for` loops inside a single comprehension makes the code nearly unreadable. If the comprehension requires mental gymnastics to parse, use standard `for` loops.
Glossary
- flatten
- To convert a complex, layered structure (like a `list` containing other `list`s) into a single, simple, one-dimensional `list`. e.g. `[[1, 2], [3, 4]]` flattened becomes `[1, 2, 3, 4]`.
- combinatorial cross-products
- Every possible combination formed by matching each item in one `list` with every item in another `list`. e.g. `[x + y for x in 'AB' for y in '12']` gives `['A1', 'A2', 'B1', 'B2']`.
Recall questions
- If you have a nested comprehension with multiple `for` clauses, what order are they evaluated in?
- What is the primary use case for `[item for sublist in matrix for item in sublist]`?
Understanding checks
Why does this code raise a `NameError: name 'row' is not defined`?
Because the multiple `for` clauses in a comprehension are evaluated from left to right. The code attempts to iterate over `row` (`for num in row`) before `row` has been defined by the outer loop (`for row in matrix`).
The correct order must match how standard nested `for` loops are written top-to-bottom. The outer loop comes first, so it should be `[num for row in matrix for num in row]`.
What is the output of this code?
[[1, 4], [9, 16]]
This is a `list` comprehension nested inside another `list` comprehension, not multiple `for` clauses in a single comprehension. The outer comprehension iterates over rows in the matrix. For each row, the inner comprehension creates a new `list` with squared numbers, thus retaining the 2D structure.
What is the value of `res` after this code executes?
[4]
The outer loop filters for rows with a length greater than `2`, keeping only `[3, 4, 5]`. The inner loop then iterates through that row and filters for even numbers, returning `4`.
Practice tasks
Flatten a list of tuples
Modify the provided code to use a nested `list` comprehension instead of nested `for` loops to flatten the `list` of `tuple`s.
Challenge
Extract positive values
Given a matrix `[[-1, 2], [3, -4]]`, write a nested `list` comprehension that flattens it AND filters it to only include positive numbers `> 0`.
Continue learning
Previous: List Comprehensions
Previous: Dictionary and set comprehensions