Iterator Protocol and StopIteration
Overview
The Iterator Protocol is Python's standard mechanism for looping over collections. Think of an iterable as a book, and an iterator as a bookmark that remembers which page you are on. When a `for` loop runs, it relies on two core built-in functions. - `iter(iterable)`: Gets an iterator from a collection. Returns a stateful iterator object, and raises a `TypeError` if the object does not support iteration. - `next(iterator, default)`: Asks the iterator for its next value. Returns the next item in the sequence. Raises a `StopIteration` exception if the iterator is empty, unless a `default` is provided (which it returns instead). When a `for` loop runs, it secretly calls these functions and catches `StopIteration` to know when to terminate. numbers = iter([1, 2]) print(next(numbers, 99)) print(next(numbers, 99)) print(next(numbers, 99)) # Prints 99 instead of crashing You can pass a default value to `next()` to safely prevent it from crashing when the iterator is empty. This is extremely useful for finding the first matching item in a sequence, covered fully in generator-expressions.
It provides a unified interface for iteration, meaning a `for` loop works identically on lists, files, network streams, or infinite sequences without needing to know their internal structure.
Where used: File reading, Database cursors, Generators
Why learn this
- Understanding how `for` loops actually work under the hood.
- Enables creating custom iterable objects that can be used in standard loops.
- Required for working with data streams that are too large to fit in memory.
Code walkthrough
nums = [1, 2]
nums_iter = iter(nums)
print(next(nums_iter))
print(next(nums_iter))
# next(nums_iter) would raise StopIteration
print('Done')Focus: nums_iter = iter(nums) # Creates the stateful iterator object
Aha moment
my_list = [1, 2, 3]
my_iter = iter(my_list)
for item in my_iter:
print(item)
print('Second loop:')
for item in my_iter:
print(item)Prediction: What is printed during the second loop?
Common guess: 1 2 3
The second loop prints nothing because the iterator `my_iter` was exhausted by the first loop. It reached the end and will continuously raise `StopIteration`, unlike the original list.
Common mistakes
- Exhausted Iterators: Iterators can only be consumed once, so attempting to loop a second time yields nothing. The fix is to recreate the iterator or convert it to a `list` using `list(iterator)`.
- Confusing Iterable and Iterator: An iterable (like a `list`) can spawn multiple iterators, but it is not an iterator itself. Calling `next()` directly on a `list` fails; you must call `iter()` on it first.
Glossary
- state
- The current condition, data, or position that an object remembers at a specific point in time. Example: an iterator tracking its state to know which item to `yield` next.
- unified interface
- A consistent and standardized way for different parts of a program to interact with various types of data. Example: the `for` loop provides a unified interface for iterating over lists, sets, and files.
Recall questions
- What exception is raised to signal the end of an iterator?
- What is the difference between an iterable and an iterator?
- How does a standard `for` loop know when to stop looping?
- How can you prevent `next()` from raising `StopIteration` when an iterator is empty?
Understanding checks
What is the output of this code?
10 Empty
`iter()` creates an iterator. The first `next()` returns 10 and prints it. The second `next()` returns 20 but isn't printed. The third `next()` raises `StopIteration`, which is caught, printing "Empty".
Why does calling `next([1, 2, 3])` result in a `TypeError`?
Because a `list` is an iterable, not an iterator.
`next()` can only be called on iterators. You must first create an iterator from the `list` by calling `iter([1, 2, 3])`, and then you can pass that iterator to `next()`.
Practice tasks
Manual Iteration Modification
The code below manually iterates over a list and prints all elements. Modify the code to ONLY print the first two elements and then stop, removing the `while` loop.
Challenge
Simulating a For Loop
Write a `while` loop that behaves exactly like a `for` loop to iterate over a list of names. Use `iter()`, `next()`, and a `try...except StopIteration` block to print each name and terminate cleanly.
Continue learning
Previous: for and while loops
Previous: try/except with specific exception types
Previous: Everything is an object in Python