Iterators & the iterator protocol
Overview
The Iterator Protocol is Python's standard mechanism for looping over collections. It consists of two steps: getting an iterator (an object that tracks state) from an iterable (like a list), and repeatedly asking the iterator for its next value until it raises a `StopIteration` exception to signal it is empty. Think of an iterable as a book, and an iterator as a bookmark that remembers which page you are on. When a `for` loop runs, it secretly catches `StopIteration` to know when to terminate.
It provides a unified interface for iteration, meaning a `for` loop works identically on lists, files, network streams, or infinite sequences without needing to know their internal structure.
Where used: File reading, Database cursors, Generators
Why learn this
- Understanding how `for` loops actually work under the hood.
- Enables creating custom iterable objects that can be used in standard loops.
- Required for working with data streams that are too large to fit in memory.
Code walkthrough
nums = [1, 2]
nums_iter = iter(nums)
print(next(nums_iter))
print(next(nums_iter))
# next(nums_iter) would raise StopIteration
print('Done')Focus: nums_iter = iter(nums) # Creates the stateful iterator object
Aha moment
my_list = [1, 2, 3]
my_iter = iter(my_list)
for item in my_iter:
print(item)
print('Second loop:')
for item in my_iter:
print(item)Prediction: What is printed during the second loop?
Common guess: 1 2 3
The second loop prints nothing because the iterator `my_iter` was exhausted by the first loop. It reached the end and will continuously raise `StopIteration`, unlike the original list.
Common mistakes
- Exhausted Iterators: Iterators can only be consumed once, so attempting to loop a second time yields nothing. The fix is to recreate the iterator or convert it to a list.
- Confusing Iterable and Iterator: An iterable (like a list) can spawn multiple iterators, but it is not an iterator itself. Calling `next()` directly on a list fails; you must call `iter()` on it first.
Glossary
- state
- The current condition, data, or position that an object remembers at a specific point in time. Example: an iterator tracking its state to know which item to yield next.
- unified interface
- A consistent and standardized way for different parts of a program to interact with various types of data. Example: the `for` loop provides a unified interface for iterating over lists, sets, and files.
Recall questions
- What exception is raised to signal the end of an iterator?
- What is the difference between an iterable and an iterator?
- How does a standard `for` loop know when to stop looping?
Understanding checks
What is the output of this code?
10 Empty
`iter()` creates an iterator. The first `next()` returns 10 and prints it. The second `next()` returns 20 but isn't printed. The third `next()` raises `StopIteration`, which is caught, printing "Empty".
Why does calling `next([1, 2, 3])` result in a `TypeError`?
Because a list is an iterable, not an iterator.
`next()` can only be called on iterators. You must first create an iterator from the list by calling `iter([1, 2, 3])`, and then you can pass that iterator to `next()`.
Practice tasks
Manual Iteration Modification
The code below manually iterates over a list and prints all elements. Modify the code to ONLY print the first two elements and then stop, removing the `while` loop.
Challenge
Simulating a For Loop
Write a `while` loop that behaves exactly like a `for` loop to iterate over a list of names. Use `iter()`, `next()`, and a `try...except StopIteration` block to print each name and terminate cleanly.
Continue learning
Previous: try / except / else / finally
Next: Generator expressions