Iterators and Generators in Python
Explain one-pass iteration and write a generator that yields values as they are requested.
Video lesson: Iterators and Generators in Python
The preview is stored on this site. YouTube loads only when you press Play. Watch on YouTube
Jump to a chapter
Read the video transcript
Select a timestamp to watch that moment on YouTube.
- 00:00
We want the nonnegative even numbers below seven. The list should contain zero, two, four, and six, and their sum should be twelve. A generator can produce those numbers one at a time without building the whole list first. That sounds like a small implementation detail until we try to use the same generator twice. The second pass can be empty, even though the first pass visibly had values. We will run that mistake, inspect the iterator, and choose the right fix for two different needs: another fresh pass or a reusable stored collection. The companion website lesson lets you implement a generator and check boundary cases in the browser console.
- 00:41
Before we run anything, distinguish a generator function from a generator object. Calling a function containing `yield` returns an object that can be iterated. It does not immediately execute the whole body or return a ready-made list. The generator remembers where it paused after each yielded value. That state is why a second pass over the same object behaves differently from a new function call. Keep the expected list and sum in mind while we test it. The first output proves that the generator did produce four values. The second result is zero because the same generator object is now exhausted. `list(numbers)` repeatedly asked it for another value until it finished. `sum(numbers)` then received no values, so its total stayed at the starting value zero.
- 01:32
The function is not broken; our use of its one-pass result is wrong for the task we imagined. This differs from a list, which stores its elements and can be iterated repeatedly. A generator is useful when values may be numerous, costly to produce, or naturally streamed. It can avoid allocating a list just to loop once. That benefit comes with a responsibility: do not assume the same generator object starts over. It will not, and there may be no way to rewind a stream. This particular failure produces a plausible number instead of an exception, which can be worse. If twelve and zero are both possible totals in your program, a missing second pass might remain unnoticed without a test.
- 02:17
`iter(numbers) is numbers` prints true: asking a generator object for an iterator gives back that same stateful object. After list consumes it, next raises StopIteration. Python's for loop catches StopIteration internally to know when a loop is done; we catch it explicitly here so you can see the mechanism. A fresh call to `even_numbers(7)` creates a different generator object with a fresh position. An iterable is something you can ask for an iterator; an iterator is the object that supplies items one by one through next. Lists are iterable but not themselves iterators.
- 02:57
This distinction explains why iterating a list twice works: each pass can request a new iterator over the stored elements. Do not confuse laziness with randomness. Our generator's values are deterministic, but they are delivered over time as the loop asks for them. If the body read a file or live data, calling it again might even produce different values. The correct fix depends on whether we want to rerun the work or preserve one snapshot. The first pair calls the generator function twice. Each call produces a fresh stream, so the list and the sum both see all four numbers. The second pair converts the stream to a list once, then reuses that stored list.
- 03:40
Both are correct here. Recomputing avoids keeping all values in memory, but it repeats the generator's work. Storing a list uses memory proportional to the output, but gives us stable, repeatable access and supports indexing. Think about the source of your values before choosing. If a generator reads a file that can change, rerunning it may not reproduce the same snapshot. If it produces a million numbers by a cheap calculation, storing all of them solely for a second simple loop may be wasteful. The lesson task asks you to implement `even_numbers(limit)` itself. `yield` makes it a generator function.
- 04:20
The loop uses `range(0, limit, 2)`, so negative and zero limits naturally produce no values. You can instead loop over all numbers and test `number % 2 == 0`; that is clear but does more work for the same result. Seven and eight have the same even values below the limit because eight itself is excluded. Zero and negative limits yield nothing. After `next(stream)` returns zero, converting the remaining stream to a list produces only two and four. That is a direct test of the paused state, not just the final values. In the website console, run the starter and then replace the placeholder with a yielding loop.
- 05:03
Complete the task and quiz, then try a generator that produces the squares of the first five nonnegative integers. Print one value with next before consuming the rest. This practice makes the one-pass behavior visible. When a later program unexpectedly reports zero rows or an empty sum, inspect whether a generator was already consumed for logging, preview, or counting. A print of `list(stream)` can be a debugging step that changes the state you are trying to diagnose. Either create a fresh source or keep the snapshot intentionally. Making that choice explicit is the core skill of this lesson.
Understand the concept
A for loop asks an iterable for an iterator, then requests one item at a time. You can see the mechanism with iterator = iter(items) and next(iterator). When no values remain, next(iterator) raises StopIteration; next(iterator, default) returns a fallback instead. A list can be iterated again, but an individual iterator is usually one-pass.
A function containing yield is a generator function. Calling it creates a generator object without running its body immediately. Each next() resumes the function until the next yield. Python keeps the local variables and position between requests. This makes generators useful when values can be produced as needed rather than stored in a full result list.
Generators are consumed. After list(generator) or a complete for loop, another pass over that same generator yields nothing. Call the generator function again to start over. A return statement inside a generator stops it; return does not deliver a normal list element. Use yield for each value the caller should receive.
The exercise yields nonnegative even integers below a limit. Build the logic from range(limit), test each number with % 2, and yield it if even. Check zero, a negative limit, an odd limit, and whether the result is a real generator. Then consume one value with next() and see that iteration continues from where it paused.
- iter() creates an iterator and next() advances it
- yield pauses and resumes a generator
- Generators are consumed after one pass
- Generator calls do not run the body immediately
See it step by step
Read the code, predict the output, then compare it with the result.
01. Advance an iterator by hand
items = iter(["Ada", "Bo"])
print(next(items))
print(next(items, "done"))
print(next(items, "done"))Ada
Bo
doneEach next call consumes one value. The last call returns the supplied default instead of raising StopIteration.
02. Yield a countdown
def countdown(start):
while start > 0:
yield start
start -= 1
print(list(countdown(3)))[3, 2, 1]The function pauses after each yield and remembers the current start value.
03. Resume a lazy squares generator
def squares(limit):
for number in range(limit):
yield number * number
values = squares(4)
print(next(values))
print(list(values))0
[1, 4, 9]The first request gets 0; converting the same generator to a list receives only its remaining values.
A closer look
Follow the reasoning, inspect each result, then try the suggested changes in the console below.
Ask an iterator for one item at a time
An iterable can provide an iterator. Calling iter on a list returns an object whose next item can be requested with next. The iterator remembers its position, so successive calls do not restart at the first element. Once every element has been delivered, another next raises StopIteration. A for loop performs that protocol for you and ends normally when it sees StopIteration.
The explicit calls below make the protocol visible. Catching StopIteration is useful in this demonstration, though most everyday code should use a for loop rather than manually advancing. An iterator is usually single-use: once exhausted, it will not refill by itself. The source list still exists, and calling iter(numbers) again would create a fresh iterator over it. Predict what a third next call will do before running.
numbers = [10, 20]
stream = iter(numbers)
print(next(stream))
print(next(stream))
try:
print(next(stream))
except StopIteration:
print("finished")10
20
finished- iter creates a stateful iterator over the existing list.
- Each next advances to the following value.
- After the last value, next raises StopIteration rather than returning None.
Yield even values from a generator function
A function containing yield is a generator function. Calling it creates a generator object without running the entire body immediately. The body advances as the caller requests values. At each yield it pauses, preserving its local variables; on the next request it resumes after that yield. This is useful for a sequence that can be produced one value at a time rather than stored as a complete list first.
The practice function even_numbers yields nonnegative even integers below limit. We take the first with next, then list consumes the remaining values. A second list call gets an empty list because that generator has been exhausted. To repeat the sequence, call even_numbers(7) again. For zero or a negative limit, range produces no values and the generator is empty. A generator can save memory for a large stream, but if you need repeated random access, a list may be clearer.
def even_numbers(limit):
for number in range(0, limit, 2):
yield number
stream = even_numbers(7)
print(next(stream))
print(list(stream))
print(list(stream))0
[2, 4, 6]
[]- The first next yields zero, the first nonnegative even integer.
- list consumes the remaining even integers below seven.
- The same generator is empty after it has been consumed.
See when a generator expression computes
Parentheses around an expression and for clause create a generator expression. It describes a sequence of calculations but does not calculate every result at creation time. Here square prints when it is called, making the timing observable. The first next computes only the square of zero. Converting the remainder to a list then computes the squares of one and two. The final printed list contains only those remaining results.
Laziness helps when an upstream source is large or values may never be requested, but it also means an error in a calculation may appear later, during consumption rather than creation. Side effects inside a generator expression make that timing harder to reason about; square prints solely as a teaching aid. For application code, prefer a calculation that returns a value without changing other state. Compare this with [square(n) for n in range(3)] to see when a list comprehension runs.
def square(number):
print(f"compute {number}")
return number * number
values = (square(n) for n in range(3))
print("created")
print(next(values))
print(list(values))created
compute 0
0
compute 1
compute 2
[1, 4]- Creating values does not call square yet.
- The first next computes only the first result.
- list evaluates the two remaining items and consumes the iterator.
Try it in Python
Edit the example and run it. Python starts in your browser the first time you click Run.
Python console
Ready to runNeed input()? Add one value per line
Your output appears here.
Need a hint?
Loop through range(limit). Inside the loop, if number % 2 == 0, use yield number. The presence of yield makes the function return a generator object.
Complete the task and select Check task to verify your code.
Quick quiz
Three questions. You can change your answers and try again.
Typical mistakes
Everyone meets these errors. See what causes them and how to fix them.
Returning a list instead of yielding values
def even_numbers(limit):
return [n for n in range(limit) if n % 2 == 0]def even_numbers(limit):
for n in range(limit):
if n % 2 == 0:
yield nWhat happens: The values are correct, but the function returns a list rather than a generator.
Use yield when the task requires items to be produced on demand.
Reusing an already consumed generator
values = (n * n for n in range(3))
print(list(values))
print(list(values))def squares():
return (n * n for n in range(3))
print(list(squares()))
print(list(squares()))What happens: The second print displays [] because the first pass consumed values.
Call the generator-producing function again to make a fresh iterator.
Calling next after exhaustion without a default
items = iter([1])
next(items)
next(items)items = iter([1])
next(items)
print(next(items, None))What happens: The second request raises StopIteration.
Use a default or a for loop if exhaustion is expected.