- Courses
- 💻Programming Languages
- 🐍Python
- Intermediate Python & Best Practices
- Generators
On this page
Generator Functions
By Enrico Hochkoeppler
Web Developer & TutoriaLibre Founder
Understanding Generator Functions
A generator function is a special kind of function that produces a sequence of values lazily, one at a time, instead of building an entire list in memory.
Generators are essential when working with:
- Large datasets
- Infinite sequences
- Streaming data
- Pipelines where each step transforms values on demand
They allow Python to compute values only when needed, improving performance and memory efficiency.
How Generator Functions Work
A generator function looks like a normal function, but it uses the yield keyword instead of return.
def count_up_to(n):
i = 1
while i <= n:
yield i
i += 1
Calling the function:
gen = count_up_to(5)
gen is not a list — it is a generator object, which produces values one at a time.
Iterating:
for num in gen:
print(num)
The function pauses at each yield and resumes from the same point on the next iteration.
Why Generators Are Memory‑Efficient
Compare:
- A list comprehension builds the entire list in memory
- A generator function produces values on demand
Example:
def squares():
n = 1
while True:
yield n * n
n += 1
This generator can produce infinite values without ever storing them.
The yield Keyword
yield transforms a function into a generator by:
- Producing a value
- Pausing execution
- Saving local state
- Resuming when the next value is requested
This makes generators ideal for pipelines and streaming.
Generator Expressions
A generator expression is the lazy counterpart of a list comprehension.
List comprehension:
squares = [n*n for n in range(10)]
Generator expression:
squares = (n*n for n in range(10))
The second version does not build a list; it yields values one at a time.
Using Generators in Pipelines
Generators can be chained to create efficient data‑processing pipelines.
Example:
def read_lines(path):
with open(path, "r", encoding="utf-8") as f:
for line in f:
yield line.strip()
def filter_nonempty(lines):
for line in lines:
if line:
yield line
def to_upper(lines):
for line in lines:
yield line.upper()
Pipeline:
lines = read_lines("data.txt")
cleaned = filter_nonempty(lines)
upper = to_upper(cleaned)
for line in upper:
print(line)
Each stage processes one line at a time — no intermediate lists.
Sending Values into Generators
Generators can receive values using .send().
def accumulator():
total = 0
while True:
value = yield total
total += value
Usage:
gen = accumulator()
next(gen) # prime the generator
gen.send(5) # returns 5
gen.send(10) # returns 15
This pattern is used in coroutines and advanced asynchronous workflows.
Stopping a Generator
A generator ends when:
- The function reaches the end
- A
returnstatement executes - A
StopIterationexception is raised internally
Example:
def limited():
yield 1
return # ends the generator
Best Practices
- Use generators for large or infinite sequences
- Prefer generator expressions for simple transformations
- Keep generator functions small and focused
- Avoid mixing heavy logic inside generators
- Use pipelines for clean, modular processing
- Remember that generators can be iterated only once
Generators are a cornerstone of Pythonic, scalable design.
Summary
Generator functions provide lazy, memory‑efficient iteration, enabling Python to handle large datasets and streaming workflows with ease.
You learned how yield works, how to build pipelines, how generator expressions differ from list comprehensions, and how to apply best practices for clean, efficient code.
Found this helpful? Share the knowledge...
... or support this project to keep it alive!
Previous
List Comprehensions
Next
Decorators
- Understanding Generator Functions
- How Generator Functions Work
- Why Generators Are Memory‑Efficient
- The
yieldKeyword - Generator Expressions
- Using Generators in Pipelines
- Sending Values into Generators
- Stopping a Generator
- Best Practices
- Summary
