- Courses
- 💻Programming Languages
- 🐍Python
- Advanced Python
- Guided Exercise: Build a High‑Performance Async Application
On this page
Guided Exercise: Build a High‑Performance Async Application
By Enrico Hochkoeppler
Web Developer & TutoriaLibre Founder
Purpose of the exercise
In this guided exercise, you will build a small but realistic asynchronous application using asyncio.
As your instructor, I will walk you through each component step by step, explaining not only what to write, but why each design choice matters.
By the end, you will understand how to:
- Structure an async codebase
- Design coroutines that cooperate efficiently
- Run large numbers of tasks concurrently
- Use async context managers for resource safety
- Build a clean, maintainable async pipeline
This mirrors the architecture of real‑world async systems such as crawlers, API clients, and distributed workers.
Application architecture
We will build an application composed of four asynchronous components:
- Fetcher — simulates asynchronous network I/O
- Processor — transforms the fetched data
- Pipeline — coordinates concurrent tasks
- Main runner — executes the full async workflow
This modular structure reflects how production async applications are organized.
Directory layout:
async_app/
fetcher.py
processor.py
pipeline.py
main.py
Each module has a single responsibility, which keeps the system easy to reason about and extend.
Implementing the asynchronous fetcher
The fetcher represents an I/O‑bound operation such as an HTTP request or database query.
We simulate network latency using asyncio.sleep().
# fetcher.py
import asyncio
import random
async def fetch_item(item_id: int) -> dict:
await asyncio.sleep(random.uniform(0.1, 0.5))
return {"id": item_id, "value": item_id * 2}
Teaching note:
The key idea is that the coroutine yields control during the sleep.
This allows the event loop to run other coroutines, enabling high concurrency.
Implementing the asynchronous processor
The processor transforms the fetched data.
In real systems, this might involve parsing, validation, enrichment, or computation.
# processor.py
import asyncio
async def process_item(data: dict) -> dict:
await asyncio.sleep(0) # cooperative yield
return {
"id": data["id"],
"processed": data["value"] + 10
}
Teaching note:
Even when the work is small, yielding control (await asyncio.sleep(0)) keeps the event loop responsive and fair.
Building the asynchronous pipeline
The pipeline coordinates the entire workflow.
It creates tasks, runs them concurrently, and gathers results.
# pipeline.py
import asyncio
from fetcher import fetch_item
from processor import process_item
async def run_pipeline(count: int) -> list[dict]:
fetch_tasks = [
asyncio.create_task(fetch_item(i))
for i in range(count)
]
fetched = await asyncio.gather(*fetch_tasks)
process_tasks = [
asyncio.create_task(process_item(item))
for item in fetched
]
results = await asyncio.gather(*process_tasks)
return results
Teaching note:
asyncio.create_task() schedules coroutines immediately.
asyncio.gather() waits for all tasks and returns their results in order.
This pattern is the foundation of high‑throughput async systems.
Adding an async context manager
Async applications often require temporary resources such as connections or sessions.
We simulate such a resource with an async context manager.
# pipeline.py (continued)
class AsyncSession:
async def __aenter__(self):
print("Opening async session")
await asyncio.sleep(0)
return self
async def __aexit__(self, exc_type, exc, tb):
print("Closing async session")
await asyncio.sleep(0)
Using the session inside the pipeline:
async def run_pipeline_with_session(count: int) -> list[dict]:
async with AsyncSession():
return await run_pipeline(count)
Teaching note:
Async context managers guarantee proper setup and cleanup even when exceptions occur.
Creating the main async entry point
The main module runs the pipeline using asyncio.run().
# main.py
import asyncio
from pipeline import run_pipeline_with_session
async def main():
results = await run_pipeline_with_session(20)
for r in results:
print(r)
if __name__ == "__main__":
asyncio.run(main())
Teaching note:
asyncio.run() creates and manages the event loop for you.
This is the recommended way to start an async program.
What you should observe when running the application
As you execute this program, pay attention to the following behaviors:
- All fetch operations run concurrently
- Processing also runs concurrently
- The event loop remains responsive
- The async session opens and closes reliably
- The total runtime is far shorter than a sequential version
This demonstrates the core advantage of asynchronous design:
high concurrency with minimal overhead.
Extension exercises
To deepen your understanding, extend the application by implementing:
- A rate limiter using asyncio.Semaphore
- A retry mechanism for failed fetches
- A streaming async iterator that yields results as soon as they complete
- A worker‑pool pattern using
asyncio.Queue - Real HTTP requests using an async client library
Each extension reinforces the principles of structured concurrency and scalable async architecture.
Found this helpful? Share the knowledge...
... or support this project to keep it alive!
Previous
Performance Optimization
Next
🛡️TypeScript
- Purpose of the exercise
- Application architecture
- Implementing the asynchronous fetcher
- Implementing the asynchronous processor
- Building the asynchronous pipeline
- Adding an async context manager
- Creating the main async entry point
- What you should observe when running the application
- Extension exercises
