On this page

Guided Exercise: Build a High‑Performance Async Application

Enrico Hochkoeppler Profile

By Enrico Hochkoeppler

Web Developer & TutoriaLibre Founder

Purpose of the exercise​

In this guided exercise, you will build a small but realistic asynchronous application using asyncio.

As your instructor, I will walk you through each component step by step, explaining not only what to write, but why each design choice matters.

By the end, you will understand how to:

This mirrors the architecture of real‑world async systems such as crawlers, API clients, and distributed workers.


Application architecture​

We will build an application composed of four asynchronous components:

This modular structure reflects how production async applications are organized.

Directory layout:


async_app/

    fetcher.py

    processor.py

    pipeline.py

    main.py

Each module has a single responsibility, which keeps the system easy to reason about and extend.


Implementing the asynchronous fetcher​

The fetcher represents an I/O‑bound operation such as an HTTP request or database query.

We simulate network latency using asyncio.sleep().


# fetcher.py

import asyncio

import random



async def fetch_item(item_id: int) -> dict:

    await asyncio.sleep(random.uniform(0.1, 0.5))

    return {"id": item_id, "value": item_id * 2}

Teaching note:

The key idea is that the coroutine yields control during the sleep.

This allows the event loop to run other coroutines, enabling high concurrency.


Implementing the asynchronous processor​

The processor transforms the fetched data.

In real systems, this might involve parsing, validation, enrichment, or computation.


# processor.py

import asyncio



async def process_item(data: dict) -> dict:

    await asyncio.sleep(0)  # cooperative yield

    return {

        "id": data["id"],

        "processed": data["value"] + 10

    }

Teaching note:

Even when the work is small, yielding control (await asyncio.sleep(0)) keeps the event loop responsive and fair.


Building the asynchronous pipeline​

The pipeline coordinates the entire workflow.

It creates tasks, runs them concurrently, and gathers results.


# pipeline.py

import asyncio

from fetcher import fetch_item

from processor import process_item



async def run_pipeline(count: int) -> list[dict]:

    fetch_tasks = [

        asyncio.create_task(fetch_item(i))

        for i in range(count)

    ]



    fetched = await asyncio.gather(*fetch_tasks)



    process_tasks = [

        asyncio.create_task(process_item(item))

        for item in fetched

    ]



    results = await asyncio.gather(*process_tasks)

    return results

Teaching note:

asyncio.create_task() schedules coroutines immediately.

asyncio.gather() waits for all tasks and returns their results in order.

This pattern is the foundation of high‑throughput async systems.


Adding an async context manager​

Async applications often require temporary resources such as connections or sessions.

We simulate such a resource with an async context manager.


# pipeline.py (continued)

class AsyncSession:

    async def __aenter__(self):

        print("Opening async session")

        await asyncio.sleep(0)

        return self



    async def __aexit__(self, exc_type, exc, tb):

        print("Closing async session")

        await asyncio.sleep(0)

Using the session inside the pipeline:


async def run_pipeline_with_session(count: int) -> list[dict]:

    async with AsyncSession():

        return await run_pipeline(count)

Teaching note:

Async context managers guarantee proper setup and cleanup even when exceptions occur.


Creating the main async entry point​

The main module runs the pipeline using asyncio.run().


# main.py

import asyncio

from pipeline import run_pipeline_with_session



async def main():

    results = await run_pipeline_with_session(20)

    for r in results:

        print(r)



if __name__ == "__main__":

    asyncio.run(main())

Teaching note:

asyncio.run() creates and manages the event loop for you.

This is the recommended way to start an async program.


What you should observe when running the application​

As you execute this program, pay attention to the following behaviors:

This demonstrates the core advantage of asynchronous design:

high concurrency with minimal overhead.


Extension exercises​

To deepen your understanding, extend the application by implementing:

Each extension reinforces the principles of structured concurrency and scalable async architecture.

Found this helpful? Share the knowledge...

... or support this project to keep it alive!

Buy Me a Coffee

Previous

Performance Optimization

Next

🛡️TypeScript