On this page

Python Threading: Concurrency and Parallelism Guide

Updated on June 25, 2026

Enrico Hochkoeppler Profile

By Enrico Hochkoeppler

Web Developer & TutoriaLibre Founder

Threading enables a Python program to run multiple tasks concurrently within the same process, making it indispensable for keeping applications responsive during I/O-bound operations such as file access, network requests, or database queries. The threading module provides a high-level interface for creating and managing threads, while synchronization primitives like Lock and thread-safe Queue objects ensure safe coordination between threads. However, the Global Interpreter Lock (GIL) in CPython limits threads to executing Python bytecode one at a time, making threading unsuitable for CPU-bound parallelism. Starting with Python 3.13, experimental free-threaded builds can disable the GIL, enabling true parallel execution of threads.


Why Threading Matters​

Threading allows a Python program to perform multiple tasks concurrently within the same process. It is essential when you need to:

Threading is a tool for concurrency, not CPU-parallelism, and is most effective when tasks spend time waiting rather than computing.


Understanding How Python Threads Work​

A thread is a lightweight unit of execution managed by the operating system. Python exposes threads through the threading module, which builds higher-level threading interfaces on top of the lower-level _thread module.

Creating and Starting a Thread​


import threading



def worker():

    print("Running in a thread")



t = threading.Thread(target=worker)

t.start()

t.join()

Code explanation: Thread(target=worker) creates a new thread object that will execute the worker function. start() begins execution in a new thread. join() waits for the thread to finish before the main program continues.

Passing Arguments to a Thread​


import threading

import time



def crawl(link, delay=3):

    print(f"crawl started for {link}")

    time.sleep(delay)  # Blocking I/O (simulating a network request)

    print(f"crawl ended for {link}")



links = [

    "https://python.org",

    "https://docs.python.org",

    "https://peps.python.org",

]



threads = []

for link in links:

    t = threading.Thread(target=crawl, args=(link,), kwargs={"delay": 2})

    threads.append(t)

    t.start()



for t in threads:

    t.join()

Code explanation: args passes positional arguments to the target function, and kwargs passes keyword arguments. Each thread runs the crawl function with a different URL. The main thread starts all threads and then waits for each to finish with join().

Alternative: Subclassing Thread​


import threading

import time



class CrawlThread(threading.Thread):

    def __init__(self, link, delay=3):

        super().__init__()

        self.link = link

        self.delay = delay



    def run(self):

        print(f"crawl started for {self.link}")

        time.sleep(self.delay)

        print(f"crawl ended for {self.link}")

Code explanation: Subclassing Thread and overriding the run() method is an alternative approach. The run() method contains the code that executes when the thread starts.


The Role of the Global Interpreter Lock (GIL)​

Python's Global Interpreter Lock (GIL) ensures that only one thread executes Python bytecode at a time. This means:

Examples of I/O-bound tasks:

Note: The GIL is a CPython implementation detail. Some performance-oriented libraries can overcome this limitation.

Free-Threading in Python 3.13+​

Starting with Python 3.13, free-threaded builds can disable the GIL, enabling true parallel execution of threads. This feature is not available by default and is governed by PEP 703.

Important: For CPU-bound parallelism, use the multiprocessing module or concurrent.futures.ProcessPoolExecutor, which bypass the GIL by using separate processes.


Designing Thread-Based Workflows​

Threads are most useful when you want to keep a program responsive while performing background work.

Example: Downloading Files with Progress Indication​


import threading

import time



def download():

    time.sleep(2)  # simulate I/O

    print("Download complete")



thread = threading.Thread(target=download)

thread.start()



print("Main thread continues running")

thread.join()

Code explanation: The main thread continues executing while the download runs in the background. join() ensures the main thread waits for the download to complete before exiting.


Coordinating Threads with Locks​

Because threads share memory, two threads may try to modify the same data at the same time, leading to race conditions. To prevent inconsistent state, Python provides Lock objects.


import threading



lock = threading.Lock()

counter = 0



def increment():

    global counter

    with lock:

        value = counter

        value += 1

        counter = value



# Create multiple threads that call increment()

Code explanation: The with lock: block ensures that only one thread updates counter at a time. The lock is automatically acquired when entering the block and released when exiting.

When to Use Locks​

Avoiding Deadlocks​

A deadlock occurs when two or more threads are waiting for each other to release locks. To avoid deadlocks:


Using Queues for Safe Communication​

The queue module provides thread-safe queues for passing data between threads.


import threading

import queue



tasks = queue.Queue()



def worker():

    while True:

        item = tasks.get()

        if item is None:

            break

        print("Processing:", item)

        tasks.task_done()



thread = threading.Thread(target=worker)

thread.start()



for i in range(5):

    tasks.put(i)



tasks.put(None)  # signal the worker to stop

thread.join()

Code explanation: tasks.get() blocks until an item is available. tasks.task_done() indicates that the processing of the item is complete. Putting None into the queue signals the worker to exit. Queues eliminate the need for manual locking and are the recommended way to coordinate threads.


Building a Worker-Thread Pattern​

A common concurrency pattern is a worker pool, where multiple threads process tasks from a shared queue.


import threading

import queue



def worker(q):

    while True:

        task = q.get()

        if task is None:

            break

        print(f"Processing: {task}")

        q.task_done()



q = queue.Queue()

threads = []



# Create worker threads

for _ in range(4):

    t = threading.Thread(target=worker, args=(q,))

    t.start()

    threads.append(t)



# Add tasks to the queue

for task in range(10):

    q.put(task)



# Signal workers to stop

for _ in threads:

    q.put(None)



# Wait for all workers to finish

for t in threads:

    t.join()

Code explanation: Four worker threads share a single queue. Each worker processes tasks as they become available. The None sentinel value signals each worker to exit. This pattern scales well for I/O-bound workloads such as downloading files, processing log entries, or handling network requests.

Using ThreadPoolExecutor​

For simpler thread pool management, use concurrent.futures.ThreadPoolExecutor:


from concurrent.futures import ThreadPoolExecutor



def process_task(task):

    return task * 2



with ThreadPoolExecutor(max_workers=4) as executor:

    results = list(executor.map(process_task, range(10)))

Code explanation: ThreadPoolExecutor manages a pool of worker threads automatically. executor.map() applies the function to each item in the iterable, distributing the work across the thread pool. The context manager ensures proper cleanup.


Daemon Threads for Background Services​

A daemon thread runs in the background and does not prevent the program from exiting.


import threading

import time



def monitor():

    while True:

        print("Monitoring...")

        time.sleep(5)



t = threading.Thread(target=monitor, daemon=True)

t.start()

Code explanation: Setting daemon=True makes the thread a daemon. Python exits when only daemon threads are left, killing any daemon threads immediately.

When to Use Daemon Threads​

When Not to Use Daemon Threads​

Important: The daemon flag must be set before calling start().


When to Use Threads​

Threads are appropriate when:

Threads are not appropriate for CPU-bound parallelism; use multiprocessing or ProcessPoolExecutor instead.


Principles for Safe and Maintainable Threaded Code​


Summary​

Python's threading module provides a robust foundation for building concurrent applications. Threads enable responsive, I/O-bound workflows while sharing memory within a single process. The GIL limits threads to concurrency rather than parallelism for CPU-bound tasks, but the multiprocessing module addresses that limitation. Starting with Python 3.13, free-threaded builds offer a path toward true parallel execution. By using locks, queues, and thread pools appropriately, you can build safe, predictable, and maintainable multithreaded programs.

Found this helpful? Share the knowledge...

... or support this project to keep it alive!

Buy Me a Coffee

Previous

Context Managers

Next

Multiprocessing