- Courses
- 💻Programming Languages
- 🐍Python
- Advanced Python
- Concurrency with Threads
On this page
Python Threading: Concurrency and Parallelism Guide
Updated on June 25, 2026
By Enrico Hochkoeppler
Web Developer & TutoriaLibre Founder
Threading enables a Python program to run multiple tasks concurrently within the same process, making it indispensable for keeping applications responsive during I/O-bound operations such as file access, network requests, or database queries. The threading module provides a high-level interface for creating and managing threads, while synchronization primitives like Lock and thread-safe Queue objects ensure safe coordination between threads. However, the Global Interpreter Lock (GIL) in CPython limits threads to executing Python bytecode one at a time, making threading unsuitable for CPU-bound parallelism. Starting with Python 3.13, experimental free-threaded builds can disable the GIL, enabling true parallel execution of threads.
Why Threading Matters
Threading allows a Python program to perform multiple tasks concurrently within the same process. It is essential when you need to:
- Keep a program responsive while performing background work
- Overlap I/O-bound operations such as file access, network requests, or waiting for external resources
- Run multiple tasks that must share memory or coordinate through shared state
Threading is a tool for concurrency, not CPU-parallelism, and is most effective when tasks spend time waiting rather than computing.
Understanding How Python Threads Work
A thread is a lightweight unit of execution managed by the operating system. Python exposes threads through the threading module, which builds higher-level threading interfaces on top of the lower-level _thread module.
Creating and Starting a Thread
import threading
def worker():
print("Running in a thread")
t = threading.Thread(target=worker)
t.start()
t.join()
Code explanation: Thread(target=worker) creates a new thread object that will execute the worker function. start() begins execution in a new thread. join() waits for the thread to finish before the main program continues.
Passing Arguments to a Thread
import threading
import time
def crawl(link, delay=3):
print(f"crawl started for {link}")
time.sleep(delay) # Blocking I/O (simulating a network request)
print(f"crawl ended for {link}")
links = [
"https://python.org",
"https://docs.python.org",
"https://peps.python.org",
]
threads = []
for link in links:
t = threading.Thread(target=crawl, args=(link,), kwargs={"delay": 2})
threads.append(t)
t.start()
for t in threads:
t.join()
Code explanation: args passes positional arguments to the target function, and kwargs passes keyword arguments. Each thread runs the crawl function with a different URL. The main thread starts all threads and then waits for each to finish with join().
Alternative: Subclassing Thread
import threading
import time
class CrawlThread(threading.Thread):
def __init__(self, link, delay=3):
super().__init__()
self.link = link
self.delay = delay
def run(self):
print(f"crawl started for {self.link}")
time.sleep(self.delay)
print(f"crawl ended for {self.link}")
Code explanation: Subclassing Thread and overriding the run() method is an alternative approach. The run() method contains the code that executes when the thread starts.
The Role of the Global Interpreter Lock (GIL)
Python's Global Interpreter Lock (GIL) ensures that only one thread executes Python bytecode at a time. This means:
- Threads do not run Python CPU-bound code in parallel
- Threads do run I/O-bound tasks concurrently
Examples of I/O-bound tasks:
- Reading files
- Making network requests
- Waiting for user input
- Communicating with external systems
Note: The GIL is a CPython implementation detail. Some performance-oriented libraries can overcome this limitation.
Free-Threading in Python 3.13+
Starting with Python 3.13, free-threaded builds can disable the GIL, enabling true parallel execution of threads. This feature is not available by default and is governed by PEP 703.
Important: For CPU-bound parallelism, use the
multiprocessingmodule orconcurrent.futures.ProcessPoolExecutor, which bypass the GIL by using separate processes.
Designing Thread-Based Workflows
Threads are most useful when you want to keep a program responsive while performing background work.
Example: Downloading Files with Progress Indication
import threading
import time
def download():
time.sleep(2) # simulate I/O
print("Download complete")
thread = threading.Thread(target=download)
thread.start()
print("Main thread continues running")
thread.join()
Code explanation: The main thread continues executing while the download runs in the background. join() ensures the main thread waits for the download to complete before exiting.
Coordinating Threads with Locks
Because threads share memory, two threads may try to modify the same data at the same time, leading to race conditions. To prevent inconsistent state, Python provides Lock objects.
import threading
lock = threading.Lock()
counter = 0
def increment():
global counter
with lock:
value = counter
value += 1
counter = value
# Create multiple threads that call increment()
Code explanation: The with lock: block ensures that only one thread updates counter at a time. The lock is automatically acquired when entering the block and released when exiting.
When to Use Locks
- Updating shared variables
- Writing to shared files
- Managing shared data structures
Avoiding Deadlocks
A deadlock occurs when two or more threads are waiting for each other to release locks. To avoid deadlocks:
- Acquire locks in a consistent order
- Use timeouts with
lock.acquire(timeout=...) - Prefer higher-level synchronization primitives like
Queue
Using Queues for Safe Communication
The queue module provides thread-safe queues for passing data between threads.
import threading
import queue
tasks = queue.Queue()
def worker():
while True:
item = tasks.get()
if item is None:
break
print("Processing:", item)
tasks.task_done()
thread = threading.Thread(target=worker)
thread.start()
for i in range(5):
tasks.put(i)
tasks.put(None) # signal the worker to stop
thread.join()
Code explanation: tasks.get() blocks until an item is available. tasks.task_done() indicates that the processing of the item is complete. Putting None into the queue signals the worker to exit. Queues eliminate the need for manual locking and are the recommended way to coordinate threads.
Building a Worker-Thread Pattern
A common concurrency pattern is a worker pool, where multiple threads process tasks from a shared queue.
import threading
import queue
def worker(q):
while True:
task = q.get()
if task is None:
break
print(f"Processing: {task}")
q.task_done()
q = queue.Queue()
threads = []
# Create worker threads
for _ in range(4):
t = threading.Thread(target=worker, args=(q,))
t.start()
threads.append(t)
# Add tasks to the queue
for task in range(10):
q.put(task)
# Signal workers to stop
for _ in threads:
q.put(None)
# Wait for all workers to finish
for t in threads:
t.join()
Code explanation: Four worker threads share a single queue. Each worker processes tasks as they become available. The None sentinel value signals each worker to exit. This pattern scales well for I/O-bound workloads such as downloading files, processing log entries, or handling network requests.
Using ThreadPoolExecutor
For simpler thread pool management, use concurrent.futures.ThreadPoolExecutor:
from concurrent.futures import ThreadPoolExecutor
def process_task(task):
return task * 2
with ThreadPoolExecutor(max_workers=4) as executor:
results = list(executor.map(process_task, range(10)))
Code explanation: ThreadPoolExecutor manages a pool of worker threads automatically. executor.map() applies the function to each item in the iterable, distributing the work across the thread pool. The context manager ensures proper cleanup.
Daemon Threads for Background Services
A daemon thread runs in the background and does not prevent the program from exiting.
import threading
import time
def monitor():
while True:
print("Monitoring...")
time.sleep(5)
t = threading.Thread(target=monitor, daemon=True)
t.start()
Code explanation: Setting daemon=True makes the thread a daemon. Python exits when only daemon threads are left, killing any daemon threads immediately.
When to Use Daemon Threads
- Monitoring
- Periodic cleanup
- Background logging
When Not to Use Daemon Threads
- Tasks that must complete before exit
- Tasks that perform critical operations
- Tasks that write data that must be saved
Important: The daemon flag must be set before calling
start().
When to Use Threads
Threads are appropriate when:
- Tasks are I/O-bound (file operations, network requests, database queries)
- Tasks must share memory
- You need responsive or asynchronous behavior
- You want lightweight concurrency without multiple processes
Threads are not appropriate for CPU-bound parallelism; use multiprocessing or ProcessPoolExecutor instead.
Principles for Safe and Maintainable Threaded Code
- Use queues instead of shared variables whenever possible
- Use locks only when necessary and keep critical sections small
- Avoid long-running operations inside locks to prevent blocking other threads
- Prefer daemon threads only for non-critical background tasks
- Always
join()non-daemon threads to ensure clean shutdown - Keep thread responsibilities simple and focused
- Use
ThreadPoolExecutorfor managing thread pools rather than raw thread creation - Handle exceptions within threads using try/except blocks
- Choose
max_workersappropriately based on expected I/O latency; more threads can help hide latency but too many add overhead
Summary
Python's threading module provides a robust foundation for building concurrent applications. Threads enable responsive, I/O-bound workflows while sharing memory within a single process. The GIL limits threads to concurrency rather than parallelism for CPU-bound tasks, but the multiprocessing module addresses that limitation. Starting with Python 3.13, free-threaded builds offer a path toward true parallel execution. By using locks, queues, and thread pools appropriately, you can build safe, predictable, and maintainable multithreaded programs.
Found this helpful? Share the knowledge...
... or support this project to keep it alive!
Previous
Context Managers
Next
Multiprocessing
-
Why Threading Matters
-
Understanding How Python Threads Work
- Creating and Starting a Thread
- Passing Arguments to a Thread
- Alternative: Subclassing Thread
-
The Role of the Global Interpreter Lock (GIL)
- Free-Threading in Python 3.13+
-
Designing Thread-Based Workflows
- Example: Downloading Files with Progress Indication
-
Coordinating Threads with Locks
- When to Use Locks
- Avoiding Deadlocks
-
Using Queues for Safe Communication
-
Building a Worker-Thread Pattern
- Using ThreadPoolExecutor
-
Daemon Threads for Background Services
- When to Use Daemon Threads
- When Not to Use Daemon Threads
-
When to Use Threads
-
Principles for Safe and Maintainable Threaded Code
-
Summary
