Threads and processes¶
In 1 minute
- Most backend work is waiting: for a supplier's server, a database, a file. Waiting doesn't need the CPU.
- Threads let a program wait for many things at the same time. 10 supplier calls of 0.2 s each: 2.0 s one by one, 0.2 s with 10 threads.
- For CPU-heavy work (calculating), threads don't help in normal Python: the GIL lets only one thread run Python code at a time.
- Processes are separate Pythons, each with its own GIL, on its own CPU core. They speed up CPU-heavy work.
- Threads share memory. Two threads changing the same stock at once can oversell it. A Lock lets only one in at a time.
Previously in Bazaar: Bazaar became an installable package, with its own virtual environment and pinned dependencies.
1. Why learn this?¶
Bazaar's price checker asks each supplier for today's price. Every request takes about 0.2 s, almost all of it waiting for the supplier's server. With 10 suppliers, one after another, that's 2 seconds of waiting. With 100 suppliers (Lesson 8's lab), 20 seconds, while the CPU sits idle.
Doing several things at once is called concurrency. Python offers three ways: threads, processes (this lesson) and asyncio (Lesson 8).
Knowing which one fits which job is a senior-level skill, and a classic interview question.
2. The concept¶
Waiting work: use threads¶
"""Asking 10 suppliers for a price. Each answer takes 0.2 s of WAITING (like a network call).
One by one, the waits add up. With threads, they overlap."""
import time
from concurrent.futures import ThreadPoolExecutor
SUPPLIERS = ["Patel Spices", "Meera Handloom", "Kumar Footwear", "Sharma Electronics", "Supplier 5",
"Supplier 6", "Supplier 7", "Supplier 8", "Supplier 9", "Supplier 10"]
def fetch_price(supplier):
time.sleep(0.2) # waiting for the supplier's server; the CPU is idle
return len(supplier) * 1000 # a pretend price in paise
start = time.perf_counter()
one_by_one = [fetch_price(s) for s in SUPPLIERS]
print(f"one by one: {time.perf_counter() - start:.1f} s")
start = time.perf_counter()
with ThreadPoolExecutor(max_workers=10) as pool: # 10 threads, one per supplier
with_threads = list(pool.map(fetch_price, SUPPLIERS)) # same function, same order of results
print(f"10 threads: {time.perf_counter() - start:.1f} s")
print("same prices?", one_by_one == with_threads)
- A thread is a separate line of work inside the same program (Module 1). The program can have many.
ThreadPoolExecutor(max_workers=10)keeps a "pool" of 10 threads ready.pool.map(fetch_price, SUPPLIERS)gives each supplier to a thread and collects the results in the original order.- While thread 1 waits for Patel Spices, thread 2 waits for Meera Handloom, and so on. All 10 waits happen together: 0.2 s in total.
with ThreadPoolExecutor(...) as pool: is a context manager (Lesson 5): at the end of the block it waits for every thread to finish.
Computing work: threads don't help¶
Now CPU-heavy work: counting prime numbers in 4 ranges. The CPU is busy the whole time; nothing is waiting.
"""CPU-heavy work: counting primes in 4 ranges. Threads don't help here (the GIL); processes do."""
import os
import time
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
CHUNKS = [(1, 150_000), (150_000, 300_000), (300_000, 450_000), (450_000, 600_000)]
def count_primes(bounds):
low, high = bounds
count = 0
for n in range(max(low, 2), high):
if all(n % d for d in range(2, int(n ** 0.5) + 1)):
count += 1
return count
def timed(label, run):
start = time.perf_counter()
total = sum(run())
print(f"{label:<12} {time.perf_counter() - start:.1f} s ({total} primes)")
return time.perf_counter() - start
if __name__ == "__main__": # needed: worker processes import this file
print("CPU cores:", os.cpu_count())
timed("one by one", lambda: [count_primes(c) for c in CHUNKS])
with ThreadPoolExecutor(max_workers=4) as pool:
timed("4 threads", lambda: list(pool.map(count_primes, CHUNKS)))
with ProcessPoolExecutor(max_workers=4) as pool:
timed("4 processes", lambda: list(pool.map(count_primes, CHUNKS)))
CPU cores: 8
one by one 1.3 s (49098 primes)
4 threads 1.3 s (49098 primes)
4 processes 0.5 s (49098 primes)
(Your times and core count may differ.) Four threads were no faster than one by one. Four processes were more than twice as fast.
Why: the GIL¶
Normal Python (CPython) has a GIL, the Global Interpreter Lock: a rule that only one thread runs Python code at any moment. It keeps Python's own memory management simple and safe.
- When a thread waits (
time.sleep, a network call, reading a file), it lets go of the GIL, so other threads run. That's why waiting work speeds up. - When threads compute, they just take turns holding the GIL. One core does all the work; the others watch.
Processes: a separate Python each¶
A process is a whole separate running program (Module 1), with its own memory and its own GIL. ProcessPoolExecutor starts several Python
processes and sends each one a chunk of work. On 8 cores, 4 processes really compute at the same time.
The price: starting a process is slower than starting a thread, and data must be copied between processes (it's pickled and sent). So use them
for big chunks of computing, not for tiny tasks. if __name__ == "__main__": (Module 4) is required, because each worker process imports your file.
| Work | Example | Use |
|---|---|---|
| mostly waiting (I/O) | supplier APIs, database, files | threads (or asyncio, Lesson 8) |
| mostly computing (CPU) | resizing images, counting, reports | processes |
Shared data: the oversell problem¶
Threads share the program's memory. That's convenient, and dangerous. Five customers try to buy the last pair of Wireless Earbuds at the same moment:
"""Two threads touching the same stock at once can oversell it. A Lock lets only one in at a time."""
import threading
import time
stock = {"Wireless Earbuds": 1}
lock = threading.Lock()
def buy(customer, sold, use_lock):
def check_and_take():
if stock["Wireless Earbuds"] > 0: # 1. check
time.sleep(0.01) # (a tiny pause, like a slow database)
stock["Wireless Earbuds"] -= 1 # 2. take
sold.append(customer)
if use_lock:
with lock: # only one thread at a time inside
check_and_take()
else:
check_and_take()
for use_lock in (False, True):
stock["Wireless Earbuds"] = 1
sold = []
threads = [threading.Thread(target=buy, args=(c, sold, use_lock)) for c in ["Asha", "Ravi", "Fatima", "John", "Priya"]]
for t in threads:
t.start()
for t in threads:
t.join() # wait for each thread to finish
print(f"lock={use_lock}: {len(sold)} sold, stock now {stock['Wireless Earbuds']}")
Without a lock, all five threads check "stock > 0?" before any of them takes one. All see 1, all sell. Bazaar now owes four customers earbuds it doesn't have. This is a race condition (Module 1, Lesson 2): the result depends on the timing of threads.
with lock: lets only one thread at a time run the check-and-take. The others wait their turn, then see stock 0.
An analogy¶
Bazaar's packing desk:
- One by one: one packer phones each supplier and waits on hold before calling the next. Most of the day is spent listening to hold music.
- Threads: one packer with 10 phones, all on hold at once. Whichever supplier answers first gets served. Fast, because the job was waiting.
- The GIL: but if the job is packing boxes, 10 phones don't help: the packer still has only two hands.
- Processes: hire 4 packers, each at their own desk. Real parallel work, but each new packer takes time to hire and needs their own copy of the order sheet.
- A lock: the single key to the stock cupboard. Only the person holding it can check and take the last pair.
Questions you may have¶
How many threads should I use? For waiting work, enough to cover the requests in flight (10–100 is common). For processes, about the number of CPU cores.
Does Django use threads? A web server runs many requests at once, using several processes, each often with several threads (Module 20). Your view code mostly waits on the database, so this works well.
Will the GIL go away? Python 3.13 added an experimental "free-threaded" build without a GIL (see Going deeper). Normal Python still has it.
3. See it in action¶
Pick the kind of work, then press Play. Time runs from left to right.
4. Code example¶
The lab's exercise: fetch 5 supplier prices in threads. It prints 0:
"""Fetch 5 supplier prices in threads. BUG: it prints an empty list.
Your task: make the program wait for every thread before printing."""
import threading
import time
prices = []
def fetch_price(supplier):
time.sleep(0.2) # waiting for the supplier
prices.append((supplier, len(supplier) * 1000))
threads = [threading.Thread(target=fetch_price, args=(s,))
for s in ["Patel Spices", "Meera Handloom", "Kumar Footwear", "Sharma Electronics", "Supplier 5"]]
for t in threads:
t.start()
print("prices received:", len(prices))
t.start() only starts each thread; it doesn't wait. The main program rushed on to print while all 5 threads were still waiting for their suppliers.
t.join() means "wait here until this thread has finished":
for t in threads:
t.start()
for t in threads:
t.join() # wait until this thread has finished
(A ThreadPoolExecutor in a with block does the joining for you. That's one reason to prefer it.)
5. Under the hood¶
Sending a lambda to a process¶
Work goes to another process by pickling it (turning it into bytes, like JSON for Python objects). A lambda has no name to find it by, so it can't be pickled.
The traceback is long; its last line:
_pickle.PicklingError: Can't pickle <function <lambda> at 0xffff80529ee0>: attribute lookup <lambda> on __main__ failed
Fix: give it a name with def, at the top level of the file. (Threads don't have this problem: they share memory, nothing is copied.)
Timing is not exact¶
time.sleep(0.2) and the operating system's scheduling are never perfectly precise. That's why the lab's test accepts a range ("between 1.9 and 3.0 s")
and never an exact time. Real network calls vary far more.
Going deeper: free-threaded Python
Python 3.13 can be built without the GIL (the "free-threaded" build, usually run as python3.13t). Then threads can compute on several cores at once.
It's still experimental: some packages don't support it yet, and single-threaded code runs a little slower. sys._is_gil_enabled() tells you which
build you're on; the lab's image prints True (normal Python). Expect this to become more common over the coming Python versions.
Going deeper: more tools
| Tool | Use |
|---|---|
pool.submit(f, x) + concurrent.futures.as_completed(...) |
handle results as they finish, not in order |
queue.Queue |
a thread-safe line of jobs between producer and worker threads |
threading.Event, Semaphore |
signal between threads; allow only N at a time |
multiprocessing.Pool |
the older process pool API |
A lock fixes the race inside one program. Bazaar's real stock lives in PostgreSQL, shared by many servers: there, the database's own locking
(SELECT ... FOR UPDATE, Module 16) does this job.
Going deeper: why the oversell is so reliable here
Without the time.sleep(0.01), the race would happen only now and then: a thread would usually finish check-and-take before the next one ran.
That's what makes race conditions so nasty: tests pass, then a busy Diwali sale finds the bug. The small sleep plays "a slow database" and makes the
timing problem show up every time, so you can see it and fix it.
6. Lab¶
Lab folder: labs/05-advanced-python/07-threads-and-processes/
Time 10 supplier calls one by one and with threads, compare threads and processes on CPU-heavy work, oversell the last pair of earbuds and fix it with a lock, see why a lambda can't go to a process, and fix the price checker that prints before its threads are done.
When you finish, run bash test_lab.sh (from your computer). You should see ALL CHECKS PASSED.
7. Common mistakes¶
- Threads for CPU-heavy work. No faster (the GIL). Fix: processes for computing, threads for waiting.
- Forgetting to wait for threads. Results are read before they exist. Fix:
join(), or aThreadPoolExecutorin awithblock. - Changing shared data without a lock. Oversold stock, lost updates. Fix:
with lock:around check-and-change, or don't share (return results instead). - Tiny tasks in processes. Start-up and copying cost more than the work. Fix: big chunks per process.
- No
if __name__ == "__main__":with processes. Worker processes re-run your top-level code. Fix: put the start-up code under it.
8. Quiz¶
9. What's next¶
Next: asyncio, a third way to wait for many things at once, with one thread and an event loop, and the plan's lab: 100 supplier APIs, three ways, timed: asyncio, and 100 supplier APIs.
In one line: threads overlap waiting (I/O), but the GIL stops them from speeding up computing; processes compute on several cores at once; threads share memory, so protect shared changes with a lock, and always wait for threads to finish.
Cheat sheet¶
| Task | Code |
|---|---|
| Many waits at once | with ThreadPoolExecutor(max_workers=10) as pool: list(pool.map(f, items)) |
| Heavy computing on several cores | with ProcessPoolExecutor() as pool: list(pool.map(f, chunks)) (under if __name__ == "__main__":) |
| A thread by hand | t = threading.Thread(target=f, args=(x,)), t.start(), t.join() |
| Only one thread at a time | lock = threading.Lock() … with lock: … |
| How many cores? | os.cpu_count() |
| Threads | Processes | |
|---|---|---|
| Good for | waiting (network, database, files) | computing |
| Memory | shared (needs locks) | separate (data is copied) |
| GIL | one shared GIL | one GIL each |
| Start-up cost | small | bigger |
Next in Bazaar: 100 supplier APIs. Threads work, but 100 threads is a lot of machinery for a lot of waiting. Python's
asynciohandles thousands of waits with just one thread. Next lesson: asyncio, and the plan's lab: the same 100 calls, three ways, timed.