Decorators¶
In 1 minute
- In Python, a function is a value: you can store it, pass it to another function, and return it from one.
- A decorator is a function that takes a function and returns a new one, a wrapper, that adds something before and/or after the original.
@timedon top ofdef fetch_prices(...)meansfetch_prices = timed(fetch_prices). Callers don't change at all.- The wrapper must call the original and return its result. Forget the
return, and every call gives backNone. @retry(times=3)is a decorator with settings. Usefunctools.wrapsin every decorator so the function keeps its own name.
Previously in Bazaar: products became dataclasses with type hints, checked by mypy. And you saw
@dataclassand@propertywithout knowing how@works.
1. Why learn this?¶
Bazaar now asks its sellers' supplier systems for prices. Some requests are slow; some fail when a supplier's server is busy. The team wants two things on every such function: print how long it took, and retry it if it fails.
Copying ten lines of timing and retry code into twenty functions is Module 4's copy-paste problem, times twenty.
A decorator writes that behaviour once, and adds it to any function with a single line. Frameworks use them everywhere:
Django's @login_required, the @dataclass and @property you've already used.
2. The concept¶
Functions are values¶
Before decorators, one idea: a function is a value, like a number or a list.
"""Before decorators: a function is a value. You can store it, pass it in, and return it."""
def rupees(paise):
return f"₹{paise // 100:,}"
show = rupees # no (): store the function itself, don't call it
print(show(29900))
def apply_to_all(func, prices): # a function that takes a function
return [func(p) for p in prices]
print(apply_to_all(rupees, [29900, 64900]))
def make_discount(percent): # a function that BUILDS and returns a function
def discount(paise):
return paise * (100 - percent) // 100
return discount
diwali = make_discount(10)
print(diwali(99900)) # the inner function still remembers percent = 10
show = rupees(no brackets) stores the function itself.show(29900)then calls it.apply_to_all(rupees, …)passes a function to another function.make_discount(10)builds and returns a new function. The inner function still rememberspercent, even aftermake_discounthas finished.
That last trick is all a decorator needs.
A decorator: wrap, don't change¶
"""A decorator: wrap a function with extra behaviour, without changing the function itself."""
import functools
def log_calls(func): # takes the original function...
@functools.wraps(func) # (keeps func's name and docstring on the wrapper)
def wrapper(*args, **kwargs): # ...and builds a new one around it
print(f"→ {func.__name__}{args}") # extra behaviour BEFORE
result = func(*args, **kwargs) # run the original, with whatever it was given
print(f"← {func.__name__} returned {result}") # extra behaviour AFTER
return result # hand the original's answer back
return wrapper # the decorator returns the wrapper
@log_calls # same as: supplier_price = log_calls(supplier_price)
def supplier_price(seller, product):
"""What a seller charges Bazaar for one product, in paise."""
prices = {("Patel Spices", "Masala Box"): 21000, ("Meera Handloom", "Cotton Kurta"): 48000}
return prices[(seller, product)]
total = supplier_price("Patel Spices", "Masala Box") + supplier_price("Meera Handloom", "Cotton Kurta")
print("total cost:", total)
print(supplier_price.__name__, "-", supplier_price.__doc__)
→ supplier_price('Patel Spices', 'Masala Box')
← supplier_price returned 21000
→ supplier_price('Meera Handloom', 'Cotton Kurta')
← supplier_price returned 48000
total cost: 69000
supplier_price - What a seller charges Bazaar for one product, in paise.
How it works, step by step:
log_calls(func)receives the original function.- It defines
wrapper, which prints a line, calls the original, prints another line, and returns the original's result. - It returns
wrapper. @log_callsabovedef supplier_pricemeanssupplier_price = log_calls(supplier_price). The name now points to the wrapper.
supplier_price itself never changed. Remove the @log_calls line, and it's back to normal.
*args, **kwargs ("any arguments, any keyword arguments", Module 4, Lesson 3) let one wrapper fit any function.
@functools.wraps(func) copies the original's name and docstring onto the wrapper, so supplier_price.__name__ is still supplier_price.
@timed: the same pattern, a useful job¶
"""@timed: measure how long any function takes. Written once, used on every slow function."""
import functools
import time
def timed(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
start = time.perf_counter() # a precise clock, in seconds
result = func(*args, **kwargs)
seconds = time.perf_counter() - start
print(f"{func.__name__} took {seconds:.1f} s")
return result
return wrapper
@timed
def fetch_supplier_prices(seller):
time.sleep(0.3) # pretend: a slow request to the supplier's API
return {"Patel Spices": [21000, 23500]}[seller]
@timed
def build_catalog_page():
time.sleep(0.1)
return "<html>…</html>"
prices = fetch_supplier_prices("Patel Spices")
page = build_catalog_page()
print(prices, len(page))
Before: start a clock. Middle: call the original. After: print the time, return the result. Two functions timed, zero timing code inside them.
(time.sleep(0.3) pretends to be a slow network request. Your times may differ by a little.)
A decorator with settings: @retry(times=3)¶
@retry(times=3) has brackets, so retry(times=3) runs first and must return the actual decorator. That's one extra level:
"""@retry(times=3): a decorator that takes its own settings. It retries a function that fails."""
import functools
def retry(times): # 1. takes the settings...
def decorator(func): # 2. ...returns the real decorator...
@functools.wraps(func)
def wrapper(*args, **kwargs): # 3. ...which returns the wrapper
for attempt in range(1, times + 1):
try:
return func(*args, **kwargs)
except ConnectionError as error:
print(f"attempt {attempt} failed: {error}")
raise ConnectionError(f"gave up after {times} attempts")
return wrapper
return decorator
calls = {"count": 0}
@retry(times=3)
def fetch_stock(product):
calls["count"] += 1
if calls["count"] < 3: # the supplier's server is busy twice, then answers
raise ConnectionError("supplier busy")
return {"Masala Box": 40}[product]
print("stock:", fetch_stock("Masala Box"))
Three levels: retry(times) returns decorator(func), which returns wrapper(...). The wrapper tries the original up to 3 times,
catching ConnectionError (Module 4, Lesson 6), and only gives up after the last attempt.
An analogy¶
A decorator is gift wrapping at Bazaar's packing desk.
- The product inside (the original function) doesn't change.
- The wrapping adds something around it: a "fragile" label (logging), a timer sticker (timing), "if delivery fails, try again" (retry).
- The customer (the caller) receives the same product, just wrapped. And the packer must remember to put the product inside the box: a wrapper
that forgets to
returnthe result delivers an empty box.
Questions you may have¶
Does the decorator run every time I call the function? The decorator (timed) runs once, when def runs. The wrapper runs on every call.
Can I put two decorators on one function? Yes. The one nearest to def wraps first. Going deeper shows the order.
Do I need to write decorators often? You'll use them every day (Django is full of them). You'll write a few: timing, logging, permissions, retries.
3. See it in action¶
Pick a decorator, then press Play.
4. Code example¶
The lab's exercise: a @with_gst decorator adds 18% GST to a price. The printed line looks right, but the customer's price is None:
"""A @with_gst decorator adds 18% GST to whatever price a function returns.
BUG: every decorated price comes out as None. Your task: make the wrapper hand back the result."""
import functools
def with_gst(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
price = func(*args, **kwargs)
price_with_tax = price + price * 18 // 100
print(f"{func.__name__}: {price} + 18% GST = {price_with_tax}")
return wrapper
@with_gst
def kurta_price():
return 64900
final = kurta_price()
print("Customer pays (paise):", final)
The wrapper calculates price_with_tax, prints it, and then simply ends. A function that ends without return gives back None (Module 4, Lesson 3),
and the wrapper is what kurta_price() now calls. The fix is one line at the end of the wrapper:
def with_gst(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
price = func(*args, **kwargs)
price_with_tax = price + price * 18 // 100
print(f"{func.__name__}: {price} + 18% GST = {price_with_tax}")
return price_with_tax # the caller gets this instead of None
return wrapper
5. Under the hood¶
Two silent mistakes¶
Forgetting functools.wraps:
"""Without functools.wraps, the decorated function loses its own name and docstring."""
def log_calls(func):
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
@log_calls
def supplier_price(seller):
"""What a seller charges Bazaar."""
return 21000
print(supplier_price.__name__, supplier_price.__doc__)
Everything still runs, but error messages, logs and tools now say wrapper instead of supplier_price. Debugging a traceback full of "wrapper" is no fun.
Forgetting a decorator's settings:
"""@retry needs its settings: @retry(times=3). Written as plain @retry, no error, but calling fetch_stock gives back a function, not the stock."""
import functools
def retry(times):
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
return decorator
@retry
def fetch_stock(product):
return 40
print(fetch_stock("Masala Box"))
The output is a function, not 40:
(The 0x… address will differ.) Plain @retry passed fetch_stock in as times, and the result was the inner decorator. Calling it with
"Masala Box" built a wrapper and returned it. Fix: @retry(times=3).
Going deeper: stacking, and ready-made decorators
"""Going deeper: stacking decorators (order matters), and a ready-made one from the standard library."""
import functools
def tag(label):
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
print(f"{label} before")
result = func(*args, **kwargs)
print(f"{label} after")
return result
return wrapper
return decorator
@tag("outer") # applied LAST, so it runs FIRST: outer(inner(checkout))
@tag("inner")
def checkout():
print(" checkout runs")
checkout()
calls = []
@functools.lru_cache(maxsize=128) # remembers answers: the same question isn't asked twice
def delivery_days(pincode):
calls.append(pincode) # pretend this is a slow lookup
return 2 if pincode.startswith("1") else 5
print(delivery_days("110001"), delivery_days("110001"), delivery_days("400001"))
print("slow lookups done:", calls)
outer before
inner before
checkout runs
inner after
outer after
2 2 5
slow lookups done: ['110001', '400001']
- Stacking: the decorator closest to
defwraps first; the top one wraps everything. So the top one's "before" runs first and its "after" runs last, like layers of an onion. In Django, the order of@login_requiredand@cache_pagematters for exactly this reason. functools.lru_cacheremembers each answer by its arguments. The seconddelivery_days("110001")didn't run the function at all. Great for slow lookups whose answers don't change; wrong for things that do (stock, prices).- You've used others:
@dataclassand@property(Lessons 2–3), and soon@pytest.fixture(Module 13) and Django's view decorators.
Going deeper: closures
The inner function in make_discount remembers percent after make_discount returned. A function that keeps names from the function
it was created in is called a closure. Every decorator relies on it: wrapper remembers func (and retry's wrapper also remembers times).
make_discount(10).__closure__[0].cell_contents shows the remembered 10.
Going deeper: decorating classes
A decorator can take a class too: @dataclass receives your class, adds __init__, __repr__ and __eq__ to it, and returns it.
Writing class decorators is rare; knowing that @dataclass is "just a function that changes a class" is enough.
6. Lab¶
Lab folder: labs/05-advanced-python/04-decorators/
Pass functions around, wrap a supplier lookup with @log_calls, time two slow functions with @timed, survive two failures with @retry(times=3),
meet two silent decorator mistakes, and fix the GST decorator whose wrapper returns None.
When you finish, run bash test_lab.sh (from your computer). You should see ALL CHECKS PASSED.
7. Common mistakes¶
- The wrapper doesn't return the result. Every decorated call gives
None. Fix:result = func(*args, **kwargs)…return result. - The wrapper doesn't call the original. The function silently never runs. Fix: make sure
func(...)is called inside. - No
functools.wraps. Tracebacks and logs saywrapper. Fix:@functools.wraps(func)on every wrapper. @retryinstead of@retry(times=3). No error, strange results. Fix: decorators with settings always need their brackets.- Caching things that change.
@lru_cacheon a stock lookup shows old stock forever. Fix: only cache answers that never change.
8. Quiz¶
9. What's next¶
Next: handling a million orders without loading them all into memory, and making sure files and connections are always closed: Generators and context managers.
In one line: a decorator takes a function and returns a wrapper that adds behaviour before and after the original; @timed means
f = timed(f); the wrapper must call the original and return its result; use functools.wraps, and brackets for decorators with settings.
Cheat sheet¶
import functools
def timed(func): # a decorator
@functools.wraps(func) # keep func's name and docstring
def wrapper(*args, **kwargs): # fits any function
result = func(*args, **kwargs) # call the original
return result # and hand back its result
return wrapper
def retry(times): # a decorator with settings: one more level
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
return decorator
| You write | Means |
|---|---|
@timed above def f |
f = timed(f) |
@retry(times=3) above def f |
f = retry(times=3)(f) |
| two decorators | the one nearest def wraps first |
Next in Bazaar: Bazaar's order log has grown to millions of lines. Loading it into one list would fill the server's memory. Next lesson: generators, which hand out one item at a time, and context managers, the idea behind
with.