Python functions: arguments, lambda, decorators and generators
Advertisement
A function is a named block of code you can run whenever you need it. Functions are how you avoid repeating yourself, how you break a big problem into small ones, and how you give a piece of logic a name that describes it. Python's functions are flexible: they accept several kinds of arguments, can return several values, can be passed around like any other value and can even be created inside other functions. This post walks through all of that in a sensible order. Every output was produced by running the code on Python 3.9.
Built-in and user-defined functions
Python ships with built-in functions such as len, max, sum, sorted, print and type. You write your own with def:
def square(n):
return n * n
print(square(5), len("hello"), max(3, 9, 4)) # 25 5 9The function has a name, parameters in brackets, an indented body and an optional return. A function that has no return (or a bare return) gives back None:
def hello():
print("hi")
print(hello()) # prints hi, then NoneA string placed first in the body is the function's docstring, which help() shows. Write one for anything other people will use.
def area(w, h):
"""Return the area of a rectangle."""
return w * h
print(area.__doc__) # Return the area of a rectangle.Parameters and arguments
Parameters are the names in the definition; arguments are the values you pass when calling. Python offers several ways to pass them.
Positional and keyword arguments
def intro(name, age):
return f"{name} is {age}"
print(intro("Asha", 30), intro(age=25, name="Ravi"))
# Asha is 30 Ravi is 25Positional arguments are matched by order. Keyword arguments are matched by name, so their order does not matter. In a call, positional arguments must come before keyword ones.
Default values
def power(base, exp=2):
return base ** exp
print(power(5), power(2, 10), power(exp=3, base=2)) # 25 1024 8Parameters with defaults must come after those without. A default is evaluated once, when the function is defined, which causes the most famous Python trap:
def add_item(x, bucket=[]):
bucket.append(x)
return bucket
print(add_item(1), add_item(2)) # [1, 2] [1, 2]Both calls share one list. The fix is to use None as the default and create the list inside:
def add_item(x, bucket=None):
if bucket is None:
bucket = []
bucket.append(x)
return bucket
print(add_item(1), add_item(2)) # [1] [2]Variable-length arguments: *args and **kwargs
*args collects extra positional arguments into a tuple; **kwargs collects extra keyword arguments into a dictionary.
def total(*nums):
return sum(nums)
def show(**info):
return info
print(total(1, 2, 3), total(), show(a=1, b=2)) # 6 0 {'a': 1, 'b': 2}The same stars work in a call, to unpack a sequence or dictionary into arguments:
values = [4, 5, 6]
config = {"base": 2, "exp": 5}
print(total(*values), power(**config)) # 15 32Keyword-only and positional-only parameters
Anything after a * must be passed by name, and anything before a / must be passed by position:
def f(a, /, b, *, c):
return a, b, c
print(f(1, 2, c=3), f(1, b=2, c=3)) # (1, 2, 3) (1, 2, 3)
try:
f(1, 2, 3)
except TypeError as e:
print(e) # f() takes 2 positional arguments but 3 were givenKeyword-only parameters make calls readable (connect(host, secure=True) is clearer than connect(host, True)).
The return statement
return ends the function immediately and sends a value back. A function can return from several places:
def grade(marks):
if marks >= 60:
return "pass"
return "fail"
print(grade(75), grade(40)) # pass failReturning several values
Python does not need a special feature for this: return a, b returns one tuple, which the caller can unpack:
def min_max(items):
return min(items), max(items)
lo, hi = min_max([4, 9, 1])
print(lo, hi, min_max([4, 9, 1])) # 1 9 (1, 9)Scope: local and global variables
A variable created inside a function is local: it exists only while the function runs. A variable at the top of the file is global. A function can read a global, but assigning to the same name creates a new local one.
count = 10
def read():
return count + 1 # reads the global
def broken():
count = count + 1 # tries to use a local before it has a value
return count
print(read()) # 11
try:
broken()
except UnboundLocalError as e:
print(e) # local variable 'count' referenced before assignmentTo change a global from inside a function, declare it with global:
def bump():
global count
count += 1
bump(); bump()
print(count) # 12Use global sparingly. Passing values in and returning results is easier to test. The built-in globals() and locals() show the two sets of names. Python looks up a name in this order, called LEGB: Local, Enclosing function, Global, Built-in. A nested function can change a variable of the enclosing function with nonlocal.
Recursion
A recursive function calls itself. It needs a base case that stops the recursion and a step that moves towards it.
def factorial(n):
if n <= 1:
return 1
return n * factorial(n - 1)
def fib(n):
return n if n < 2 else fib(n - 1) + fib(n - 2)
print(factorial(5), [fib(i) for i in range(10)])
# 120 [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]Python limits the depth to about 1,000 calls, after which it raises RecursionError. Recursion is natural for trees and nested data, but a loop is usually better for simple counting. The plain fib above repeats work; add functools.lru_cache to make it fast:
from functools import lru_cache
@lru_cache(maxsize=None)
def fib2(n):
return n if n < 2 else fib2(n - 1) + fib2(n - 2)
print(fib2(80)) # 23416728348467685Functions are objects
A function is a value. You can give it another name, store it in a list or pass it to another function:
def hello(): return "hi"
greet = hello # function aliasing
print(greet(), greet is hello, greet.__name__) # hi True hello
ops = {"sq": lambda n: n * n, "neg": lambda n: -n}
print(ops["sq"](4), ops["neg"](4)) # 16 -4Deleting the old name (del hello) does not affect greet, since both names point to the same function object.
Anonymous functions: lambda
A lambda is a small function written in one expression, without a name: lambda parameters: expression.
add = lambda a, b: a + b
print(add(2, 3), (lambda x: x * 10)(4)) # 5 40Lambdas are mostly used as short arguments, especially as key= for sorting:
people = [("Asha", 30), ("Ravi", 25), ("Mina", 28)]
print(sorted(people, key=lambda p: p[1]))
# [('Ravi', 25), ('Mina', 28), ('Asha', 30)]If the logic needs more than one expression, write a normal def. A lambda is not "better"; it is only shorter.
filter(), map() and reduce()
These three apply a function across a sequence.
nums = [1, 2, 3, 4, 5, 6]
print(list(filter(lambda n: n % 2 == 0, nums))) # [2, 4, 6]
print(list(map(lambda n: n * n, nums))) # [1, 4, 9, 16, 25, 36]
from functools import reduce
print(reduce(lambda a, b: a + b, nums)) # 21
print(reduce(lambda a, b: a * b, nums, 1)) # 720filter keeps the items for which the function is true, map transforms every item, and reduce (in functools) folds the sequence into a single value. Both filter and map return lazy iterators, so wrap them in list() to see the items. For simple cases, list comprehensions say the same thing more clearly: [n * n for n in nums if n % 2 == 0].
Nested functions and closures
A function can be defined inside another. The inner function remembers the variables of the outer one, even after the outer function has finished. This is a closure:
def make_counter(start=0):
count = start
def step():
nonlocal count
count += 1
return count
return step
c = make_counter()
print(c(), c(), c()) # 1 2 3Each call to make_counter() creates a separate counter with its own count. Closures are the basis of decorators.
Decorators
A decorator is a function that takes a function and returns an improved one. It lets you add behaviour, such as timing, logging or checking, without editing the original.
import functools
def shout(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
result = func(*args, **kwargs)
return result.upper()
return wrapper
@shout
def greet(name):
"""Return a greeting."""
return f"hello {name}"
print(greet("asha"), greet.__name__) # HELLO ASHA greet@shout above def greet is shorthand for greet = shout(greet). The wrapper accepts *args, **kwargs so it works with any function, and functools.wraps keeps the original name and docstring, which helps debugging.
Decorator chaining
You can stack decorators. They apply from the bottom up:
def bold(f):
return lambda *a: "<b>" + f(*a) + "</b>"
def italic(f):
return lambda *a: "<i>" + f(*a) + "</i>"
@bold
@italic
def word(s):
return s
print(word("hi")) # <b><i>hi</i></b>italic wraps word first, then bold wraps the result. A decorator that takes its own settings is one more level of nesting:
def repeat(times):
def deco(f):
def wrapper(*a, **k):
return [f(*a, **k) for _ in range(times)]
return wrapper
return deco
@repeat(3)
def ping():
return "ping"
print(ping()) # ['ping', 'ping', 'ping']Generators
A generator is a function that uses yield instead of return. It produces values one at a time, pausing between them and remembering where it was.
def countdown(n):
while n > 0:
yield n
n -= 1
g = countdown(3)
print(next(g), next(g), next(g)) # 3 2 1
try:
next(g)
except StopIteration:
print("finished") # finished
print(list(countdown(4))) # [4, 3, 2, 1]The big advantage is memory: a generator never holds the whole sequence. Compare the sizes:
import sys
big_list = [n for n in range(1_000_000)]
big_gen = (n for n in range(1_000_000))
print(sys.getsizeof(big_list) > 1_000_000, sys.getsizeof(big_gen) < 200) # True True
print(sum(n for n in range(1_000_000))) # 499999500000The list needs several megabytes; the generator expression takes a few hundred bytes however long the range is. Generators are perfect for reading a big file line by line, producing an endless sequence, or building a pipeline where each step handles one item at a time. The trade-off is that a generator can be used once and does not support indexing or len().
Common mistakes
- A mutable default argument such as
def f(x, bucket=[]). - Forgetting
returnand gettingNoneback. - Assigning to a global name inside a function without
global, which makes it local. - Writing
print(result)inside the function when the caller needs the value returned. - A recursive function without a base case.
- Using lambda for logic that deserves a name and a docstring.
- Calling a decorator-wrapped function with arguments the wrapper does not accept.
Practice
Write average(*numbers) that returns None for no arguments. Write a decorator timed that measures how long a function takes with time.perf_counter(). Write a generator evens(limit) and use it with sum. Finally, rewrite your average function with a keyword-only round_to=2 parameter.
Continue
Functions work with the data structures in lists and dictionaries, and the exception handling post shows how functions report errors. Group functions into modules, use them as methods in object-oriented programming, and practise in the Python video course.