Advanced Function Usage in Python

We continue to explore knowledge related to defining and using functions. Through previous learning, we know that functions have independent variables (parameters) and dependent variables (return values). Independent variables can be of any data type, and dependent variables can also be of any data type. So here’s a small question: can we use functions as function parameters and functions as function return values? Let me give you the conclusion first: Functions in Python are “first-class functions”. So-called “first-class functions” means that functions can be assigned to variables, functions can be used as function parameters, and functions can also be used as function return values. The usage of using one function as a parameter or return value of other functions is usually called “higher-order functions”.

Higher-Order Functions

Let’s return to an example we talked about before: design a function that takes any number of parameters and implements sum operations on elements of int or float type. We’ll slightly adjust the previous code to make the entire code more compact, as shown below.

def calc(*args, **kwargs):
    items = list(args) + list(kwargs.values())
    result = 0
    for item in items:
        if type(item) in (int, float):
            result += item
    return result

If we want the calc function above to not only do multi-parameter summation, but also implement more or even custom binary operations, what should we do? The code above can only sum because the function uses the += operator, which couples the function with addition operations. If we can decouple this relationship, the function’s versatility and flexibility will be better. The way to decouple is to turn the + operator into a function call and design it as a function parameter, as shown in the following code.

def calc(init_value, op_func, *args, **kwargs):
    items = list(args) + list(kwargs.values())
    result = init_value
    for item in items:
        if type(item) in (int, float):
            result = op_func(result, item)
    return result

Note that the function above has added two parameters, where init_value represents the initial value of the operation, and op_func represents the binary operation function. To call the modified function, we first define functions for addition and multiplication operations, as shown in the following code.

def add(x, y):
    return x + y


def mul(x, y):
    return x * y

If we want to do sum operations, we can call the calc function in the following way.

print(calc(0, add, 1, 2, 3, 4, 5))  # 15

If we want to do product operations, we can call the calc function in the following way.

print(calc(1, mul, 1, 2, 3, 4, 5))  # 120

The calc function above achieves decoupling from addition operations by turning the operator into a function parameter. This is a very clever and practical programming technique, but it may be difficult for beginners to understand. I suggest you think about it carefully. It should be noted that in the code above, there is a significant difference between passing a function as a parameter to other functions and directly calling a function. Calling a function requires parentheses after the function name, while passing a function as a parameter only requires the function name.

If we haven’t defined the add and mul functions in advance, we can also use the add and mul functions provided by the operator module in Python’s standard library. They represent binary operations for addition and multiplication respectively, and we can use them directly, as shown in the following code.

import operator

print(calc(0, operator.add, 1, 2, 3, 4, 5))  # 15
print(calc(1, operator.mul, 1, 2, 3, 4, 5))  # 120

There are quite a few higher-order functions among Python’s built-in functions. The filter and map functions we mentioned before are higher-order functions. The former can filter elements in a sequence, and the latter can map elements in a sequence. For example, if we want to remove odd numbers from an integer list and square all even numbers to get a new list, we can use these two functions directly, as shown in the following code.

def is_even(num):
    """Determine if num is even"""
    return num % 2 == 0


def square(num):
    """Calculate square"""
    return num ** 2


old_nums = [35, 12, 8, 99, 60, 52]
new_nums = list(map(square, filter(is_even, old_nums)))
print(new_nums)  # [144, 64, 3600, 2704]

Of course, to complete the functionality of the code above, you can also use list comprehension, which is simpler and more elegant.

old_nums = [35, 12, 8, 99, 60, 52]
new_nums = [num ** 2 for num in old_nums if num % 2 == 0]
print(new_nums)  # [144, 64, 3600, 2704]

Let’s discuss another built-in function sorted, which can sort elements of container data types (such as lists, dictionaries, etc.). We talked about the sort method of the list type before, which sorts list elements. The sorted function is functionally no different from the list’s sort method, but it returns a sorted list object instead of directly modifying the original list. This is what we call side-effect-free function design, meaning that calling the function produces a return value without affecting the program’s state or external environment in any other way. When using the sorted function for sorting, you can customize sorting rules through higher-order functions. We’ll illustrate this with the following example.

old_strings = ['in', 'apple', 'zoo', 'waxberry', 'pear']
new_strings = sorted(old_strings)
print(new_strings)  # ['apple', 'in', 'pear', 'waxberry', 'zoo']

The code above is not unfamiliar to everyone, but if we want to sort list elements by string length rather than alphabetical order, we can pass a parameter named key to the sorted function, assigning the key parameter to the function len that gets string length. We talked about this function in previous lessons, as shown in the following code.

old_strings = ['in', 'apple', 'zoo', 'waxberry', 'pear']
new_strings = sorted(old_strings, key=len)
print(new_strings)  # ['in', 'zoo', 'pear', 'apple', 'waxberry']

Note: The sort method of the list type also has the same key parameter. Interested readers can try it themselves.

Lambda Functions

When using higher-order functions, if the function used as a parameter or return value is very simple and can be completed in one line of code, and we don’t need to consider function reuse, we can use lambda functions. Lambda functions in Python are functions without names, so many people also call them anonymous functions. Lambda functions can only have one line of code, and the result produced by the expression in the code is the return value of this anonymous function. In the previous code, the is_even and square functions we wrote only have one line of code, so we can consider replacing them with lambda functions, as shown in the following code.

old_nums = [35, 12, 8, 99, 60, 52]
new_nums = list(map(lambda x: x ** 2, filter(lambda x: x % 2 == 0, old_nums)))
print(new_nums)  # [144, 64, 3600, 2704]

From the code above, you can see that the keyword for defining lambda functions is lambda, followed by the function’s parameters. If there are multiple parameters, separate them with commas; the part after the colon is the function’s execution body, usually an expression. The result of the expression is the return value of the lambda function, and you don’t need to write the return keyword.

We said before that functions in Python are “first-class functions”, and functions can be directly assigned to variables. After learning about lambda functions, some of the functions we wrote before can be implemented in one line of code. Let’s see if you can understand the following factorial and prime number checking functions.

import functools
import operator

# Implement factorial function in one line of code
fac = lambda n: functools.reduce(operator.mul, range(2, n + 1), 1)

# Implement prime checking function in one line of code
is_prime = lambda x: all(map(lambda f: x % f, range(2, int(x ** 0.5) + 1)))

# Call Lambda functions
print(fac(6))        # 720
print(is_prime(37))  # True

Tip 1: The reduce function used above is a function in Python’s standard library functools module. It can implement reduction operations on a set of data, similar to the calc function we defined before. The first parameter is the function representing the operation, the second parameter is the data for the operation, and the third parameter is the initial value of the operation. Obviously, the reduce function is also a higher-order function. Together with the filter and map functions, they constitute three very key actions in data processing: filtering, mapping, and reduction.

Tip 2: The lambda function for checking prime numbers above constructs a range from 2 to $\small{\sqrt{x}}$ through the range function, checking if there are factors of x in this range. The all function is also a Python built-in function. If all boolean values in the passed sequence are True, the all function returns True, otherwise it returns False.

Partial Functions

Partial functions refer to fixing certain parameters of a function to generate a new function, so that you don’t need to pass the same parameters every time you call the function. In Python, we can use the partial function from the functools module to create partial functions. For example, the int function by default treats strings as decimal integers for type conversion. If we modify its base parameter, we can define three new functions for converting binary, octal, and hexadecimal strings to integers, as shown in the following code.

import functools

int2 = functools.partial(int, base=2)
int8 = functools.partial(int, base=8)
int16 = functools.partial(int, base=16)

print(int('1001'))    # 1001

print(int2('1001'))   # 9
print(int8('1001'))   # 513
print(int16('1001'))  # 4097

I wonder if you’ve noticed that the first parameter and return value of the partial function are both functions. It processes the passed function into a new function and returns it. By constructing partial functions, we can combine actual usage scenarios to turn the original function into a new function that is more convenient to use. I wonder if you find this operation interesting.

Summary

Functions in Python are first-class functions and can be assigned to variables, used as function parameters and return values, which means we can use higher-order functions in Python. The concept of higher-order functions is not friendly to beginners, but it brings flexibility in function design. If the function we want to define is very simple, only one line of code, and we don’t need a function name to reuse it, we can use lambda functions.