Advanced Function Usage in Python

We continue to explore the knowledge related to defining and using functions. Through previous learning, we know that functions have independent variables (parameters) and dependent variables (return values), where independent variables can be of any data type, and dependent variables can also be of any data type. This raises a small question: can we use functions as function parameters and use functions as function return values? Here we’ll state the conclusion first: Functions in Python are “first-class functions”. So-called “first-class functions” means that functions can be assigned to variables, functions can be used as function parameters, and functions can also be used as function return values. The practice of using a function as a parameter or return value of other functions is commonly referred to as “higher-order functions”.

Higher-Order Functions

Let’s return to an example we discussed earlier: design a function that accepts any number of parameters and performs a sum operation on elements of int or float type. We’ll slightly adjust the previous code to make it more compact, as shown below.

def calc(*args, **kwargs):
    items = list(args) + list(kwargs.values())
    result = 0
    for item in items:
        if type(item) in (int, float):
            result += item
    return result

If we want the calc function above to not only perform summation of multiple parameters but also implement more or even custom binary operations, what should we do? The code above can only sum because the function uses the += operator, which creates a coupling relationship between the function and addition operations. If we can break this coupling, the function’s versatility and flexibility will be much better. The way to break the coupling is to turn the + operator into a function call and design it as a function parameter, as shown in the code below.

def calc(init_value, op_func, *args, **kwargs):
    items = list(args) + list(kwargs.values())
    result = init_value
    for item in items:
        if type(item) in (int, float):
            result = op_func(result, item)
    return result

Note that the function above has added two parameters, where init_value represents the initial value of the operation, and op_func represents the binary operation function. To call the modified function, we first define functions for addition and multiplication operations, as shown in the code below.

def add(x, y):
    return x + y


def mul(x, y):
    return x * y

If we want to perform a sum operation, we can call the calc function as follows.

print(calc(0, add, 1, 2, 3, 4, 5))  # 15

If we want to perform a product operation, we can call the calc function as follows.

print(calc(1, mul, 1, 2, 3, 4, 5))  # 120

The calc function above achieves decoupling from addition operations by turning the operator into a function parameter. This is a very clever and practical programming technique, but it may be difficult for beginners to understand. We recommend that you carefully consider this. It’s important to note that in the code above, passing a function as a parameter to other functions is significantly different from directly calling a function - calling a function requires parentheses after the function name, while passing a function as a parameter only requires the function name.

If we haven’t predefined the add and mul functions, we can also use the add and mul functions provided by Python’s standard library operator module, which represent binary operations for addition and multiplication respectively. We can use them directly, as shown in the code below.

import operator

print(calc(0, operator.add, 1, 2, 3, 4, 5))  # 15
print(calc(1, operator.mul, 1, 2, 3, 4, 5))  # 120

Among Python’s built-in functions, there are quite a few higher-order functions. The filter and map functions we mentioned earlier are higher-order functions. The former can filter elements in a sequence, and the latter can map elements in a sequence. For example, if we want to remove odd numbers from an integer list and square all even numbers to get a new list, we can directly use these two functions to do it, as shown below.

def is_even(num):
    """Check if num is even"""
    return num % 2 == 0


def square(num):
    """Calculate square"""
    return num ** 2


old_nums = [35, 12, 8, 99, 60, 52]
new_nums = list(map(square, filter(is_even, old_nums)))
print(new_nums)  # [144, 64, 3600, 2704]

Of course, to accomplish the functionality of the code above, we can also use list comprehension, which is simpler and more elegant.

old_nums = [35, 12, 8, 99, 60, 52]
new_nums = [num ** 2 for num in old_nums if num % 2 == 0]
print(new_nums)  # [144, 64, 3600, 2704]

Let’s discuss another built-in function sorted, which can sort elements of container data types (such as lists, dictionaries, etc.). We previously discussed the sort method of the list type, which sorts list elements. The sorted function is functionally no different from the list’s sort method, but it returns a sorted list object rather than directly modifying the original list. This is called side-effect-free function design, meaning that calling the function produces a return value without causing any other effects on the program’s state or external environment. When using the sorted function for sorting, we can customize the sorting rules through higher-order functions, as illustrated in the example below.

old_strings = ['in', 'apple', 'zoo', 'waxberry', 'pear']
new_strings = sorted(old_strings)
print(new_strings)  # ['apple', 'in', 'pear', 'waxberry', 'zoo']

The code above is not unfamiliar to everyone, but if we want to sort list elements by string length rather than alphabetical order, we can pass a parameter named key to the sorted function, assigning the key parameter to the function len that gets string length, which we discussed in previous lessons, as shown in the code below.

old_strings = ['in', 'apple', 'zoo', 'waxberry', 'pear']
new_strings = sorted(old_strings, key=len)
print(new_strings)  # ['in', 'zoo', 'pear', 'apple', 'waxberry']

Note: The sort method of the list type also has the same key parameter. Interested readers can try it themselves.

Lambda Functions

When using higher-order functions, if the function used as a parameter or return value is very simple and can be completed in one line of code, and we don’t need to consider function reuse, we can use lambda functions. Lambda functions in Python are functions without names, so many people also call them anonymous functions. Lambda functions can only have one line of code, and the result produced by the expression in the code is the return value of this anonymous function. In the previous code, the is_even and square functions we wrote only have one line of code, so we can consider replacing them with lambda functions, as shown in the code below.

old_nums = [35, 12, 8, 99, 60, 52]
new_nums = list(map(lambda x: x ** 2, filter(lambda x: x % 2 == 0, old_nums)))
print(new_nums)  # [144, 64, 3600, 2704]

From the code above, we can see that the keyword for defining lambda functions is lambda, followed by the function’s parameters. If there are multiple parameters, they are separated by commas; the part after the colon is the function’s execution body, usually an expression, and the result of the expression is the return value of the lambda function, without needing to write the return keyword.

As we said earlier, functions in Python are “first-class functions”, and functions can be directly assigned to variables. After learning about lambda functions, some of the functions we wrote earlier can be implemented in one line of code. Let’s see if you can understand the following functions for calculating factorials and determining prime numbers.

import functools
import operator

# Implement factorial calculation function in one line
fac = lambda n: functools.reduce(operator.mul, range(2, n + 1), 1)

# Implement prime number determination function in one line
is_prime = lambda x: all(map(lambda f: x % f, range(2, int(x ** 0.5) + 1)))

# Call Lambda functions
print(fac(6))        # 720
print(is_prime(37))  # True

Tip 1: The reduce function used above is from Python’s standard library functools module. It can perform reduction operations on a set of data, similar to the calc function we defined earlier. The first parameter is the function representing the operation, the second parameter is the data for the operation, and the third parameter is the initial value of the operation. Obviously, the reduce function is also a higher-order function. Together with the filter and map functions, they constitute three very important actions in data processing: filtering, mapping, and reduction.

Tip 2: The lambda function for determining prime numbers above constructs a range from 2 to $\small{\sqrt{x}}$ through the range function, checking if there are any factors of x in this range. The all function is also a Python built-in function. If all boolean values in the passed sequence are True, the all function returns True, otherwise it returns False.

Partial Functions

Partial functions refer to fixing certain parameters of a function to generate a new function, so that we don’t need to pass the same parameters every time we call the function. In Python, we can use the partial function from the functools module to create partial functions. For example, the int function by default treats strings as decimal integers for type conversion. If we modify its base parameter, we can define three new functions for converting binary, octal, and hexadecimal strings to integers, as shown in the code below.

import functools

int2 = functools.partial(int, base=2)
int8 = functools.partial(int, base=8)
int16 = functools.partial(int, base=16)

print(int('1001'))    # 1001

print(int2('1001'))   # 9
print(int8('1001'))   # 513
print(int16('1001'))  # 4097

I wonder if everyone has noticed that the first parameter and return value of the partial function are both functions. It processes the passed function into a new function and returns it. By constructing partial functions, we can combine the original function with actual usage scenarios to create new functions that are more convenient to use. I wonder if everyone finds this operation interesting.

Summary

Functions in Python are first-class functions, which can be assigned to variables and can also be used as function parameters and return values. This means we can use higher-order functions in Python. The concept of higher-order functions is not friendly to beginners, but it brings flexibility to function design. If the function we want to define is very simple, only one line of code, and we don’t need a function name to reuse it, we can use lambda functions.