Functional programming is a programming paradigm in which the primary method of computation is the evaluation of functions. But how does Python support functional programming?
In this tutorial, you’ll learn:
- What the functional programming paradigm entails
- What it means to say that functions are first-class citizens in Python
- How to define anonymous functions with the
lambdakeyword - How to implement functional code using
map(),filter(), andreduce()
Functional programming typically plays a minor role in Python code, but it’s still good to be familiar with it. You’ll probably encounter it from time to time when reading code written by others. And you may even find situations where it’s advantageous to use Python’s functional programming capabilities in your own code.
What Is Functional Programming?
A pure function is a function whose output value follows solely from its input values without any observable side effects. In functional programming, a program consists primarily of the evaluation of pure functions. Computation proceeds by nested or composed function calls without changes to state or mutable data.
The functional paradigm is popular because it offers several advantages over other programming paradigms. Functional code is:
- High level: You describe the result you want rather than explicitly specifying the steps required to get there. Single statements tend to be concise but pack a lot of punch.
- Transparent: The behavior of a pure function can be described by its inputs and outputs, without intermediary values. This eliminates the possibility of side effects and facilitates debugging.
- Parallelizable: Routines that don’t cause side effects can more easily run in parallel with one another.
Many programming languages support some degree of functional programming. In some languages, virtually all code follows the functional paradigm. Haskell is one such example. Python, by contrast, does support functional programming but contains features of other programming models as well.
While it’s true that an in-depth description of functional programming is somewhat complex, the goal here isn’t to present a rigorous definition but to show you what you can do by way of functional programming in Python.
How Well Does Python Support Functional Programming?
To support functional programming, it’s beneficial if a function in a given programming language can do these two things:
- Take another function as an argument
- Return another function to its caller
Python plays nicely in both respects. Everything in Python is an object, and all objects in Python have more or less equal stature. Functions are no exception.
In Python, functions are first-class citizens. This means that functions have the same characteristics as values like strings and numbers. Anything you would expect to be able to do with a string or number, you can also do with a function.
For example, you can assign a function to a variable. You can then use that variable the same way you would use the function itself:
The assignment another_name = func on line 8 creates a new reference to func() named another_name. You can then call the function by either of the two names, func or another_name, as shown on lines 5 and 9.
You can display a function to the console with print(), include it as an element in a composite data object like a list, or even use it as a dictionary key:
In this example, func() appears in all the same contexts as the values "cat" and 42, and the interpreter handles it just fine.
For present purposes, what matters is that functions in Python satisfy the two criteria beneficial for functional programming listed above. You can pass a function to another function as an argument:
Here’s what’s happening in the above example:
- The call on line 9 passes
inner()as an argument toouter(). - Within
outer(), Python bindsinner()to the function parameterfunction. outer()can then callinner()directly withfunction.
This is known as function composition. Keep in mind that you’re passing the function object as an argument. If you would call the function object using parentheses, then you wouldn’t pass the function object but instead its return value.
When you pass a function to another function, the passed-in function is sometimes referred to as a callback because a call back to the inner function can modify the outer function’s behavior.
A good example of this is the Python function sorted(). Ordinarily, if you pass a list of string values to sorted(), then it sorts them in lexical order:
However, sorted() takes an optional key argument that specifies a callback function that can serve as the sorting key. So, for example, you can sort by string length instead:
sorted() can also take an optional argument that specifies sorting in reverse order. But you could manage the same thing by defining your own callback function that reverses the sense of len():
Just as you can pass a function to another function as an argument, a function can also specify another function as its return value:
Here’s what’s going on in this example:
- Lines 2 to 3:
outer()defines the local functioninner(). - Line 5:
outer()passesinner()back as its return value. - Line 8: You assign the return value from
outer()to the variablefunction.
Following this, you can call inner() indirectly through function, as shown on line 11. You can also call it indirectly using the return value from outer() without intermediate assignment, as on line 14.
As you can see, Python has the pieces in place to support functional programming nicely. But before you jump into functional code, there’s one more concept that will be helpful for you to explore: the lambda expression.
Defining an Anonymous Function With lambda
Functional programming is all about calling functions and passing them around, so it naturally involves defining a lot of functions. You can always define a function in the usual way using the def keyword.
Sometimes, it’s convenient to be able to define an anonymous function on the fly without having to give it a name. In Python, you can do this with a lambda expression.
The syntax of a lambda expression is as follows:
The following table summarizes the parts of a lambda expression:
| Component | Meaning |
|---|---|
lambda | The keyword that introduces a lambda expression |
<parameter_list> | An optional comma-separated list of parameter names |
: | Punctuation that separates <parameter_list> from <expression> |
<expression> | An expression usually involving the names in <parameter_list>, which represents the lambda function’s return value |
The value of a lambda expression is a callable function, just like a function defined with the def keyword. It takes arguments, as specified by <parameter_list>, and returns a value, as indicated by <expression>.
Here’s a quick first example:
The statement on line 1 is just the lambda expression by itself. On line 2, the Python REPL displays the value of the expression, which is a function.
The built-in Python function callable() returns True if the argument passed to it appears to be callable and False otherwise. Lines 4 and 5 show that the value returned by the lambda expression is in fact callable, as a function should be.
In this case, the parameter list consists of the single parameter s. The subsequent expression s[::-1] is slicing syntax that returns the characters in s in reverse order. So this lambda expression defines a temporary, nameless function that takes a string argument and returns the argument string with the characters reversed.
The object created by a lambda expression is a first-class citizen, just like a standard function or any other object in Python. You can assign it to a variable and then call the function using that name:
This is functionally—no pun intended—equivalent to defining reverse() with the def keyword:
The calls on lines 4 and 8 behave in the exact same way.
However, it’s not necessary to assign a variable to a lambda expression before calling it. You can also call the function defined by a lambda expression directly:
You wrapped the lambda expression into parentheses to clarify where it ends, then appended another set of parentheses and passed "I am a string" as an argument to your anonymous function. Python assigned the string argument to the parameter s, then your lambda function reversed the string and returned the result.
Here’s another example that builds on the same concept but is more complex because it uses multiple arguments in the lambda expression:
In this case, the parameters are x1, x2, and x3, and the expression is x1 + x2 + x3 / 3. This is an anonymous lambda function to calculate the average of three numbers.
The true advantage of using lambda expressions shows when you use them for short and straightforward logic. Recall when you defined reverse_len() above to serve as a callback function to sorted():
Instead of defining reverse_len, you could write a short and straightforward lambda expression:
A lambda expression will typically have a parameter list, but it’s not required. You can define a lambda function without parameters. The return value is then not dependent on any input parameters:
Note that you can only define fairly rudimentary functions with lambda. The return value from a lambda expression can only be one single expression. A lambda expression can’t contain statements like assignment or return, nor can it contain control structures such as for, while, if, else, or def.
When defining a Python function with def, you can effectively return multiple values. If a return statement in a function contains several comma-separated values, then Python packs them and returns them as a tuple:
This implicit tuple packing doesn’t work with an anonymous lambda function:
But you can explicitly return a tuple from a lambda function. You just have to denote the tuple with parentheses. You can also return a list or a dictionary from a lambda function:
A lambda expression has its own local namespace, so the parameter names don’t conflict with identical names in the global namespace. A lambda expression can access variables in the global namespace, but it can’t modify them.
There’s one final oddity to be aware of. If you find a need to include a lambda expression in a formatted string literal, or f-string, then you’ll need to enclose it in explicit parentheses:
Now you know how to define an anonymous function with lambda. Next, it’s time to delve into functional programming in Python. You’ll see how lambda functions are particularly convenient when writing functional code.
Python offers two built-in functions, map() and filter(), that fit the functional programming paradigm. A third function, reduce(), is no longer part of the core language but is still available in a module called functools. Each of these three functions takes another function as one of its arguments.
Applying a Function to an Iterable With map()
The first function on the docket is map(), which is a Python built-in function. With map(), you can apply a function to each element in an iterable in turn. The map() function will return an iterator that yields the results. This can allow for some very concise code because a map() statement can often take the place of an explicit loop.
Calling map() With a Single Iterable
You can call map() with one iterable or with many iterables. You’ll start by looking at the syntax for calling map() on a single iterable:
map(<f>, <iterable>) returns in iterator that yields the results of applying function <f> to each element of <iterable>.
Here’s an example. Suppose you’ve defined reverse(), which is a function that takes a string argument and returns its reverse using your old friend the [::-1] string slicing mechanism:
If you have a list of strings, then you can use map() to apply reverse() to each element of the list:
But remember, map() doesn’t return a list. It returns a map object, which is an iterator. To obtain the values from the iterator, you need to either iterate over it or use list():
Iterating over iterator yields the items from the original list, animals, with each string reversed by reverse(). In the second example, you collect all items that the iterator yields into a new list using list().
In this example, reverse() is a fairly short function and one you might not need outside of this use with map(). Rather than cluttering up the code with a throwaway function, you could use an anonymous lambda function instead:
If the iterable contains items that aren’t suitable for the specified function, then Python raises an exception:
In this case, the lambda function expects a string argument, which it tries to slice. The third element in the list, 3.14159, is a float object. Because it isn’t sliceable, a TypeError occurs.
Here’s a somewhat more real-world example. One of Python’s built-in string methods, str.join(), concatenates strings from an iterable, separated by the string that you call it on:
This works fine if the objects in the list are strings. If they aren’t, then str.join() raises a TypeError exception:
One way to remedy this is with a loop. Using a for loop, you can create a new list that contains string representations of the numbers in the original list. Then you can pass the new list to .join():
However, because map() applies a function to each object of a list in turn, it can often eliminate the need for an explicit loop. In this case, you can use map() to apply str() to the elements in the list before joining them:
The call to map(str, [1, 2, 3, 4, 5]) returns an iterator. This iterator, when consumed, yields the string representations of each element in the list [1, 2, 3, 4, 5], resulting in ["1", "2", "3", "4", "5"].
The "+".join() method then takes this iterator and concatenates its elements with a "+" delimiter, producing the string "1+2+3+4+5". It works and allows for less code without the need to write an explicit loop.
However, although map() accomplishes the desired effect in the example above, it would be more Pythonic to use a list comprehension instead of an explicit loop in a case like this.
Calling map() With Multiple Iterables
There’s another way that you can use map() when you’re passing more than one iterable after the function argument:
In this example, map(<f>, <iterable1>, <iterable2>, ..., <iterablen>) applies <f> to the elements in each <iterablei> in parallel and returns an iterator that yields the results.
The number of <iterablei> arguments specified to map() must match the number of arguments that <f> expects. <f> acts on the first item of each <iterablei>, and that result becomes the first item that the return iterator yields. Then, <f> acts on the second item in each <iterablei>, and that becomes the second yielded item, and so on.
A detailed example should help clarify:
In this case, add_three() takes three arguments. Correspondingly, there are three iterable arguments to map(). In this case, all are lists:
[1, 2, 3][10, 20, 30][100, 200, 300]
The first item that map() yields is the result of applying add_three() to the first element in each list:
The second item is the result of calculating add_three(2, 20, 200), and the third item is the result of calculating add_three(3, 30, 300). This is shown in the following diagram:
The return value from map() is an iterator that yields the items 111, 222, and 333. You again use list() to collect these items in a list.
Because add_three() is so short, you could readily replace it with a lambda function instead:
In this example, you’ve added a fourth element to each iteratble, which yields a fourth sum. Keep in mind that the length of the iterables isn’t relevant for this approach to work. It’s only important that you pass as many iterables as your function takes inputs.
This means that each list could have only one element or one thousand elements—this same approach still works. Try changing the number of elements in each list and running the code another time.
Additionally, this example uses implicit line continuation inside parentheses. This isn’t necessary, but it helps make the code easier to read.
If you’d like to learn more about processing iterables without a loop using map(), then check out the Python’s map(): Processing Iterables Without a Loop tutorial.
Selecting Elements From an Iterable With filter()
filter() allows you to select—or filter—items from an iterable based on evaluation of the given function. Its function signature is shown below:
filter(<f>, <iterable>) applies function <f> to each element of <iterable> and returns an iterator that yields all items for which <f> is truthy. Conversely, it filters out all items for which <f> is falsy.
In the following example, greater_than_100(x) is truthy if x > 100:
In this case, greater_than_100() is truthy for items 111, 222, and 333, so these items remain, whereas filter() discards 1, 2, and 3. As in previous examples, greater_than_100() is a short function, and you could replace it with a lambda expression instead:
The next example features range(). range(n) produces an iterator that yields the integers from 0 to n - 1. The following example uses filter() to select only the even numbers from the list and filter out the odd numbers:
You can also use filter() with other data types, such as strings. In the next example, you want to filter a list of animals so that only uppercase values remain. You can do that using filter() and a built-in string method, either with a helper function or using a lambda expression:
This works because the string method .isupper() returns True if all alphabetic characters in the string that you call it on are uppercase. If any of the characters aren’t uppercase, then .isupper() returns False.
As mentioned, the function that you pass to filter() doesn’t need to return True and False explicitly. It also works with functions that return truthy and falsy values:
In this example, you used the lambda expression lambda s: s as the function argument. This anonymous function returns the string without any changes. Because empty strings ("") are falsy in Python, filter() only keeps the non-empty strings which you then use to create a new list, ["cat", "dog"].
If you want to dive deeper into use cases for filter(), then you can read about how to extract values from iterables using filter().
Reducing an Iterable to a Single Value With reduce()
reduce() applies a function to the items in an iterable two at a time, progressively combining them to produce a single result.
As you learned earlier, reduce() is no longer part of the core language but was once a built-in function. Apparently, Guido van Rossum—the creator of Python—rather disliked reduce() and advocated for its removal from the language entirely. Here’s what he had to say about it:
So now
reduce(). This is actually the one I’ve always hated most, because, apart from a few examples involving+or*, almost every time I see areduce()call with a non-trivial function argument, I need to grab pen and paper to diagram what’s actually being fed into that function before I understand what thereduce()is supposed to do.So in my mind, the applicability of
reduce()is pretty much limited to associative operators, and in all other cases it’s better to write out the accumulation loop explicitly. (Source)
Guido actually advocated for the elimination of all three functions, reduce(), map(), and filter(), as well as lambda expressions from Python. He supported using the more Pythonic list comprehensions and generator expressions instead.
As you’ve seen, map() and filter() have remained in Python. reduce() is no longer a built-in function, but it’s still available for import from a standard-library module called functools.
There are several ways to import reduce(), but the following approach is the most straightforward:
When Python executes this line of code, the interpreter places reduce() into the global namespace and makes it available for use. The examples you’ll see in the next section will import reduce() from functools as shown above.
Calling reduce() With Two Arguments
The most straightforward reduce() call takes one function and one iterable:
In a call to reduce(<f>, <iterable>), the function <f> must be a function that takes exactly two arguments. reduce() will then progressively combine the elements in <iterable> using <f>. To start, reduce() invokes <f> on the first two elements of <iterable>. That result is then combined with the third element, then that result with the fourth, and so on, until the list is exhausted. Then, reduce() returns the final result.
Guido was right when he said that the most straightforward applications of reduce() are those using associative operators—for example, the plus operator (+):
This call to reduce() produces the result 15 from the list [1, 2, 3, 4, 5] as follows:
This is a rather roundabout way of summing the numbers in the list. While this works fine, there’s a more direct way. Python’s built-in sum() function returns the sum of the numeric values in an iterable:
Remember that the binary plus operator also concatenates strings. So this same example will progressively concatenate the strings in a list as well:
Again, there’s a way to accomplish this that many would consider more typically Pythonic using str.join():
Now consider an example using the binary multiplication operator (*). The factorial of the positive integer n is defined as follows:
You can implement a factorial function using reduce() and range() as shown below:
Once again, there’s a more straightforward way to do this. You can use factorial() provided by the standard math module:
As a final example, suppose you need to find the maximum value in a list. Python provides the built-in function max() to do this, but you could use reduce() as well:
Notice that in each of the above examples, the function passed to reduce() is a one-line function. In each case, you could have used a lambda function instead:
This may be a convenient way to avoid placing an otherwise unneeded function into the namespace. On the other hand, it may be a little harder for someone reading the code to determine your intent when you use lambda instead of defining a separate function. As is often the case, it’s a balance between readability and convenience.
Calling reduce() With an Initial Value
There’s another way to call reduce() that specifies an initial value for the reduction sequence:
When present, <initializer> specifies an initial value for the combination. In the first call to <f>, the arguments are <initializer> and the first element of <iterable>. That result is then combined with the second element of <iterable>, and so on:
Consider this diagram to better understand the sequence of function calls that Python goes through when you call reduce() with an initializer:
Again, reduce() isn’t the only way to make this calculation happen. You could also achieve the same result without reduce():
As you’ve seen in the above examples, even in cases where you can accomplish a task using reduce(), it’s often possible to find a more straightforward and Pythonic way to accomplish the same task without it. Maybe it’s not so hard to understand why reduce() was removed from the core language after all.
Despite the fact that reduce() isn’t necessary to write your Python code, it’s a remarkable function. The description at the beginning of this section states that reduce() combines elements to produce a single result.
But that result doesn’t have to be a single value, like in the examples shown above. It can also be a composite object like a list or a tuple. For that reason, reduce() is a very generalized higher-order function from which you can implement many other functions.
For example, you can implement map() in terms of reduce():
You can implement filter() using reduce() as well:
In fact, you can express any operation on a sequence of objects as a reduction.
At this point, you’ve increased your knowledge about reduce() and know why the Python community decided to hide it away in the functools module. You also better understand how to use reduce() and where to import it from if you decide to experiment with it.
If you want to learn more about how to move from a functional to a Pythonic coding style, then you can read the dedicated tutorial on Python’s reduce().
Conclusion
Functional programming is a programming paradigm in which the primary method of computation is the evaluation of pure functions. Even though Python isn’t primarily a functional language, you can still write Python following functional programming principles.
To do this, it’s a good idea to be familiar with lambda, map(), filter(), and reduce(). They can help you write concise, high-level, parallelizable code. You may also see these functions used in code that others have written, so it’s good to understand how they work.
In this tutorial, you learned:
- What functional programming is
- How functions in Python are first-class citizens, and how that makes them suitable for functional programming
- How to define a simple anonymous function with
lambda - How to implement functional code with
map(),filter(), andreduce()
Incorporating functional programming concepts into your Python code may help you write more efficient, readable, and maintainable programs. Keep experimenting, and don’t hesitate to combine functional programming with other paradigms to create robust and versatile applications.
If you have any questions, comments, or examples of how you’ve used these concepts in your own projects, please share them in the comments section below. Your feedback and experiences can help others in the community learn and grow.
Copy
Happy Pythoning!