The Walrus Operator: Python's Assignment Expressions

Hello, Walrus!

To get a first impression of what assignment expressions are all about, start your REPL and play around with the following code:

Line 1 shows a traditional assignment statement where the value False is assigned to walrus. Next, on line 5, you use an assignment expression to assign the value True to walrus. After both lines 1 and 5, you can refer to the assigned values by using the variable name walrus.

You might be wondering why you’re using parentheses on line 5, and you’ll learn why the parentheses are needed later on in this tutorial.

There’s a subtle—but important—difference between the two types of assignments with the walrus variable. An assignment expression returns the value, while a traditional assignment doesn’t. You can see this in action when the REPL doesn’t print any value after walrus = False on line 1 but prints out True after the assignment expression on line 5.

You can see another important aspect about walrus operators in this example. Though it might look new, the := operator does not do anything that isn’t possible without it. It only makes certain constructs more convenient and can sometimes communicate the intent of your code more clearly.

Now you have a basic idea of what the := operator is and what it can do. It’s an operator used in assignment expressions, which can return the value being assigned, unlike traditional assignment statements. To get deeper and really learn about the walrus operator, continue reading to see where you should and shouldn’t use it.

Implementation

Like most new features in Python, assignment expressions were introduced through a Python Enhancement Proposal (PEP). PEP 572 describes the motivation for introducing the walrus operator, the details of the syntax, and examples where the := operator can be used to improve your code.

This PEP was originally written by Chris Angelico in February 2018. Following some heated discussion, PEP 572 was accepted by Guido van Rossum in July 2018.

Since then, Guido announced that he was stepping down from his role as benevolent dictator for life (BDFL). Since early 2019, the Python language has been governed by an elected steering council instead.

The walrus operator was implemented by Emily Morehouse, and made available in the first alpha release of Python 3.8.

Motivation

In many languages, including C and its derivatives, assignment statements are also expressions. This can be both very powerful and a source of confusing bugs. For example, the following code is valid C but doesn’t execute as intended:

Here, if (x = y) will evaluate to true, and the code snippet will print out x and y are equal (x = 8, y = 8). Is this the result you were expecting? You were trying to compare x and y. How did the value of x change from 3 to 8?

The problem is that you’re using the assignment operator (=) instead of the equality comparison operator (==). In C, x = y is an expression that evaluates to the value of y. In this example, x = y is evaluated as 8, which is considered truthy in the context of the if statement.

Take a look at a corresponding example in Python. This code causes a SyntaxError:

Unlike the C example, this Python code gives you an explicit error instead of a bug.

The distinction between assignment statements and assignment expressions in Python is useful in order to avoid these kinds of hard-to-find bugs. PEP 572 argues that Python is better suited to having different syntax for assignment statements and expressions instead of turning the existing assignment statements into expressions.

One design principle underpinning the walrus operator is that there are no identical code contexts where both an assignment statement using the = operator and an assignment expression using the := operator would be valid. For example, you can’t do a plain assignment with the walrus operator:

In many cases, you can add parentheses (()) around the assignment expression to make it valid Python:

Writing a traditional assignment statement with = isn’t allowed inside such parentheses. This helps you catch potential bugs.

Later on in this tutorial, you’ll learn more about situations where the walrus operator isn’t allowed, but first you’ll learn about the situations where you might want to use it.

Debugging

Arguably one of the best use cases for the walrus operator is when debugging complex expressions. Say that you want to find the distance between two locations along the earth’s surface. One way to do this is to use the haversine formula:

ϕ represents the latitude, and λ represents the longitude of each location. To demonstrate this formula, you can calculate the distance between Oslo (59.9°N 10.8°E) and Vancouver (49.3°N 123.1°W) as follows:

As you can see, the distance from Oslo to Vancouver is just under 7,200 kilometers.

Now, say that you need to double-check your implementation and want to see how much the haversine terms contribute to the final result. You could copy and paste the term from your main code to evaluate it separately. However, you could also use the := operator to give a name to the subexpression that you’re interested in:

The advantage of using the walrus operator here is that you calculate the value of the full expression and keep track of the value of ϕ_hav at the same time. This allows you to confirm that you didn’t introduce any errors while debugging.

Lists and Dictionaries

Lists are powerful data structures in Python that often represent a series of related attributes. Similarly, dictionaries are used all over Python and are great for structuring information.

Sometimes when setting up these data structures, you end up performing the same operation several times. As a first example, calculate some basic descriptive statistics of a list of numbers and store them in a dictionary:

Note that both the sum and the length of the numbers list are calculated twice. The consequences are not too bad in this simple example, but if the list were larger or the calculations were more complicated, you might want to optimize the code. To do this, you can first move the function calls out of the dictionary definition:

The variables num_length and num_sum are only used to optimize the calculations inside the dictionary. By using the walrus operator, you can make this role clearer:

You’ve now defined num_length and num_sum inside the definition of description. This is a clear hint to anybody reading this code that these variables are just used to optimize these calculations and aren’t used again later.

In the next example, you’ll work with a bare-bones implementation of the wc utility for counting lines, words, and characters in a text file:

This script can read one or several text files and report how many lines, words, and characters each of them contains. Here’s a breakdown of what’s happening in the code:

• Line 4 loops over each filename provided by the user. The sys.argv list contains each argument given on the command line, starting with the name of your script. For more information about sys.argv, you can check out Python Command Line Arguments.
• Line 5 converts each filename string to a pathlib.Path object. Storing a filename in a Path object allows you to conveniently read the text file in the next lines.
• Lines 6 to 10 construct a tuple of counts to represent the number of lines, words, and characters in one text file.
• Line 7 reads a text file and calculates the number of lines by counting newlines.
• Line 8 reads a text file and calculates the number of words by splitting on whitespace.
• Line 9 reads a text file and calculates the number of characters by finding the length of the string.
• Line 11 prints all three counts together with the filename to the console. The *counts syntax unpacks the counts tuple. In this case, the print() statement is equivalent to print(counts[0], counts[1], counts[2], path).

To see wc.py in action, you can use the script on itself as follows:

In other words, the wc.py file consists of 11 lines, 32 words, and 307 characters.

If you look closely at this implementation, then you’ll notice that it’s far from optimal. In particular, it repeats the call to path.read_text() three times. That means that the program reads each text file three times. You can use the walrus operator to avoid the repetition:

You assign the contents of the file to text, which you reuse in the next two calculations. Note the placement of parentheses that help scope that text will refer to the text in the file and not the number of lines.

The program still functions the same, although the word and character counts have changed:

As in the earlier examples, an alternative approach is to define text before the definition of counts:

While this is one line longer than the previous implementation, it probably provides the best balance between readability and efficiency. The := assignment expression operator isn’t always the most readable solution even when it makes your code more concise.

List Comprehensions

List comprehensions are great for constructing and filtering lists. They clearly state the intent of the code and will usually run quite fast.

There’s one list comprehension use case where the walrus operator can be particularly useful. Say that you want to apply some computationally expensive function, slow(), to the elements in your list and filter on the resulting values. You could do something like the following:

Here, you filter the numbers list and leave the positive results from applying slow(). The problem with this code is that this expensive function is called twice.

A very common solution for this type of situation is rewriting your code to use an explicit for loop:

This will only call slow() once. Unfortunately, the code is now more verbose, and the intent of the code is harder to understand. The list comprehension had clearly signaled that you were creating a new list, while this is more hidden in the explicit for loop since several lines of code separate the list creation and the use of .append(). Additionally, a list comprehension runs faster than the repeated calls to .append().

You can code some other solutions by using a filter() expression or a kind of double list comprehension:

The good news is that there’s only one call to slow() for each number. The bad news is that the code’s readability has suffered in both expressions.

Figuring out what’s actually happening in the double list comprehension takes a fair amount of head-scratching. Essentially, the second for statement is used only to give the name value to the return value of slow(num). Fortunately, that sounds like something that you can accomplish with an assignment expression!

You can rewrite the list comprehension using the walrus operator as follows:

Note that the parentheses around value := slow(num) are required. This version is effective and readable, and it communicates the intent of the code well.

Next, look at a slightly more involved and practical example. Say that you want to use the Real Python feed to find the titles of the last episodes of the Real Python Podcast.

You can use the Real Python Feed Reader to download information about the latest Real Python publications. In order to find the podcast episode titles, you’ll use the third-party Parse package. Start by creating a virtual environment and installing both packages:

You can now read the latest titles published by Real Python:

Podcast titles start with "The Real Python Podcast", so here you can create a pattern that Parse can use to identify them:

Compiling the pattern beforehand speeds up later comparisons, especially when you want to match the same pattern over and over. You can check if a string matches your pattern using either pattern.parse() or pattern.search():

Note that Parse is able to pick out the podcast episode number and the episode name. The episode number is converted to an integer data type because you used the :d format specifier.

Time to get back to the task at hand. In order to list all the recent podcast titles, you need to check whether each string matches your pattern and then parse out the episode title. A first attempt may look something like this:

Though it works, you might notice the same problem you saw earlier. You’re parsing each title twice because you filter out titles that match your pattern and then use that same pattern to pick out the episode title.

Like you did earlier, you can avoid the double work by rewriting the list comprehension using either an explicit for loop or a double list comprehension. Using the walrus operator, however, is even more straightforward:

Assignment expressions work well to simplify these kinds of list comprehensions. They keep your code readable and save you from doing a potentially expensive operation twice.

In this section, you’ve focused on examples where you can rewrite list comprehensions using the walrus operator. The same principles also apply if you see that you need to repeat an operation in a dictionary comprehension, a set comprehension, or a generator expression.

The following example uses a generator expression to calculate the average length of episode titles that are over 50 characters long:

The generator expression uses an assignment expression to avoid calculating the length of each episode title twice.

While Loops

Python has two different loop constructs: for loops and while loops. You typically use a for loop when you need to iterate over a known sequence of elements. A while loop, on the other hand, is for when you don’t know beforehand how many times you’ll need to repeat the loop.

In while loops, you need to define and check the ending condition at the top of the loop. This sometimes leads to some awkward code when you need to do some setup before performing the check. Here’s a snippet from a multiple-choice quiz program that asks the user to answer a question with one of several valid answers:

This works but has an unfortunate repetition of two identical input() lines. It’s necessary to get at least one answer from the user before checking whether it’s valid or not. You then have a second call to input() inside the while loop to ask for a second answer in case the original user_answer wasn’t valid.

If you want to make your code more maintainable, it’s quite common to rewrite this kind of logic with a while True loop. Instead of making the check part of the main while statement, the check is performed later in the loop together with an explicit break:

This has the advantage of avoiding the repetition. However, the actual check is now harder to spot.

Assignment expressions can simplify these kinds of loops. In this example, you can now put the check back together with while where it makes more sense:

The while statement is a bit denser, but the code now communicates the intent more clearly without repeated lines or seemingly infinite loops.

You can expand the box below to see the full code of the multiple-choice quiz program and try a couple of questions about the walrus operator yourself.

See Build a Quiz Application With Python if you want to dive deeper into using Python to quiz yourself or your friends. You can also quiz yourself on your knowledge of the walrus operator:

You can often simplify while loops by using assignment expressions. The original PEP shows an example from the standard library that makes the same point.

Witnesses and Counterexamples

In the examples you’ve seen so far, the := assignment expression operator does essentially the same job as the = assignment operator in your old code. You’ve seen how to simplify code, and now you’ll learn about a different type of use case that this operator makes possible.

In this section, you’ll learn how you can find witnesses when calling any() by using a clever trick that isn’t immediately possible without using the walrus operator. A witness, in this context, is an element that satisfies the check and causes any() to return True.

By applying similar logic, you’ll also learn how you can find counterexamples when working with all(). A counterexample, in this context, is an element that doesn’t satisfy the check and causes all() to return False.

In order to have some data to work with, define the following list of city names:

You can use any() and all() to answer questions about your data:

In each of these cases, any() and all() give you plain True or False answers. What if you’re also interested in seeing an example or a counterexample of the city names? It could be nice to see what’s causing your True or False result:

• Does any city name start with "B"?

  Yes, because "Berlin" starts with "B".

• Do all city names start with "B"?

  No, because "Oslo" doesn’t start with "B".

In other words, you want a witness or a counterexample to justify the answer.

Capturing a witness to an any() expression has not been intuitive in earlier versions of Python. If you were calling any() on a list and then realized you also wanted a witness, you’d typically need to rewrite your code:

Here, you first capture all city names that start with "B". Then, if there’s at least one such city name, you print out the first city name starting with "B". Note that here you’re actually not using any() even though you’re doing a similar operation with the list comprehension.

By using the := operator, you can find witnesses directly in your any() expressions:

You can capture a witness inside the any() expression. The reason this works is a bit subtle and relies on any() and all() using short-circuit evaluation: they only check as many items as necessary to determine the result.

You can more clearly see what’s happening by wrapping .startswith("B") in a function that also prints out which item is being checked:

Note that any() doesn’t actually check all the items in cities. It only checks items until it finds one that satisfies the condition. Combining the := operator and any() works by iteratively assigning each item that is being checked to witness. However, only the last such item survives and shows which item was last checked by any().

Even when any() returns False, a witness is found:

However, in this case, witness doesn’t give any insight. 'Belgrade' doesn’t contain ten or more characters. The witness only shows which item happened to be evaluated last.