One of the senior devleopers on our team has been organizing learning sessions on general Python/programming topics and this blog post is inspired by a recent session we had on comprehensions in Python.

Note that the below code was run on Python 3.5.

Why use list comprehensions?

So I had been exposed to list comprehensions before and a great blog post on the topic is Trey Hunner's Python List Comprehensions: Explained Visually.

However, I had never been big on using them because with the CS program I am doing and all the languages we use at work, I could sometimes be coding in 5-6 different languages at once!

I had always thought that the small amount of space I would be saving would not outweigh the fact that other languages don't support them and that it can make your code harder to understand.

List comprehensions can make your code EASIER to read

A coworker hit on idea I had not thought of and that is that a good list comprehension can make your code more expressive and actually easier to read. The key though is to not let your comprehensions get too complex.

Comprehensions are one line for loops

So, I think its helpful to think of comprehensions simply as one line for loops that result in the building of a list. As Trey Hunner said in his blog post, "Every list comprehension can be rewritten as a for loop but not every for loop can be rewritten as a list comprehension."

To start writing list comprehensions always write your template first

I got this idea from a coworker and really like it.

The basic idea is to make comprehensions easier to write by first start with a simple comprehension template.

This gets especially helpful with the dictionary comprehensions.

Here is the template for a list comprehension:

new_list = [x for x in some_list]

Doubling every item in a list

Now we can take our simple template above and modify it a little to double every item in a list:

some_list = [1,2,3,4]
new_list = [x * 2 for x in some_list]
print(new_list)

>
[2, 4, 6, 8]

Note that the * 2 is the only change we made to the template.

You can run the above example here: https://repl.it/FsqJ/1

Doubling every item in a list that is larger than 2

So with just a little more additional code added to our list comprehension template we can add an if statement:

some_list = [1,2,3,4]
new_list = [x * 2 for x in some_list if x > 2]
print(new_list)

>
[6, 8]

Note that the * 2 and if x > 2 are the only changes we made to the template.

Dictionary comprehensions are just like list comprehensions but for dictionaries

Let's start out with a very simple dictionary:

d = {'a':1,'b':2,'c':3}

To understand dictionary comprehensions you need to know that you can access each of the key:value pairs in a dictionary by using the items() method.

d = {'a':1,'b':2,'c':3}
for pair in d.items():
  print(pair)

>
('c', 3)
('a', 1)
('b', 2)

We will be able to use this idea once he have a simple dictionary comprehension template that we can build from each time.

A dictionary comprehension template

d = {k:v for (k,v) in d.items()}

Note that the variables k and v represent key and value. You can rename them to whatever you want, but these seem to make a lot of sense.

Creating a new dictionary with only pairs where the value is larger than 2

Let's use our template to add an if statement to do this for us:

d = {'a':1,'b':2,'c':3}
d = {k:v for (k,v) in d.items() if v > 2}
print(d)

>
{'c': 3}

We can also perform operations on the key value pairs

d = {'a':1,'b':2,'c':3}
d = {k + 'c':v * 2 for (k,v) in d.items() if v > 2}
print(d)

>
{'cc': 6}

Above we are concatenating a c to each key and and doubling the corresponding value, if the value of that key-value pair is greater than 2.

Don't let your comprehensions get too complex

So the key with these comprehensions is to not let them get so complex that your head spins when you try to decipher what they are actually doing. This is obviously subjective but it may be good to avoid anything more complex than if statements and simple mathematical operations.

If you find yourself wanting to throw in explanatory comments, then it might be a good idea to just refactor your comprehension as a normal for loop.

I have a lot of prior experience with people I worked with writing overly complex (but possibly necessary) Excel formulas. So the idea of crazy, long list comprehensions is the stuff of nightmares. Just don't do it!

Comprehensions are possibly faster

A final thing I want to mention is that comprehensions are a little faster than the same code that uses a traditional for loop. I am not saying this to get you to rewrite all of your for loops as comprehensions but to simply let you know that you are not losing any speed by using a comprehension.

I have done some research into why the comprehensions are faster and have found some conflicting ideas. One article said that list comprehensions and for loops compile down to the same Python for loop byte code so there should not be any difference in speed.

I may do more research on this topic and update this post if I have time. It's an interesting question.

Why are they called "comprehensions"?

I believe they are called "comprehensions" because the idea of set-comprehensions where a "comprehension of a set" is a short and non-exhaustive description of a set.

Let me know if you have some other ideas or come across some better reasoning as to the origin of the name.