A nice example of when to use reduce in python
I just ran building a query in django that forced me to OR together a bunch of Q objects and I immediately realized that it was a great example for when reduce is nice.
Basically, I have a list of names as strings.
names = ["foo", "bar", "cake", "banana"] # example
Each of these represent a name of an object in a database. For each name I should build a Q object with it’s name argument set to that name. Then finally these queries should be OR’ed together to form a filter for later use.
Without using reduce it’s probably best done like this
def get_filter(names):
queries = [Q(name = name) for name in names] # build Q objects
name_filter = queries[0] # set up first Q to enable OR - loop
for query in queries[1:]: # OR loop through the rest
name_filter |= query
return name_filter
With reduce it is reduced to this
def get_filter(names): queries = [Q(name = name) for name in names] # build Q objects return reduce(Q.__or__, queries) # reduce the Q objects using unbound OR
Much nicer, although the function for reduce can be debated. Importing operator.or_ or using a lambda x,y: x|y works as well.
Setting the filter to the first variable and then iterating over the rest OR’ing as we loop seems a lot more messy then just reducing it using OR 
Since OR’ing a Q object to anything else than a Q object (or an QOr or QAnd which is the result of doing OR or AND on Q objects…) will throw an exception it is needed though.
Also, both of these functions will throw exceptions when names is empty. Just returning an empty filter might be nicer, which of course is solved by simply checking if names is empty in the beginning of the function and in that case return an empty filter.
Edit: Please read comments for more info.
Good to use with http://www.djangosnippets.org/snippets/273/
You do make a good case for it, however, it’s disappearing for Python 3000. Using it will make your code less portable. Is it really worth saving the few keystrokes?
reduce() isn’t disappearing in Py3; it’s just getting moved into functools.
Why not use the “in” field lookup?
http://www.djangoproject.com/documentation/db-api/#in
Jacob, is there some discussion on the Py3 mailing list or something that I don’t know about? I’m going off of PEP 3100, which lists reduce() as “to be removed”.
Nope, there’s still no good use for
reduce. As a one liner:def get_filter(names): return any(Q(name= name) for name in names)anyis a function that works basically like an short circuited or: it returns true and quits if any of the objects in the list return true. If not, it returns false.Oh, also, my version short circuits when it gets a true. Yours goes through the whole list no matter what.
“any is a function that works basically like an short circuited or: it returns true and quits if any of the objects in the list return true. If not, it returns false.”
Well… that has nothing to do with what my functions do.
The code I wrote uses the __or__ from the Q objects to get a new Q object. It does not return a boolean, but rather a filter used for queries.
any does also not seem to be a part of python 2.4, is it new in 2.5?
Also… since I’m guessing the any function is lazy (wherever it exists) it would terminate with a boolean or a value before actually OR’ing all of the Q objects. This should not happen.
So yeah… there is still use for reduce.
“Why not use the “in†field lookup?”
Because I did not know it existed
Thanks a lot.
This however is still a great example in how reduce can be a nice improvement, if this convienience functions hadn’t exist (since it is a very special case) it would be a lot nicer than doing the loop. If you for an example have a lot of queries that aren’t all related to the same field that together describes a final query (when OR’ed together), then this is the way to go.
It is also not the first time I’ve been through something like this. Last time was multiplication of matrices. Reduce can quite often make your code a lot prettier.
In this case though it seems that using objects.filter(name__in = names) is clearly better.
“Oh, also, my version short circuits when it gets a true. ”
And that will help you build a query object in exactly what way?
Hehe, I even proposed a QuerySet method for something similar: http://code.djangoproject.com/ticket/1807
I love the reduce solution except it’s throwing me some errors when its output is used directly in combination with a filter() call because reduce will automatically make QOr instances of all items and filter() expects the first argument to be a Q instance
i’ve put up the code and solution here: http://dpaste.com/hold/16016/
Anythoughts on how to make this prettier?
Try replacing models.Q.__or__ with lambda x,y:x|y and see if it helps
(or operator.or_ from the operator module)
To be honest, this might actually be an issue with the code I posted as well. I actually used the operator.or_ function for OR’ing but wrote the Q.__or__ variant here thinking that they would do the same thing, but I guess that QOr maybe doesn’t inherit Q (too lazy to check right now) and in that case fails the __or__ since the instance is not a Q.
I replaced my code to use the builtin __in variable in django now though.
reduce(operator.or_, Qobjects) should work just fine though (don’t forget to import operator).
Hey thanks, operator.or_ works great. Thanks!