Elf Sternberg

Actually, building a QuerySet can be expensive even if it doesn't involve any database I/O. Part of it is the fact that when you do:

my_query = \
models.Foo.objects.filter(field1=val1) \
.exclude(field2=val2, field3=val3) \
.extra(where='func(field4, %s) < 0',
params=[val4])

you're actually creating (and garbage collecting) two QuerySets that you don't care about, before you create the one that you do care about. Django doesn't know that the intermediate ones are useless, so it has no choice but to flesh them out fully enough so that they could be evaluated if necessary. One of the "fleshing out" things it does is to call copy.deepcopy() on the data it inherited from its parent. For instance, in the example above, when it created the second, it would call copy.deepcopy() on val1. When it created the third, it would call copy.deepcopy() on val1, val2, and val3.

We profiled one particularly database-heavy piece of our code and found that it was spending 30% of its time in copy.deepcopy(), because of this!

Creating Q objects and and-ing and or-ing them together to reduce the number of times you have to refine a given QuerySet doesn't seem to help, either.