Query API¶

At the moment, lifter is read-only, meaning no write query can be issued to a store.

QuerySet is the class used for read-related queries.

Paths¶

Paths refers to the fields of a model involved in a query. For example, take the following queryset:

import lifter.models

class User(lifter.models.Model):
    pass

manager.filter(User.age > 13, User.is_active == True)

In the previous example, two paths are used in the query: User.age and User.is_active.

Thanks to a little magic, these paths are actually represented as plain python objects:

User.age
>>> <Path: age>

Paths can actually be nested. If you have the following data source:

users = [
    {
        'id': 1,
        'name': 'Jeff Winger',
        'company': {
            'id': 3,
            'name': 'Greendale',
        }
    }
]

You can totally create paths such as User.company.id or User.company.name.

Query nodes¶

Paths are used to create query nodes. A QueryNode is a Python object, used to represent a condition:

path = User.age
qn = path < 30

Of course, in real life, you’ll use the shorter (and also more readable notation):

User.age < 30
>>> <QueryNode age, <built-in function lt>, [30], {}>

In the previous example, we have created a query node representing the condition age < 30. We can now use this node to fetch data:

qn = User.age < 30
manager.all().filter(qn)
>>> Returns all user matching the condition

Combining nodes¶

It is possible to combine nodes together using python bitwise operators to build more complex queries:

qn = (User.age < 30) & (User.is_active == True)
>>> a node matching User.age < 30 AND User.is_active == True

qn = (User.age < 30) | (User.is_active == True)
>>> a node matching User.age < 30 OR User.is_active == True

# use ~ to invert a query node
qn = ~(User.age < 30)

Queries¶

Queries are higher-level objects that describe an action to run on the data store:

import lifter.query

qn = (User.age < 30) & (User.is_active == False)
query = lifter.query.Query(action='select', filters=qn)

QuerySets¶

Don’t worry, you won’t have to instanciate all of these objects by hand to use lifter.

QuerySets are here to provide the high-level API for interacting with data stores.

Once you have a manager instance, issuing query is done easily with querysets:

import lifter.models
from lifter.backends.python import IterableStore

class User(lifter.models.Model):
    pass

data = [
    {
        'age': 27,
        'is_active': False,
        'email': 'kurt@cobain.music',
    },
    {
        'age': 687,
        'is_active': True,
        'email': 'legolas@deepforest.org',
    },
    {
        'age': 34,
        'is_active': False,
        'email': 'golgoth@lahorde.org',
    }
]

store = IterableStore(data)
manager = store.query(User)

# Here you pass query nodes directly to the queryset to obtain results from the store
manager.filter(User.age < 30)

QuerySet methods¶

filter(*explicit_queries, **keyword_queries)¶

Return a set of objects that match one or multiple queries.

Simple example using one query:

# return all 42 years-old users
manager.filter(User.age == 42)

Providing multiple queries to this method will merge all of them using AND operator:

manager.filter(User.age >= 42, User.age <= 56)

The previous example will return only objects that match both queries.

This is equivalent of writing:

manager.filter((User.age >= 42) & (User.age <= 56))

exclude(*explicit_queries, **keyword_queries)¶

This method is the exact opposite of filter(). It will return objects that do not match the provided queries:

# Exclude inactive users
manager.exclude(User.is_active == False)

# Exclude only inactive users that are 42 years-old
manager.exclude(User.is_active == False, User.age == 42)

Providing multiple queries to this method will merge all of them using AND operator:

This is equivalent of writing:

manager.exclude((User.age >= 42) & (User.age <= 56))

get(*explicit_queries, **keyword_queries)¶

This method retrieve a single object that match all of the given queries:

kurt = manager.get(User.id == 447)

Get will raise lifter.exceptions.DoesNotExist if no object is found, and lifter.exceptions.MultipleObjectsReturned if multiple objects are found:

import lifter.exceptions

try:
    kurt = manager.get(User.first_name == 'Kurt')
except lifter.exceptions.DoesNotExist:
    print('Sorry, no user found, try something else')
except lifter.exceptions.MultipleObjectsReturned:
    print('Multiple users are named Kurt, please precise your query')

This method will retrieve the final object among the queryset values:

>>> manager.get(User.first_name == 'Kurt')
# Retrieve among all manager loaded objects
>>> manager.filter(User.age == 42).get(User.first_name == 'Kurt')
# Retrieve among 42 years-old users

order_by(*paths)¶

Order the queryset results using the provided attribute(s):

>>> manager.order_by(User.age)
# Returns a queryset of users, from younger to older

You can reverse the ordering using python invert operator:

>>> manager.order_by(~User.age)
# Returns a queryset of users, from older to younger, this time

It’s possible to sort using multiple paths:

>>> manager.order_by(User.is_active, User.age)
# Sort by is_active then by age

Finally, you can also use random sorting, by passing a question mark instead of a path:

>>> manager.order_by('?')
# Random order
>>> manager.order_by(User.age, '?')
# Sort by age then randomly

values(*paths)¶

Use this method if you only want to retrieve specific values from your object list, instead of the objects themselves. It will return a list of dictionaries, with the requested values as keys:

>>> manager.values(User.age, User.email)
[{'age': 36, 'email': 'benard@blackbooks.com'}, {'age': 33, 'email': 'manny@blackbooks.com'}]

values_list(*paths, flat=False)¶

This method works as values(), but instead of of list of dictionaries, it returns a list of tuples.

>>> manager.values_list(User.age, User.email)
[(36, 'benard@blackbooks.com'), (33, 'manny@blackbooks.com')]

If you’re only requesting a single value and want a flat list (no tuples in it), you can set the flat parameter to True:

>>> manager.values_list(User.email, flat=True)
['benard@blackbooks.com', 'manny@blackbooks.com']

count()¶

A helper method that return the number of objects inside the queryset:

>>> manager.filter(User.age == 42).count()
56

You can achieve the same result using len:

qs = manager.filter(User.age == 42)
print(len(qs))

first()¶

A helper method that return the first object of the queryset or None if it’s empty:

>>> manager.filter(User.age == 42).first()
<User object>
>>> manager.filter(User.age == 666).first()
None

last()¶: Works as first() but returns the last object of the queryset.

exists()¶

A helper method that return True if the queryset has at least one result, False otherwise:

>>> manager.filter(User.age == 666).exists()
False

distinct()¶

A method that remove duplicates from a queryset:

>>> manager.values_list(User.eye_color, flat=True)
['green', 'brown', 'green', 'red', 'brown', 'red']
>>> manager.values_list(User.eye_color, flat=True).distinct()
['green', 'brown', 'red']

aggregate(*aggregates, **named_aggregates, flat=False)¶

Extract data from the queryset objects and return it as a dictionary.

A simple example to retrieve the average age of all users:

>>> import statistics
>>> manager.aggregate((User.age, statistics.mean))
{'age__mean': 44.2}

Under the hood, the previous example will loop on all loaded users, grab the age attribute, append the age to a list, then pass this list to the mean function and return the final result.

The method expect (path, callable) tuples as parameters. The path is the object attribute you want to gather, and the callable is the function that will return a value from the gathered data.

You can request multiple aggregates at once:

>>> manager.aggregate((User.age, statistics.mean), (User.age, min))
{'age__mean': 44.2, 'age__min': 12}

Bind them to specific keys:

>>> manager.aggregate(average_age=(User.age, statistics.mean))
{'average_age': 44.2}

And return aggregates as a list instead of a dictionary using the flat parameter:

>>> manager.aggregate((User.age, statistics.mean), (User.age, min), flat=True)
[44.2, 12]

Chaining querysets¶

Some of the previously described methods allow chaining You can chain querysets at will using filter() and/or exclude():

manager.exclude(User.age == 34).filter(User.is_active == True).filter(User.has_beard == False)

The previous example tranlates to:

In all users, exclude then one where age equals 34
Then, from the previous queryset, keep only active users
Then, from the previous queryset, leave only users with no beard

Querysets are lazy¶

No matter how much time you chain filter() and/or exclude() calls, the final query will only be actually applied when you try to access the queryset data:

# This will be instant, even if your user list has 1,000,000,000 entries in it
queryset = manager.exclude(User.age == 16)

# however, calling one of the following will apply the filter
queryset.count()
for user in queryset:
    print(user.age)

Once a queryset is evaluated (when queries have been applied), results are stored internally, and the queryset can be looped has many times as you want at no cost.