Django: Foreign Keys, Views, Managers, Methods, Mixins and the MRO

Jan 31, 2021

Let's say you are creating a Django-based application and you want to store some data that is tied to a user. In your views you need to make sure that all user-owned data is displayed and created only for the currently logged-in user. Here are three techniques you can use to achieve that goal cleanly:

Queryset method in each view
Queryset in a view mixin
Custom manager on the model that filters by foreign key

A little example project

In our little example project we're going to record weight measurements for a user each time they weigh themselves. Imagine a "self-made fitbit without the hardware". So we're going to gate all views for logged-in users, and when a user is logged-in, they must only see their own data, and must only be able to create entries for themselves.

The application is simple, there's just a log-in view, a paginated list view and a create-new-entry view. Everything else can feature-creep in later.

You can see the project in the repository real soon now.

The central item of the application is the model class, which looks somewhat like this:

from django import models

class Measurement(models.Model):

    user = models.ForeignKey(User, on_delete=models.CASCADE)
    weight = models.FloatField()

    created = models.DateTimeField(auto_now_add=True)

Simple enough, right? Now let's do something with that data.

A queryset method in each view

The first and most straightforward is to simply have each View class define an appropriate queryset method. So your code will look somewhat like this:

from django.http import HttpResponseRedirect
from django.views.generic import ListView, CreateView
from .models import Measurement

class List(ListView):

    def get_queryset(self):
        return Measurement.objects.filter(user=self.request.user)

# ...

class Create(CreateView):

    def form_valid(self):
        data = form.validated_data
        measurement = Measurement.objects.create(user=self.request.user, **data)
        return HttpResponseRedirect(self.get\_success\_url())

This is completely fine, and you should absolutely start out with code like that. There is no problem in this code or in this situation, but you'll find out once you start the inevitable process of adding features, that this technique isn't really scalable: you'll have to repeat writing those filter methods every time you add a view, and if your lookup logic ever changes, you'll need to change it everywhere. Which isn't DRY.

So, advantages here are:

quickly try things out, just write the queryset and be done
ideal first iteration

Disadvantages are:

you need to repeat the filter/creation every time you add a view
if you need to change the lookup logic, you need to change several places; it's not DRY.

Queryset mixins

One simple solution to that problem is to extract the filter and creation logic into their own functions. Fortunately, we're working with class-based views here, so we can put them into a class. This class will be only responsible for looking up the correct items and creating the correct measurements, nothing else (which is very SOLID). It could look like this:

from django.http import HttpResponseRedirect
from django.views.generic import ListView, CreateView
from .models import Measurement

class MeasurementForUserMixin:
    """A mixin for getting/creating user-specific measurements."""

    def get_queryset(self):
        return Measurement.objects.filter(user=self.request.user)

    def create(self, data):
        return Measurement.objects.create(user=self.request.user, **data)


class List(MeasurementForUserMixin, ListView):
    pass

# ...

class Create(MeasurementForUserMixin, CreateView):

    def form_valid(self):
        measurement = self.create(form.validated_data)
        return HttpResponseRedirect(self.get\_success\_url())

There are a few things to note here.

The first is that the MeasurementForUserMixin does not inherit from any other class. Since a mixin is not supposed to work in isolation, it doesn't need to inherit anything. That mixin is only useful in combination with any of the usual View classes. You can also see this in its use of self.request.user (your IDE might not like that self.request is not defined), which is never defined, but will come from the classes that it's mixed into.

The second, very important thing to note here is that both List and Create now inherit from both their view base classes and the mixin -- and the order of that inheritance is important.
To understand how and why that works, we need to take a quick detour.

A detour through the MRO

This section does not describe all there is to know about multiple inheritance and the MRO, but it covers most use cases. You can find out more about the MRO in more in-depth articles.

When a python class inherits from another class, it gets access to all the attributes and methods of its parent class. So, if you ask an instance for an attribute, it will check itself, then its parent (which is its class), then the parent class of the class, then the parent class of that class, ..., and so on until object, which is the class all other classes inherit from. This is the way inheritance works in most languages, and so it does in Python.

So far, so good.

But what happens when a class inherits from more than one class? Then we need a way to resolve where attributes and methods come from. We need to know, where to look for attributes and methods. This is what the method resolution order does. Its a list of classes that need to be checked whenever you ask for an attribute.

Let's do a quick example:

>>> class Example:
        def our_own_function(self):
            print("this is us")

>>> Example.mro()
[__main__.Example, object]

>>> Example.our_own_function
<function __main__.Example.our_own_function(self)>

>>> Example.__str__
<slot wrapper '__str__' of 'object' objects>

Here, we've created a simple class. It's MRO is [__main__.Example, object]. This means that when we ask for an attribute on the class (a value or a function or anything else that has a name), Python will first look whether that name is found in __main__.Example, and then in str. You can see this in the two names we ask for. The method called our_own_function is found in the class itself, as <function __main__.Example.our_own_function(self)>. While the method __str__, which we didn't define, is inherited from str as <slot wrapper '__str__' of 'object' objects> (ignore the weird naming for today).

So, the MRO defines where names are looked up, in order from left to right.

Now what about multiple inheritance? In that case, the MRO will contain more classes, including those we explicitly inherit from, again from left to right. The order is important here, so we know which methods will be used.

We need another example, slightly more complicated this time.

class Grandparent1:
    pass

class Grandparent2:
    pass

class Base1(Grandparent1):
    def base1_func(self):
        print("base1_func")

    def shared_hidden(self):
        print("hidden: base1")

    def shared_visible(self):
        print("visible: base1")
        super().shared_visible()

class Base2(Grandparent2):
    def base2_func(self):
        print("base2_func")

    def shared_hidden(self):
        print("hidden: base2")

    def shared_visible(self):
        print("visible: base2")

class Example(Base1, Base2):
    def example_func(self):
        print("example_func")

Let's go to the interpreter and examine that situation a bit.

>>> Example.mro()
[<class '__main__.Example'>, <class '__main__.Base1'>, <class '__main__.Grandparent1'>, <class '__main__.Base2'>, <class '__main__.Grandparent2'>, <class 'object'>]

First off, the MRO. It's Example, Base1, Grandparent1, Base2, Grandparent2, object. We can see that the MRO is depth-first, but only until a common ancestor is found. This thing is called the dependency diamond or the "Deadly Diamond of Death" and is one of the reasons why multiple inheritance is not present in many languages. In any case, Python will search in Base1 before it goes to Base2

>>> Example.base1_func
<function Base1.base1_func at 0x7fadd1fa4700>

>>> Example.base2_func
<function Base2.base2_func at 0x7fadd1fa48b0>

No surprises here, inherited functions are inherited from their definitions.

>>> Example.shared_hidden
<function Base1.shared_hidden at 0x7fadd1fa4790>

>>> Example.shared_visible
<function Base1.shared_visible at 0x7fadd1fa4820>

But these two functions, which are present in both parent classes are now only inherited from Base1, because it comes first in the MRO. This means that inheriting from a base class overwrites inheritance from "further-right" base classes, and this is exactly what makes mixins work: instead of creating a new "kind" of View, we simply overwrite some parts of it in a mixin -- in our case, the queryset method.

You might now under the impression that overwritten methods are "lost", or inaccessible. And at first, it does look like that.

>>> e = Example()
>>> e.shared_hidden()
hidden: base1

Here we can see that only the shared function in Base1 was executed. Since execution stopped there, the shared_hidden method from Base2 is essentially gone.

But we have a way to get there:

>>> e.shared_visible()
visible: base1
visible: base2

Using the keyword super gives us access to methods further up in the inheritance hierarchy, ie. further right in the MRO. super essentially means "look up this name, but only to the right of the current position in the MRO".

This is excellent! We can now overwrite methods in a class by inheriting from a mixin. If that mixin terminates the super chain, that means that the behaviour is fully defined by that mixin.
If a mixin calls super, it's now part of the chain and only enhances the behaviour.

Everyone uses the super keyword, mostly when doing initialisation in the __init__ method. Knowing these details gives you the ability to write super-powerful pun absolutely intended and useful mixins.

Mixing in those classes

How does all of that help with our original problem, getting Views to show the correct things? Let's have another look at our previous example:

from django.http import HttpResponseRedirect
from django.views.generic import ListView, CreateView
from .models import Measurement

class MeasurementForUserMixin:
    """A mixin for getting/creating user-specific measurements."""

    def get_queryset(self):
        return Measurement.objects.filter(user=self.request.user)

    def create(self, data):
        return Measurement.objects.create(user=self.request.user, **data)


class List(MeasurementForUserMixin, ListView):
    pass

# ...

class Create(MeasurementForUserMixin, CreateView):

    def form_valid(self):
        measurement = self.create(form.validated_data)
        return HttpResponseRedirect(self.get\_success\_url())

Here, List inherits from MeasurementForUserMixin and ListView in that order. This means that any method defined in MeasurementForUserMixin will be found before the same name is searched for in ListView. We have selectively overwritten the get_queryset and create functions with specific functions. These don't call super anywhere, so they are definitive.

Any call in an instance of List to get_queryset will now be directed to MeasurementForUserMixin, even if that code originally comes from ListView Yes, I should have added that to the example above, but I noticed too late..

This is what we wanted to achieve.

So, advantages here are:

easily composable, scales well with complexity
DRY: permission logic is collected in one place and easily swappable

Disadvantages are:

logic is tied to views, not models, which can make finding the functions harder
can be hard to grasp when there are too many mixins

Managers and their methods

There is one more place where one could put methods pertaining to controlling retrieval of database objects You could also put that stuff directly into the model, but that wouldn't be SOLID and clutter up the model too much, and that is the model manager. You know this as .objects, it's the object responsible for giving you access to the database.

Let's look at the code, then go through it slowly and understand what's happening.

from django import models

class MeasurementForUserManager(models.Manager):

    def filter(self, user, **params):
        return super.filter(user=user, **params)

    def create(self, user, **data):
        return super().create(user=user, **data)

Here, we define a new Manager class, simply as a subclass of the regular manager it is very important that we can not change the constructor for that manager, because that manager gets instantiated with the model class, and manages all database access for that class. This also means we can not have a manager that "knows" about a specific user or a request. We have to pass in the user/request in the specific functions like filter and create . We overwrite two functions, one will give us a specific queryset and the other will create a correct object note the use of super in these methods: these point to the methods of the base class, models.Manager. So far, so good. Now we need to hook up this manager in our model class.

class Measurement(models.Model):

    user = models.ForeignKey(User, on_delete=models.CASCADE)
    weight = models.FloatField()

    created = models.DateTimeField(auto_now_add=True)

    objects = MeasurementForUserManager()

We could also add the user parameter to the constructor __init__ of the class for creation of instances, if we wanted to make sure each Measurement is attached to the correct user. Since the foreign key is not optional, we'd get error messages for that case anyway.

Now, using the model in the views looks like this:

from django.http import HttpResponseRedirect
from django.views.generic import ListView, CreateView
from .models import Measurement

class List(ListView):

    def get_queryset(self):
        return Measurement.objects.filter(user=self.request.user)

class Create(CreateView):

    def form_valid(self):
        data = form.validated_data
        measurement = Measurement.objects.create(user=self.request.user, **data)
        return HttpResponseRedirect(self.get\_success\_url())

Not that different from doing it manually, right? The difference is that you cannot do it wrong any more, since the parameter user is now required, both on filter and create to be complete, we could also make it a parameter to all. Trying to use any of these methods without passing a user will result in an error.

Also, the actual code that does the filtering is now collected in the manager, close to the actual model. This means that you don't have to remember how the filtering is done each time you use it, and it's close to the model definition. When/if your lookups change, these are easy to find and easy to change in a single place.

We've essentially made a manager that filters by a foreign key for us, but it could be whatever you need.

So, advantages here are:

DRY: permission logic is in a single place
model logic lives close to the model
can be combined with view mixins

Disadvantages are:

you still need to write queryset retrieval for each view
no "automatic" composition like in mixins

Summary

We have seen three ways to write permission logic for our views: writing it by hand for each view, writing mixins that compose into the views, and writing model managers that are attached to the model. Each one has their own advantages and disadvantages. In a young project, you'll probably start with writing your views directly. When your project matures, you'll probably start creating some view abstractions into mixins, and write specific logic in model managers. If your project leans more towards view logic, you'll have more mixins. If your project is more database-oriented, you'll have more manager methods.

So know the tool set, apply it freely and find your own solutions!