How to Aggregate Related Objects in Django

Django's models are one of the best features that the framework provides. For most use cases, the default functionality is sufficient and simple to use. Though, for special requirements, the classes are just as easily customized.

In a recent project, I needed to compile a list of one-to-many Foreign Key relationships for one of my models. This is simple enough with a small, finite number of related models. However, for this specific project, there could be many more relations added in the future. My problem was this:

How can I create a method that will aggreate all related objects, and require no additional code if a new relation is added?

This was not something that Django's models could do out of the box. I was, however, able to create a solution that worked exactly as I needed it to.

Before I get to my solution, there are a few concepts that I need to introduce.

Suppose you have a models.py that looks like this:

class Magazine(models.Model):
    issue = models.IntegerField()
    published = models.DateTimeField(default=timezone.now)


class Article(models.Model):
    magazine = models.ForeignKey(Magazine, on_delete=models.CASCADE)
    title = models.CharField(max_length=127)
    author = models.CharField(max_length=127)
    text = models.TextField()


class Advertisement(models.Model):
    magazine = models.ForeignKey(Magazine, on_delete=models.CASCADE)
    vendor = models.CharField(max_length=127)
    text = models.TextField()


class Image(models.Model):
    magazine = models.ForeignKey(Magazine, on_delete=models.CASCADE)
    image = models.ImageField()

Article, Advertisement, and Image all have many-to-one relationship with Magazine (i.e. One Magazine can have many Articles, etc).

If you wanted to find which magazine that an Article is in, the query is simple:

>>> Article.objects.get(pk=article_id).magazine

This works really well, and is the basic pattern for finding the "parent" relation from an object. Though you might typically find that you have an instance of Magazine instead of Article.

So, how do you find all of the Articles in a Magazine?

It's fairly simple, and there are a couple of approaches available:

You could filter for the magazine on the Article model

This query would look like:

>>> Article.objects.filter(magazine=magazine_id)

This, again, assumes that you have an instance of Article to work with, but it might be more common of a pattern in your code to have the "parent" instance instead. It makes more sense to start with a Magazine and then work your way down the hierarchy.

To use an instance of Magazine, you would want to opt for using the article_set attribute of the Magazine model. This would look like:

>>> Magazine.objects.get(pk=magazine_id).article_set.all()

In each case, the return would be a list of Article objects:

[
  <Article: Article object (1)>,
  <Article: Article object (2)>,
  <Article: Article object (3)>,
]

These same queries could be used for the Advertisement and Images models as well.

Advertisement.objects.filter(magazine=magazine_id)
Image.objects.filter(magazine=magazine_id)

# or

Magazine.objects.get(pk=magazine_id).advertisement_set.all()
Magazine.objects.get(pk=magazine_id).image_set.all()

Now, if you wanted to find all articles, advertisements, and images associated with a magazine, you'd generally have to write three seperate queries, no matter which of the above methods you choose. This is fine for the most part, but becomes cumbersome with each additional relation.

This is the main problem that I faced when trying to create an aggregated list of all related objects, regardless of the relationships defined.

I needed to create a function that could consolidate all of these queries into one return value.

Something that would look like:

[
  <Article: Article object (1)>,
  <Advertisement: Advertisement object (1)>,
  <Advertisement: Advertisement object (2)>,
  <Image: Image object (1)>,
]

There wasn't any documentation that I could find for this functionality, so I started searching through Django's source code and frequent dir() calls for answers. After a little bit of digging, I was able to come up with a single function that creates a list of all the related objects.

Here it is:

def get_related_items(self):
    response = []
    for rel_obj in self._meta.related_objects:
        response += [
             instance
             for instance in rel_obj.related_model.objects.filter(
                 magazine = self
             )
        ]
    return response

If you're confused as to where this would go in your code, this would belong to the Magazine class. Check out my article on model methods for more information.

This might look a bit intimidating, but I'll walk through it:

We have a couple of layers to get through:

The list of related models
The list of related objects

Models, being the target of queries. Objects, the results.

To do this, I needed to create two for loops. The first loop will iterate through all of the related models, and the second loop will iterate through all of the related model's instances, appending the instance to the aggregate list.

A quick note on syntax:

Wherever I can, I tend to prefer Python's shorthand list population. If you haven't seen it before, this article is a decent resource for explaining it. The basic list can be populated with [x for x in y]. It is possible to do single-line nested for loops, but it's not recommended due to being more difficult to read.

Ok, so, here's the step-by-step explanation:

def get_related_items(self):
    # The final list must be outside of the for loops since we don't want to do
    # single-line nested loops
    response = []

    # Iterate through self._meta.related_objects, which contains the total list
    # of models that have ForeignKey relations with this model
    for rel_obj in self._meta.related_objects:
        # Generate a list of the objects in the related model that match the
        # FK relation, and append it to the aggregate list.
        response += [
            instance
            for instance in rel_obj.related_model.objects.filter(
                magazine = self
            )
        ]

    return response

It's really rather simple once you break it up. I don't know if this could've been possible without self._meta.related_objects. It provides a list of all related models, therefore making it easy to fulfill the requirements of the project:

Create a method that will aggreate all related objects, and require no additional code if a new relation is added.

Now, you should now be able to simply call magazine_instance.get_related_items() which, given my database, will return:

[
  <Article: Article object (1)>,
  <Article: Article object (2)>,
  <Advertisement: Advertisement object (1)>,
  <Advertisement: Advertisement object (2)>,
  <Advertisement: Advertisement object (3)>,
  <Image: Image object (1)>
]

The one caveat that I might add is that I didn't really spend the time to figure out how to abstract the filter query (magazine = self) further so that this could be used in any model with no configuration. So, if you want to use this in your code, be sure to update that line with the filter query relevant to your code.