Building Scalable Django Microservices: Best Practices and Folder Structure

In modern web development, microservices have become a popular architectural pattern. They allow different parts of an application to be built, deployed, and maintained independently. In a Django project, structuring your apps as independent microservices can help you achieve better scalability, maintainability, and performance.

In this blog, we will walk through how to build a demo Django Microservices project with a clean folder structure, modular apps, and best microservices communication practices. We will also show how to optimize performance using custom serializers and direct database queries, avoiding the overhead of HTTP requests.

Project Setup: Microservices Architecture

To demonstrate the benefits of Django Microservices, we will divide our project into two distinct yet interconnected applications (microservices):

app_1: Handles products.
app_2: Manages orders that depend on product data from app_1.

Related read: How We Created Scalable And Secure Microservices-Based Architecture For A Finance Application

Why Use Microservices in Django?

Django, by default, encourages a monolithic structure, but you can structure your project as microservices by splitting it into smaller, independent apps. Each app should focus on a specific domain and can communicate with other apps via APIs or direct database queries.

Here’s the general structure for our demo project:

django_microservices_project/
│
├── apps/
│ ├── app_1/ # Product microservice
│ │ ├── migrations/
│ │ ├── models.py
│ │ ├── serializers.py
│ │ ├── views.py
│ │ ├── urls.py
│ │ └── tests.py
│ │
│ └── app_2/ # Order microservice
│ ├── migrations/
│ ├── models.py
│ ├── serializers.py
│ ├── views.py
│ ├── urls.py
│ └── tests.py
│
├── config/
│ ├── __init__.py
│ ├── settings.py
│ ├── urls.py
│ └── wsgi.py
│
├── manage.py
└── requirements.txt

This folder structure ensures a clear separation between different microservices (apps) while maintaining a shared project configuration under the config/ folder.

Best Practices for Folder Structure

When structuring your Django project, follow these guidelines for a maintainable and scalable codebase:

➡️ Group by Domain

Instead of placing all files related to views, models, and serializers in the same folder, group them based on their respective domains (e.g., app_1 for products, app_2 for orders). This helps in managing the app’s functionality more effectively.

➡️ Keep App Independence

Each microservice app should be able to function independently. Keep models, views, serializers, and tests related to the domain within the app folder.

➡️ Modularize Shared Logic

If you have shared functionality, like database queries between apps, abstract this logic into separate files (e.g., microservice_communication.py) to avoid duplication and promote reusability.

Communication Between Microservices

➡️ Direct Database Queries

While it’s common for microservices to communicate over HTTP, in a Django project with shared databases, we can bypass HTTP calls and directly query the database of another app. This avoids the overhead of network requests and is more performant.

In our project, app_2 (Orders) needs to retrieve product data from app_1 (Products). Instead of using HTTP requests, we will import models directly and perform queries via Django’s ORM.

Here’s how we set up direct communication:

File: app_2/microservice_communication.py

# app_2/microservice_communication.py

from apps.app_1.models import Product # Import models directly from app_1

def get_products_by_name(name_filter=None):
    """
    Function to query the Product model in app_1 and return filtered products.
    Optionally accepts a name filter to search for products by name.
    """
    queryset = Product.objects.all()

    if name_filter:
        queryset = queryset.filter(name__icontains=name_filter)

    return queryset

def get_product_by_id(product_id):
    """
    Fetch a specific product from app_1 by its ID.
    """
    try:
        product = Product.objects.get(id=product_id)
        return product
    except Product.DoesNotExist:
        return None

In the above code:

• get_products_by_name filters products by name and returns a queryset.
• get_product_by_id retrieves a product by its ID.

This file acts as a communication layer between the two microservices, allowing app_2 to directly access data from app_1.

➡️ Models for app_1 (Product Microservice)

File: app_1/models.py

# app_1/models.py

from django.db import models

class Product(models.Model):
   name = models.CharField(max_length=255)
   description = models.TextField()
   price = models.DecimalField(max_digits=10, decimal_places=2)
   stock_quantity = models.IntegerField()

   def __str__(self):
       return self.name

   @classmethod
   def create_product(cls, name, description, price, stock_quantity):
       """
       Custom method to create a product instance.
       """
       product = cls.objects.create(
           name=name,
           description=description,
           price=price,
           stock_quantity=stock_quantity
       )
       return product

   def update_product(self, name=None, description=None, price=None, stock_quantity=None):
       """
       Custom method to update product information.
       """
       if name:
            self.name = name
       if description:
            self.description = description
       if price:
            self.price = price
       if stock_quantity:
            self.stock_quantity = stock_quantity

      self.save()
      return self

In this Product model:

🔹Fields: We define basic fields like name, description, price, and stock_quantity that represent each product.
🔹Methods: We include two methods:
• create_product: A class method to handle the creation of new products. This allows us to easily create products via a method rather than relying on Django’s default manager.
update_product: An instance method to update product details, such as price or stock quantity.

➡️ Models for app_2 (Order Microservice)

In app_2, we will define an Order model that keeps track of customer orders. Each order will be associated with a product and contain information like the order quantity, the product, and the status.

File: app_2/models.py

# app_2/models.py

from django.db import models
from apps.app_1.models import Product # Direct import from app_1

class Order(models.Model):
    PENDING = 'pending'
    SHIPPED = 'shipped'
    DELIVERED = 'delivered'
    CANCELED = 'canceled'

    STATUS_CHOICES = [
        (PENDING, 'Pending'),
        (SHIPPED, 'Shipped'),
        (DELIVERED, 'Delivered'),
        (CANCELED, 'Canceled'),
    ]

    product = models.ForeignKey(Product, on_delete=models.CASCADE)
    quantity = models.PositiveIntegerField()
    status = models.CharField(
       max_length=10,
       choices=STATUS_CHOICES,
       default=PENDING
    )
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

    def __str__(self):
        return f"Order #{self.id} - {self.product.name}"

    @classmethod
    def create_order(cls, product_name, quantity):
        """
        Custom method to create an order for a product.
        """
        product = Product.objects.filter(name=product_name).first()
        if product:
             order = cls.objects.create(
                 product=product,
                 quantity=quantity,
                 status=cls.PENDING
             )
             return order
        return None

     def update_order(self, quantity=None, status=None):
         """
         Custom method to update order details.
         """
         if quantity:
             self.quantity = quantity
         if status:
             self.status = status
         self.save()
         return self

In the Order model:

🔹ForeignKey: We link the Order model to the Product model from app_1 using a ForeignKey relationship. This allows each order to be associated with a specific product.
🔹Status Choices: The order can have multiple states (pending, shipped, delivered, canceled), which are defined as STATUS_CHOICES.
🔹Methods:

create_order: A class method that creates an order for a given product name and quantity. It checks whether the product exists in app_1 and then creates the order.
update_order: An instance method to update the order’s quantity and status.

➡️Database Relationship Between app_1 and app_2

The Order model in app_2 has a foreign key relationship with the Product model in app_1. This means that every order in app_2 will reference a product from app_1.
This relationship allows app_2 to access the product’s data and manage orders based on product information.

Enhance Your Success with Our Product Services Today

Using Custom Serializers for Better Performance

Django REST Framework’s ModelSerializer is convenient but can introduce performance overhead because it automatically introspects the model and generates validation code. For performance reasons, we can avoid using ModelSerializer and instead use regular serializers, where we manually handle object creation and updates.

Why Use Regular Serializers?

🔹Performance: Regular serializers are faster because they don’t use the extra functionality of ModelSerializer.
🔹Custom Logic: You can define custom create and update methods that interact directly with the model’s methods.

File: app_2/serializers.py

# app_2/serializers.py

from rest_framework import serializers
from .models import Order
from app_2.microservice_communication import get_product_by_id

class OrderSerializer(serializers.Serializer):
    id = serializers.IntegerField(read_only=True)
    product_id = serializers.IntegerField()
    quantity = serializers.IntegerField()

    def create(self, validated_data):
        product_id = validated_data['product_id']
        product = get_product_by_id(product_id)

        if product is None:
           raise serializers.ValidationError("Product does not exist.")

        return Order.create_order(
           product_name=product.name,
           quantity=validated_data['quantity']
        )

    def update(self, instance, validated_data):
        product_id = validated_data.get('product_id', instance.product_id)
        product = get_product_by_id(product_id)

        if product is None:
            raise serializers.ValidationError("Product does not exist.")

        return instance.update_order(
            product_name=product.name,
            quantity=validated_data.get('quantity', instance.quantity)
        )

In this example:

🔹We directly check if the product exists by querying app_1’s product data before creating or updating an order.
🔹By using regular serializers, we gain full control over how objects are created or updated and avoid the overhead of ModelSerializer.

Setting Up Views and URLs

Now let’s define the views that will handle API requests in app_2 (Order microservice).

File: app_2/views.py

# app_2/views.py

from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status
from .serializers import OrderSerializer

class OrderCreateView(APIView):
    def post(self, request, *args, **kwargs):
        serializer = OrderSerializer(data=request.data)

        if serializer.is_valid():
            serializer.save()
            return Response(serializer.data, status=status.HTTP_201_CREATED)
        else:
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

Here, we use the OrderSerializer to handle the creation of orders. If the product ID exists in app_1, the order will be created successfully. Otherwise, it raises a ValidationError.

File: app_2/urls.py

# app_2/urls.py

from django.urls import path
from .views import OrderCreateView

urlpatterns = [
    path('orders/', OrderCreateView.as_view(), name='order-create'),
]

This route listens for POST requests to create orders.

Benefits of This Approach

🔹Clear Separation of Concerns

Each microservice (app_1 and app_2) has a clear domain responsibility. This modularity makes it easier to scale and maintain different parts of the application.

🔹Performance

By using regular serializers and direct database queries, you minimize overhead. There’s no need for HTTP requests between services or automatic model serialization, which improves response times.

🔹Flexibility

You can easily add more microservices or change the database schema without disrupting the entire system. Each app remains independent and follows the Single Responsibility Principle.

🔹Simplified Communication

Instead of relying on external HTTP APIs for inter-service communication, direct database access through Django’s ORM reduces complexity and increases performance.

coma

Conclusion

Adopting a microservices architecture in Django enhances scalability, maintainability, and performance. Structuring the project into independent apps with a well-organized folder hierarchy ensures a clear separation of concerns. Direct database queries between microservices reduce the overhead of HTTP requests, optimizing communication. Additionally, using custom serializers over ModelSerializer improves performance by eliminating unnecessary processing.

Implementing these best practices allows developers to build robust Django microservices that scale efficiently. Keeping each microservice independent yet seamlessly connected ensures flexibility and modularity. A structured approach enhances code reusability, simplifies debugging, and improves testing. Ultimately, this leads to a high-performing application that is easier to maintain and expand.

Keep Reading

Keep Reading

  • Service
  • Career
  • Let's create something together!

  • We’re looking for the best. Are you in?