When building scalable web applications, one common challenge is managing time-consuming operations without impacting the user experience. Tasks like sending emails, processing images, or generating reports can take time, so they’re better handled asynchronously. A powerful solution is using NestJS Redis for background task processing.
In this guide, we’ll show how to integrate NestJS Redis with Bull queues to create a production-ready system for handling background jobs. You’ll learn to configure Redis, register queues, build processors, and manage jobs efficiently using NestJS.
NestJS Redis offers several compelling benefits when implementing background task processing:
• Persists jobs: Tasks survive server restarts
• Supports multiple workers: Scale horizontally by adding more processors
• Provides atomic operations: Ensures data consistency
• Offers excellent performance: In-memory operations with optional persistence
• Has a mature ecosystem: Well-tested libraries like Bull for Node.js
First, let’s set up our NestJS project with the required Redis and Bull dependencies:
# Create a new NestJS project
npm i -g @nestjs/cli
nest new redis-background-tasks
cd redis-background-tasks
# Install Bull and Redis packages
npm install @nestjs/bull bull redis
npm install @types/bull --save-dev
# Additional helpful packages
npm install @nestjs/schedule @nestjs/configYou have several options for running Redis:
Option 1: Local Installation
You can run Redis locally or through Docker to power your NestJS Redis queues:
# macOS with Homebrew
brew install redis
redis-server
# Ubuntu/Debian
sudo apt-get install redis-server
sudo systemctl start redis
# Windows (using WSL or Redis for Windows)Option 2: Docker
docker run -d -p 6379:6379– name redis redis:7-alpine
Option 3: Docker Compose (recommended for development)
# docker-compose.yml
version: '3.8'
services:
redis:
image: redis:7-alpine
ports:
- "6379:6379"
volumes:
- redis_data:/data
command: redis-server --appendonly yes
volumes:
redis_data:Create a .env file to store your Redis connection details:
# .env
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=
NODE_ENV=developmentThe key to integrating Redis with NestJS is configuring the BullModule in your main application module:
// src/app.module.ts
import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import { BullModule } from '@nestjs/bull';
import { ScheduleModule } from '@nestjs/schedule';
@Module({
imports: [
// Global configuration module
ConfigModule.forRoot({
isGlobal: true,
}),
// Enable scheduled jobs
ScheduleModule.forRoot(),
// Configure Bull with Redis connection
BullModule.forRoot({
redis: {
host: process.env.REDIS_HOST || 'localhost',
port: parseInt(process.env.REDIS_PORT) || 6379,
password: process.env.REDIS_PASSWORD,
// Optional: Additional Redis configuration
maxRetriesPerRequest: 3,
retryDelayOnFailover: 100,
lazyConnect: true,
},
}),
],
})
export class AppModule {}Before we begin queuing jobs, we’ll define our task types and Data Transfer Objects (DTOs) using TypeScript. This structure makes it easier to process different job types within our NestJS Redis system.
// src/tasks/dto/create-task.dto.ts
import { IsString, IsOptional, IsObject, IsEnum } from 'class-validator';
export enum TaskType {
EMAIL = 'email',
IMAGE_PROCESSING = 'image_processing',
REPORT_GENERATION = 'report_generation',
}
export class CreateTaskDto {
@IsEnum(TaskType)
type: TaskType;
@IsString()
title: string;
@IsOptional()
@IsString()
description?: string;
@IsOptional()
@IsObject()
payload?: any;
@IsOptional()
priority?: number;
@IsOptional()
delay?: number;
}To register queues in your NestJS Redis application, create a dedicated TasksModule and define each queue using the BullModule.registerQueue() method.
Create a tasks module that registers multiple Redis-backed queues:
// src/tasks/tasks.module.ts
import { Module } from '@nestjs/common';
import { BullModule } from '@nestjs/bull';
@Module({
imports: [
// Email processing queue
BullModule.registerQueue({
name: 'email',
defaultJobOptions: {
removeOnComplete: 10, // Keep 10 completed jobs
removeOnFail: 5, // Keep 5 failed jobs
attempts: 3, // Retry failed jobs 3 times
backoff: {
type: 'exponential',
delay: 2000,
},
},
}),
// Image processing queue
BullModule.registerQueue({
name: 'image-processing',
defaultJobOptions: {
removeOnComplete: 10,
removeOnFail: 5,
attempts: 5, // More retries for image processing
backoff: {
type: 'exponential',
delay: 5000,
},
},
}),
// Report generation queue
BullModule.registerQueue({
name: 'report-generation',
defaultJobOptions: {
removeOnComplete: 5, // Fewer completed jobs (reports are large)
removeOnFail: 3,
attempts: 2, // Fewer retries (reports take longer)
},
}),
],
controllers: [TasksController],
providers: [TasksService, EmailProcessor, ImageProcessor, ReportProcessor],
exports: [TasksService],
})
export class TasksModule {}The service layer handles Redis queue operations:
// src/tasks/tasks.service.ts
import { Injectable, Logger } from '@nestjs/common';
import { InjectQueue } from '@nestjs/bull';
import { Queue } from 'bull';
import { CreateTaskDto, TaskType } from './dto/create-task.dto';
@Injectable()
export class TasksService {
private readonly logger = new Logger(TasksService.name);
constructor(
@InjectQueue('email') private emailQueue: Queue,
@InjectQueue('image-processing') private imageQueue: Queue,
@InjectQueue('report-generation') private reportQueue: Queue,
) {}
async createTask(createTaskDto: CreateTaskDto) {
const { type, title, description, payload, priority = 0, delay = 0 } = createTaskDto;
// Select the appropriate Redis queue based on task type
let queue: Queue;
const jobData = {
title,
description,
payload,
createdAt: new Date(),
};
switch (type) {
case TaskType.EMAIL:
queue = this.emailQueue;
break;
case TaskType.IMAGE_PROCESSING:
queue = this.imageQueue;
break;
case TaskType.REPORT_GENERATION:
queue = this.reportQueue;
break;
default:
throw new Error(`Unknown task type: ${type}`);
}
// Add job to Redis queue with configuration
const job = await queue.add(type, jobData, {
priority, // Higher number = higher priority
delay, // Delay execution (milliseconds)
attempts: 3, // Number of retry attempts
backoff: { // Retry strategy
type: 'exponential',
delay: 2000,
},
removeOnComplete: 10, // Cleanup policy
removeOnFail: 5,
});
this.logger.log(`Created ${type} task with ID: ${job.id} in Redis`);
return {
id: job.id,
type,
title,
status: 'queued',
createdAt: new Date(),
};
}
// Monitor Redis queue statistics
async getQueueStats() {
const emailStats = await this.getQueueStatus(this.emailQueue, 'email');
const imageStats = await this.getQueueStatus(this.imageQueue, 'image-processing');
const reportStats = await this.getQueueStatus(this.reportQueue, 'report-generation');
return {
email: emailStats,
imageProcessing: imageStats,
reportGeneration: reportStats,
totalQueues: 3,
redisConnection: 'active',
};
}
private async getQueueStatus(queue: Queue, name: string) {
const waiting = await queue.getWaiting();
const active = await queue.getActive();
const completed = await queue.getCompleted();
const failed = await queue.getFailed();
return {
name,
waiting: waiting.length,
active: active.length,
completed: completed.length,
failed: failed.length,
total: waiting.length + active.length + completed.length + failed.length,
};
}
}Now we’ll implement the actual job processors. Each processor listens to a specific Redis queue and handles the job data. This pattern is common in NestJS Redis applications using Bull.
// src/tasks/processors/email.processor.ts
import { Processor, Process } from '@nestjs/bull';
import { Logger } from '@nestjs/common';
import { Job } from 'bull';
@Processor('email') // Links to Redis queue named 'email'
export class EmailProcessor {
private readonly logger = new Logger(EmailProcessor.name);
@Process('email')
async handleEmailJob(job: Job) {
this.logger.log(`Processing email job ${job.id} from Redis`);
const { title, payload } = job.data;
try {
// Simulate email processing with progress updates stored in Redis
await job.progress(25);
await this.simulateWork(1000);
await job.progress(50);
this.logger.log(`Preparing email: ${title}`);
await this.simulateWork(1000);
await job.progress(75);
this.logger.log(`Sending email to: ${payload?.recipient || 'default@example.com'}`);
await this.simulateWork(1000);
await job.progress(100);
this.logger.log(`Email job ${job.id} completed successfully`);
// Return result that will be stored in Redis
return {
status: 'sent',
recipient: payload?.recipient || 'default@example.com',
sentAt: new Date(),
jobId: job.id,
};
} catch (error) {
this.logger.error(`Email job ${job.id} failed:`, error);
throw error; // Bull will handle retry logic based on queue configuration
}
}
private simulateWork(ms: number): Promise<void> {
return new Promise(resolve => setTimeout(resolve, ms));
}
}// src/tasks/processors/image.processor.ts
import { Processor, Process } from '@nestjs/bull';
import { Logger } from '@nestjs/common';
import { Job } from 'bull';
@Processor('image-processing')
export class ImageProcessor {
private readonly logger = new Logger(ImageProcessor.name);
@Process('image_processing')
async handleImageProcessing(job: Job) {
this.logger.log(`Processing image job ${job.id} from Redis queue`);
const { title, payload } = job.data;
try {
// Simulate complex image processing with detailed progress
await job.progress(10);
this.logger.log(`Loading image: ${payload?.filename || 'default.jpg'}`);
await this.simulateWork(2000);
await job.progress(30);
this.logger.log(`Resizing image...`);
await this.simulateWork(3000);
await job.progress(60);
this.logger.log(`Applying filters...`);
await this.simulateWork(2000);
await job.progress(80);
this.logger.log(`Optimizing image...`);
await this.simulateWork(1500);
await job.progress(100);
this.logger.log(`Image processing job ${job.id} completed`);
return {
status: 'processed',
originalFile: payload?.filename || 'default.jpg',
processedFile: `processed_${payload?.filename || 'default.jpg'}`,
processedAt: new Date(),
optimizationRatio: '75%',
jobId: job.id,
};
} catch (error) {
this.logger.error(`Image processing job ${job.id} failed:`, error);
throw error;
}
}
private simulateWork(ms: number): Promise<void> {
return new Promise(resolve => setTimeout(resolve, ms));
}
}Create endpoints to monitor your Redis queues:
// src/tasks/tasks.controller.ts
import { Controller, Post, Get, Body, Param } from '@nestjs/common';
import { TasksService } from './tasks.service';
import { CreateTaskDto } from './dto/create-task.dto';
@Controller('tasks')
export class TasksController {
constructor(private readonly tasksService: TasksService) {}
@Post()
async createTask(@Body() createTaskDto: CreateTaskDto) {
return this.tasksService.createTask(createTaskDto);
}
@Get('stats')
async getQueueStats() {
return this.tasksService.getQueueStats();
}
@Get(':queueName/:jobId')
async getJobDetails(
@Param('queueName') queueName: string,
@Param('jobId') jobId: string,
) {
return this.tasksService.getJobDetails(queueName, jobId);
}
}For production environments, enhance your Redis configuration:
// src/app.module.ts - Enhanced Redis configuration
BullModule.forRoot({
redis: {
host: process.env.REDIS_HOST || 'localhost',
port: parseInt(process.env.REDIS_PORT) || 6379,
password: process.env.REDIS_PASSWORD,
// Connection pool settings
maxRetriesPerRequest: 3,
retryDelayOnFailover: 100,
lazyConnect: true,
keepAlive: 30000,
// Connection limits
family: 4,
maxMemoryPolicy: 'allkeys-lru',
// Cluster support (if using Redis Cluster)
// enableReadyCheck: false,
// maxRetriesPerRequest: null,
},
// Global default job options
defaultJobOptions: {
removeOnComplete: 100,
removeOnFail: 50,
attempts: 3,
backoff: {
type: 'exponential',
delay: 2000,
},
},
}),Implement cron jobs that create background tasks:
// src/jobs/jobs.service.ts
import { Injectable, Logger } from '@nestjs/common';
import { Cron, CronExpression } from '@nestjs/schedule';
import { TasksService } from '../tasks/tasks.service';
import { TaskType } from '../tasks/dto/create-task.dto';
@Injectable()
export class JobsService {
private readonly logger = new Logger(JobsService.name);
constructor(private readonly tasksService: TasksService) {}
// Scheduled job that adds tasks to Redis every 5 minutes
@Cron(CronExpression.EVERY_5_MINUTES)
async handleScheduledReports() {
this.logger.log('Adding scheduled report to Redis queue...');
await this.tasksService.createTask({
type: TaskType.REPORT_GENERATION,
title: 'Scheduled System Health Report',
description: 'Automated system health report generation',
payload: {
type: 'system_health',
automated: true,
timestamp: new Date(),
},
});
}
}Test your Redis-backed background tasks:
# Create an email task
curl -X POST http://localhost:3000/api/tasks \
-H "Content-Type: application/json" \
-d '{
"type": "email",
"title": "Welcome Email",
"description": "Send welcome email to new user",
"payload": {
"recipient": "user@example.com",
"template": "welcome"
},
"priority": 5
}'
# Check queue statistics
curl http://localhost:3000/api/tasks/stats
# Monitor job progress
curl http://localhost:3000/api/tasks/email/1Monitor your queues directly in Redis:
# Connect to Redis CLI
redis-cli
# View all keys (queues)
KEYS *
# Monitor queue activity
MONITOR
# Check specific queue length
LLEN "bull:email:waiting"
# View job data
HGETALL "bull:email:1"• Use separate queues for different task types
• Configure appropriate cleanup policies
• Set reasonable retry limits
• Implement exponential backoff for retries
• Log detailed error information
• Set up dead letter queues for persistent failures
• Monitor Redis memory usage
• Use appropriate data expiration policies
• Consider Redis persistence settings
• Use Redis AUTH (password)
• Configure firewall rules
• Consider Redis over TLS for production
• Track queue lengths and processing times
• Set up alerts for queue backlogs
• Monitor the Redis server health
# Redis configuration for production
maxmemory 2gb
maxmemory-policy allkeys-lru
save 900 1
save 300 10
save 60 10000version: '3.8'
services:
app:
build: .
ports:
- "3000:3000"
environment:
- REDIS_HOST=redis
- NODE_ENV=production
depends_on:
- redis
redis:
image: redis:7-alpine
volumes:
- redis_data:/data
- ./redis.conf:/usr/local/etc/redis/redis.conf
command: redis-server /usr/local/etc/redis/redis.conf
ports:
- "6379:6379"
volumes:
redis_data:
Integrating Redis for background task processing in NestJS offers a robust and scalable way to handle time-consuming operations without blocking the main application thread. By combining Bull queues, Redis persistence, and NestJS’s modular architecture, you can efficiently manage everything from sending emails to processing images and generating reports. This setup enables high performance and reliability, making it ideal for modern web applications.
With features like job persistence, automatic retries, real-time monitoring, and horizontal scalability, NestJS Redis task queues provide the foundation for a production-ready background processing system. Follow the best practices outlined in this guide—such as configuring cleanup policies, monitoring queue health, and handling errors gracefully—to ensure long-term stability and responsiveness in your app.
We worked with Mindbowser on a design sprint, and their team did an awesome job. They really helped us shape the look and feel of our web app and gave us a clean, thoughtful design that our build team could...
The team at Mindbowser was highly professional, patient, and collaborative throughout our engagement. They struck the right balance between offering guidance and taking direction, which made the development process smooth. Although our project wasn’t related to healthcare, we clearly benefited...
Founder, Texas Ranch Security
Mindbowser played a crucial role in helping us bring everything together into a unified, cohesive product. Their commitment to industry-standard coding practices made an enormous difference, allowing developers to seamlessly transition in and out of the project without any confusion....
CEO, MarketsAI
I'm thrilled to be partnering with Mindbowser on our journey with TravelRite. The collaboration has been exceptional, and I’m truly grateful for the dedication and expertise the team has brought to the development process. Their commitment to our mission is...
Founder & CEO, TravelRite
The Mindbowser team's professionalism consistently impressed me. Their commitment to quality shone through in every aspect of the project. They truly went the extra mile, ensuring they understood our needs perfectly and were always willing to invest the time to...
CTO, New Day Therapeutics
I collaborated with Mindbowser for several years on a complex SaaS platform project. They took over a partially completed project and successfully transformed it into a fully functional and robust platform. Throughout the entire process, the quality of their work...
President, E.B. Carlson
Mindbowser and team are professional, talented and very responsive. They got us through a challenging situation with our IOT product successfully. They will be our go to dev team going forward.
Founder, Cascada
Amazing team to work with. Very responsive and very skilled in both front and backend engineering. Looking forward to our next project together.
Co-Founder, Emerge
The team is great to work with. Very professional, on task, and efficient.
Founder, PeriopMD
I can not express enough how pleased we are with the whole team. From the first call and meeting, they took our vision and ran with it. Communication was easy and everyone was flexible to our schedule. I’m excited to...
Founder, Seeke
We had very close go live timeline and Mindbowser team got us live a month before.
CEO, BuyNow WorldWide
If you want a team of great developers, I recommend them for the next project.
Founder, Teach Reach
Mindbowser built both iOS and Android apps for Mindworks, that have stood the test of time. 5 years later they still function quite beautifully. Their team always met their objectives and I'm very happy with the end result. Thank you!
Founder, Mindworks
Mindbowser has delivered a much better quality product than our previous tech vendors. Our product is stable and passed Well Architected Framework Review from AWS.
CEO, PurpleAnt
I am happy to share that we got USD 10k in cloud credits courtesy of our friends at Mindbowser. Thank you Pravin and Ayush, this means a lot to us.
CTO, Shortlist
Mindbowser is one of the reasons that our app is successful. These guys have been a great team.
Founder & CEO, MangoMirror
Kudos for all your hard work and diligence on the Telehealth platform project. You made it possible.
CEO, ThriveHealth
Mindbowser helped us build an awesome iOS app to bring balance to people’s lives.
CEO, SMILINGMIND
They were a very responsive team! Extremely easy to communicate and work with!
Founder & CEO, TotTech
We’ve had very little-to-no hiccups at all—it’s been a really pleasurable experience.
Co-Founder, TEAM8s
Mindbowser was very helpful with explaining the development process and started quickly on the project.
Executive Director of Product Development, Innovation Lab
The greatest benefit we got from Mindbowser is the expertise. Their team has developed apps in all different industries with all types of social proofs.
Co-Founder, Vesica
Mindbowser is professional, efficient and thorough.
Consultant, XPRIZE
Very committed, they create beautiful apps and are very benevolent. They have brilliant Ideas.
Founder, S.T.A.R.S of Wellness
Mindbowser was great; they listened to us a lot and helped us hone in on the actual idea of the app. They had put together fantastic wireframes for us.
Co-Founder, Flat Earth
Ayush was responsive and paired me with the best team member possible, to complete my complex vision and project. Could not be happier.
Founder, Child Life On Call
The team from Mindbowser stayed on task, asked the right questions, and completed the required tasks in a timely fashion! Strong work team!
CEO, SDOH2Health LLC
Mindbowser was easy to work with and hit the ground running, immediately feeling like part of our team.
CEO, Stealth Startup
Mindbowser was an excellent partner in developing my fitness app. They were patient, attentive, & understood my business needs. The end product exceeded my expectations. Thrilled to share it globally.
Owner, Phalanx
Mindbowser's expertise in tech, process & mobile development made them our choice for our app. The team was dedicated to the process & delivered high-quality features on time. They also gave valuable industry advice. Highly recommend them for app development...
Co-Founder, Fox&Fork
