32.8. Background Workers
Table of Contents
Why Background Workers Matter in a Production Backend
Most real applications perform tasks that are too slow or too unreliable to run inside the normal HTTP request. Examples:
- Sending emails after user actions
- Generating PDFs or reports
- Processing images or videos
- Syncing data with external APIs
- Rebuilding search indexes or caches
If you try to do all of this inside your API handler, your responses will be slow, users will retry, and your servers will be fragile.
Background workers solve this by moving work out of the request, into separate processes that can run independently and be scaled separately.
In this chapter you will design how background workers fit into your final project, and what you need to implement to run them reliably in production.
Key idea: Any task that is slow, unreliable, or not critical for the immediate user response should be moved to a background worker.
Role of Background Workers in Your Final Project
For the final production backend, you should explicitly decide:
- Which features use background tasks
- Which parts stay inside the main API process
- How the API and workers communicate
- How workers are deployed, monitored, and scaled
Common use cases that you can integrate into your project:
| Use case | Why background worker? |
|---|---|
| Sending registration emails | External SMTP latency, possible timeouts |
| Password reset emails | Same as above, plus rate limiting per user / IP |
| Generating invoices | Can take time, often CPU heavy (PDF generation) |
| Image processing | CPU heavy, can be batched |
| Cleaning old sessions | Periodic maintenance, no user waiting |
| Syncing with payment APIs | External API latency, retries needed |
| Cache warmup & rebuild | Expensive DB or search queries, can run in the background |
When you design your final project, pick at least 2 or 3 of these to implement with workers, so you exercise:
- Fire‑and‑forget jobs (e.g. send email)
- Retriable jobs (e.g. sync with payment provider)
- Scheduled jobs (e.g. nightly reports, periodic cleanup)
Architecture: How Workers Integrate with Your System
A typical production setup for background workers in your project:
- FastAPI application
Handles HTTP requests and does only the minimal work needed to respond quickly. When it needs a background task, it publishes a job into a broker (typically Redis). - Message broker (Redis)
Stores queued jobs. The API writes jobs, workers read jobs. It also stores job states, retries, and schedules (depending on tool). - Worker processes
Separate processes, often separate containers, that: - Subscribes to certain queues
- Processes jobs
- Handles retries, backoff, and failure logging
- Scheduler (optional but recommended)
To run periodic jobs, such as hourly or daily maintenance tasks.
A simple conceptual diagram:
- User → HTTP request → FastAPI → enqueue job in Redis → respond 200
- Worker listens to Redis → pulls job → executes task (e.g. send email)
In your final deployment, you will likely run:
- One or more API containers
- One or more worker containers
- One Redis container or managed Redis service
- Optional: a scheduler container
Choosing a Background Task Tool
You already saw background jobs and queues in earlier chapters. For the final project, choose a tool and stick with it.
Most common Python choices:
| Tool | Broker | Pros | Cons |
|---|---|---|---|
| Celery | Redis/RabbitMQ | Mature, powerful, widely used | Heavier, more configuration |
| RQ | Redis | Simple, easy to understand | Fewer advanced features |
| Dramatiq | Redis | Clean API, good performance | Smaller ecosystem than Celery |
FastAPI BackgroundTasks | None (same process) | Very simple, no broker needed | Not for heavy or long running tasks |
For a production‑style final project, prefer a real queue + worker such as:
- Celery + Redis, or
- RQ + Redis
Using BackgroundTasks from FastAPI is better suited for very small or demo projects because it runs in the same process as the API and does not survive restarts.
Whatever you choose, your architecture will look like:
- API adds job → Broker → Worker handles job
Designing Your Background Job APIs
You must decide how the rest of your application asks for background work. The main rule:
Your application code should call clean, simple functions like send_welcome_email(user_id)
These functions should not know if a task runs synchronously or in the background.
A typical pattern:
- Define a "task" function in a
tasksorjobsmodule. - Expose two interfaces:
- A Python function to call directly in unit tests or simple environments.
- A queued version to use in production.
Example design idea (Celery style, simplified code):
# app/tasks/email.py
from .celery_app import celery_app
@celery_app.task(name="send_welcome_email")
def send_welcome_email_task(user_id: int) -> None:
# Real email sending logic here
...
def send_welcome_email(user_id: int, use_background: bool = True) -> None:
if use_background:
send_welcome_email_task.delay(user_id=user_id)
else:
send_welcome_email_task(user_id=user_id)Then, in your FastAPI endpoint:
from app.tasks.email import send_welcome_email
@app.post("/register")
def register_user(...):
user = user_service.register(...)
send_welcome_email(user.id, use_background=True)
return {"id": user.id}
During unit tests, you might call send_welcome_email(user.id, use_background=False) so the email logic runs immediately and is easy to assert.
Identifying Tasks to Move to Workers in Your Project
Go through your final project features and list:
- Which operations can be slow or unpredictable
- Which do not need to finish before the user sees a response
- Which can be batched or scheduled
Example for an e‑commerce style project:
| Feature | Should be backgrounded? | Reason |
|---|---|---|
| Send order confirmation | Yes | External email service, user does not need to wait |
| Charge payment | No (usually) | User must know immediately if the payment succeeded |
| Generate invoice PDF | Yes | Can be slow, user can download later |
| Update analytics/event log | Yes | Non‑critical, can be processed asynchronously |
| Update product search index | Yes | Often slow updates, can lag slightly behind reality |
Then, for each "Yes", design:
- A dedicated task function
- A queue name (for priority or separation)
- Retry strategy
- Logging and metrics
Queue Design and Priorities
You can have a single queue, or multiple queues with different priorities. For the final project, multiple queues are a good exercise.
Example design:
| Queue name | What goes here | Priority |
|---|---|---|
emails | All email sending tasks | Medium |
critical | Payment callbacks, security related tasks | Highest |
maintenance | Cleanup tasks, slow reports | Lowest |
Practical guidance:
- Start with one queue if you are unsure.
- Once things work, split into critical and non‑critical queues.
- Run more worker processes for critical queues if needed.
With Celery, you can run workers that listen to specific queues:
celery -A app.celery_app worker -Q critical,emails -n worker_critical@%h
celery -A app.celery_app worker -Q maintenance -n worker_maintenance@%hError Handling, Retries, and Idempotency
In production, some jobs will fail:
- SMTP server returns 5xx
- External API is down
- Temporary network issues
Your worker system must handle this automatically.
Retries and Backoff
Most queue systems provide retry configuration. For example:
- Maximum retries: 5
- Delay between retries: exponential backoff, such as 10s, 30s, 60s, 5min, 15min
Example concept with Celery (conceptual code):
@celery_app.task(
name="send_welcome_email",
max_retries=5,
default_retry_delay=30 # seconds
)
def send_welcome_email_task(user_id: int):
try:
send_email(...)
except SomeTemporaryError as exc:
raise send_welcome_email_task.retry(exc=exc)
Always design background tasks to be idempotent as much as possible.
If a task runs twice, it should not corrupt data.
Idempotency examples:
- Sending the same welcome email twice is usually acceptable.
- Charging a payment twice is not acceptable. For this, you must:
- Use idempotency keys with the payment provider.
- Check your own database state before performing the action.
In your final project, explicitly decide:
- Which tasks are idempotent by design.
- Which tasks must check state before performing any external or database action.
Handling Permanent Failures
What happens after all retries fail?
At least:
- The job should end in a "failed" state, not be lost.
- You should have logs with enough context to debug.
- Optionally, store failed job info in your database.
Ideas for your project:
- A
failed_jobsdatabase table for very important tasks (e.g. payment webhooks). - An admin API endpoint to inspect failed jobs or to retry them manually.
- At minimum, structured logs that include:
- Job name
- Job ID
- Parameters (safe subset, no secrets)
- Exception message and stack trace
Monitoring and Observability for Workers
You cannot treat workers as "fire and forget" in production. You need to know:
- Are workers running?
- Is the queue growing too large?
- Are many jobs failing?
- How long do jobs take?
For your final project, plan:
- Health checks
At least one endpoint or command that tells you if the worker system is healthy. This can be: - A specific route in your application that tries a sample Redis operation.
- A separate process that checks worker status.
- Metrics
At minimum: - Number of jobs processed
- Number of failed jobs
- Average processing time per job type
- Queue length
If you integrate Prometheus, you can expose metrics like:
background_jobs_total{task="send_email"}background_jobs_failed_total{task="send_email"}background_jobs_duration_seconds_bucket{task="send_email", ...}
- Logging
Log at least: - Start of job with job type and ID
- Success with duration
- Failures with stack traces
This is essential when something "mysteriously" stops happening, like "users stopped receiving emails."
Deployment of Workers in Your Production Setup
In your final project, Docker and Compose (or Kubernetes) will orchestrate services. For workers, you typically:
- Use the same image as the API container.
- Change the entrypoint or command to run the worker instead of the API server.
Example docker-compose.yml snippet idea:
services:
api:
build: .
command: uvicorn app.main:app --host 0.0.0.0 --port 8000
depends_on:
- redis
worker:
build: .
command: celery -A app.celery_app worker -l info
depends_on:
- redis
scheduler:
build: .
command: celery -A app.celery_app beat -l info
depends_on:
- redis
redis:
image: redis:7-alpineKey points to plan for your project:
- The worker process must have the same configuration as the API:
- Environment variables (database URL, email config, 3rd party API keys)
- Same code version
- You can scale the worker independently:
- For Compose:
docker-compose up --scale worker=3 - On Kubernetes, you would use a separate Deployment for workers.
Make sure your CI/CD pipeline:
- Builds a single image
- Uses it for both API and worker deployments
- Migrates the database before scaling workers that might depend on schema changes
Scheduled and Periodic Jobs
Many background tasks are not triggered by a user action, but by time:
- Daily database cleanup
- Recalculate statistics
- Close unpaid orders after 24 hours
- Send daily digest emails
You can implement scheduling in several ways:
| Approach | Pros | Cons |
|---|---|---|
| External cron + API calls | Very simple, language agnostic | Requires a server cron or external service |
| Celery beat or worker scheduler | Integrated with the queue system | More configuration |
| Hosted schedulers (e.g. GitHub Actions cron, cloud schedulers) | Managed service | Another dependency |
Example conceptual Celery beat schedule (not full code):
from celery.schedules import crontab
celery_app.conf.beat_schedule = {
"cleanup-expired-sessions": {
"task": "cleanup_expired_sessions",
"schedule": crontab(minute=0, hour="*/1"), # every hour
},
"send-daily-report": {
"task": "send_daily_report",
"schedule": crontab(minute=0, hour=0), # every day at midnight
},
}For your final project, pick at least one scheduled job, for example:
- Cleanup expired refresh tokens every hour.
- Mark stale orders as "expired" daily.
- Send a daily summary to admin.
This will force you to handle:
- A task that is not triggered by an HTTP request.
- Code that must run correctly even if there are no users active.
Data Access and Transactions in Background Tasks
Background workers still need to access your database, cache, and configuration, just like the API does.
Key rules:
- Reuse the same database access layer
Do not write separate raw SQL for workers. Use the same ORM models and repository classes. - Handle transactions correctly
A task should: - Open a database session
- Perform its updates
- Commit or roll back on error
- Close the session
- Be careful with long‑running transactions
Do not hold a DB transaction open while doing slow external calls. Pattern: - Read necessary data from DB, then close the session.
- Perform external call.
- Open a new session to update final status.
Example pattern in pseudocode:
def process_invoice_task(invoice_id: int):
# 1. Load data
with db_session() as session:
invoice = repo.get_invoice(session, invoice_id)
data = build_invoice_data(invoice)
# 2. Slow external work (no DB transaction open)
pdf_bytes = generate_pdf(data)
# 3. Save result
with db_session() as session:
repo.save_invoice_pdf(session, invoice_id, pdf_bytes)
session.commit()For your final project, review each background task and ensure you do not keep transactions open longer than necessary.
Configuration and Secrets for Workers
Workers need access to the same sensitive configuration as your API:
- Database URL
- Redis URL
- Email SMTP credentials
- Third‑party API keys
Rules:
- Load configuration from environment variables in both API and worker.
- Do not hardcode secrets in code or Dockerfiles.
- In Docker, use environment sections or
.envfiles that are not committed to Git. - In CI/CD, use the platform's secrets manager to inject environment variables.
Example variables that both API and worker should read:
| Variable name | Used by |
|---|---|
DATABASE_URL | API, workers |
REDIS_URL | API, workers |
SMTP_HOST | Workers (email tasks) |
SMTP_USER | Workers |
SMTP_PASSWORD | Workers |
APP_ENV | API, workers (dev/prod) |
It is very common to forget that workers need the same configuration as the main app. Plan for this early in your project structure.
Testing Background Workers
For a production‑ready project, only testing HTTP endpoints is not enough. You should also test:
- The task functions themselves
- How your API enqueues jobs
- How tasks behave with failure and retries (at least at a unit level)
Practical testing strategies:
- Unit test the task logic synchronously
Call the underlying function directly, not through the broker:
def test_send_welcome_email_sends_email(mocker):
mock_send = mocker.patch("app.services.email.send_email")
send_welcome_email(user_id=1, use_background=False)
mock_send.assert_called_once()- Test that API endpoints add jobs
For example, patch the.delaymethod (for Celery) or enqueue method for your chosen library. - Integration testing with a real broker (optional but ideal)
- Start Redis in your test environment.
- Start a worker process.
- Enqueue a job from the test.
- Wait and then assert that the expected side effect occurred.
Even if full integration tests are complex, unit tests for the task functions are very important.
Putting It All Together for Your Final Project
When you design and implement the final production backend, summarize your background worker plan in a short internal document or README section:
Example checklist:
- Tooling
- [ ] Background processing library chosen (e.g. Celery + Redis)
- [ ] Common task base module created
- Use cases
- [ ] Registration email implemented as background job
- [ ] Password reset email implemented as background job
- [ ] At least one longer running task (e.g. invoice generation) in background
- Queues
- [ ] At least one queue
- [ ] Optional: separate queues for critical and non‑critical tasks
- Retries and idempotency
- [ ] Retry policy configured per task
- [ ] All tasks safe to rerun or protected by idempotency logic
- Scheduling
- [ ] At least one scheduled job (e.g. cleanup)
- Deployment
- [ ] Worker container defined in Docker Compose or Kubernetes
- [ ] Same image and configuration as API
- [ ] Easy to scale worker count
- Monitoring
- [ ] Logs for job start, success, failure
- [ ] Basic metrics or at least queue length visibility
- [ ] Health checks or manual commands to verify workers are alive
By following this plan, your final project will not just have background tasks as a side feature. It will have a real, production‑style background processing system, with proper architecture, configuration, deployment, and monitoring.
Views: 0
Where to Move
Move chapter:
- ☰ 1. Introduction to Backend Development
- ☰ 1.1. What Is Backend Development?
- ☰ 1.2. Frontend vs Backend
- ☰ 1.3. How Web Applications Work
- ☰ 1.4. Client-Server Architecture
- ☰ 1.5. Request-Response Cycle
- ☰ 1.6. Static vs Dynamic Websites
- ☰ 1.7. Backend Developer Responsibilities
- ☰ 1.8. Backend Development Roadmap
-
- ☰ 2. How the Web Works
- ☰ 2.1. The Internet and the Web
- ☰ 2.2. IP Addresses
- ☰ 2.3. Domain Names
- ☰ 2.4. DNS
- ☰ 2.5. Ports
- ☰ 2.6. TCP/IP Basics
- ☰ 2.7. HTTP and HTTPS
- ☰ 2.8. HTTP Requests
- ☰ 2.9. HTTP Responses
- ☰ 2.10. HTTP Methods
- ☰ 2.11. HTTP Status Codes
- ☰ 2.12. HTTP Headers
- ☰ 2.13. Cookies
- ☰ 2.14. Sessions
- ☰ 2.15. What Happens When You Enter a URL?
-
- ☰ 3. Development Environment
- ☰ 3.1. Choosing an Operating System
- ☰ 3.2. Linux Basics for Backend Developers
- ☰ 3.3. Command Line Basics
- ☰ 3.4. Environment Variables
- ☰ 3.5. Installing Development Tools
- ☰ 3.6. Code Editors and IDEs
- ☰ 3.7. Git
- ☰ 3.8. GitHub and GitLab
- ☰ 3.9. Creating Your First Repository
- ☰ 3.10. Branches and Commits
- ☰ 3.11. Basic Git Workflow
-
- ☰ 4. Programming Fundamentals
- ☰ 4.1. Variables and Data Types
- ☰ 4.2. Operators
- ☰ 4.3. Conditions
- ☰ 4.4. Loops
- ☰ 4.5. Functions
- ☰ 4.6. Data Structures
- ☰ 4.7. Error Handling
- ☰ 4.8. Modules and Packages
- ☰ 4.9. File Handling
- ☰ 4.10. Object-Oriented Programming
- ☰ 4.11. Classes and Objects
- ☰ 4.12. Inheritance and Composition
- ☰ 4.13. Clean Code Principles
-
- ☰ 5. Python for Backend Development
- ☰ 5.1. Introduction to Python
- ☰ 5.2. Installing Python
- ☰ 5.3. Virtual Environments
- ☰ 5.4. pip and Package Management
- ☰ 5.5. Python Project Structure
- ☰ 5.6. Functions and Type Hints
- ☰ 5.7. Classes and Dataclasses
- ☰ 5.8. Exceptions
- ☰ 5.9. Working with Files
- ☰ 5.9.1. File Downloads
- ☰ 5.9.2. File Validation
- ☰ 5.9.3. Image Uploads
- ☰ 5.9.4. Local File Storage
- ☰ 5.9.5. Object Storage
- ☰ 5.9.6. S3-Compatible Storage
- ☰ 5.9.7. Presigned URLs
- ☰ 5.9.8. Secure File Handling
-
- ☰ 5.10. JSON
- ☰ 5.11. Date and Time
- ☰ 5.12. Logging
- ☰ 5.13. Configuration
- ☰ 5.14. Environment Variables
- ☰ 5.15. Async Programming
- ☰ 5.16. Python Backend Best Practices
-
- ☰ 6. Introduction to Web Backends
- ☰ 6.1. Building a Simple Web Server
- ☰ 6.2. Routing
- ☰ 6.3. URL Parameters
- ☰ 6.4. Query Parameters
- ☰ 6.5. Request Bodies
- ☰ 6.6. Form Data
- ☰ 6.7. JSON Requests
- ☰ 6.8. Returning Responses
- ☰ 6.9. Templates
- ☰ 6.10. Static Files
- ☰ 6.11. Middleware
- ☰ 6.12. Application Configuration
-
- ☰ 7. REST APIs
- ☰ 7.1. What Is an API?
- ☰ 7.2. What Is REST?
- ☰ 7.3. Resources and Endpoints
- ☰ 7.4. RESTful URL Design
- ☰ 7.5. HTTP Methods in REST
- ☰ 7.6. GET Requests
- ☰ 7.7. POST Requests
- ☰ 7.8. PUT Requests
- ☰ 7.9. PATCH Requests
- ☰ 7.10. DELETE Requests
- ☰ 7.11. Request Validation
- ☰ 7.12. Response Models
- ☰ 7.13. HTTP Status Codes
- ☰ 7.14. Error Responses
- ☰ 7.15. Pagination
- ☰ 7.16. Filtering
- ☰ 7.17. Sorting
- ☰ 7.18. Searching
- ☰ 7.19. API Versioning
- ☰ 7.20. API Documentation
- ☰ 7.21. OpenAPI and Swagger
-
- ☰ 8. FastAPI
- ☰ 8.1. Introduction to FastAPI
- ☰ 8.2. Routes and Path Operations
- ☰ 8.3. Path Parameters
- ☰ 8.4. Pydantic Models
- ☰ 8.5. Data Validation
- ☰ 8.6. Dependency Injection
- ☰ 8.7. Exception Handling
- ☰ 8.8. Background Tasks
- ☰ 8.9. File Uploads
- ☰ 8.10. Async Endpoints
- ☰ 8.11. FastAPI Project Structure
- ☰ 8.12. Building a Complete REST API
-
- ☰ 9. Databases
- ☰ 9.1. Introduction to Databases
- ☰ 9.2. Relational vs NoSQL Databases
- ☰ 9.3. Tables, Rows, and Columns
- ☰ 9.4. Primary Keys
- ☰ 9.5. Foreign Keys
- ☰ 9.6. Relationships
- ☰ 9.7. One-to-One Relationships
- ☰ 9.8. One-to-Many Relationships
- ☰ 9.9. Many-to-Many Relationships
- ☰ 9.10. Database Schema Design
- ☰ 9.11. Normalization
- ☰ 9.12. Indexes
- ☰ 9.13. Transactions
- ☰ 9.14. ACID Properties
-
- ☰ 10. SQL
- ☰ 10.1. Introduction to SQL
- ☰ 10.2. CREATE
- ☰ 10.3. INSERT
- ☰ 10.4. SELECT
- ☰ 10.5. UPDATE
- ☰ 10.6. DELETE
- ☰ 10.7. WHERE
- ☰ 10.8. ORDER BY
- ☰ 10.9. GROUP BY
- ☰ 10.10. Aggregate Functions
- ☰ 10.11. JOINs
- ☰ 10.12. Subqueries
- ☰ 10.13. Constraints
- ☰ 10.14. Indexes
- ☰ 10.15. Transactions
- ☰ 10.16. Common Table Expressions
- ☰ 10.17. SQL Performance Basics
-
- ☰ 11. PostgreSQL
- ☰ 11.1. Introduction to PostgreSQL
- ☰ 11.2. Installing PostgreSQL
- ☰ 11.3. Creating Databases
- ☰ 11.4. Users and Roles
- ☰ 11.5. Permissions
- ☰ 11.6. PostgreSQL Data Types
- ☰ 11.7. Constraints
- ☰ 11.8. Indexes
- ☰ 11.9. Transactions
- ☰ 11.10. JSON Data
- ☰ 11.11. Backup and Restore
- ☰ 11.12. PostgreSQL Performance Basics
-
- ☰ 12. ORM and Database Integration
- ☰ 12.1. What Is an ORM?
- ☰ 12.2. SQLAlchemy
- ☰ 12.3. Models
- ☰ 12.4. Database Sessions
- ☰ 12.5. Creating Records
- ☰ 12.6. Reading Records
- ☰ 12.7. Updating Records
- ☰ 12.8. Deleting Records
- ☰ 12.9. Relationships
- ☰ 12.10. Queries
- ☰ 12.11. Transactions
- ☰ 12.12. Connection Pooling
- ☰ 12.13. Database Migrations
- ☰ 12.14. Alembic
- ☰ 12.15. Repository Pattern
-
- ☰ 13. Authentication
- ☰ 13.1. Authentication vs Authorization
- ☰ 13.2. Password Storage
- ☰ 13.3. Password Hashing
- ☰ 13.4. Registration
- ☰ 13.5. Login
- ☰ 13.6. Tokens
- ☰ 13.7. JSON Web Tokens
- ☰ 13.8. Access Tokens
- ☰ 13.9. Refresh Tokens
- ☰ 13.10. OAuth 2.0
- ☰ 13.11. OpenID Connect
- ☰ 13.12. Social Login
- ☰ 13.13. Logout
- ☰ 13.14. Password Reset
- ☰ 13.15. Email Verification
- ☰ 13.16. Multi-Factor Authentication
-
- ☰ 14. Authorization
- ☰ 14.1. Users and Permissions
- ☰ 14.2. Role-Based Access Control
- ☰ 14.3. Permission-Based Access Control
- ☰ 14.4. Protecting API Endpoints
- ☰ 14.5. Admin Permissions
- ☰ 14.6. Resource Ownership
- ☰ 14.7. Authorization Best Practices
-
- ☰ 15. Backend Security
- ☰ 15.1. Introduction to Web Security
- ☰ 15.2. HTTPS and TLS
- ☰ 15.3. SQL Injection
- ☰ 15.4. Cross-Site Scripting
- ☰ 15.5. Cross-Site Request Forgery
- ☰ 15.6. CORS
- ☰ 15.7. Authentication Attacks
- ☰ 15.8. Brute-Force Protection
- ☰ 15.9. Rate Limiting
- ☰ 15.10. Input Validation
- ☰ 15.11. Secure Password Storage
- ☰ 15.12. Secrets Management
- ☰ 15.13. Secure Cookies
- ☰ 15.14. Security Headers
- ☰ 15.15. File Upload Security
- ☰ 15.16. Dependency Vulnerabilities
- ☰ 15.17. OWASP Top 10
-
- ☰ 16. Caching
- ☰ 16.1. Why Caching Matters
- ☰ 16.2. Application Caching
- ☰ 16.3. Redis
- ☰ 16.3.1. Introduction to Redis
- ☰ 16.3.2. Installing Redis
- ☰ 16.3.3. Keys and Values
- ☰ 16.3.4. Expiration
- ☰ 16.3.5. Distributed Locks
-
- ☰ 16.4. Cache Keys
- ☰ 16.5. Cache Invalidation
- ☰ 16.6. Caching Database Queries
- ☰ 16.7. Distributed Caching
-
- ☰ 17. Background Processing
- ☰ 17.1. Synchronous vs Asynchronous Tasks
- ☰ 17.2. Background Jobs
- ☰ 17.3. Message Queues
- ☰ 17.4. Celery
- ☰ 17.5. Redis as a Message Broker
- ☰ 17.6. Scheduled Tasks
- ☰ 17.7. Retry Strategies
- ☰ 17.8. Handling Failed Jobs
- ☰ 17.9. Email Jobs
- ☰ 17.10. Long-Running Tasks
-
- ☰ 18. Email
- ☰ 18.1. How Email Works
- ☰ 18.2. SMTP
- ☰ 18.3. Sending Emails
- ☰ 18.4. HTML Emails
- ☰ 18.5. Transactional Emails
- ☰ 18.6. Password Reset Emails
- ☰ 18.7. Background Email Delivery
- ☰ 18.8. Email Delivery Best Practices
-
- ☰ 19. Testing
- ☰ 19.1. Why Backend Testing Matters
- ☰ 19.2. Unit Testing
- ☰ 19.3. Integration Testing
- ☰ 19.4. API Testing
- ☰ 19.5. pytest
- ☰ 19.6. Fixtures
- ☰ 19.7. Mocking
- ☰ 19.8. Testing Databases
- ☰ 19.9. Testing Authentication
- ☰ 19.10. Testing FastAPI
- ☰ 19.11. Test Coverage
- ☰ 19.12. Automated Testing
-
- ☰ 20. Logging and Monitoring
- ☰ 20.1. Application Logging
- ☰ 20.2. Log Levels
- ☰ 20.3. Structured Logging
- ☰ 20.4. Request Logging
- ☰ 20.5. Error Logging
- ☰ 20.6. Metrics
- ☰ 20.7. Health Checks
- ☰ 20.8. Monitoring
- ☰ 20.9. Prometheus
- ☰ 20.10. Grafana
- ☰ 20.11. Alerting
- ☰ 20.12. Error Tracking
- ☰ 20.13. Observability Basics
-
- ☰ 21. Docker
- ☰ 21.1. Introduction to Containers
- ☰ 21.2. Docker Architecture
- ☰ 21.3. Images
- ☰ 21.4. Containers
- ☰ 21.5. Dockerfiles
- ☰ 21.6. Building Images
- ☰ 21.7. Running Containers
- ☰ 21.8. Volumes
- ☰ 21.9. Networks
- ☰ 21.10. Docker Compose
- ☰ 21.11. FastAPI with Docker
- ☰ 21.12. PostgreSQL with Docker
- ☰ 21.13. Redis with Docker
- ☰ 21.14. Multi-Container Applications
- ☰ 21.15. Production Docker Best Practices
-
- ☰ 22. Web Servers and Reverse Proxies
- ☰ 22.1. Application Servers
- ☰ 22.2. Uvicorn
- ☰ 22.3. Gunicorn
- ☰ 22.4. Reverse Proxies
- ☰ 22.5. Nginx
- ☰ 22.6. Traefik
- ☰ 22.7. HTTPS
- ☰ 22.8. TLS Certificates
- ☰ 22.9. Let's Encrypt
- ☰ 22.10. Load Balancing
-
- ☰ 23. Deployment
- ☰ 23.1. Development vs Production
- ☰ 23.2. Preparing an Application for Production
- ☰ 23.3. Linux Server Setup
- ☰ 23.4. Deploying with Docker
- ☰ 23.5. Environment Configuration
- ☰ 23.6. Managing Secrets
- ☰ 23.7. HTTPS Configuration
- ☰ 23.8. Production Logging
- ☰ 23.9. Backups
- ☰ 23.10. Zero-Downtime Deployment
- ☰ 23.11. Rollbacks
-
- ☰ 24. CI/CD
- ☰ 24.1. Introduction to CI/CD
- ☰ 24.2. Continuous Integration
- ☰ 24.3. Continuous Delivery
- ☰ 24.4. Continuous Deployment
- ☰ 24.5. CI/CD Pipelines
- ☰ 24.6. GitHub Actions
- ☰ 24.7. GitLab CI/CD
- ☰ 24.8. Building Docker Images
- ☰ 24.9. Container Registries
- ☰ 24.10. Deployment Pipelines
- ☰ 24.11. Staging Environments
- ☰ 24.12. Production Environments
- ☰ 24.13. Rollback Strategies
-
- ☰ 25. Backend Architecture
- ☰ 25.1. Monolithic Architecture
- ☰ 25.2. Modular Monoliths
- ☰ 25.3. Microservices
- ☰ 25.4. Layered Architecture
- ☰ 25.5. Service Layer
- ☰ 25.6. Separation of Concerns
- ☰ 25.7. Domain-Driven Design Basics
- ☰ 25.8. Event-Driven Architecture
- ☰ 25.9. Choosing an Architecture
-
- ☰ 26. Performance and Scalability
- ☰ 26.1. Understanding Backend Performance
- ☰ 26.2. CPU-Bound vs I/O-Bound Work
- ☰ 26.3. Asynchronous Programming
- ☰ 26.4. Database Optimization
- ☰ 26.5. Query Optimization
- ☰ 26.6. Indexing
- ☰ 26.7. Load Balancing
- ☰ 26.8. Horizontal Scaling
- ☰ 26.9. Vertical Scaling
- ☰ 26.10. Stateless Applications
- ☰ 26.11. Performance Testing
- ☰ 26.12. Load Testing
-
- ☰ 27. Advanced API Development
- ☰ 27.1. WebSockets
- ☰ 27.2. Server-Sent Events
- ☰ 27.3. GraphQL
- ☰ 27.4. Webhooks
- ☰ 27.5. Idempotency
- ☰ 27.6. API Rate Limiting
- ☰ 27.7. API Keys
- ☰ 27.8. API Gateways
- ☰ 27.9. Distributed APIs
- ☰ 27.10. Real-Time Applications
-
- ☰ 28. Production Backend Engineering
- ☰ 28.1. Configuration Management
- ☰ 28.2. Database Backups
- ☰ 28.3. Disaster Recovery
- ☰ 28.4. Graceful Shutdown
- ☰ 28.5. Fault Tolerance
- ☰ 28.6. Circuit Breakers
- ☰ 28.7. High Availability
- ☰ 28.8. Monitoring Production Systems
- ☰ 28.9. Incident Response
-
- ☰ 29. Project 1: Task Management API
- ☰ 29.1. Project Requirements
- ☰ 29.2. Project Structure
- ☰ 29.3. Database Design
- ☰ 29.4. CRUD Operations
- ☰ 29.5. Validation
-
- ☰ 30. Project 2: Authentication System
- ☰ 30.1. User Registration
- ☰ 30.2. JWT Authentication
- ☰ 30.3. Role-Based Authorization
- ☰ 30.4. Security Testing
-
- ☰ 31. Project 3: E-Commerce Backend
- ☰ 31.1. Architecture
- ☰ 31.2. User Management
- ☰ 31.3. Product Management
- ☰ 31.4. Categories
- ☰ 31.5. Shopping Cart
- ☰ 31.6. Orders
- ☰ 31.7. Inventory
- ☰ 31.8. Payments
- ☰ 31.9. Admin API
-
- ☰ 32. Final Project: Production Backend
- ☰ 32.1. Planning the Application
- ☰ 32.2. Designing the Architecture
- ☰ 32.3. Designing the Database
- ☰ 32.4. Building the REST API
- ☰ 32.5. Authentication and Authorization
- ☰ 32.6. PostgreSQL Integration
- ☰ 32.7. Redis Integration
- ☰ 32.8. Background Workers
- ☰ 32.9. File Storage
- ☰ 32.10. Dockerizing the Application
- ☰ 32.11. CI/CD Pipeline
- ☰ 32.12. Production Deployment
- ☰ 32.13. HTTPS and Domain Setup
- ☰ 32.14. Performance Optimization
- ☰ 32.15. Security Review
- ☰ 32.16. Documentation
-
- ☰ 33. Backend Developer Roadmap
- ☰ 33.1. What You Should Know Now
- ☰ 33.2. Building a Portfolio
- ☰ 33.3. Creating Backend Projects
- ☰ 33.4. Contributing to Open Source
- ☰ 33.5. Preparing for Backend Interviews
- ☰ 33.6. Backend Interview Questions
- ☰ 33.7. Database Interview Questions
- ☰ 33.8. API Interview Questions
- ☰ 33.9. Linux and Docker Interview Questions
- ☰ 33.10. Where to Go Next
-
Comments
Please login to add a comment.
Don't have an account? Register now!