Part 3: Scalability¶
13. Scalable Folder Structure for FastAPI¶
“When APIs grow, folder structure becomes your defense against entropy.”
Why Scalability Hits the Backend First¶
React can get away with a little mess. FastAPI cannot.
As your backend evolves to support:
- Auth and permission layers
- Vector DBs, async GPT pipelines, OCR parsing
- Multiple API versions
- Streaming, WebSockets, background jobs
…your old api/
, services/
, schemas/
layout starts to strain.
You need a layered backend architecture that:
- Supports independent deployments
- Reduces coupling between services
- Enables async, task-driven workflows
- Maintains testability and refactor-safety
Recommended Scalable FastAPI Folder Layout¶
app/
├── api/ # Route definitions (per version, grouped by domain)
│ ├── v1/
│ │ ├── chatbot.py
│ │ ├── invoice.py
│ │ └── upload.py
├── services/ # Business logic (GPT, processing, workflows)
│ ├── gpt_service.py
│ ├── invoice_parser.py
│ └── embedding_service.py
├── schemas/ # Pydantic models for API input/output
│ ├── chatbot.py
│ ├── invoice.py
│ └── shared.py
├── core/ # App settings, middleware, auth guards
│ ├── config.py
│ ├── security.py
│ └── exceptions.py
├── infra/ # External systems (DB, VectorDB, S3, Celery)
│ ├── database.py
│ ├── supabase.py
│ ├── weaviate.py
│ └── celery_worker.py
├── tasks/ # Background task definitions
│ └── process_invoice.py
├── shared/ # Utilities used across layers
│ ├── ocr.py
│ ├── image_utils.py
│ └── logging.py
├── main.py # FastAPI app entry point
└── routers.py # Route inclusion logic
This structure mirrors clean architecture and DDD layering, while staying idiomatic to FastAPI.
Key Layers and Responsibilities¶
Layer | Purpose |
---|---|
api/ |
Route definitions, versioning, and decorators |
services/ |
Business logic, orchestration, pipeline steps |
schemas/ |
Data contracts between client and backend |
core/ |
Security, config, middleware, exception handling |
infra/ |
External resources like DBs, VectorStores, file storage |
tasks/ |
Async background jobs (Celery, BackgroundTasks) |
shared/ |
Utility functions reused across modules |
tests/ |
May live locally or in root folder, depending on scale |
Clean Separation of Router, Service, and Schema¶
FastAPI makes it easy to blur lines—resist the temptation.
Bad Pattern:
@app.post("/chat")
def chat_handler(input: str = Body(...)):
response = openai.ChatCompletion.create(...)
return response
Scalable Pattern:
# api/v1/chatbot.py
@router.post("/chat", response_model=ChatResponse)
def handle_chat(input: ChatInput, svc: GPTService = Depends()):
return svc.generate_response(input)
# services/gpt_service.py
class GPTService:
def generate_response(self, input: ChatInput) -> ChatResponse:
...
Why?
- Routes should be thin
- Services should be testable
- Schemas should be reusable
Versioned Routing and Scalable Endpoints¶
Structure your api/
like this:
api/
├── v1/
│ ├── chatbot.py
│ ├── invoice.py
│ └── upload.py
├── v2/
│ └── chatbot.py
In main.py
or routers.py
:
app.include_router(v1.chatbot.router, prefix="/v1/chatbot")
app.include_router(v2.chatbot.router, prefix="/v2/chatbot")
Benefits:
- Non-breaking upgrades
- Parallel development of new features
- Easier deprecation and API sunset cycles
Supporting Background Jobs¶
Background processing becomes essential at scale:
- GPT calls that exceed request timeouts
- OCR-heavy image parsing
- Long-running embedding workflows
Structure:
tasks/
├── embed_document.py
├── process_invoice.py
└── reindex_vector_db.py
Use:
BackgroundTasks
(FastAPI native, simple cases)Celery
+ Redis (robust task queue)RQ
,dramatiq
, orHuey
(lightweight alternatives)
Keep tasks/
decoupled from api/
—triggered by services/
.
Auth, Permissions, and Role-Based Modules¶
Place guards and JWT logic in:
core/
├── security.py # Auth check functions, token decode/validate
├── permissions.py # Role-based access checks
└── dependencies.py # Dependency injection helpers
In route:
@router.post("/secure-action", dependencies=[Depends(require_admin)])
def admin_only_action():
...
- Clean
- Declarative
- Testable
Repository Pattern for Data Access¶
To decouple DB from services:
infra/
├── database.py # Engine + session
├── repositories/
│ ├── user_repo.py
│ └── invoice_repo.py
Services call repositories, not Session
or raw SQL:
class InvoiceService:
def __init__(self, repo: InvoiceRepository):
self.repo = repo
def create_invoice(...):
return self.repo.insert_invoice(...)
This enables:
- Swappable DBs (e.g., PostgreSQL → Supabase)
- Easier mocking in tests
- Separation of query logic from business flow
Summary: FastAPI at Scale¶
Concern | Where It Lives |
---|---|
Routes | api/v1/ |
Business logic | services/ |
Schema validation | schemas/ |
Config/auth/middleware | core/ |
VectorDB, Celery, S3 | infra/ |
Async tasks | tasks/ |
Shared utils | shared/ |
API versioning | api/v1/ , api/v2/ |
Role-based access | core/security.py , permissions.py |
This structure prepares your backend to:
- Handle growing teams and responsibilities
- Expand into multi-feature orchestration
- Scale horizontally (via services) and vertically (via workers/tasks)
In the next chapter, we’ll zoom out to deployment concerns—and how structure directly impacts your CI/CD, Dockerization, and scaling workflows.