Secure, Test and Deploy a Production FastAPI Application

Production changes the question

The prototype question is “Does it answer?” The production questions are:

• Who is allowed to ask and which documents may they access?
• What prevents one user from exhausting model cost?
• Can success, refusal and provider failure be reproduced in tests?
• Can the service be observed and rolled back?

Authenticate, then authorise

Authentication proves identity. Authorisation decides what that identity may do.

01from typing import Annotated02from fastapi import Depends, HTTPException, status03from fastapi.security import OAuth2PasswordBearer0405oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/auth/token")0607async def get_current_user(token: Annotated[str, Depends(oauth2_scheme)]):08    claims = verify_token_signature_and_claims(token)09    if claims is None:10        raise HTTPException(11            status_code=status.HTTP_401_UNAUTHORIZED,12            detail="Invalid authentication credentials",13        )14    return User.from_claims(claims)

Token verification should validate signature, issuer, audience, expiry and required claims. Do not merely decode a token and trust its content.

Authorisation happens afterward:

01def require_permission(user: User, permission: str) -> None:02    if permission not in user.permissions:03        raise HTTPException(status_code=403, detail="Permission denied")

For RAG, use the verified tenant and groups to filter retrieval before the model sees evidence.

Limit cost and abuse

AI endpoints can be expensive even when CPU usage is small. Control:

• requests per identity and tenant;
• concurrent generations;
• maximum input length and retrieved context;
• model/token budget per request;
• upload size and document count; and
• timeout and retry budget.

Return 429 Too Many Requests with safe retry guidance. Distributed deployments need a shared limiter rather than memory local to one process.

Log events, not secrets

Use structured logs:

01logger.info("assistant_request_completed", extra={02    "request_id": request_id,03    "tenant_id": user.tenant_id,04    "route": "/assistant/stream",05    "latency_ms": latency_ms,06    "retrieved_count": len(evidence),07    "model": model_name,08    "input_tokens": usage.input_tokens,09    "output_tokens": usage.output_tokens,10})

Do not log bearer tokens, API keys or complete private documents. Decide whether prompts and answers may be logged, how they are redacted and how long traces are retained.

Test the public contract

01from fastapi.testclient import TestClient02from main import app0304client = TestClient(app)0506def test_health():07    response = client.get("/health")08    assert response.status_code == 20009    assert response.json() == {"status": "healthy"}1011def test_question_rejects_empty_input():12    response = client.post("/questions", json={"question": "  "})13    assert response.status_code == 422

Override dependencies so tests do not call a live model:

01def fake_generator():02    return StubGenerator(answer="Five days", source_ids=["hr-4.2"])0304app.dependency_overrides[get_generator] = fake_generator

Test more than the happy path:

Containerise the application

01FROM python:3.12-slim0203WORKDIR /app0405COPY requirements.txt .06RUN pip install --no-cache-dir -r requirements.txt0708COPY app ./app0910RUN useradd --create-home apiuser11USER apiuser1213EXPOSE 800014CMD ["fastapi", "run", "app/main.py", "--port", "8000"]

Pin tested dependency versions, use a small trusted base image, run as a non-root user and scan the final image. Inject secrets at runtime—never use ENV MODEL_API_KEY=... in the Dockerfile.

Deploy the right responsibilities

01load balancer → FastAPI replicas → database/vector store02                               └→ model provider0304document upload → ingestion worker → versioned index

Keep interactive answering and document ingestion separate. A large file should not consume the request worker needed to stream answers.

Configure readiness and liveness carefully. Liveness asks whether the process should be restarted. Readiness asks whether it can safely receive traffic. Do not make every health probe call the paid LLM provider.

Real-time scenario: a safe release

Production release gate

1. Authentication and document-isolation tests pass.
2. Request, response and SSE event contracts are versioned and tested.
3. Provider timeout, retry and circuit behaviour are bounded.
4. Secrets are injected at runtime and excluded from logs/images.
5. Cost, latency, error and token metrics are visible.
6. Container runs without root and passes scanning.
7. Database/index migrations and rollback are tested.

Common mistakes

Interview bit

Key takeaways

Frequently asked questions

Is authentication enough to secure an AI API? No. The service also needs authorisation, input limits, cost controls, secret management, safe logging and dependency maintenance.

What should production tests cover beyond successful responses? Test invalid input, denied access, timeouts, dependency failures, streaming interruption and boundaries around expensive or consequential operations.

Should a health endpoint call every external dependency? Usually separate basic process health from deeper readiness checks so temporary downstream problems do not create misleading restart loops.

Interactive Knowledge Check

Primary sources

• FastAPI security tutorials
• FastAPI testing
• FastAPI in containers
• FastAPI deployment concepts