Practical outcome
You will persist assistant conversations, call external AI services with bounded failure handling and keep configuration outside application code.
Follow one request across database and external-service boundaries
Keep the route thin
An early prototype often puts SQL, API calls, prompt construction and response formatting inside one route. It works until you need to test, retry or replace one dependency.
A clearer design assigns one responsibility to each layer:
01route → application service → repository02└──→ external provider client
- Route: HTTP input, dependency wiring and response.
- Service: use-case workflow and business decisions.
- Repository: persistent data operations.
- Provider client: external API contract, timeout and error translation.
Configure the application safely
01from functools import lru_cache02from pydantic_settings import BaseSettings, SettingsConfigDict0304class Settings(BaseSettings):05database_url: str06model_api_key: str07model_timeout_seconds: float = 20.008environment: str = "development"0910model_config = SettingsConfigDict(11env_file=".env",12env_file_encoding="utf-8",13extra="ignore",14)1516@lru_cache17def get_settings() -> Settings:18return Settings()
Use .env only for local development and exclude it from Git. In deployed environments, inject secrets through the platform or a secret manager.
Use dependencies to make boundaries visible
01from typing import Annotated02from fastapi import Depends0304SettingsDep = Annotated[Settings, Depends(get_settings)]0506@app.get("/runtime")07def runtime_info(settings: SettingsDep):08return {"environment": settings.environment}
Dependencies can provide settings, database sessions, authenticated users and services. They also make tests easier because FastAPI can override them.
Persist conversations through a repository
01from dataclasses import dataclass0203@dataclass04class Conversation:05id: str06user_id: str07title: str0809class ConversationRepository:10def create(self, conversation: Conversation) -> Conversation:11# Execute parameterised database work here.12return conversation1314def find_for_user(self, conversation_id: str, user_id: str):15# Scope every query to the authorised user or tenant.16...
The repository contract keeps storage decisions away from HTTP. Whether you use SQLAlchemy, SQLModel or a direct database driver, manage one session per request or explicit unit of work and close it reliably.
Call an external service with limits
01import httpx0203class ModelProvider:04def __init__(self, api_key: str, timeout: float):05self.api_key = api_key06self.timeout = timeout0708async def generate(self, prompt: str) -> str:09async with httpx.AsyncClient(timeout=self.timeout) as client:10response = await client.post(11"https://provider.example/v1/generate",12headers={"Authorization": f"Bearer {self.api_key}"},13json={"prompt": prompt},14)15response.raise_for_status()16return response.json()["text"]
Production clients normally reuse connection pools rather than creating a client per call. The important beginner mental model is explicit timeout, authentication, status check and controlled response parsing.
Translate provider failure at the boundary
Do not leak a vendor’s error body to your API consumer. Translate it into your application language:
| Provider failure | API decision |
|---|---|
| Timeout | Retry within a strict budget or return 503 |
| Rate limit | Respect backoff; return 429 or 503 based on ownership |
| Authentication rejected | Alert configuration owner; return safe 503 |
| Invalid provider response | Record request ID; fail validation safely |
Retry only operations that are safe to repeat. Use exponential backoff with jitter and a maximum attempt/time budget.
Real-time scenario: save before generating
A recoverable assistant request
The API validates a question, creates a request record with status pending, calls retrieval and the model, then stores the final answer and citation IDs. If the provider times out, the request becomes failed_retryable with a safe reason. The user receives a request ID instead of losing all evidence of the attempt.
Async does not make blocking work disappear
Use async def when the libraries you call support awaiting network or database I/O. A blocking library inside an async route can still block the event loop. CPU-heavy document processing and long ingestion jobs usually belong in a worker rather than the request process.
Common mistakes
Common mistakes
- Opening a new provider connection without timeout on every request.
- Hardcoding database URLs or model keys in Python files.
- Sharing one unrestricted database query across tenants.
- Retrying every error, including invalid requests and non-idempotent writes.
- Placing long PDF ingestion inside the interactive answer endpoint.
Interview bit
How do you structure a FastAPI service?
I keep route handlers focused on HTTP and inject application services. Services coordinate the use case, repositories own persistence and provider clients own external contracts, timeouts and error translation. Configuration comes from validated settings and deployed secrets. This separation makes failure handling, testing and provider replacement much safer.
Key takeaways
Key takeaways
- Routes coordinate HTTP; services and adapters own application work.
- Inject configuration and dependencies so they are explicit and replaceable.
- Scope database operations by verified user or tenant identity.
- Every external call needs a timeout and controlled error translation.
- Move long-running ingestion or CPU work away from interactive requests.
Frequently asked questions
Where should database sessions be created and closed? Manage them through a clear dependency or unit-of-work boundary so cleanup still occurs when a request fails.
Should an API retry every failed external call? No. Retry only transient, safe-to-repeat operations with limits and backoff; otherwise retries can duplicate actions or amplify an outage.
Where should API keys and database passwords live? Keep secrets outside source code and images, inject them through approved configuration and rotate them when exposure is suspected.
Interactive Knowledge Check
Choose an answer, inspect the explanation and explain the idea in your own words.