← Back|CLOUD-DEVOPS›Section 1/15

0 of 15 completed

Docker for AI apps

Intermediate⏱ 14 min read📅 Updated: 2026-02-17

Introduction

"My machine la work aagudhu, unga machine la yenda work aagala?" — Developer life oda biggest problem! 😤

AI apps ku idhu worse — TensorFlow version mismatch, CUDA driver issues, Python package conflicts. Oru developer machine la train aagura model vera developer machine la load aagaadhu.

Docker indha problem completely solve pannum! Unga app, dependencies, model — ellam oru container la pack panniduveenga. Enga run pannaalum same result! 🐳

Indha article la Docker basics, AI-specific Dockerfiles, multi-stage builds, GPU containers — ellam paapom! 📦

Docker Core Concepts

Docker la 4 key concepts:

1. Image 📸

Container oda blueprint/template
Read-only
Dockerfile la irundhu build aagum
Example: python:3.11-slim, nvidia/cuda:12.0

2. Container 📦

Image oda running instance
Isolated environment
Start, stop, delete pannalam
Multiple containers from same image

3. Dockerfile 📝

Image build panna instructions
Step-by-step recipe
FROM, RUN, COPY, CMD commands

4. Registry 🏪

Images store pannra place
Docker Hub (public, free)
ECR (AWS), GCR (Google), ACR (Azure)

Analogy 🍰:

Dockerfile = Recipe
Image = Frozen cake (ready to use)
Container = Cake cut panni serve pannradhu
Registry = Bakery shop (cakes store)

Docker Architecture

🏗️ Architecture Diagram

┌─────────────────────────────────────────────────┐
│              DOCKER ARCHITECTURE                  │
├─────────────────────────────────────────────────┤
│                                                   │
│  ┌──────────────────────────────────────────┐    │
│  │            Docker Host (Your Machine)     │    │
│  │                                           │    │
│  │  ┌─────────┐ ┌─────────┐ ┌─────────┐   │    │
│  │  │Container│ │Container│ │Container│   │    │
│  │  │ AI App  │ │ Redis   │ │Postgres │   │    │
│  │  │ + Model │ │ Cache   │ │   DB    │   │    │
│  │  └────┬────┘ └────┬────┘ └────┬────┘   │    │
│  │       └───────────┼───────────┘         │    │
│  │            ┌──────▼──────┐              │    │
│  │            │Docker Engine│              │    │
│  │            └──────┬──────┘              │    │
│  │            ┌──────▼──────┐              │    │
│  │            │  Host OS    │              │    │
│  │            │  (Linux)    │              │    │
│  │            └─────────────┘              │    │
│  └──────────────────────────────────────────┘    │
│                                                   │
│  vs Virtual Machine:                             │
│  ┌────────┐ ┌────────┐ ┌────────┐               │
│  │ App    │ │ App    │ │ App    │               │
│  │ Guest  │ │ Guest  │ │ Guest  │ ← Full OS    │
│  │  OS    │ │  OS    │ │  OS    │   each!      │
│  └───┬────┘ └───┬────┘ └───┬────┘               │
│      └──────────┼──────────┘                     │
│           Hypervisor (VMware/KVM)                │
│                                                   │
└─────────────────────────────────────────────────┘

AI App Dockerfile — Step by Step

✅ Example

Complete Dockerfile for AI inference app:

dockerfile

# Base image — slim version (smaller)
FROM python:3.11-slim

# Working directory set
WORKDIR /app

# System dependencies (for ML libraries)
RUN apt-get update && apt-get install -y \
    gcc g++ && \
    rm -rf /var/lib/apt/lists/*

# Python dependencies first (Docker caching!)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# App code copy
COPY . .

# Model download (or COPY from local)
RUN python download_model.py

# Port expose
EXPOSE 8080

# Health check
HEALTHCHECK CMD curl -f http://localhost:8080/health

# Run the app
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8080"]

Key tip: requirements.txt COPY first, then code. Dependencies change aagama iruntha cached layer use aagum — build fast! ⚡

Multi-Stage Builds (Smaller Images)

AI Docker images romba perusa aagum — 5GB+! Multi-stage builds use pannina size 50-70% reduce aagum:

dockerfile

# Stage 1: Build (install everything)
FROM python:3.11 AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --user -r requirements.txt

# Stage 2: Runtime (only needed files)
FROM python:3.11-slim
WORKDIR /app
COPY --from=builder /root/.local /root/.local
COPY . .
ENV PATH=/root/.local/bin:$PATH
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8080"]

Size comparison:

Approach	Image Size
Single stage (python:3.11)	3.5 GB
Single stage (slim)	2.1 GB
Multi-stage (slim)	1.2 GB
Multi-stage + .dockerignore	800 MB

Always use multi-stage for production AI apps! Storage cost um reduce aagum, deploy um fast aagum. 📦

GPU Support in Docker

AI model inference ku GPU venum na Docker la um GPU use pannalam!

Prerequisites:

NVIDIA GPU installed
NVIDIA Container Toolkit installed
Docker 19.03+

GPU Dockerfile:

dockerfile

FROM nvidia/cuda:12.0-runtime-ubuntu22.04

RUN apt-get update && apt-get install -y python3 python3-pip
COPY requirements.txt .
RUN pip3 install -r requirements.txt
COPY . .
CMD ["python3", "app.py"]

Run with GPU:

bash

docker run --gpus all -p 8080:8080 my-ai-app
# Specific GPU:
docker run --gpus '"device=0"' my-ai-app

Verify GPU inside container:

bash

docker run --gpus all nvidia/cuda:12.0-base nvidia-smi

Common CUDA images:

Base Image	Size	Use
nvidia/cuda:12.0-base	200MB	Runtime only
nvidia/cuda:12.0-runtime	1.5GB	+ CUDA runtime
nvidia/cuda:12.0-devel	3.5GB	+ compilation tools

Docker Compose for AI Stack

Multiple containers manage panna Docker Compose use pannunga:

yaml

# docker-compose.yml
version: '3.8'
services:
  ai-app:
    build: .
    ports:
      - "8080:8080"
    environment:
      - MODEL_PATH=/models/bert
      - REDIS_URL=redis://cache:6379
    volumes:
      - ./models:/models
    depends_on:
      - cache
      - db

  cache:
    image: redis:7-alpine
    ports:
      - "6379:6379"

  db:
    image: postgres:15-alpine
    environment:
      - POSTGRES_DB=aiapp
      - POSTGRES_PASSWORD=secret
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

One command la entire stack up:

bash

docker compose up -d    # Start all services
docker compose logs -f  # View logs
docker compose down     # Stop all

AI app + Redis cache + PostgreSQL — full stack 30 seconds la up! 🚀

Docker Optimization for AI

AI Docker images optimize panna tips:

1. .dockerignore file 📋

code

__pycache__
*.pyc
.git
.env
data/raw/
notebooks/
*.ipynb

2. Layer caching strategy 📦

Rarely changing layers first (OS, system deps)
Dependencies next (requirements.txt)
Code last (changes most often)

3. Slim base images 🏋️

python:3.11 → 1.0GB
python:3.11-slim → 150MB
python:3.11-alpine → 60MB (compatibility issues possible)

4. Model optimization 🧠

ONNX format use pannunga (smaller, faster)
Quantized models use pannunga (INT8)
Model weights separate volume la mount pannunga

5. Multi-stage builds 🏗️

Build dependencies final image la include aagaadhu
Compiler, dev tools — build stage la mattum

Result: 5GB image → 800MB image. Deploy 6x faster! ⚡

Common Docker Issues (AI Apps)

⚠️ Warning

AI apps la common Docker issues & solutions:

⚠️ Issue: Image too large (5GB+)

→ Multi-stage build, slim base, .dockerignore

⚠️ Issue: Build takes 30+ minutes

→ Layer caching optimize, requirements.txt separate COPY

⚠️ Issue: Out of memory (OOM killed)

→ Docker memory limit increase: docker run -m 4g

⚠️ Issue: GPU not detected

→ nvidia-container-toolkit install, --gpus all flag

⚠️ Issue: Model file not found

→ COPY path check, or volume mount use

⚠️ Issue: Permission denied

→ RUN chown or USER directive add

⚠️ Issue: Slow inference in container

→ CPU cores limit check, OMP_NUM_THREADS set

Essential Docker Commands

Daily use pannra Docker commands:

Command	Purpose
`docker build -t name .`	Image build
`docker run -p 8080:8080 name`	Container start
`docker ps`	Running containers list
`docker logs container_id`	View logs
`docker exec -it container bash`	Shell access
`docker stop container_id`	Stop container
`docker images`	List images
`docker rmi image_id`	Delete image
`docker system prune -a`	Cleanup everything
`docker compose up -d`	Start all services
`docker compose down`	Stop all services
`docker stats`	Resource usage monitor

Pro tip: docker system prune -a regularly run pannunga — unused images, containers disk space eat pannudum! 🧹

Prompt: Docker Setup

📋 Copy-Paste Prompt

You are a Docker expert specializing in AI applications.

I have a Python FastAPI app that:
- Loads a HuggingFace BERT model (500MB)
- Serves text classification API
- Uses Redis for caching
- Needs to run on CPU (no GPU)

Create:
1. Optimized multi-stage Dockerfile
2. docker-compose.yml (app + Redis)
3. .dockerignore file
4. Build and run commands
5. Production optimization tips

Target: Image size under 1GB, startup time under 30 seconds.

✅ Key Takeaways

✅ Docker Solves Reproducibility — App + dependencies package. Same image enga run pannaalum consistent result. "Works on my machine" forever solve

✅ Dockerfile Concepts — Image (blueprint), Container (running instance), Layer (caching). FROM slim base, COPY dependencies first, code last (caching efficiency)

✅ Multi-Stage Builds — Build stage (install everything) + runtime stage (only needed). 5GB image → 800MB possible. AI apps mandatory for large models

✅ Optimization Techniques — .dockerignore (unnecessary files exclude), layer caching strategy, slim base images, non-root user (security), health checks

✅ GPU in Docker — nvidia/cuda base image, nvidia-container-toolkit install, --gpus all flag. Model inference GPU-accelerated containers possible

✅ Docker Compose — Multiple containers (app, Redis, database) single YAML file. Local development perfect; production Kubernetes recommended

✅ Common Issues — Image too large (multi-stage), build slow (cache), OOM killed (memory limit), GPU not detected (toolkit), model missing (volume mount)

✅ AI App Pattern — Python:slim base, requirements.txt separate COPY (caching), app code last, ONNX models volume mount, health endpoint, proper signals handling

🏁 🎮 Mini Challenge

Challenge: Containerize an AI App (HuggingFace Model)

Real Docker practice — sentiment analysis model containerize pannu:

Step 1: Python App Create Pannunga 🐍

python

# app.py
from transformers import pipeline
from fastapi import FastAPI

app = FastAPI()
classifier = pipeline("sentiment-analysis")

@app.post("/analyze")
def analyze(text: str):
    result = classifier(text)
    return {"text": text, "sentiment": result}

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)

Step 2: Requirements Create Pannunga 📋

bash

# requirements.txt
torch==2.0.0
transformers==4.28.0
fastapi==0.104.0
uvicorn==0.24.0

Step 3: Dockerfile Write Pannunga 🐳

dockerfile

FROM python:3.10-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app.py .

EXPOSE 8000

CMD ["python", "app.py"]

Step 4: Build & Run Locally 🚀

bash

# Build image
docker build -t sentiment-ai:latest .

# Run container
docker run -p 8000:8000 sentiment-ai:latest

# Test endpoint
curl -X POST "http://localhost:8000/analyze?text=I love machine learning"

# Stop container
docker ps  # see running container
docker stop <container-id>

Step 5: Push to Docker Hub 📦

bash

# Create Docker Hub account (free)
docker tag sentiment-ai:latest your-username/sentiment-ai:latest
docker push your-username/sentiment-ai:latest

Step 6: Run from Registry 🌐

bash

# Anyone can now run:
docker run -p 8000:8000 your-username/sentiment-ai:latest

Completion Time: 90 minutes

Key Skills: Docker build, tagging, pushing, running

Real-world project ⭐

💼 Interview Questions

Q1: Docker vs Virtual Machine — difference? When use?

A: VM = full OS (heavy, slow). Docker = lightweight container (fast, efficient). Docker: modern applications, microservices, CI/CD. VM: legacy apps, full OS needed, security isolation critical. AI apps: Docker perfect — quick deployment, reproducible.

Q2: Docker image size large issue — optimization?

A: Use slim/alpine base images. Multi-stage build (build stage, runtime stage). Remove unnecessary packages. .dockerignore use (large files exclude). Model caching (separate layer). Tool: docker-slim (automatic optimization). Large models: volume mount kara, image la store panna vendaam.

Q3: Docker security concerns — best practices?

A: Scan images (vulnerabilities). Don't run as root. Read-only filesystem. Secrets management (environment variables, not hardcoded). Registry authentication. Regular updates. Minimal images (less attack surface). Private registries for sensitive models.

Q4: Docker volume vs bind mount — AI apps la which use?

A: Volume = Docker-managed, portable, consistent. Bind mount = host directory directly mount. AI apps: models (volume, persistent), code (bind mount, local dev). Production: volumes recommended. Development: bind mounts faster iteration.

Q5: Multi-GPU Docker setup — how?

A: ```bash

# Dockerfile

FROM nvidia/cuda:12.0-runtime

# Rest setup same

code

Run: `docker run --gpus all -it image-name`
All GPUs automatically visible. CUDA runtime take care. Multi-GPU model training: torch.distributed use, GPUs automatically distributed.

Frequently Asked Questions

❓ Docker na enna?

Docker is a containerization platform — unga app and all dependencies oru "container" la pack panniduveenga. Any machine la same ah run aagum.

❓ Docker vs Virtual Machine enna difference?

VM full OS run pannum (heavy, slow). Docker OS kernel share pannum (lightweight, fast). Docker containers seconds la start aagum, VMs minutes edukum.

❓ AI apps ku Docker yenda special?

AI apps ku specific Python versions, CUDA drivers, ML libraries venum. Docker la ivanga ellam package pannidalam — "works on my machine" problem solve!

❓ Docker free ah?

Yes! Docker Engine completely free and open-source. Docker Desktop free for personal use and small businesses (<250 employees).

🧠Knowledge Check

Quiz 1 of 1

Docker la multi-stage build use pannra main reason enna?

0 of 1 answered

← Previous ByteCI/CD basics Next Byte →Kubernetes basics

Courses

Learning Paths

Exam Prep

Docker for AI apps

Introduction

Docker Core Concepts

Docker Architecture

AI App Dockerfile — Step by Step

Multi-Stage Builds (Smaller Images)

GPU Support in Docker

Docker Compose for AI Stack

Docker Optimization for AI

Common Docker Issues (AI Apps)

Essential Docker Commands

Prompt: Docker Setup

✅ Key Takeaways

🏁 🎮 Mini Challenge

💼 Interview Questions

Frequently Asked Questions