Docker and Kubernetes in Production

Docker: Application Packaging and Isolation

Before Docker, deploying an application meant managing dependencies on the host machine — a Python version here, a library there, a system configuration setting that only works on Ubuntu 18. The "it works on my machine" problem was endemic. Docker solves this by packaging the application and all its dependencies into a single, portable unit called a container.

A Docker container is a running instance of a Docker image. An image is a layered filesystem snapshot built from a Dockerfile — a sequential list of instructions that defines exactly what goes into the container. The image can be pushed to a registry (Docker Hub, AWS ECR, Google Artifact Registry) and pulled on any machine that runs Docker. The container behaves identically regardless of the host OS.

Key Docker Concepts

• Image — Immutable snapshot of the filesystem and runtime configuration. Built by docker build.
• Container — A running instance of an image. Ephemeral by default — data is lost when the container is removed unless you use volumes.
• Volume — Persistent storage that outlives a container's lifecycle. Essential for databases and stateful applications.
• Network — Docker creates software-defined networks that allow containers to communicate with each other using service names as hostnames.
• Docker Compose — A YAML-based tool for defining and running multi-container applications on a single machine. Perfect for local development and simple production setups.

Kubernetes: Orchestration at Scale

Docker Compose works well for running a few containers on one machine. But what happens when you need to run your application across ten servers? When a server goes down? When you need to roll out a new version with zero downtime? When a container crashes at 3am and needs to be automatically restarted? This is where Kubernetes comes in.

Kubernetes (often abbreviated as K8s) is a container orchestration platform. You describe the desired state of your application (I want 5 replicas of this container running, exposed on port 80, with at least 512MB of RAM each), and Kubernetes continuously works to ensure that state is maintained, even as nodes fail, containers crash, or load spikes.

Core Kubernetes Resources

• Pod — The smallest deployable unit in Kubernetes. Usually one container, sometimes a sidecar or init container alongside it.
• Deployment — Manages a set of identical pod replicas. Handles rolling updates and rollbacks automatically.
• Service — A stable network endpoint (IP address and DNS name) that routes traffic to pods. Abstracts away the fact that pods are ephemeral.
• Ingress — Manages HTTP/S routing from outside the cluster to services inside it. Typically backed by Nginx or a cloud load balancer.
• ConfigMap / Secret — Externalizes configuration and sensitive credentials from container images.
• HorizontalPodAutoscaler (HPA) — Automatically scales the number of pod replicas based on CPU usage or custom metrics.

How Docker and Kubernetes Work Together

The workflow in a Kubernetes-based production system looks like this:

1. Develop — Write your application code and a Dockerfile that packages it.
2. Build — CI system builds a Docker image and tags it with the git commit hash (my-app:a3f2c9d).
3. Push — The image is pushed to a container registry.
4. Deploy — The Kubernetes Deployment manifest is updated to reference the new image tag. Kubernetes pulls the image onto its nodes and performs a rolling update.
5. Run — Kubernetes schedules pods onto nodes based on resource availability, maintains the desired replica count, restarts failed pods, and routes traffic via Services.
6. Scale — When CPU usage spikes, HPA automatically adds more replicas. When load drops, it scales back down.

Kubernetes does not run containers directly — it delegates to a container runtime (containerd or CRI-O in modern clusters, Docker in older ones). This is why Docker images work seamlessly in Kubernetes: both use the OCI (Open Container Initiative) image format as a common standard.

Production Patterns We Use

Health Checks (Liveness and Readiness Probes)

Kubernetes can automatically restart containers that are unhealthy and withhold traffic from containers that are not yet ready to serve. Configure a liveness probe that calls your /health endpoint periodically. If it fails three times in a row, Kubernetes restarts the pod. Configure a readiness probe (also on /health or a separate /ready endpoint) to control when traffic is routed to a newly started pod — so you never send traffic to a pod that is still loading its model or warming up its cache.

Resource Requests and Limits

Always set CPU and memory requests and limits on your pods. Requests tell Kubernetes how much to reserve when scheduling. Limits prevent a misbehaving pod from starving its neighbors. Without these, a memory leak in one pod can crash an entire node.

Rolling Updates and Pod Disruption Budgets

Kubernetes Deployments perform rolling updates by default, replacing pods one at a time. Combine this with a PodDisruptionBudget that says "always keep at least 2 replicas running" to ensure zero-downtime deployments even during cluster maintenance.

GitOps with Argo CD or Flux

In a GitOps workflow, your Kubernetes manifests live in a git repository. Argo CD or Flux continuously syncs the cluster state to match the git state. Any change to the cluster goes through pull requests, giving you a full audit trail, easy rollback, and automatic drift detection.

When to Skip Kubernetes

Kubernetes is powerful but complex. It is overkill for many applications. If you are running a small number of containers on a single server, Docker Compose is a perfectly valid production solution and significantly simpler to operate. If you want managed orchestration without the K8s learning curve, consider AWS ECS, Google Cloud Run, or Railway.

Consider Kubernetes when you have true multi-service applications across multiple nodes, when you need advanced auto-scaling, or when your team has the operational bandwidth to manage a cluster. Managed offerings like GKE Autopilot, EKS Fargate, or AKS reduce but do not eliminate the operational complexity.

Back to all articles