Docker & Kubernetes for Backend Engineers: The Practical Guide

Almost every backend service at Indian product companies now ships as a container image and runs on Kubernetes, whether that is EKS, GKE, AKS or a self-managed cluster. You do not need to be a platform engineer, but you do need to write a Dockerfile that does not leak secrets or weigh 1.5 GB, and understand why your pod got OOMKilled, why a deploy dropped requests, and how autoscaling actually decides. This guide covers exactly that layer: the things a backend engineer touches, with working examples.

Containers vs Virtual Machines: What Is Actually Different

A container is not a lightweight VM. It is a process (or process tree) that the kernel shows a restricted view of the world:

• Namespaces isolate what a process can see: PID (its own process tree where it is PID 1), network (own interfaces and ports), mount (own filesystem view), UTS (hostname), IPC, user (UID mapping) and cgroup namespaces.
• cgroups limit what it can use: CPU shares and quotas, memory (exceeding the limit triggers the OOM killer), I/O, PIDs.
• Union/overlay filesystem stacks read-only image layers with a thin writable layer on top.

In the cloud you typically get both: containers running on VMs (your Kubernetes nodes are EC2 or GCE instances). For stronger isolation of untrusted code, sandboxed runtimes such as gVisor or Firecracker micro-VMs sit between the two, which is how online judges and serverless platforms run arbitrary user code.

Images, Layers and the Build Cache

An image is an ordered stack of read-only layers plus metadata (entrypoint, env, exposed ports), following the OCI image spec. Each filesystem-changing Dockerfile instruction (RUN, COPY, ADD) creates a layer identified by a content hash. Two consequences matter daily:

1. Cache invalidation is top-down. If a layer changes, every layer after it is rebuilt. Copy files that change rarely (dependency manifests) before files that change constantly (source code).
2. Deleting in a later layer does not shrink the image. RUN rm secret.pem after COPY secret.pem leaves the file in the earlier layer, readable by anyone who pulls the image. This is a real way credentials leak.

Writing a Production-Grade Dockerfile

Java (Spring Boot) with a multi-stage build

# ---- build stage: has Maven and the JDK, never shipped ----
FROM maven:3.9-eclipse-temurin-21 AS build
WORKDIR /src
# 1. dependencies first: this layer is cached until pom.xml changes
COPY pom.xml .
RUN mvn -q -B dependency:go-offline
# 2. then source code
COPY src ./src
RUN mvn -q -B package -DskipTests

# ---- runtime stage: JRE only ----
FROM eclipse-temurin:21-jre-alpine
WORKDIR /app
RUN addgroup -S app && adduser -S app -G app
COPY --from=build /src/target/order-service.jar app.jar
USER app
EXPOSE 8080
# container-aware heap sizing instead of a hardcoded -Xmx
ENV JAVA_TOOL_OPTIONS="-XX:MaxRAMPercentage=75"
# exec form: java is PID 1 and receives SIGTERM directly
ENTRYPOINT ["java", "-jar", "/app/app.jar"]

Python (FastAPI)

FROM python:3.12-slim AS base
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
WORKDIR /app

FROM base AS deps
COPY requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt

FROM base
COPY --from=deps /install /usr/local
COPY app ./app
RUN useradd --create-home --uid 10001 appuser
USER 10001
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

.git
**/__pycache__
**/*.pyc
.env
*.pem
target/
node_modules/
tests/

The checklist behind those files

Kubernetes Architecture in One Picture

The key idea is declarative reconciliation. You kubectl apply a manifest saying "4 replicas of image v42". The API server stores it in etcd. The Deployment controller notices desired ≠ actual and creates a ReplicaSet, whose controller creates Pod objects. The scheduler picks nodes based on resource requests, affinity and taints. Each node's kubelet sees pods assigned to it and asks containerd to start them. If a node dies, controllers notice missing pods and create replacements elsewhere. Nobody runs "start 4 containers" imperatively.

Since Kubernetes 1.24 removed dockershim, nodes use containerd or CRI-O directly. Your Docker-built images still work because they are OCI images.

Pods, ReplicaSets and Deployments

• Pod: the smallest schedulable unit; one or more containers sharing a network namespace (same IP, talk over localhost) and volumes. Usually one app container plus optional sidecars (log shipper, service-mesh proxy).
• ReplicaSet: keeps N identical pods running. You rarely create one directly.
• Deployment: manages ReplicaSets to give rolling updates and rollbacks.
• StatefulSet: stable pod names and per-pod persistent volumes for databases and brokers.
• DaemonSet: one pod per node (log collectors, node exporters). Job/CronJob: run-to-completion work.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-service
  labels: { app: order-service }
spec:
  replicas: 4
  revisionHistoryLimit: 5
  selector:
    matchLabels: { app: order-service }
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1          # at most 5 pods during rollout
      maxUnavailable: 0    # never drop below 4 ready pods
  template:
    metadata:
      labels: { app: order-service }
    spec:
      terminationGracePeriodSeconds: 40
      containers:
        - name: app
          image: registry.example.com/order-service:1.42.0
          ports: [{ containerPort: 8080 }]
          envFrom:
            - configMapRef: { name: order-service-config }
            - secretRef:    { name: order-service-secrets }
          resources:
            requests: { cpu: "500m", memory: "768Mi" }
            limits:   { memory: "768Mi" }
          readinessProbe:
            httpGet: { path: /health/ready, port: 8080 }
            periodSeconds: 5
            failureThreshold: 3
          livenessProbe:
            httpGet: { path: /health/live, port: 8080 }
            periodSeconds: 10
            failureThreshold: 6
          startupProbe:
            httpGet: { path: /health/live, port: 8080 }
            periodSeconds: 5
            failureThreshold: 24    # up to 2 minutes to boot
          lifecycle:
            preStop:
              exec: { command: ["sleep", "10"] }

A rolling update with maxSurge: 1, maxUnavailable: 0 creates one new pod, waits until it is ready, then terminates one old pod, and repeats. That is why a correct readiness probe is the foundation of zero-downtime deploys. kubectl rollout undo deployment/order-service flips back to the previous ReplicaSet in seconds.

Services, DNS and Ingress

Pods are ephemeral and their IPs change. A Service gives a stable virtual IP and DNS name (order-service.payments.svc.cluster.local) that load-balances across pods matching a label selector. Only ready pods are in its endpoints.

apiVersion: v1
kind: Service
metadata: { name: order-service }
spec:
  selector: { app: order-service }
  ports: [{ port: 80, targetPort: 8080 }]
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata: { name: public-api }
spec:
  ingressClassName: nginx
  tls: [{ hosts: [api.example.com], secretName: api-tls }]
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /orders
            pathType: Prefix
            backend: { service: { name: order-service, port: { number: 80 } } }

An Ingress (or the newer Gateway API) provides HTTP routing by host and path, plus TLS termination, through one external load balancer instead of one per service. In a system design discussion this is the layer where an API gateway handles auth, rate limits and routing.

One gotcha: Kubernetes Services balance per connection (kube-proxy iptables/IPVS rules), not per request. Long-lived HTTP/2 or gRPC connections from a client stick to one pod, so load can be badly skewed. Fixes: client-side load balancing with a headless Service, or a service mesh (Envoy-based) that balances per request.

Probes, Requests and Limits: Where Most Production Bugs Live

Readiness vs liveness vs startup

Requests and limits

• Requests are what the scheduler reserves. A node with 4 CPUs fits eight pods requesting 500m each. HPA CPU utilization is computed against requests.
• CPU limits are enforced by CFS throttling: exceed your quota in a 100 ms period and the container is paused until the next period, which shows up as mysterious p99 latency spikes. Many teams set CPU requests but no CPU limits for latency-sensitive services; know your organization's policy.
• Memory limits are hard: exceed them and the container is OOMKilled (exit code 137). For the JVM, size the heap relative to the container limit (MaxRAMPercentage) and leave room for metaspace, threads and direct buffers.
• Setting requests equal to limits for all resources gives the Guaranteed QoS class, last to be evicted under node pressure.

Horizontal Pod Autoscaler: The Math and the Gotchas

The HPA controller periodically (every 15 seconds by default) computes:

desiredReplicas = ceil( currentReplicas × currentMetricValue / targetMetricValue )

Worked example: 4 pods averaging 90% CPU utilization (of their requests), target 60%: ceil(4 × 90 / 60) = 6 pods. Later at 30%: ceil(6 × 30 / 60) = 3 pods, but scale-down is deliberately slow: the default stabilization window keeps the highest recommendation from the last 5 minutes to avoid flapping. Changes within a 10% tolerance of the target are ignored.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: { name: order-service }
spec:
  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: order-service }
  minReplicas: 4
  maxReplicas: 40
  metrics:
    - type: Resource
      resource:
        name: cpu
        target: { type: Utilization, averageUtilization: 60 }
  behavior:
    scaleUp:
      policies: [{ type: Percent, value: 100, periodSeconds: 30 }]   # can double every 30s
    scaleDown:
      stabilizationWindowSeconds: 300

Autoscaling lessons for spiky Indian traffic

• HPA is reactive. Metrics lag, pods take time to pull images and warm up, and new nodes from the cluster autoscaler take minutes. For known spikes (IPL toss, Big Billion Days at midnight, Tatkal at 10 AM), pre-scale with a scheduled bump to minReplicas.
• CPU is a poor signal for I/O-bound services. Scale on requests per second or queue depth via custom/external metrics (e.g. KEDA for Kafka lag).
• Scaling pods does not scale your database. 40 pods × 20 connections = 800 connections; make sure the database and connection poolers can take it.
• Vertical Pod Autoscaler adjusts requests instead of replica count; do not let VPA and HPA both act on CPU for the same workload.

Graceful Shutdown and Zero-Downtime Deploys

When a pod is deleted (rollout, scale-down, node drain), two things happen in parallel: the kubelet starts the shutdown sequence, and the endpoints controller removes the pod from Services, which kube-proxies and ingress controllers learn about a moment later. If your app exits instantly, requests still routed to it fail.

1. Pod marked Terminating. Removal from endpoints begins propagating.
2. preStop hook runs (e.g. sleep 10) so load balancers stop sending new traffic before the app stops accepting.
3. SIGTERM sent to PID 1. The app stops accepting new connections, finishes in-flight requests, commits Kafka offsets, closes pools.
4. After terminationGracePeriodSeconds (default 30, counting from the start including preStop), SIGKILL.

import signal, threading

stop = threading.Event()

def on_sigterm(signum, frame):
    stop.set()                      # stop taking new work

signal.signal(signal.SIGTERM, on_sigterm)

def run_worker(consumer, handle):
    while not stop.is_set():
        batch = consumer.poll(timeout_ms=500)
        for record in batch:
            handle(record)
        consumer.commit()           # commit what we finished
    consumer.close()                # leave the group cleanly, triggers rebalance

Spring Boot supports this with server.shutdown=graceful; uvicorn and gunicorn handle SIGTERM by draining. Pair graceful shutdown with retries and idempotency on callers; see Resilience Patterns.

Debugging Pods: A Field Guide

Other indispensable commands: kubectl get events --sort-by=.lastTimestamp, kubectl exec -it POD -- sh, kubectl debug for ephemeral debug containers in distroless images, and kubectl top pods with metrics-server. Logs and metrics only get you so far across many services; that is where observability with traces comes in.

What Interviews Ask About Docker and Kubernetes

For backend SDE roles, expect conceptual questions plus "how would you deploy this service?" inside system design. A longer question bank lives in Docker & Kubernetes Interview Questions; these are the ones that separate prepared candidates:

• "How is a container isolated?" Namespaces for visibility, cgroups for resource limits, shared kernel. Bonus: seccomp, capabilities, user namespaces.
• "Your image is 1.2 GB. Fix it." Multi-stage, slim/distroless base, .dockerignore, combine and clean package-manager layers.
• "How do you deploy without downtime?" Rolling update with maxUnavailable: 0, readiness probes, preStop delay, SIGTERM handling, backward-compatible DB migrations (expand then contract).
• "Deployment vs StatefulSet?" Identity and storage, ordered rollout.
• "How does service discovery work?" Service ClusterIP plus CoreDNS; endpoints track ready pods.
• "How do you manage config and secrets?" ConfigMaps, Secrets (base64 is not encryption; enable etcd encryption at rest or use an external secret manager such as Vault or a cloud secrets manager).
• "Blue-green vs canary?" Two full environments with a switch vs gradually shifting a percentage of traffic while watching error rates; canaries need traffic splitting from an ingress or mesh.
• "Should we move our monolith to Kubernetes and microservices?" Kubernetes runs monoliths fine; decomposition is a separate decision. See Microservices vs Monolith.

Frequently Asked Questions

Pranjal Jain

Ex-Microsoft SDE · IIT Kanpur · Founder of Prepflix. Helps engineers crack startup and product company interviews across India.