Containers vs Virtual Machines

Virtual Machine (VM): Emulates hardware. Runs a full Guest OS on top of a Hypervisor. Heavy (GBs). Container: Shares the Host OS Kernel. Lightweight (MBs). Fast startup (ms).

Containers are process isolation mechanisms provided by the Linux Kernel.

The Three Pillars of Containers

How does docker run actually work? It uses three Linux features:

1. Namespaces (Isolation)

Namespaces limit what a process can see.

Try it: unshare --fork --pid --mount-proc /bin/bash creates a new PID namespace.

2. Control Groups (Cgroups) (Resource Limiting)

Cgroups limit how much a process can use.

• Memory: "You get 512MB RAM." (OOM Kill if exceeded)
• CPU: "You get 0.5 CPU cores."
• Block I/O: "Limit disk read speed to 10MB/s."

Accessed via file system: /sys/fs/cgroup/.

3. Union Filesystem (Storage)

How are images so efficient?

Docker Images are built from Layers.

• Layers: Read-only changesets (e.g., Ubuntu Base -> Add Python -> Add App Code).
• Union Mount: Merges all layers into a single view.
• Copy-on-Write (COW): If a container modifies a file from a lower layer, it copies it up to the writable top layer first.

OverlayFS is the modern implementation.

Docker Architecture

Client-Server model.

1. Docker CLI: Sends commands (docker run) to Daemon via REST API (Unix Socket).
2. Docker Daemon (dockerd): Manages objects (images, containers, networks).
3. Containerd: High-level runtime. Manages image pull/push and lifecycle.
4. Runc: Low-level runtime. Spawns the actual process using syscalls.

Building Efficient Images

1. Multi-Stage Builds

Build in a fat image (Go compiler), copy binary to tiny image (Alpine/Scratch).

# Stage 1: Build
FROM golang:1.21 AS builder
WORKDIR /app
COPY . .
RUN go build -o myapp main.go

# Stage 2: Run
FROM alpine:latest
WORKDIR /root/
COPY --from=builder /app/myapp .
CMD ["./myapp"]
# Result: 10MB image (instead of 800MB)

2. Layer Caching

Order matters! Put frequently changing instructions (COPY code) at the bottom.

# BAD: Breaks cache if source code changes
COPY . .
RUN npm install

# GOOD: Caches dependencies unless package.json changes
COPY package.json .
RUN npm install
COPY . .

Security: Container Breakout

If a container runs as root, and a vulnerability exists in the Kernel... Container Breakout: Attacker escapes the container and gains access to the Host.

Defenses:

• Run as Non-Root: USER appuser in Dockerfile.
• Read-Only Root: docker run --read-only.
• Capabilities: Drop unused Linux privileges (--cap-drop ALL --cap-add NET_BIND_SERVICE).

Containers vs. VMs: Where the Line Actually Is

"Containers are lightweight VMs" is the most common — and most wrong — mental model. The difference is architectural:

A container is just a normal Linux process wearing three disguises: namespaces (it sees its own filesystem, network, and PIDs), cgroups (it's limited in CPU/memory/IO), and a union filesystem (its disk is stacked image layers plus a writable top). Run ps aux on a Docker host and every containerized process is right there in the list.

This is also why the security note above matters so much: all containers on a host share one kernel. For hostile multi-tenancy (running customers' arbitrary code), the industry adds a stronger boundary back — gVisor (userspace kernel proxy) or Firecracker microVMs (AWS Lambda's approach: real VM isolation with ~125ms boot times) — landing deliberately between the two columns of the table.

What Actually Happens on docker run

Demystifying the flow end-to-end:

1. CLI → daemon: docker run nginx is an API call to dockerd.
2. Image resolution: missing layers are pulled from the registry, verified by digest, and stored content-addressed (a layer shared by 50 images exists on disk once).
3. Filesystem assembly: overlayfs mounts the image's read-only layers with a fresh writable layer on top.
4. Delegation: dockerd hands the spec to containerd, which invokes runc — the small OCI runtime that actually creates the namespaces and cgroups, then execs your entrypoint as PID 1 inside them. runc exits; a tiny shim stays as the process's parent.

The layering explains real operational facts: Kubernetes talks to containerd directly (Docker the daemon is unnecessary in production clusters), a dockerd restart doesn't kill running containers (the shims own them), and "distroless" images work because a container needs no OS — just your binary and its libraries, since the kernel comes from the host.

One production gotcha worth internalizing: your entrypoint runs as PID 1, and PID 1 has special signal semantics — default handlers don't apply. Naive entrypoints ignore SIGTERM, so docker stop waits 10 seconds then SIGKILLs them — that's where mysterious slow shutdowns and dropped in-flight requests come from. Use a tiny init (--init/tini) or handle signals explicitly.

Related Concepts

• Kubernetes Architecture (Orchestrates containers)
• CI/CD Pipelines (Builds images)
• Microservices (Deployed in containers)
• Process vs Thread (The primitives underneath)
• Serverless Architecture (Firecracker microVMs in action)