Field guide
New here and not sure where to start? This is your map: how to approach a broken system, the tools you'll reach for, and the docs worth bookmarking — organised around what the scenarios actually teach.
The debugging loop
- 1Read the signals — then distrust them
The ticket is vague, and sometimes wrong, on purpose — that's how real tickets arrive. Note what's claimed versus what you can actually verify.
- 2Observe the real state before theorising
Don't guess first; look. What's running, restarting, pending? Statuses and restart counts tell you where to point your attention.
- 3Ask: what would have to be true?
For this exact symptom, what could cause it? Form two or three hypotheses before you change anything.
- 4Follow the system's own breadcrumbs
The answer is usually already printed somewhere — in logs, in describe output, in events. Read them before you theorise further.
- 5Change one thing, then re-verify
One variable at a time. Re-deploy your edit and run bin/sim verify. If it didn't move the needle, undo it before trying the next idea.
- 6Solved? Read the postmortem
The senior writeup — and especially the 'class of bug' it belongs to — is the real lesson. The fix is the smallest part.
Your toolkit
A container packages an app with everything it needs to run. Compose runs a multi-container app from a single YAML file. The 0xx scenarios live here.
Kubernetes runs and heals containers across a cluster. kubectl is how you talk to it; kind spins up a throwaway cluster on your laptop in ~30s. The 1xx/2xx scenarios live here.
First moves when something's broken
# docker / compose $ docker compose ps # what's up / restarting? $ docker compose logs -f <service> # why did it die? $ docker inspect <container> # env, mounts, config # kubernetes — your first three moves $ kubectl get pods -n <ns> # status, restarts, age $ kubectl describe pod <pod> -n <ns> # events at the bottom = gold $ kubectl logs <pod> -n <ns> # add -p (previous crash), -c (container) $kubectl get events -n <ns> --sort-by=.lastTimestamp
Full reference: Debugging running pods ↗.
What the scenarios teach
Every problem falls into one of four buckets. Here's what each covers, which scenarios drill it, and where to read up.
Lifecycle
It isn't running — or won't stay running.
- ▸Reading startup errors and restart/crash loops
- ▸Environment variables & Secrets
- ▸Init containers and pod phases
- ▸Liveness / readiness / startup probes
- ▸Rollouts, rollback, and desired-vs-live state
- ▸Scheduling — why a pod sits in Pending
- ▸Config changes that don't take effect
Networking
It's running, but can't reach its dependencies.
- ▸Service discovery: how one service finds another
- ▸Cluster DNS and service naming
- ▸Selectors, labels, and Service endpoints
- ▸Silent dependency failures (it 'works' but nothing connects)
State
It runs and connects, but loses your data.
- ▸Ephemeral vs. persistent storage
- ▸Docker volumes; restart vs. recreate
- ▸PersistentVolumeClaim ↔ PersistentVolume binding
- ▸StorageClasses, access modes & binding modes
- ▸StatefulSets and volumeClaimTemplates
Efficiency
It works, but wastes time, memory, or money.
- ▸Image size: multi-stage builds, base images, layer caching
- ▸Resource requests & limits; OOMKilled; QoS classes
- ▸Picking the right workload type (a scheduled Job vs. an always-on pod)