Observability and incident response

Observability is what separates services you can run in production from services you can only run locally. The three pillars are logs, metrics, and traces. Each answers a different question. Logs tell you what happened. Metrics tell you how much and how fast. Traces tell you how one request flowed through the system.

The golden signals are the dashboard pulse you watch: latency (p50, p95, p99), traffic (requests per second), errors (error rate), and saturation (how full your queues and pools are). Track all of them, set alerts on the ones that matter, and most incidents get caught before users notice.

After every real incident, write a blameless postmortem. What happened, when, why, what made it worse, what made it better, what you will change. No names, no fingers. The point is not to punish anyone. The point is to stop the same incident from happening twice.