The Facebook Outage — When BGP Goes Wrong, Everything Goes Dark
·993 words·5 mins
Facebook, WhatsApp, and Instagram went down for six hours due to a BGP misconfiguration, exposing how fragile the internet’s routing infrastructure really is.
- Osmond van Hemert — Senior Software Engineer/
- Blog Series: In-Depth Tech Coverage on AI, Security & Cloud/
- Cloud Operations/
Cloud Operations
Overview#
Building cloud infrastructure is one thing; operating it in production is another. This series covers cloud operations—the practices, tools, and strategies that keep systems reliable, cost-effective, and resilient. Topics include observability and monitoring, cost optimization at scale, incident response patterns, and the operational culture that separates well-run systems from perpetual firefighting.
Whether you’re responsible for reliability, reducing cloud costs, or leading incident response, these insights apply directly to your work.
What You’ll Find Here#
Observability & Monitoring: Building effective monitoring systems, understanding metrics, logs, and traces. Knowing what to watch tells you when to care.
Cost Optimization: Understanding cloud billing, rightsizing, reserved instances, and spot instances. Reducing costs without sacrificing reliability requires strategy.
Incident Response: Preparing for failures, detecting issues early, response playbooks, and learning from incidents without blame.
Infrastructure as Code: Declarative infrastructure, drift detection, configuration management, and making infrastructure auditable and reproducible.
Capacity Planning: Predicting growth, autoscaling strategies, and ensuring infrastructure scales smoothly with demand.
Disaster Recovery: Backup strategies, failover mechanisms, multi-region concerns, and recovery time objectives.
Learning Path#
- Master observability fundamentals — understand what metrics and logs actually tell you
- Implement comprehensive monitoring — dashboards, alerting, and actionable signals
- Learn to optimize costs — understand billing, identify waste, and right-size resources
- Build incident response muscle — playbooks, on-call rotations, and blameless postmortems
- Plan for growth — capacity planning, autoscaling, and handling traffic spikes
Key Topics Covered#
- Monitoring & Observability: Prometheus, Datadog, New Relic, logs, metrics, traces, and SLOs
- Cloud Cost Management: RI analysis, spot pricing, reserved capacity, workload migration, and FinOps
- Incident Management: PagerDuty, alerting rules, runbooks, postmortem processes, and on-call culture
- Infrastructure as Code: Terraform, CloudFormation, Pulumi, Ansible, and drift detection
- Autoscaling & Performance: Load balancing, horizontal scaling, vertical scaling, and performance testing
- Disaster Recovery: Backup strategies, RTO/RPO targets, multi-region failover, and testing DR
Related Series#
Explore complementary areas: Cloud Platform Watch (new AWS/Azure/GCP features and pricing), Kubernetes & Containers (container orchestration operations)
Terraform 1.0 — Infrastructure as Code Reaches a Milestone
·1064 words·5 mins
After years of 0.x releases, Terraform hits 1.0 with stability guarantees. What this means for the IaC ecosystem and your existing workflows.
The Fastly Outage — A Masterclass in Single Points of Failure
·990 words·5 mins
When a single configuration change at Fastly took down half the internet, it exposed uncomfortable truths about how we build on CDN infrastructure.
GitOps Goes Mainstream — ArgoCD, Flux, and the CNCF Bet
·980 words·5 mins
With ArgoCD accepted into CNCF incubation and Flux reaching its own milestones, GitOps is transitioning from buzzword to standard practice for Kubernetes deployments.
Microsoft Build 2021 — The Developer Platform Play Deepens
·963 words·5 mins
Microsoft Build 2021 doubled down on the developer platform strategy with Azure improvements, deeper GitHub integration, and a clearer vision for the cloud-native developer workflow.
OVHcloud Strasbourg Fire — When 'The Cloud' Literally Burns Down
·907 words·5 mins
A catastrophic fire at OVHcloud’s Strasbourg datacenter destroys thousands of servers and raises hard questions about cloud resilience and backup strategies.
When the Grid Goes Down — Cloud Resilience Lessons from the Texas Power Crisis
·1041 words·5 mins
The Texas power grid failure is knocking out data centers and cloud services, offering hard lessons about infrastructure resilience, multi-region architecture, and the physical realities underlying our digital systems.
HashiCorp Launches Waypoint and Boundary — Closing the Developer Experience Gap
·916 words·5 mins
HashiCorp announced two new open-source tools at HashiConf Digital — Waypoint for application deployment and Boundary for secure remote access. Here’s why they matter.
Terraform 0.13 — Module-Level For Each and the Provider Story
·961 words·5 mins
Terraform 0.13 brings count and for_each to modules, automatic provider installation, and custom validation rules. A look at what changes in practice.
Redis 6.0 in Production — ACLs, Threading, and What Actually Matters
·945 words·5 mins
Redis 6.0 brings ACLs and I/O threading to the world’s most popular in-memory data store. Here’s what the changes mean in practice.
Your CI/CD Pipeline Is Your New Attack Surface — And Remote Work Made It Worse
·980 words·5 mins
As teams rushed to enable remote development workflows, CI/CD pipelines became a prime target. Here’s what’s going wrong and how to harden your build infrastructure.
Infrastructure as Code Under Pressure — Lessons from Pandemic-Scale Scaling
·1116 words·6 mins
The sudden shift to remote work has stress-tested Infrastructure as Code practices at unprecedented scale. Here’s what’s working, what’s breaking, and what we should learn.