New ask Hacker News story: Ask HN: What are the most interesting or worst outages you have seen?
Ask HN: What are the most interesting or worst outages you have seen?
2 by ozten | 0 comments on Hacker News.
Inspired by comments in the Klarna incident, please share your most interesting / worst outage stories from your past. I'll go first: We had a project to re-write the cart UI of a large e-commerce platform. The project was later cancelled. I requested to de-provision all the new project hardware, but accidentally used the backend infrastructure name due to muscle memory and being tired. Infra techs didn't double check that these hosts didn't take traffic and I accidentally removed all capacity for our shopping cart fleets in all regions :| It took about 45 minutes to recover, I wanted to crawl into a hole and disappear.
2 by ozten | 0 comments on Hacker News.
Inspired by comments in the Klarna incident, please share your most interesting / worst outage stories from your past. I'll go first: We had a project to re-write the cart UI of a large e-commerce platform. The project was later cancelled. I requested to de-provision all the new project hardware, but accidentally used the backend infrastructure name due to muscle memory and being tired. Infra techs didn't double check that these hosts didn't take traffic and I accidentally removed all capacity for our shopping cart fleets in all regions :| It took about 45 minutes to recover, I wanted to crawl into a hole and disappear.
Comments
Post a Comment