The Four Agreements of Incident Response

PEACETIME WARTIME @mattstratton Once an incident is triggered, we need to switch our mode of thinking. We need a mentality shift. We want a distinction between “normal operations” and “there’s an incident in progress”. We need to switch decision making from peacetime to wartime. From day-today operations, to defending the business. Something that would be considered completely unacceptable during normal operations, such as deploying code without running any tests, might be perfectly acceptable during a major incident when you need to restore service quickly. The way you operate, your role hierarchy, and the level of risk you’re willing to take will all change as we make this shift. “Fire isn’t an emergency to the fire department. You expect a rapid response from a group of professionals, skilled in the art of solving whatever issues you are having.” [Quote from Blackrock 3] - http://www.blackrock3.com/blog/incident-management-meets-it-operations

National Incident Management System (NIMS) Coordinated Incident Management System (CIMS) Australasian Inter-Service Incident Management System (AIIMS) Gold-Silver-Bronze Command Structure (GSB) Incident Command System (ICS) … and many other similar systems used in around the world. @mattstratton It’s worth noting that even though our process is based on the US systems, NIMS and ICS, there are many similar systems in use all over the world. While many are also based on ICS, some were developed separately, yet offer many of the same features. I particularly like the UK system, simply because it has a role called the “Gold Commander”, which just sounds like a Bond villain. When developing our process at PagerDuty, we looked at a few of the other systems in use around the world, and chose the bits we liked the most to add to our own system. There’s also a book available from the US FEMA website, called “Comparative Emergency Management: Understanding Disaster Policies, Organizations, and Initiatives from Around the World” if you’re interested in learning more about the systems in use. It compares the systems used by about 30 different countries.

@mattstratton It’s OK to panic on the inside. We’re only human after all. It’s a natural reaction to panic in these sorts of situations a little bit. Everything about getting paged is designed to get adrenaline flowing. Loud pager sounds and so on. Just don’t ever outwardly show panic, because it will cause others to do the same. Act calm, and others will follow suit. We trained with some ex-firefighters to learn about incident response, and something they mentioned stuck with me. They would often come to a house fire, and the owner would be historical “Oh my god, you have to help…”, etc. It’s understandable of course, but they were actually hindering their progress. The firefighters would have to tell them, “This might be your first fire, it’s not ours”. Fire isn’t an emergency to the fire department, it’s routine. Those with experience will stay calm, and that can make the difference between a chaotic incident, and one that resolves smoothly. So don’t panic.