When It All Goes Pete Tong…

Murphy’s Law states:

“If something can go wrong, it will go wrong”

Many CISOs will also state:

“it is not a case of if you have been breached, but rather that you have, you just don’t know it yet”

Depressing as both statements sound by themselves, put them together, and you enter into a worldview of doom and gloom from which it is hard to crawl. It doesn’t matter what you do; there will always be a breach and multiple mistakes in your team. These factors create a perfect storm for finding a new job relatively quickly.

But there is hope that when you start a new role or join a new company, there is one thing that needs to be in place before anything else; the Incident Management Plan*. In all but the most security mature organisations, any improvements put into place by you will take months and years to bear fruit, during which time a disaster can strike without notice (the unknown unknowns hitting at an unknown time, if you will.) So making sure you have a plan to fall back on at a moment’s notice gives you space and time to respond appropriately while still being able to focus on the more fundamental changes you have in mind for the organisation.

But what to put into these plans? There are a few key points that should always be adhered to whenever writing a response plan;

Keep it Simple

Human beings are emotional sacks of meat and adrenalin when things go wrong. They can simultaneously be forgetful, angry, scared, sad, and even stupid. Therefore your plans, and by association, your writing and grammar, need to be as simple as possible. It’s not an easy task and will require many edits, reviews and rewrites, but simplicity is your friend during a confusing and rapidly changing situation. 

Keep it Flexible

Extending the first point, you also cannot create a prescriptive document. If you define every action based on a specific input, your plan will fail when that particular input isn’t happening. The plan needs to work on the principles of what must occur during an incident rather than the specifics of what needs to be done. It is useful, for instance, to focus on roles and responsibilities rather than activities; in this way, someone is accountable for “public communications”; how they achieve that is up to them, but the plan does not define it.

Know What’s Important

This is another way of saying, “Understand your critical services”. These services could be technology-based, process focussed or even role/person-specific. During an incident, the immediate focus is to get the bare minimum of services/capabilities/business operating again as quickly and safely as possible. Going back to Business As Usual is for later on. You need to know what the bare minimum is to achieve it.

The ISO 22301:2019 – Security & Resilience – Business continuity management systems standard is a great place to start to understand the mechanics of this element in more detail (and great for this topic as a whole).

Collaborate While Creating

It never ceases to amaze me how often plans like this get created in isolation across companies, divisions and departments. What that means, more often than not, is a competition for resources because they all assume they will have exclusive access to the resources required to see them through a crisis just because they have a plan.

Ideally, there should be a single master plan for the organisation that allows each discrete business area to manage their plans (essential in larger organisations). Then, all of these plans and their requirements are fed back into the overarching strategy to carry out capacity planning and coordination more effectively and efficiently.

Multi-channel Sharing and Education

This is the one time I will permit using a few trees to print out your plans. Electronic documents are still valuable and should be saved in different formats and on other devices and platforms (for redundancy, obvs). Having paper copies of the entire document, in addition to aide memoirs, laminated “cheat sheets”, credit card numbers and any other creative approaches to ensuring the needed information is always available. Remember, this is a time of crisis; your laptop may be burning down with your building, and your phone may be out of battery with nowhere to charge. Base your communication and distribution methods on the assumption of Murphy’s Law above.

Test the Plan, Learn and Review

You must test the plan as much as possible, especially when creating it. If you feel brave enough, you can have a tabletop walkthrough or pull the plug on a data centre. Some third-party services allow you to test your plan in a virtual space using specialised communications tools that are even more realistic. Whatever the case, every time you check it, review it and feed the findings back into the plan. Even a slight improvement could make all the difference.

Test the Plan Again

Did I mention testing? Even if you have a real-life crisis, use the learnings and feedback to improve the plan again. Every opportunity to stress the crisis plan, people and procedures must happen.

Test it Again

It must be tested, whatever happens, at least once a year, and reviewed yearly. You will be surprised at how much your business changes over a year; a process may be updated, people and roles change, and telephone numbers and email addresses frequently updated. If your plan doesn’t reflect even these simple changes, it is more likely to fail.

The Holy Trinity Mantra

Finally, if in doubt, remember these three elements of your plan. I like to ensure they are seen through in this order, but you may feel differently according to your business and how it operates. (If people don’t list as number one on your list, take a long, hard look at yourself.) Nonetheless, The Trinity remains the same.

  1. Focus on People – without your people, you have no business to speak of, recovered or otherwise.
  2. Focus on Facilities – even with just a pen, paper, telephone, and somewhere to work, your people can work miracles in keeping the business afloat. Keep them safe, secure and happy.
  3. Focus on Technology – get the systems running to take the strain off the people. This may have taken days or weeks, depending on the incident. Ensure your critical systems are running first, and that includes payroll. Paid people pull together in a crisis. Unpaid people don’t.

Hopefully, you will never have to use the plan, but if you do, feeling prepared for anything is a powerful way to ensure your best work on everything else on your list. Knowing that you have it ready to go is like remembering to take your umbrella with you when you leave the house. Because you have it, it isn’t going to rain; mildly annoying but so much better than getting caught in a monsoon in your best work attire.

*Also known as the Crisis Management Plan, Business Continuity Plan, When It Hits The Fan Plan, or any other variable that works for you, your company, and your business culture.

Links to other interesting stuff on the web (affiliate links)

How to Upskill Your Cybersecurity Team

The AWS Security Cheat Sheet

Think Before You Share The Link