Agile? Or FrAgile?

(I found this piece deep in the vaults at (TL)2 Towers, so I figured I would break my non-blogging streak.)

Sensitive client code has been discovered on a GitHub repository, and it looks like one of our developers put it there. The client is upset, and their Chief Information Security Officer (CISO) wants to meet with you in New York to discuss what happened.”

Not the best email start to a Tuesday morning, I grant you, but I did at least have two things in my favour; six days before the meeting with the client and an excellent Cyber Incident Response Team (CIRT) headed up by a very talented and focussed individual based out of Miami. So I felt confident we could get to the bottom of the incident.

Fast forward to the following Monday afternoon (don’t worry, this isn’t a blog about cyber forensic investigations), and I am walking out to the cab back to the airport with a thankful, happy client and the promise of more billable work in the form of security testing.

It sounds like the beginning and end of two different stories, but this is a real example from back in 2016 during my tenure as a CISO. What initially felt like a nightmare scenario was quickly turned around, and three things became apparent very quickly:

  1. Proper incident management is crucial because it is not a case of “if we have a data breach” nowadays but “when we have a data breach”.
  2. Agile working methods are a double-edged sword; we get work done quickly and effectively, but if we are not careful, boy, can it bite us in the ass.
  3. Focusing on the client, their perceptions, need for honesty, demands, and wishes are paramount.

I want to focus on 2 and 3 from above because let’s face it, incident management is often carried out by the cool-looking security folks who swoop in when it all goes wrong.

Is it Agile or FrAgile?

Agile working methodologies are, I am told, wonderful ways to get work done quickly and effectively. Flexible working, the heavy reliance on technologies, still often in early release stages and updated daily. You can reuse and share code through various repositories, test things out with the rallying cry of “Fail fast! Fail often!”, pull out your credit card and spin up servers on the other side of the world by the time the kettle has boiled or the coffee machine dripped its last drop.

The problem with this is that if we are not careful, the environment becomes the wild west, with code flying around various platforms containing hardcoded admin credentials and schematics shared openly as you struggle to squash that last bug with the help of the internet. (I am having palpitations just thinking about it.)

Sticking code up onto GitHub doesn’t make you agile; it makes you fragile. You must prepare and plan, work as a cohesive unit with your team and have established (and secure) ways of working with these new and funky tools that are not only effective but also in the best interests of the client.

The client is King AND Queen.

I recall another incident where a client contacted us to say they had found their code and credentials on a GitHub-type site. They were (unsurprisingly) very upset and demanded our response. Upon investigation, it turned out the data in question was uploaded there some two years earlier, was out of date, and the credentials were changed long ago anyway. It was severe, but we were off the hook with confirmation that third parties downloaded none of the data!

Famous last words. The client was livid.

The incident escalated to their board, and we were bought to task for the security lapse in no uncertain terms. However, we had forgotten one crucial thing in all of this; it wasn’t our data to lose in the first place. 

There is a saying in information security circles; “my risk model is not your risk model”. It means that what risks affect you are not the same as those that affect me. Precisely the mistake we made with this client. It didn’t matter that nothing had happened; it didn’t matter that the data and credentials were stale; we were sloppy and slapdash with their intellectual property. We had broken their trust.

In the middle of the high-pressure project environment, losing sight of what the client holds dear versus what you are trying to achieve in the here and now is easy.

Making the Leap

Do you need to address your agile working practises and understanding of your client’s motivations and risks? Here are two data points to consider:

  1. A constant increase in the inclusion of unlimited liability clauses in your contracts. Right or wrong, they aren’t going away, and as you access more of a client’s sensitive and confidential data to “make digital work” for them, it will continue to do so.
  2. When it happens, the cost of a data breach is increasing. According to the Ponemon Institute/IBM Security report “2019 Cost of a Data Breach Report – A comprehensive analysis of data breaches reported in 2018,” the average cost of data breaches has increased by 12%, a whopping $3.92M in the healthcare industry alone. You won’t get much change out of that for a satisfactory bonus pool.

When you leave your house in the morning, you lock your door, maybe enable an alarm system. When you leave your car, park it in a reputable garage and lock the door, immobilising the engine and setting the alarm.

Then you go to work and copy someone else’s confidential data and credentials to an open file share. Can you see the paradox?

Taking that leap in maintaining client confidence, assuring them of intelligent decision-making, agile (not fragile) working practises and mental rigour to maintaining their crown jewels is challenging. But it is essential.

Why did that first client give me more business after a breach? Demonstrating that we were in control of our processes and ensuring he and his risks were obviously the centre of our world helped: that and a high degree of transparency. But that initial breach was too high a price for extra business and a warm handshake.


All Fun & Games

Business Continuity Plans; probably the most important, yet undervalued and underfunded, part of your security team. This is the team that deals with what might happen to kill you tomorrow, versus what is actually killing us today. A justifiable investment is very hard to make, because they prove their worth when nothing happens; much like the rest of security, but that nothing is going to happen at some unspecified time in the future.

And then something happens, and the leadership are baying for your blood, crying “why didn’t we do something about this before?”. After an initial flurry of investment and interest, it dies down again to pre-crisis levels, and trhe sequence continues.

Maintaining that level of interest is very difficult in virtually any modern business because of the common demands on any listed company; quarterly earnings reports that continually drive down general and administration costs (you are an overhead there, Mr Security), and lurching from one poor investment briefing to another mean there is little room for “what if” investment.

So let’s play some games instead. If they won’t take its seriously, then neither will we. (That’s supposed to be sardonic, by the way.)

How to test your plans!

Doing tabletop exercises and practising the the plans you have in place is a great way of gaining interest in what it is you are doing, but can be very challenging g to start. The people you are targeting are, after all, the most senior and time poor people in the company. So, let’s start small.

Start with a team within your sphere of influence that has a role to play; maybe the SOC team, and include if you can the departments of peers, such as Legal or Communications. Run a scenario over an hour, record it, document it, create a transcript if need be, and share that report as widely as possible. Make sure you clearly record somewhere that you carried out the test as well, it’s useful fro compliance reasons.

Then rinse and repeat, and each time rely ion the success of the most recent exercise to build the scale and seniority of the exercise. It always surprises me frankly, ho much senior executive try and avoid the exercises, but thoroughly enjoy them when they finally submit to one. it is like they finally see the real world impact of what it is they are doing and the influence they can leverage during times of crisis. I could theorise about the egotistical nature of the phenomenon, but i will leave that to the psychologists and other trick-cyclists.

As the scale of the tests get larger, consider not only running them over longer periods of time and bringing in third parties to manages. This helps in two ways:

  1. You get to be directly involved in the exercise without knowing all the “answers”.
  2. They can bring a level of expertise you won’t have had, as well as tools and bespoke environments to practise with.

These can be run over extended periods, normally no more than a day, but can go beyond if supported. Four hours is a good place to start, with a working lunch in the middle (it helps attract people; everyone loves a free lunch). These third parties may be able to bring additional technology such as a dedicated virtual environment that includes a physically separate network, dedicated laptops, tablets and phones, that ensure the environment is carefully tracked and recorded, and no real world disruptions are encountered. Finally, they can also add real people to interact with, actually phoning the participants, “tweeting” or posting on other social media as part of the exercise, giving an even more realistic feel.

If you want to go extra fancy, you can even run them over multiple geographies, but make sure you can walk before you run!

Given recent circumstances with COVID-19, the lockdown and massive changes to working practises, being able to respond quickly to dramatic changes in the working environment is no longer an exercise in the impossible future, but rather planning on how to operate in a fast moving, ever changing and dangerous environment whilst still maintaining a running and profitable business.

This could be your next tabletop exercise.

That doesn’t sound like a game to me.

Are you trying to get your Business continuity and Crisis Management plans out of the document and into an actual exercise for your business but don’t know how to start? (TL)2 Security can help with everything from your initial plan to a full day exercise. Partnering with industry leading organisations to bring the Situation Room to your business, and ensuring you have real world and actionable improvements and observations at the end of the process, contact (TL)2 Security for more information.


Command, Control, and Conquer

Back in the ’90s, there was a game released called Command and Conquer, a strategic game whereby you had to manage resources, build, train and mobilise armies and conquer the neighbouring armies. It was a classic that spawned many spin-offs, sequels and addons for decades. What struck me about it though was how multi-skilled you had to be, especially in the later levels.

You couldn’t just be an excellent Field Marshall as you also had to manage resources, cash and other materials to create your buildings and structures that allowed you to create your army in the first place. You had to know logistics, how long something would take to build, train and mobilise, look into the future at new locations for better access to materials, and also have plans in place if the enemy attacked before you were ready.

Essentially, you were skipping from one crisis to the next, finely balancing between success and crashing failure. It sounds a lot like any modern-day incident management situation really.

In this week’s The Lost CISO (season 2), I take a quick look at incident management and highlight four key points to remember during an incident. In case you haven’t seen it yet. here it:

The bottom line is that, much like in the Command & Conquer game, you could plan ahead what you were doing because the environment was constantly changing, the unknowns were stubbornly remaining unknowns and the literal (in the case of the game) fog of war meant you can’t see more than just a few steps ahead. There are though some keys to success.

The first key point is that having a plan is all well and good, but as my military friend regularly tell me;

no plan survives contact with the enemy

Why? Because the enemy much like life does random, unexpected and painful things on a regular basis. Incidents have a habit of doing the same thing, so if your plan is rigid, overly explicit and has little room to ad-lib or manoeuvre in, it will fail.

Therefore, my approach has always been to build any kind of plan around four simple areas:

  • Command
  • Control
  • Communication
  • Collaboration

In other words, decide who is in charge, decide who is responsible for what areas, ensure everyone knows how to talk to each other, ensure everyone works openly and honestly with everyone else. There may be some other details in there as well, but really, if you have these four areas covered your plans will remain flexible and effective, and you may find yourself being able to close incidents more quickly and efficiently.

With all that extra time on your hands, you can then spend some time basking under the Tiberian sun.