Risk Appetite – managing feast and famine

Leave a comment August 2, 2013 Thom Langford

I was able to attend the RANT forum a few nights ago, and watch an excellent presentation by Sarb Sembhi. However, and this is no insult to the speakers at the RANT forums (being one myself) the most valuable part of the evening is the socialising with colleagues and peers before and after.

I was talking to a couple of people who were recounting the challenges they face with their leadership regarding their risk management activities. I paraphrase greatly, but the gist of the issue was

Highlighting risks to them is all well and good, but then suddenly they tell us that another activity needs to be escalated up the risk matrix, or that there is a hot topic that they want pushed to the top of the risks list so it gets more attention. How are we supposed to manage a risk programme with any credibility when risks get artificially prioritised or de prioritised according to the mood of management?

We came to the conclusion that the risk appetite of the management team in question was a very flexible and fluid thing that changed quite frequently, and seemed entirely disconnected from the risk management activities being carried out.

This is a complex issue, and not one that can be solved in a single blog post, but there are a few guidelines and concepts that may be pertinent to heading off this kind of behaviour.

Listen to them. On the whole an organisations management know what activities and changes will affect the business more than you. If they are highlighting something it is not to mess you around but because they are genuinely concerned about it. Look at your risk programme; does it squarely address the risks they are highlighting? Are they new risks, old risks, or poorly understood risks? Perhaps you have already found them and they need to be reviewed under the new light cast on it by management.
Educate them. How much does your management team actually understand about the risk work you are doing? Do they really know what the scope of your remit is, how you go about finding risks, and more importantly how you measure them? ISO27005 is often described as an arbitary way of measuring risk, but it does a good job of explaining how you can approach and understand it. If you use that standard in your programme, make sure they understand how you measure them, and get their buy in to the approach. This way, when you disagree with their analysis of a “new” risk you can explain in agreed terms why.
Use your governance structure. Your management team should only be looking at risks that are escalated to them, that is to say residual risks that are still considered as “high” (or whatever parlance you use). Every other risk below that should be managed and dealt with by the governance structure in place. Certain lower risks can be mitigated (managed, avoided or transferred) by people closer to that risk; a developer could change a portion of code, a project manager could remove or add contractors or a team member could go through more awareness training. Changing the course of a project or increasing the staffing costs by 50% is beyond their remit and they are therefore not able (or authorised) to treat them effectively; these risks get passed up your governance chain until they reach a point at which they can be dealt with. At the very top I would estimate they should be seeing no more than 0.1% of total risks escalated to them. Any more and it may be that the structure underneath is not doing their job.
Understand their appetite. One of the standard ISO 27005 risk acceptance approaches provides a matrices for what is acceptable and what isn’t. It is provided as an example only, and should not be used out of the box without considering the risk appetite of your organisation. If you are a risk averse organisation, the yellow and red band move down to the lower left, thereby meaning more “red” risks will need to be addressed. A risk taking organisation will move the green and yellow band up, thereby ensuring fewer “red” risks will need to be addressed. The risk profile of an organisation is something that is rarely understood by those that measure risk, and therein lies the problem. Only if the risk profile is drawn up, understood (including the approach to measure the risks in the first place) and signed off can risks be identified, “measured” and addressed in a way that meets the organisations business objectives.
Accept that the appetite changes. if you review your risks annually (as a bare minimum) that is also a cue to review the risk appetite. If incidents throughout the year affect the business for the good or bad, that is a cue to review the risk appetite. If the organisation management suddenly think something is a big risk and needs to be addressed, that is a cue to review the risk appetite. And when I say review, I mean with the management, and not just in isolation.

There… simple! Well, not at all when you face these challenges every day, but if you can start that dialogue with your management and start to understand the business as they understand it you will be a long way towards heading off the “the sky is falling, fix it now!” response to risks.

The EU, Porn, and Hollywood

1 Comment May 18, 2013 Thom Langford

And if that title doesn’t attract attention I don’t know what will…

Unfortunately (for you) while this title is accurate the rest of this post may not quite deliver what you are expecting or hoping for. Just a few days ago (Thursday 16th May) I attended for the first time an ISSA-UK chapter meeting in Bristol where Marcus Alldrick, Richard Hollis and myself were presenting (in that order) to the great and the good of the south west infosec community.

Marcus Alldrick emphasises…

Marcus’ presentation of The EU’s Proposed Data Protection Regulation, It’s Life Jim But Not As We Know It was very well received with a huge amount of interaction to the point of a twenty minute overrun. I have tended to avoid expending too much energy on draft legislation like this as it often changes dramatically the closer it gets to publication (MA201 CMR 17 is a good example of this), and so the view that Marcus presented was a welcome one. Although his deck was content rich he put it across in his own inimitable style and I found it hugely educational. One point that came across loud and clear is that if it gets enacted in its current format one of the most sought after roles in any company will be that of Chief Privacy Officer for the job security alone (the role must be filled by the same person for a minimum of two years!).

…and Richard Hollis boasts

Second up was Richard Hollis with his hotly anticipated Deep Threat – Top 10 Lessons to Learn from the Online Adult Entertainment Industry. While the expected jokes and euphemisms came thick and fast underneath it were some startling and very interesting lessons, but namely that the adult entertainment industry simply does information security far better than the rest of us; they are single minded, have a lot to lose, and ultimately see the “battle” with maintaining security as just that… it’s a war which they are determined to win. A fascinating insight into an often overlooked industry with some great lessons summarising the underlying security ethos of this industry.

I’m a little teapot

Finally it was my turn. To be honest I was somewhat apprehensive following these two presentations; there was a huge amount of interaction to this point and while my presentations somewhat relied on audience participation the main points I was raising were quite high level and in some cases not often talked about. I shouldn’t have worried. I had an absolute blast talking about different elements of risk management and getting some excellent feedback, comments, questions and of course different opinions. My case was obviously helped by the fact that I was handing out prizes for each correct answer identifying a quote to a film! The presentation itself is below along with a few snippets of the presentation itself taken from the back of the room.

I have always been impressed with the ISSA-UK meetings, the quality of the discussion between people and to be honest the great value that membership of this association brings. I am very much looking forward to more of these, and if asked to present again at one of their sessions. My thanks to Alan and Gabe (@infoseccrow) for giving me the opportunity to present here.

UFOs Dirty Dancing and Exploding Helicopters (PDF)

Use Your Nose and Gut to See The Real Picture

1 Comment May 3, 2013 Thom Langford

After the high energy of the conferences last week it was always going to be a challenge coming back to the humdrum of day to day work. Reviewing someone else’s audit findings was never going to be the quickest way to get those energy levels up!

This was compounded somewhat by what I found myself reading of course; this was a audit report on an environment that had a very limited scope, i.e. type of work being carried out, type of data being handled, type of resources required to complete the task. The auditors however were coming in from a very strictly controlled, somewhat binary view of the world. The upshot of this was that there were a lot of findings along the lines of:

Workstations have access to the internet.
Physically secured environment within the office (of the same company) required.
Firewall must separate development environment from the rest of the office.

On the face of it these findings are perfectly acceptable, but what they don’t do is take into account the bigger picture.

The group that was being audited did not have access to any sensitive information, PI or even intellectual property. They required access to the internet as they were a creative group that uses multiple types of resources from the web, and they were already on a secured VLAN.

Unfortunately they failed to understand what was in front of their faces throughout the entire audit and assessment process (in fact, they remind me of the type of auditor that Javvad recently showed us in his latest video) They didn’t observe their surroundings fully, understand the working environment, nor comprehend the true purpose of the audit, namely to reduce risk not squash the life out of some very expensive resources and make it difficult to do their job.

They did everything by the book.

There is always a time and a place for a slightly more maverick approach in my opinion. There are times when as an auditor you need to go with what your nose tells you is bad, or your gut tells you isn’t right. No kind of by-the-book approach will let this happen. Let’s elaborate on these two approaches a little more:

Using your nose

This is quite literally “smelling” out the findings. Just because a document has been presented and all seems in order, or just because an activity is shown to be in normal use doesn’t always mean everything is in order. I have spent many enjoyable hours discussing with colleagues the tricks and traps that people use to fool auditors and assessors (some of the simpler ones are in Javvad’s video!). I even heard one where freshly printed documents were deliberately given coffee stains to give the impression that they had been around for some time, or people being sent home for the day when the auditor was around. Smelling this out requires a slightly cynical nature and a “poacher-turned-gamekeeper” approach. You might see a name occur too often, or the same approval date on documents that were obviously written at different times and approved by different approvers, but they are all indicators that something may be amiss.

Using your gut

A “gut feeling” is a very difficult thing to define, and to be honest not always as reliable. i often think it is because you have observed something subconsciously that make it a gut feeling. Using your nose is based upon an observable phenomenon whereas using your gut is not. They can be very good indicators that something is not quite right and deserve to be investigated further; the real skill however is knowing when to stop. Burning up half of your audit time because of a gut feeling is unprofessional, a waste of time and is doing both you and the auditees a huge disservice. However it can pay off huge dividends when you get it right in what is uncovered.

I want to caveat the above however; I don’t want to come across as though auditing is some kind of cat and mouse arms race (or any other kind of mixed metaphor). Any good audit or assessment is always going to be open, collaborative and educational and this needs to be the goal from the outset. However, many auditees are placed under huge pressure to pass an audit and sometimes will feel a high risk, deceptive, strategy is the only way to retain their jobs. I myself was once told in no uncertain terms “do whatever it takes to pass the audit” (and of course did).

What I really want to see in the industry is a move away from the checkbox and clipboard approach to auditing and assessing as the natural conclusion of that is a deeply unpleasant homogenisation of controls and environments that stifles creativity, and ultimately reduces the ability of a business to deliver to its clients and to its shareholders.

Lies, Damned Lies, and Statistics

1 Comment March 29, 2013 Thom Langford

Originally attributed to Mark Twain, who subsequently attributed it to Benjamin Disreali (although no evidence has been found that he actually said it), the above quote sums up how the use of statistics can blur the lines between powerful argument supporter and simple use of numbers to confuse and deceive.

When used properly, statistics in your risk management programme help support your recommendations, allow you to build effective business cases and even allow for a certain amount of self-analysis and performance reporting. When used badly, you run the risk of undermining the credibility of your entire risk management programme.

Consider the following two statements made by security awareness training companies:

“Reduce phishing click-throughs by 75%!”

KnowBe4 Internet Security Awareness Training

“…successfully trained over 7000 employees” (Fox Entertainment)

TerraNova Security Awareness

In the first instance there is a bold claim in that click-through rates reduce by 75% which on the face of it sounds great. When reading in details there are some more impressive results but I can’t help thinking of the somewhat artificial nature of the test, i.e. “I have just taken anti phishing training and I am suddenly getting five phishing type emails, hmmmmmm”. Perhaps a more suitable test would have been to wait two months before sending the test emails? (The time between training and testing is unfortunately not specified however). There was also no mention of any feedback given in between each test. Security awareness training is such a hot topic however that I will leave that well alone for now!

In the second case the banner across the top of the website proudly announces how many people have been successfully trained; unfortunately it makes no mention of the other 5,500 employes who were not trained in Fox Entertainment (headcount checked at 12,500).

Now this is just standard sales patter and I certainly don’t mean to pick on these two companies specifically, but both statements illustrate the point perfectly. In both cases the products are probably very good in their own field, but when you “reverse” what it is they are saying they speak volumes. Some foods for instance are labelled as 90% fat-free, but in reality that means they contain 10% fat, and so here therefore there is still a 25% sized group of people who did click through and there are still 5,500 people who were not trained (and why not?). This is related to the fear, uncertainty and doubt that is often touted in the industry and can be used to scare and subsequently encourage people to buy products.

As risk professionals we need to take a more balanced, calmer route. We need to use statistics more carefully and responsibly, especially when what it is we are presenting makes its way into the core of the business, the leadership, the board, and ends up being used to make business decisions with serious implications. We can’t take sales statistics for instance on face value and use them to recommend a product or emphasise a point.

A Google search of “risk management statistics” produces over a billion results (in of itself a bad and useless statistic to present) so there is plenty of work out there on how to present your work, so I won’t be suggesting anything specific here. There are also plenty of other issues with statistics, for instance causation and inference which can be looked at in more detail at a later date.

i will however close on three key points I use whenever I am producing statistics for anything that comes out of the data gathered by a risk management programme:

“Reverse” the statistic (see above). If you don’t like what you see, don’t use it.
Be careful of your sample size; too small and the statistics are meaningless, too big and the resulting statistic you are focussing on is still a big and scary number even though you are potentially trying to emphasise quite the reverse.
Look at what you come up with cynically; is it a lie, or a damn lie?

And to underscore how statistics can mess with your head, statistically there are six Popes per square mile in the Vatican; go figure.

Getting Your Hands Dirty

1 Comment March 21, 2013 Thom Langford

In my last post I referred to ensuring that your risk management programme is producing the quality of output to ensure the business information it feeds into is of the highest quality; maintaining the integrity of your programme.

If there is one thing that can be done to improve the integrity of your risk assessments it is simply to get your hands dirty during them. I have had a number of conversations with people who have been on the receiving end of an assessment where the assessor simply sits at the table and asks for evidence in the form of documentation, verbal responses or even just PowerPoint presentations to confirm the effectiveness of the information security programme in question. Personally I have sat in a conference room for one or two days at a time and only left the room for a short thirty minute ‘walkabout’. Quite how the assessor felt they were getting a representative view of what we were doing was beyond me.

There are a number of problems with this hands off approach:

The ability of those being assessed to ‘play’ the assessor increases with their reluctance to physically move around the organisation. Pre-prepared evidences (the so called “audit box” as was once described to me) can be made available, the organisations SME’s can be wheeled in to ensure the right things are said at the right time and the people who never seem able to say the right thing at the right time (and every organisation has them!) can be told to work in a different building that day.

Secondly, unless the assessor is actually looking at the evidence first hand, even down to rifling through the physical pieces of paper or reviewing server logs, there is absolutely no way any kind of discrepancy will ever be found. Of course this is a sampling exercise, and of course there is no way every single piece of evidence, paper or electronic can be reviewed, but some kind of benefit can be gleaned from going though them. Quite apart form anything else it gives the clear impression that “no stone is unturned” during the assessment process. I have come up with a surprising number of findings from simply taking a few minutes to look through large piles of paper records.

Finally, and perhaps slightly more esoterically, the action of a walkabout can give a very good “feel” for a place. If the presence of the auditor brings hurried and furtive glances everywhere they go, it may give the indication of nervousness or unwillingness regarding the assessment (or of course just a healthy distrust of strangers). If there are rows of empty desks that are obviously normally in use but seem to be vacated for the day this may give the indication that special plans have been laid on for the assessment (or that the sales team are in a meeting). This last point is not so clear cut as the other two, and should only be used as an indicator of what is already coming out of your assessment, but it is a useful one nonetheless.

I have a colleague who every time he enters a “serious” meeting, he undoes his cufflinks and rolls up his cuffs a couple of times; this is his way of mentally preparing for the challenge ahead by literally rolling up his sleeves. When it comes to risk assessments that is exactly what you need to do, and then prepare yourself to get your hands dirty.