Lies, Damned Lies, and Statistics

1 Comment

Pac-ManOriginally attributed to Mark Twain, who subsequently attributed it to Benjamin Disreali (although no evidence has been found that he actually said it), the above quote sums up how the use of statistics can blur the lines between powerful argument supporter and simple use of numbers to confuse and deceive.

When used properly, statistics in your risk management programme help support your recommendations, allow you to build effective business cases and even allow for a certain amount of self-analysis and performance reporting. When used badly, you run the risk of undermining the credibility of your entire risk management programme.

Consider the following two statements made by security awareness training companies:

“Reduce phishing click-throughs by 75%!”

KnowBe4 Internet Security Awareness Training

“…successfully trained over 7000 employees”  (Fox Entertainment)

TerraNova Security Awareness

In the first instance there is a bold claim in that click-through rates reduce by 75% which on the face of it sounds great. When reading in details there are some more impressive results but I can’t help thinking of the somewhat artificial nature of the test, i.e. “I have just taken anti phishing training and I am suddenly getting five phishing type emails, hmmmmmm”. Perhaps a more suitable test would have been to wait two months before sending the test emails? (The time between training and testing is unfortunately not specified however). There was also no mention of any feedback given in between each test. Security awareness training is such a hot topic however that I will leave that well alone for now!

In the second case the banner across the top of the website proudly announces how many people have been successfully trained; unfortunately it makes no mention of the other 5,500 employes who were not trained in Fox Entertainment (headcount checked at 12,500).

Now this is just standard sales patter and I certainly don’t mean to pick on these two companies specifically, but both statements illustrate the point perfectly. In both cases the products are probably very good in their own field, but when you “reverse” what it is they are saying they speak volumes. Some foods for instance are labelled as 90% fat-free, but in reality that means they contain 10% fat, and so here therefore there is still a 25% sized group of people who did click through and there are still 5,500 people who were not trained (and why not?). This is related to the fear, uncertainty and doubt that is often touted in the industry and can be used to scare and subsequently encourage people to buy products.

As risk professionals we need to take a more balanced, calmer route. We need to use statistics more carefully and responsibly, especially when what it is we are presenting makes its way into the core of the business, the leadership, the board, and ends up being used to make business decisions with serious implications. We can’t take sales statistics for instance on face value and use them to recommend a product or emphasise a point.

A Google search of “risk management statistics” produces over a billion results (in of itself a bad and useless statistic to present) so there is plenty of work out there on how to present your work, so I won’t be suggesting anything specific here. There are also plenty of other issues with statistics, for instance causation and inference which can be looked at in more detail at a later date.

i will however close on three key points I use whenever I am producing statistics for anything that comes out of the data gathered by a risk management programme:

  1. “Reverse” the statistic (see above). If you don’t like what you see, don’t use it.
  2. Be careful of your sample size; too small and the statistics are meaningless, too big and the resulting statistic you are focussing on is still a big and scary number even though you are potentially trying to emphasise quite the reverse.
  3. Look at what you come up with cynically; is it a lie, or a damn lie?

And to underscore how statistics can mess with your head, statistically there are six Popes per square mile in the Vatican; go figure.

Getting Your Hands Dirty

1 Comment

dirty-handsIn my last post I referred to ensuring that your risk management programme is producing the quality of output to ensure the business information it feeds into is of the highest quality; maintaining the integrity of your programme.

If there is one thing that can be done to improve the integrity of your risk assessments it is simply to get your hands dirty during them. I have had a number of conversations with people who have been on the receiving end of an assessment where the assessor simply sits at the table and asks for evidence in the form of documentation, verbal responses or even just PowerPoint presentations to confirm the effectiveness of the information security programme in question. Personally I have sat in a conference room for one or two days at a time and only left the room for a short thirty minute ‘walkabout’. Quite how the assessor felt they were getting a representative view of what we were doing was beyond me.

There are a number of problems with this hands off approach:

The ability of those being assessed to ‘play’ the assessor increases with their reluctance to physically move around the organisation. Pre-prepared evidences (the so called “audit box” as was once described to me) can be made available, the organisations SME’s can be wheeled in to ensure the right things are said at the right time and the people who never seem able to say the right thing at the right time (and every organisation has them!) can be told to work in a different building that day.

Secondly, unless the assessor is actually looking at the evidence first hand, even down to rifling through the physical pieces of paper or reviewing server logs, there is absolutely no way any kind of discrepancy will ever be found. Of course this is a sampling exercise, and of course there is no way every single piece of evidence, paper or electronic can be reviewed, but some kind of benefit can be gleaned from going though them. Quite apart form anything else it gives the clear impression that “no stone is unturned” during the assessment process. I have come up with a surprising number of findings from simply taking a few minutes to look through large piles of paper records.

Finally, and perhaps slightly more esoterically, the action of a walkabout can give a very good “feel” for a place. If the presence of the auditor brings hurried and furtive glances everywhere they go, it may give the indication of nervousness or unwillingness regarding the assessment (or of course just a healthy distrust of strangers). If there are rows of empty desks that are obviously normally in use but seem to be vacated for the day this may give the indication that special plans have been laid on for the assessment (or that the sales team are in a meeting). This last point is not so clear cut as the other two, and should only be used as an indicator of what is already coming out of your assessment, but it is a useful one nonetheless.

I have a colleague who every time he enters a “serious” meeting, he undoes his cufflinks and rolls up his cuffs a couple of times; this is his way of mentally preparing for the challenge ahead by literally rolling up his sleeves. When it comes to risk assessments that is exactly what you need to do, and then prepare yourself to get your hands dirty.

Probably not a serious breach, but definitely a serious failure

Leave a comment

The Twitterverse, online and traditional media worlds were if not alight then certainly smouldering with the news of a security breach as a result of pictures being published showing the Prince in a normal day at the office. At first I couldn’t work out why the press was saying that username and passwords were at risk, especially as the main photograph showed the Prince at a computer screen. Surely passwords are always obscured at a login prompt? Even the MOD can’t have such bespoke systems that they clearly show passwords on a screen? I even Tweeted that surely this must have been, therefore, a Post It fail rather than technology fail. Thankfully there were further Tweets and further analysis of the situation, and it was the Naked Software blog that finally made sense of it all.

Unbelievably it was a Post It fail… or at least a piece of A4 taped to the wall fail. 

My personal analysis of this may be a little different from most infosec professionals, in that what was exposed was probably not that serious. A username and password was effectively leaked for what was probably an unclassified part of the MOD network (or whatever the correct terminology is). This physical network is probably behind fences and locks and soldiers with guns (or heaven forbid, the MOD Police), and probably didn’t even have anything interesting on it. I do of course think those in charge were right to change the password and username though, as that is obviously  sensible precaution, but after that point, so what?

That said, what i think this does highlight is a dreadful failure of the security “attitude test” by the personnel and leadership of that base. How on earth it could have been deemed as acceptable to have a username and password, of any description, taped to a wall, no matter how secure the environment, is beyond me. Firstly, this means that a generic account is in use, a fundamental no-no in anyone’s book, but also it indicates that it is acceptable to do other things born of convenience. Share files on a USB between here and home – no problems! Carry printed flight rosters and contact details in your manilla envelope out of the base – of course! The mere act of allowing this to happen means there are already shoddy security practises at work in this base and their head of security should investigate immediately (and be slightly ashamed. As an aside I was also surprised at the Prince to be honest; here is someone who must have had security training to the nth degree given his position, and he is stood, smiling, right next to the picture.

It reminds me of why I make such a big deal of using lock leads in the office. The actual risk of having a laptop stolen from your own office in the middle of the day is fairly low (overnight the risk rises of course, but we don’t leave laptops out overnight do we?!). I often cite the example of a fire alarm and subsequent evacuation, and laptops being removed/stolen by the last person on the floor, but again, this is an unlikely event. my main driver for the lock lead is because the very physical act of attaching your laptop to a lock lead first thing in the morning is a strong reminder of the need for security, and puts that person into a more security aware frame of mind. If they take their laptop into a meeting room, again the act of unlocking it is a reminder again. I have argued before that security awareness training does not interact with people often enough to influence their behaviour in any measurable way, but if we can encourage the use of lock leads throughout the organisation much of the battle is won.

Really, if the MOD gets this wrong, what hope is there for the rest of us?

 

 

May I Ask YOU A Question Or Two…?

1 Comment

The iPhone5 launch is very exciting for many people, and I have to admit myself included. Whatever your opinion of that particular can of worms, one thing is for sure, and that is many people will be parting with a lot of money in the next week or two in order to get hold of the latest piece of geek chic.

When there is a likelihood of a money changing hands, scammers and criminals will never be far behind.

I took a phone call (from a UK 0845 number) on my mobile phone on Saturday from someone claiming to be from O2, with an offer to get the new iPhone5 on the day of release without having to queue for hours at my local O2 store. They would even honour the lower retail store price compared to the order online price; on my tariff that meant £70 for the handset rather than £100 because I was a good customer (which I am). What an offer!

Without thinking, I confirmed the first line of my address… and then thought “Oh crap, shouldn’t have done that”; I got a bit carried away. They had called me, not the other way around, I really had no idea who they were!

Cast your mind back a few years ago, and there was a semi legal scam whereby people would take calls from “a representative from <insert mobile provider here>”. They would entice the individual with early upgrades and a new phone, get the verbal agreement, and then shift the contract to a new, third party provider. The downside was that this provider had many hidden charges and an average £25 bill would become £125 overnight partnered with a legally binding contract. This was soon clamped down upon, but this example starting to ring through my mind!

It was at this point that I had verbally agreed that I wanted the new iPhone delivered to my door on a new and cheaper contract this coming Friday… Oh dear God, Have I just committed professional suicide here?!

I turned on my professional brain, and then asked the person at the end if she really was from O2, and obviously she replied “yes!”. So I asked her if she would mind if I asked her a few security questions “of course not, I would do the same!”. i logged onto my O2 account and asked her for my account number, last bill amount and how long I had been a customer. She had all of the information to hand, I was happy, and I am now looking forward to a new phone on Friday (either that or this blog will be closed down on Saturday!).

It did occur to me however that I felt a little awkward asking these questions. How many other people in a similar position, offered an enticing deal would do the same thing? And how often would someone be ripped off as a result. We receive phone calls all the time from our service providers, and very often just asking for innocent information or making sure you are happy with their current deal, but sometimes the first question they ask is a “security” question to confirm you are the correct person. This normal procedure is easily hijacked by social engineers who could over the course of a few months gather a vast amount of information just from phoning you and asking you outright!

Has anybody else experienced this kind of thing? Have you missed some great deals because you missed the opportunity to grab it because you were too suspicious or have you thrown caution to wind only to regret it later, if only for a short period of time? How cautious do we need to be in these circumstances?

One thing I learnt however is that in the middle of a conversation, it is very easy to forget who called who; remembering that if you answer the call you haven’t confirmed their identity and therefore need to ask some security questions of your own is probably  the best way of keeping you out of trouble!