What 80’s pop can teach us about Rocket failure and incident management


Most accidents originate in actions committed by reasonable, rational individuals who were acting to achieve an assigned task in what they perceived to be a responsible and professional manner.

(Peter Harle, Director of Accident Prevention,Transportation Safety Board of Canada and former RCAF pilot, ‘Investigation of human factors: The link to accident prevention.’ In Johnston, N., McDonald, N., & Fuller, R. (Eds.), Aviation Psychology in Practice, 1994)

I don’t just read infosec blogs or cartoons that vaguely related to infosec, I also read other blogs from “normal” people. One such blog is from a chap called Wayne Hale who was a Fligh Director (amongst other things) at NASA until fairly recently. As a career NASA’ite he saw NASA from it’s glory days through the doldrums and back to the force it is today. There are a number of reasons I like his blog, but mostly I have loved the idea of space since I was a little kid – I still remember the first space shuttle touching down, watching it on telly, and whooping with joy much to my mother’s consternation and chagrin. The whole space race has captured my imaginaion, as a small child and an overweight adult. I encourage anyone to head to his blog for not only fascinating insider stories of NASA, but also of the engineering behind space flight.

What Wayne’s blog frequently shows is one thing; space is hard. It is an unforgiving environment that will take advantage of every weakness, known and unknown, to take advantage and destroy you. Even just getting into space is hard. Here is Wayne describing a particular incident the Russians had;

The Russians had a spectacular failure of a Proton rocket a while back – check out the video on YouTube of a huge rocket lifting off and immediately flipping upside down to rush straight into the ground. The ‘root cause’ was announced that some poor technician had installed the guidance gyro upside down. Reportedly the tech was fired. I wonder if they still send people to the gulag over things like that.

This seems like such a stupid mistake to make, and one that is easy to diagnose; the gyro was in stalled upside down by an idiot engineer. Fire the engineer, problem solved. But this barely touches the surface of root cuse analysis. Wayne coniTunes;

better ask why did the tech install the gyro upside down? Were the blueprints wrong? Did the gyro box come from the manufacturer with the ‘this side up’ decal in the wrong spot? Then ask – why were the prints wrong, or why was the decal in the wrong place. If you want to fix the problem you have to dig deeper. And a real root cause is always a human, procedural, cultural, issue. Never ever hardware.

What is really spooky here is that the latter part of the above quote could so easily apply to our industry, especially the last sentence – it’s never the hardware.

A security breach could be traced back to piece of poor coding in an application;

1. The developer coded it incorrectly. Fire the developer? or…

2. Ascertain that the Developer had never had secure coding training. and…

3. The project was delivered on tight timelines and with no margins, and…

4. As a result the developers were working 80-100 hrs a week for three months, which…

5. Resulted in errors being introduced into the code, and…

6. The errors were not found because timelines dictated no vulnerabiliy assessments were carried out, but…

7. A cursory port scan of the appliction by unqualified staff didn’t highlight any issues.

It’s a clumsy exampe I know, but there are clearly a number of points (funnily enough, seven) throughout the liufecycle of the environment that would have highlighted the possibility for vulnerabilities, all of which should have been acknowledged as risks, assessed and decisions made accordingly. Some of these may fall out of the direct bailiwick of the information security group, for instance working hours, but the impact is clearl felt with a security breach.

A true root cause analysis should always go beyond just the first response of “what happened”? If in doubt, just recall the eponymous words of Bronski Beat;

“Tell me why? Tell me why? Tell me why? Tell me why….?”

Making the world angrier, one process at a time

Angry Thom BlogI have recently set up Family Sharing on my iOS devices, so that I can monitor and control what apps go on my kids devices without having to be in the room with them. Previously they would ask for an app, and I would type in my AppleID password and that was  that. Unfortunately with my new role I am travelling so much now that the thought of waiting a week before they can get an apps was causing apoplectic grief with my kids. Family Sharing was the solution, and when I had finally worked it out, we were goood to go and it works well. I can now authorise a purchase from anywhere in the world. I get woken up at 3am with a request for a BFF makeover or car crash game (one girl, one boy) but my kids are happy.

One problem however was that for some reason my daughters date of birth was incorrect, therefore indicating that she was an adult, and thereby breaking the whole “app approval” process. Straightforward to fix? Not at all.

I won’t bore you with the details, but it was the most frustrating process I have encountered in a long time. I admit, I misinterpreted the instructions along the way (they were a bit asinine in my defence), but it came down to the fact that I had to have a credit card as my default payment method for my family account, not a debit card, simply to authorise the change of status of my daughter from an adult to a child. In other words, I had to jump through hoops to restrict her  account rather than give it more privilege. Not only that, but from an account that already had the privileges in the first place. There didn’t seem to be any element of trust along the way.

I am sure there is a good, formal response from Apple along the lines of “take your security seriously”, “strong financial controls” etc, but as an experience for me it sucked, and if I could have worked around it I would have. Thankfully not all of Apple’s ecosystem works like this!

This is a problem for many information security organisations when they introduce procedures to support organisational change or request mechanisms. For instance, how many times have you seen a change request process require CISO, CIO and potentially even higher approvals for even simple changes? Often this is due to a lack of enablement in the organisation, the ability to trust people at all levels, and often it is a simple lack of accountability. It seems we regularly don’t trust either our own business folks as well as our own employees to make the right decisions.

Procedures like this fail in a number of places:

  1. They place huge pressure on executives to approve requests they have little context on, and little time to review.
  2. The operational people in the process gain no experience in investigting and approving as they simply escalate upwards.
  3. The original requestors are frustrated by slow progress and no updates as the requests are stuck in senior management and above queues.
  4. The requestors often work aroun d the procedure, avoid it, or simply do the opposite of what finally comes out of the request as work pressures dictate a quicker response.
  5. The owners of the procedure respond with even tighter regulations and processes in order to reduce the ability nof the nrequestor to wotk around them.

And so the cycle continues.

The approach I have regularly used in situations like this comprises of two tenets:

  1. Consider the experience of the user first, then the desirable outcomes of the process second.
  2. Whatever process you then come up with, simplify it further. And at least once more.

Why should you consider the expoerience of the user first? Who is the process for the benfit of, you as in formation secuity, or them as the end user? If you answered the former, then go to the back of the class. We are not doing security for our benefit, it is not security for the sake of security, it is to allow the user, our customers, to do more. If we make their experience bad as they do their best to make more money, sell more beer, do more whatever, security becomes an irellevance at best and a barrier to successful business at worst.

Making the requstors exoerience as painless and as straightforward as possible (perhaps eeven throw in a bit of education in there?) they are encouraged to not only see the long term benefits of using the procedure as we defined, but also become fanatical advocates of it.

Secondly, why should we keep it simple? Well not only to support the above points, but also because guess who is going to have to support the process when it is running? Of course, you and your team. If the process itself is bulky and unmanageable then more time will be spent running the process than doing the work that the process needs to support. If that amount of time becomes too onerous over time, then the process itself breaks down, the reporting on the process becomes outdated, and ultimately the process itself becomes irrelevant and considered a waste of time by those it affects.

Putting your requestors at the centre of your simplified process universe will always make that process more robust, more understood, more beneficial and of course more relevant to the business, and who can argue with that?

InfoSecurity Europe

I spoke at this years InfoSecurity Europe in London a few months back on articulating risk to senior management. Peter Wood, the moderator, did an excellent job as moderator of the panel, and even revitalised my faith in them after too many very poor experiences earlier this year.

Your InfoSec premiums have increased by 20% this year. Are we worth it?

High-insurance-PremiumsMy annual home insurance quote came through this morning, with the usual 10-20% uplift that I know I can remove again through simply phoning the provider and threatening to leave. It is a pretty standard technique in the industry that has been going on for years, and that preys upon the lazy people in the world who can’t be bothered to look for a better deal.

Rewind a few months when I spoke with a very senior executive who admitted that he saw information security as a form of insurance.

“I don’t want to have to pay for it, but I do because I know that when I need it you guys come and fix the problems we are in”

This is a somewhat common and fair attitude to information security given our background as an industry and how we often interact with the business (a particularly large topic that this entire blog is really about). yet what was so interesting was his follow on comment:

“the things is, I am sure there is so much more information security can do for us, I just don’t know what it is”

When I first took out home insurance, I was most concerned about getting the cheapest quote. I was young, free and almost single, but all of the extras that the larger insurance companies were offering (and charging for) did not concern me. If my house burnt down I would find somewhere else to live while the insurance company sorted everything out, what do I need a hotel for?  Lost my house keys? I will change the crappy lock on the front door myself when I get round to it, I don’t need a locksmith from the insurance company to do it for me.

Fast forward to today, and I live a far more complex busy life, cash rich (relatively speaking), time poor, with responsibilities to my children and wife, and a lifetime of memories in my house that are virtually irreplaceable. if things go wrong, I need it fixed quickly and easily and with the minimum of impact to me and my family. I even have proactive services, such as boiler cover and servicing to reduce the likelihood of things going wrong in the first place. Therefore I am leveraging every aspect of what the insurance company can give me even before something goes wrong, and the peace of mind that I get knowing they are looking out for me even prior to disaster striking is worth (almost!) every penny.

An information security programme must be able to sell every aspect of its services to the business, and not just be seen as a reactionary force. if it does that, every time something goes wrong, both the financial and emotional premiums of paying for your services will increase time over time until the point the programme is seen as imply an overhead like paying the rent and keeping the plant watered, i.e. when the time comes, costs to be reduced.

Look at how you provide service before the fact; risk assessments, security testing, awareness and education can all be seen as services that prevent and/or add value to the business. What about the day to day? Consultancy to the business to do things securely without them even thinking about it; it doesn’t have to have “security” written on it to be a win for you and the business. And of course don’t forget after the event; incident management, business continuity, or even helping in the quality acceptance environments after something has been developed.

The key is to be involved in the full lifecycle of your business, whatever they are. They will be different from business to business and industry to industry, so it may not always be easy to identify, but it is extremely valuable.

And the prices we quote every year? Unlike insurance premiums, we are worth every penny.

Note: I don’t actually like the analogy of infosec and insurance, but it is one I regularly hear, so I decided to try and embrace it in this blog. I still don’t like it, but I can see how it could be useful for a simple elevator pitch or short conversation. There are plenty of analogies out there, and the best place for them in my humble opinion is at The Analogies Project. Check them out, and use them wherever possible. Even better, think about becoming a contributor.


The Consistency of Plastique

51lIxdlS2nL._SX300_As I said in my last post I have been travelling quite extensively recently, but this weekend I was able to take a long weekend in Oslo with my wife just before the Nordic CSA Summit where I was invited to speak on “the CISO Perspective”. As a gift for speaking, each of us was given a block of Norwgian cheese, in a roughly square shape, that really did seem to have the consistancy, weight and look of a lump of plastique (I imagine…). It did occur to me that in the spirit of all good 44CON prizes, it was intended to get you stopped at the airport.

On my return home yesterday, I was pret sure my bag would be picked up for secondary screening given the presence of this lump of cheesy explosive in my bag (although apparently @digininja tells me a malt loaf has the same effect as well). Sure enough, my bag was selected, I presented to the good natured security folks the block of cheese, and with a wry smile they let my bag through. The same could not be said of my carry on bag though.

5piecelockpicktoolI was asked quite curtly if I had a penknife or similar in this bag; now I am getting more forgetful, but I was pretty sure I hadn’t. The security guy really did not look like he believed me, so we started to empty my bag. Then I remembered, I had a pick lock set that I had put into  zipped pocket in my bag about nine months ago, intending to give it to my good friend Akash in Boston who had expressed an interest in that particular art. Remember I just said I am getting forgetful? That’s why it has been in my bag for so long having seen Akash many times this last nine months. Oh well.

But it also occurred to me that I had been through about ten different airports in that time, and this was the first time it had been picked up, let alone even identified as a possible penknife (understandable as the picks fold into the main body).

This underscores to me the inconsistency of the security scanning at virtually every airport. Shoes on or off? Belts on or off? IPads as well as laptops taken out? Kindles, in the bag or out? My bag of cables that you tell me to keep in my bag at one airport, and then getting admonished for not pulling it out of the bag at the next? As an end user of these services (and I am fully supportive of them despite this I must say) it is extremely frustrating. There seem to be too many exceptions in place without clear reason, and without tying back to a singular way of doing things. The shoe bomber, Richard Reid, saw to it we have to take our shoes off going through security… except of course when you don’t.

Consistency in an information security programme is obviously key. But sometimes the pendulum swings too far the other way. Any policy that ends with “There are no exceptions to this policy” is asinine at best,  and crippling to the business at worst. There will always be a need for an exception in order to ensure business can be carried out effectively. As long as the risks are understood and communicated effectively, then move on and do it.

It certainly doesn’t mean that the exception can be used as an excuse to carry on working like that. There is no concept of precedence in this case. If there was the natural end state would be complete mayhem as every exception is used to the point where there is no policy left. An exception is just what it says on the tin, a one off easing off the rules for business to to operate effectively and efficiently. It should be time based, must be reviewed regularly, and where possible repealed if alternative approaches have come to light.

Consistency is important when applying policies, especially across a large organisation, but for goodness sake, don’t forget that change is an important part of business and needs to be embraced. But please do a better job of managing that change, and the subsequent exceptions, than airport security does.

Conferences and Presentations

What with InfoSec Europe, BSides, RSA Unplugged and the just attended Nordic CSA Summer conference it has been busy on the presentation front again. I have a few more presentation to upload to this site as well as some footage. I am hoping to make it to Blackhat in Vegas for the first time this year, and speak on behalf of friendly vendor who I have always enjoyed working with.


Diligently preparing for the conference

As I also mentioned in my last post, my employer became a sponsor of the European Security Blogger Awards, something I hope we will be for future events as well. Unfortunately I lost my best personal blogger award crown this year to Lee Munson of Security faq’s. I can’t help but feel that if I have to lose to someone, Lee would be top of my list as he consistently outshines me in both quality and volume of blogging. As a community we are lucky to have someone like Lee and if you haven’t already done so please do reach out to him and congratulate him.



Direct Hit, Near Miss or Remote Miss? Why you are more confident than you should be.

_39166788_blitz416_gettyIn the years running up to the beginning of the second world war the British government was extremely concerned that in the event of hostilities breaking out, the german Luftwaffe would launch significant attacks against Britain and especially London. With an estimated 250,000 casualties in the first week alone, the consensus was that millions of Londoners would flee, leaving the industrial war engine to grind to a halt. Several psychiatric hospitals were even set up on the outskirts of London to handle the huge numbers of casualties psychologically affected by the bombing.

History tells us this was not the case, despite horrific numbers of casualties and extensive damage to homes, property and businesses throughout London.

A Canadian psychiatrist, J. T. MacCurdy, in his book The Structure of Morale postulated this was because the effect of a bomb falling on a population splits them into three groups:

1. The people killed by the bomb. As MacCurdy puts it

the morale of the community depends on the reaction of the survivors, so from that point of view, the killed do not matter. Put this way the fact is obvious, corpses do not run about spreading panic.

Harsh, but true in this model.

2. The Near Misses, the ones that

feel the blast, … see the destruction… but they survive, deeply impressed. It may result in ‘shock’…and a preoccupation with he horrors that have been witnessed.

3. The Remote Misses. These are the people who hear the sirens, the bombs explode, watch the aircraft overhead, but the bombs explode down the street. For them the experience of the bombing is that they survived easily, unlike the Near Miss group. The emotion as a result of the attack…

is a feeling of excitement with a flavor of invulnerability.

Near miss = trauma, remote miss = invulnerability.

Diaries and recollections of the period certainly support these theories. For instance, when a laborer was asked if he wanted to be evacuated to the countryside (after being bombed out of his house twice) he replied;

What, and miss all this? Not for all the tea in China!

The reason for this attitude, the sense of invulnerability, is that they have been through the very worst of time… and survived. They had faced their fears, and realized they were not as bad as they thought they were going to be, and in fact the result of surviving had given them a sense of elation that made them feel even more alive than before.

This is a very long way of saying that we may very easily view security incidents and breaches like this. Sony (perhaps) are the ones right at the centre of the blast. they are affected directly, and don’t even run around spreading panic because they are too busy dealing with the incident itself.

The near misses, Sony’s vendors, suppliers and partners are probably reeling from the near miss and are probably doing all they can to ensure it doesn’t happen to them. in short why are traumatized.

Finally, there is the rest of us. Yeah baby! Another breach, and it wasn’t us! We are invincible! We don’t need to do anything different at all, because we are survivors!

I think I see an issue here. Every time we are not breached, we become more confidant that we will not be breached, and become over confident and convinced we are having the time of our lives doing great stuff in the infosec world and not being breached. let’s hope that bomb doesn’t drop too close to home to burst that bubble, otherwise Careers is So over ceases to be a funny industry joke and very much a reality. Take the precautions now, take the threat seriously, and do what you can now, before it is too late.

I would strongly recommend reading the Book David & Goliath by Malcolm Gladwell if you would like to read more about this concept as well as others along the same lines.

A personal note…

PubGr_logoI am now under new employment as a result of an acquisition of my previous employer, and I have been fortunate enough to be elevated to Group CISO of the acquiring company. Unsurprisingly this has resulted in a massive new workload, travel schedule and responsibilities, and hence my distinct lack of posts this last few months. Despite this I have still been nominated for European Personal Security Blog 2015 in this years Blogger Awards; thank you!

Additionally, I am so proud to say that not only is my new employer keen to promote this blog internally in the new company, but also thrilled to say we have become the newest sponsor of the European Security Blogger Network.

Finally, I have been on the road a huge amount the last few weeks, including at RSA USA where I was very happy with my presentation at the RSA Studio; I spoke about how we have changed our approach to security awareness, and the use of the Restricted Intelligence product to catalyse it.

There were also talks at Munich Identity Management Conference, although the talks are not public yet.

Next week, Bsides London, InfoSec Europe, European Blogger Awards and RSA Unplugged. I am mentoring a rookie at Bsides, Speaking at infoSec, as well as at the Tripwire booth, sponsoring (and nominated!) at the Blogger Awards, and just watching at RSA Unplugged.

It’s has been a busy few months!

%d bloggers like this: