Monday, June 8, 2020

“Purchase Decision”



Note from Tom: If you’re only looking for my latest pandemic post, please go to my Pandemic Blog. If you’re looking for my cyber/NERC CIP post, you’ve come to the right place.


In NATF’s recent webinar on their Supply Chain Risk Questionnaire – which I’ve written about in four posts, starting with this one – they briefly discussed not only the questionnaire but their Supplier Cyber Security Assessment Model, which forms the basis for their supply chain/CIP-013 program. It was previously discussed (briefly) in their webinar in March.

The model is summarized in a diagram on page 6. The heart of the diagram is the circle at the bottom: “Conduct Risk Assessment”. The process of assessing the supplier’s cyber risk (which is of course the focus of NATF’s program, including the Criteria and the questionnaire) is the biggest input (physically) to the Risk Assessment in the circle. However, in two little boxes on the side we find two more types of input: “Other factors…(financial, operational, reputational, regulatory, etc.)” and “The purchasing entity’s inherent risk and risk appetite”. The output of the circle (and of the whole process) is a green box to the right of the circle that reads “Purchase Decision”.

What this diagram essentially says is that the goal of CIP-013, and of supply chain cybersecurity risk management for the BES in general, is to help NERC entities make well-informed decisions on whether or not to purchase particular products or services. That decision is based on a number of factors, of which the vendor’s cyber security is important but hardly the only one (even though the cyber assessment takes up at least ten times as much space as the little box of “other factors” does).

There’s only one problem I have with this diagram and the model behind it: I simply don’t think it’s realistic for the domain it’s trying to address, which is supply chain security for hardware and software components of BES Cyber Systems.

When it comes to purchase decisions for hardware and software that go into BCS, there are two considerations that outweigh all others – including cyber security – by far. The first is “Does this product do everything we need it to?” If another product doesn’t do something the engineers consider is important, it won’t matter how low the price is, how great a reputation the supplier has, how “compliant” their product is, etc. That product won’t be purchased.

The second consideration – and I’d say this is the one that outweighs all the others by far – is “Do we have experience with this supplier (and if possible with this product), and has it been good?” If the answers to both parts of this question are yes, it will require an act of Congress, a Papal Encyclical, and a Presidential Directive for that supplier not to get this new order. And that makes perfect sense: For systems that literally run the BES, an electric utility will almost never take a flier on a different supplier unless they’re having some problems with the current one – and unless there are at least ten other utilities on the same block that use a different supplier, and will promise them on a stack of Bibles that they won’t be sorry they switched.

Of course there’s another consideration which makes a NERC entity strongly inclined to stick with their existing supplier: installed base. If a utility uses an EMS from supplier X, the chances are strong they’ll still be with that supplier ten years from now, and the chances are just about 100% that they won’t change suppliers for at least a few years, even if they make the irrevocable decision today to change.

What this comes down to is: For BCS hardware and software, NERC entities almost never make “purchase decisions”. That is, they almost never find themselves in the situation where they need to compare a number of suppliers on various factors, including cybersecurity. The decision to purchase product A from supplier B has almost always been made before cybersecurity even comes into consideration.[i]

Does this mean the industry is wasting its time worrying about supply chain security? Or, more cynically, can it be said that the main purpose of supplier security assessments is to find evidence to back up a purchase decision that has already been made?

Fortunately, no. It means that, in the great majority of BCS procurements, the risk assessment begins after the purchase decision has been made. The question the assessment needs to answer isn’t “Should we buy from these guys or not?”. It’s “Since we’ve decided to buy from these guys once again, what are the cybersecurity risks we need to be concerned about, and how can we mitigate those risks to the best of our ability (and accept the rest)?”

What does this mean for a CIP-013 program? I’ve had this opinion for a long time, so it’s baked into my methodology. The biggest consequence is that I don’t think it’s useful to spend time developing overall supplier risk scores, or even scores for general areas like identity and access management or remote access. Instead, you need a score for every cyber risk that applies to a supplier.

So far, I and my clients have identified about 90 cyber risks that apply to suppliers and vendors. I want to know where the supplier stands on each of those risks – are they high, medium or low risk? In other words, I want a risk score for each risk that applies to a supplier. My questionnaire asks one or two questions for each supplier risk, and from the answer(s) I develop a score for that risk.

What’s the point of having all those scores? The main use for them is in the procurement risk assessment, which is the heart of my methodology, and also is the focus of the NERC Evidence Request Spreadsheet for CIP-013-1. The supplier’s scores tell you which risks still need to be mitigated during the procurement or installation of the product. For each unmitigated risk, you need to identify one or more mitigations that will reduce the risk during the procurement or installation – and if there is no possible mitigation at that point, you need to accept the risk.



Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com. Are you working on your CIP-013 plan and you would like some help on it? Or would you like me to review what you’ve written so far and let you know what could be improved? Just drop me an email!

[i][i][i] One of my clients – a medium-to-large municipal utility – told me they do a competitive RFP for BCS hardware or software once every five years, if that.

No comments:

Post a Comment