What I planned to discuss at GridSecCon

The panel I moderated on supply chain security yesterday at GridSecCon was very successful. All of the panelists had very interesting things to say, and the audience asked some great questions – most of which we were able to answer during our allotted hour. Since I hate the idea that everything that was said has just evaporated into the ether, I am going to ask all of the panelists to summarize what they said – both in their initial short presentations and in response to questions. I will publish what they send to me within a couple weeks (but I really hope the E-ISAC will look into recording panels and presentations next year, and making them available on the web site. Slides are always posted, but panelists are only allowed one slide apiece).

I gave each of the panelists (including me) four minutes up front for their presentations – in order to leave as much time as possible for Q&A. I said we each should choose a topic (or two) that the audience wouldn’t already know about. I didn’t want to waste time on having each panelist discuss their organization and describe how committed they are to grid security, so I stated up front that all of their organizations (SEL, INL, OSISoft, Hydro Quebec, and NERC) are very committed to grid security and doing all they can to achieve that end – and that includes Tom Alrich, LLC! I gave them all a chance to disagree with that statement, but none of them did, for some reason.

For my four minutes, I had prepared a discussion of vendor questionnaires, which I think is a very important topic for supply chain security and CIP-013 compliance. However, during conversations at GridSecCon as well as before, I realized there was something more important I’d like to say, specifically regarding CIP-013 compliance – so I switched to that. I will do a post on what I actually said next week, but here is what I intended to say:

My topic is supplier or vendor security questionnaires. I believe these are a very important tool for supply chain cyber risk management – and they’re often overlooked. These are questionnaires that an entity sends to its vendors or suppliers on a regular basis (hopefully at least annually).

My guess is that most supplier questionnaires are mainly used for the purpose of deciding whether to buy from the supplier in the first place, or whether to drop them if their level of security changes for the worse.

That’s certainly a worthy initial goal, but as we all know, supplier relationships in this industry often go on for decades (if not centuries), and they’re never terminated lightly. So the important question is: There are a lot of security risks that a supplier could pose for our BES Cyber Systems. For this particular supplier, which of these risks do you need to worry about - and perhaps take additional mitigations to protect yourself from - and which risks has the supplier already sufficiently mitigated, so that you don’t have to take additional steps to mitigate them?

This means the question isn’t “Does this supplier have good or bad security?” It’s “Which supply chain risks has this supplier mitigated, and which ones haven’t they mitigated?” A properly-designed questionnaire can elicit this information. 

The questions in the questionnaire need to be specific to particular threats, like “What measures do you have in place to monitor your network for unusual activity?” You will assign the vendor or supplier a risk score for each threat, not just one overall score. For example, if the supplier has a good SIEM system and they’re managing it well, they’ve probably sufficiently mitigated this threat, and you will probably give them a low risk score for the threat.

But if they answer “Why would we want to monitor our network? We’ve never had a breach?”, that’s a red flag that they haven’t mitigated the risk at all. They would probably receive a high risk score for this threat. You need to have a documented process for evaluating the supplier’s answers to these questions. This is needed for overall consistency, but also for protection against lawsuits from losing suppliers in RFPs – especially if you work for a public entity subject to FOIA requests.

You’ll use the risk scores to guide your risk remediation efforts with the supplier. Armed with these scores, you can have a conversation with the supplier about what they’re doing well and what they’re not doing well, and suggest steps they can take to improve their scores. You may also use the scores to determine terms to include the next time you renew their contract.

The risk scores will be most helpful when you’re doing a procurement risk assessment, at the beginning of each procurement. NERC’s most recent Evidence Request Tool makes clear that CIP 13 audits will look at individual procurements. You’ll need to show evidence that you conducted a risk assessment, taking into account the threats you have identified as important to mitigate.

You’ll need to show that you’ve considered all of these threats in your procurement risk assessment. If a supplier or vendor has low risk scores for some of these threats, this means they’ve already mitigated those threats, and you don’t need to consider these further in your assessment. On the other hand, if they have high or medium scores for other threats, you do need to consider those threats – and you will probably need to identify additional mitigations that you or the supplier can take to reduce the residual risk in this procurement.

For example, if you believe a software supplier has security problems in their development environment and there’s a non-zero probability that malware or a backdoor may have been planted in their software, you might ask them to conduct a code review or do a full vulnerability assessment before sending you the software. Or you – the NERC entity - might take extra steps to look for vulnerabilities and backdoors before you install their software. Or you might do both.