The NATF Questionnaire: Deciding which questions to use (part III)

Note from Tom: If you’re only looking for today’s pandemic post, please go to my new blog. If you’re looking for my cyber/NERC CIP post, you’ve come to the right place.

In my first post in this series, I pointed out that NATF’s recent supplier questionnaire has a lot of questions, many of which I don’t think are needed. In that post, I pointed out three reasons why NERC entities should strongly consider paring down the list of questions to the ones that do seem to be valuable – I found 83 of the 228 (approximately) questions to be “keepers”. I discussed the first reason in that post, and the second reason in this post last week.

Now it’s time to discuss the third reason, which might well be the most important if you’re using this questionnaire as part of compliance with NERC CIP-013. I stated this reason in the first post as “…asking unnecessary questions increases compliance risk for CIP-013…”

I realize this one may require some explanation. Let’s look at a question that I don’t believe should be asked in a CIP-013 questionnaire, that I used as an example in the first post: “Describe how you perform security assessments of third-party companies with which you share data (i.e., hosting providers, cloud services, PaaS, IaaS, SaaS, etc.). Provide a summary of your practices and/or controls that assure the third party will be subject to the appropriate standards regarding security, service recoverability, and confidentiality.”

My problem with this question is that I don’t think it addresses a significant BES risk. BES risks are all about control systems, not information systems. The latter process information, whereas control systems…well, they control. They don’t store information that is of value, other than their own configuration. The only other information that’s significant for the BES is what falls under the definition of BCSI. In my opinion, the NATF Criteria themselves do a good job of addressing any information risks that are important for the BES – there are nine criteria specifically on that subject. Since I already have supplier questions – in the list that I and my clients have compiled – that cover every one of the NATF criteria, I simply don’t see the need for the above question. Given that the supplier should be storing little if any information about a NERC entity’s systems anyway, it’s very hard to see how adding this question to the nine that are based on Criteria helps mitigate any additional BES supply chain risk.

But what’s the harm in asking this question? Specifically related to the reason stated above, how might asking this question increase CIP-013 compliance risk? Here’s how:

1. As I pointed out in this post last year, you shouldn’t be fooled by the fact that you have a tremendous amount of freedom in how you develop your supply chain cybersecurity risk management plan in R1. This is because the plan you develop with so much freedom in R1 becomes your jail when you get to R2.
2. R2 says you need to implement your R1 plan, period. It doesn’t say “Follow the general tenor of your R1 plan” or anything like that. If you said you would do something in your plan, you’d better do it when you implement the plan – and if you’re not sure whether or not you’ll be able to do something in R2, don’t put it in the R1 plan in the first place. Otherwise, you could be found in violation (in fact, I think this idea in general will be by far the biggest source of compliance risk for CIP-013).
3. In R1.1, you’re supposed to “identify and assess” supply chain cybersecurity risks (to the BES). This means you’ll first identify the set of possible risks, then you’ll assess how significant they are. These risks can apply to vendors or suppliers or they can apply to your entity. For the former, you’ll need to assess how likely it is that each vendor/supplier risk you’ve identified applies to each of your vendors or suppliers of hardware or software components of BES Cyber Systems (in other words, you’ll assess whether the vendor or supplier has or hasn’t already mitigated each of the risks you’ve identified, although in some cases a risk won’t apply to them at all). In my opinion, the best way to do this assessment is a questionnaire. My rule is you should never ask a question in your questionnaire that doesn’t address a risk you’ve identified as important – and if you see a question that you want to add to your questionnaire, you need to acknowledge that this means there’s a risk you haven’t identified so far (I’ve identified a number of risks that way).
4. But there’s another word that was left out of R1.1, which you nevertheless need to consider as if it were there: “mitigate”. If you’ve identified a risk and assessed that it applies to a particular supplier, you’re required to do something to mitigate it. Specifically, you need to try to get the supplier to mitigate it, using contract language or some other means (maybe you just get them to commit verbally to mitigating a risk, and you write a memo to yourself about that conversation). But if all of those efforts fail and you can’t get the supplier to do what’s needed to mitigate a particular risk (say, the risk that they won’t do background checks on their software developers, and one of them will plant a backdoor in a software product that you install on a BCS), it’s incumbent on you to take steps on your own to mitigate the risk (using the same example, by requiring the supplier to do background checks on their developers) – as NERC discussed in their CIP-013 FAQ earlier this year. Your plan needs to describe how you will do all of these things.
5. So let’s say you have identified your risks in R1.1 and you’ve mapped all of them to particular questions in the NATF questionnaire. However, when you develop your own supplier questionnaire, you decide to include some of the other NATF questions, that don’t correspond to the risks you identified; you do this because the questions “sound like they should be asked”. One of the NATF questions you include is the one about assessing cloud providers.
6. Now let’s say one of your important suppliers answers the above question by saying they don’t perform assessments of third party suppliers with which they share data. You might be tempted to brush this off, saying something like “Well, they don’t have much of our data and they’ve already told us it isn’t stored in the cloud, so we don’t have to worry about this.”
7. But there’s a problem with this: The question doesn’t say anything about what they’re doing with your data, or even whether they hold any of it. It just asks what they do if they store any data at all in the cloud. If you don’t follow up with them, and document that you did so and what the result was, a future auditor might say this is a potential non-compliance. They would do this because your plan (hopefully) said you’d follow up whenever a supplier gave an answer that indicated they may have a high likelihood of risk in the subject matter of the question. But you didn’t follow up in this case.
8. And there’s another problem with this question: it’s an essay question. This means you should have documented before you issued the questionnaire what your strategy would be for evaluating the answer, meaning how you’ll determine whether the answer indicates a low, medium or high likelihood that the risk in question (storing data with cloud providers who don’t have good security) is present with regard to this supplier. And if you didn’t do that, you might be in for another PNC.

Of course, if it turned out that this question really does address an important risk, you might rationalize getting a PNC by saying something like “It’s just as well that we got it - hopefully it won’t turn into an actual violation. This really is a risk we should be concerned about.” Unfortunately, you can’t say that in this case, since you probably already had nine other questions in your questionnaire having to do with information protection (the questions based on NATF criteria); the fact is that you didn’t need this question in the first place, and now it’s led you to a PNC.

The moral of this story: Only include questions in your questionnaire that you think address important risks to the BES (i.e. that are listed in your list of risks, even if you added the risk retroactively to justify a question you decided was important to ask). And then make sure you follow up and resolve any cases where the supplier’s answer to a question indicates there’s more than a low likelihood that this risk might apply to them.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com. Are you working on your CIP-013 plan and you would like some help on it? Or would you like me to review what you’ve written so far and let you know what could be improved? Just drop me an email!