Two Lessons from CIP-014: First Lesson

I have recently been wondering how CIP-013 will be enforced, since this is a non-prescriptive, objectives-based standard. I recently concluded that it in effect wouldn’t be “audited” at all, since there will be no way to find an entity in non-compliance with the requirements[i]. Of course, the auditor will still review the plan, and its implementation, from a general supply chain security perspective. If he or she feels that there are parts that need to be improved, they will issue an Area of Concern – which the entity would be well-advised to take to heart. However, I don’t believe there will be any Potential Non-Compliance (PNC) findings issued for R1 or R2 as a whole, unless the entity has simply done nothing or very little to comply with these requirements – and I find it impossible to believe that a NERC entity with Medium or High impact assets would do that.

However, CIP-013 isn’t the first CIP standard that is non-prescriptive and objectives-based. CIP-014 (the physical security standard that applies to certain important substations) is in principle the same (although I would say that CIP-013 goes further in that direction, but not by much). While CIP-014 certainly isn’t being audited yet, there has been a lot more opportunity for entities to talk with their regions about auditing and other compliance questions. What have they found?

I haven’t done any sort of scientific survey, but I did have a long conversation with a NERC physical security compliance person at one of the largest utilities in the US, about his experience so far with CIP-014. He had two stories to tell, which illustrate the challenges ahead for both CIP-014 and CIP-013 compliance enforcement. They also relate very directly to the larger question of how, if all of the CIP standards were re-cast in a non-prescriptive, objectives-based format, they would be complied with and enforced.

The First Lesson

This utility is putting a lot of money into CIP-014 compliance. There was one particular investment of $80 million that was being strongly considered. However, before the powers that be would commit to this investment, they asked my friend to find out whether this investment would enhance their chances of being found compliant with the requirements of CIP-014.

Since some of you may not be familiar with CIP-014, the standard requires the entity to (among other things):

1. Conduct a risk evaluation[ii] to identify which of its facilities (control centers and transmission substations) meet the criteria for inclusion in this standard;
2. Have a qualified third party validate that evaluation;
3. For the substations and control centers that are in scope, conduct an assessment of the facilities’ “potential threats and vulnerabilities” to physical attack;
4. For each facility in scope, develop and implement a physical security plan that will, among other things, address the threats and vulnerabilities identified in the assessment; and
5. Have a qualified third party validate both the assessment in step 3 and the plan developed in step 4. The third party may recommend changes in either document; the entity must change the plan to reflect those recommendations, or document why it did not. And since the plan has to be implemented, these changes will also need to be implemented.

Now that you know how CIP-014 works, you’ll be able to understand my friend’s problem. The plan in step 4 has identified the $80 million investment in question as being required to address one or more threats and vulnerabilities identified in the assessment in step 3. However, since no NERC entity has unlimited funds to address each threat and fix each vulnerability, there need to be trade-offs. This $80 million investment undoubtedly came at the expense of spending an equivalent amount of money to address some threats and vulnerabilities that were not considered to have such a high impact. But the entity – and the third party that reviewed both the assessment and the plan – determined that the impact of the threats addressed by the $80 million investment was sufficiently greater than that of the other threats, that this was the proper way to spend the money.

But management’s concern is this: NERC will give the final “assessment” of the plan when they come for an audit. What if they make the investment, then in a later audit NERC decides that they had their priorities wrong? In other words, that they should have spent the $80 million addressing some of what they thought were lower-impact threats, meaning NERC disagrees with them on their assessment of the impact of the threats in question. Will NERC then order them to spend an additional $80 million addressing these other threats?

It’s certainly a reasonable question, and my friend was tasked with asking it of their Regional Entity; in effect, he was going to ask the region whether they could review their assessment and plan, at least as they pertained to this particular issue. What do you think was their answer? There really was only one thing the region could say: For us to review your plan before you implement it would be a compromise of the time-honored principle of auditor independence. If we tell you how to comply up front, then when we come back to audit we will simply be auditing ourselves.

I don’t think any final decision has been made on the $80 million investment, but my friend thought it very possible he wouldn’t be allowed to proceed with it without some sort of nod from NERC or the region. So the threats and vulnerabilities addressed by that investment will likely remain unaddressed, until NERC audits them and decides they need to make the investment; hopefully, this finding won’t come with a Potential Non-Compliance finding, but just an Area of Concern.

I hope you understand that I’m not in any way saying the region had a choice in how they responded to this entity. Under the NERC Rules of Procedure and Compliance Monitoring and Enforcement Plan, the auditors must maintain strict independence. But that independence comes with a cost. In this case, the cost is a set of substations that are probably not going to be as physically secure as they might be, if the entity had gone ahead with the investment.

How does this relate to CIP-013? CIP-013 asks the entity to develop and implement a supply chain cyber security risk management plan (SCCSRMP). There is a 10-page Implementation Guidance that addresses what should be in the plan, which will – according to NERC’s current views on guidance - carry weight with the auditors (unlike any other guidance that may come out). It’s a very good document, but it could be 1,000 pages and still not cover everything needed to develop and implement a good plan.  

As with CIP-014, the entity will have to develop the plan and implement it, without any official guidance from NERC on whether it’s a good plan or not. As with CIP-014, the entity could go for years believing their plan is good, only to have all of this contradicted years later by a NERC auditor. Since the CIP-013 plan also must address security threats (although this time it’s a question of cyber threats to the supply chain, vs. physical security threats in CIP-014), it’s very possible that a putative future CIP-013 auditor will also disagree with the entity’s assessment of the relative impact of the threats they face. Finally, it’s possible that the CIP-013 auditor will issue a PNC due to this disagreement.[iii]

So it’s very possible that the same thing will happen to you, if you’re involved with CIP-013 compliance, as happened to my friend who was involved with CIP-014 compliance: an important project (or section of a project) will be cancelled or greatly delayed because there is no way that NERC auditors can provide the sort of pre-implementation assurance of compliance that would allow management to feel completely assured in making their investments.

How could management feel assured? What if, before an entity starts implementing a plan (either a physical security plan in CIP-014 or a SCCSRMP in CIP-013), they had to submit it to their NERC region? The region would review it thoroughly, identify any problems they find with it, then point these out to the entity; the entity would then need to change their plan. The entity would then have the comfort of knowing, when they approve a large investment, that it will improve their chances of being compliant.

As you can guess, this wouldn’t be possible in the current NERC environment. First, as I’ve said before, NERC’s CMEP (and probably the NERC Rules of Procedure) needs to be revised to allow for this. But there is another, even more fundamental, change that would need to happen: The NERC auditors would have to turn into something like cyber security consultants. They would review the entity’s assessment and plan, then come back later to review how the entity implemented the plan. They simply couldn’t be called auditors any more (maybe “assessors”, as in PCI).

This might seem like a radical change, but really it’s not. It would simply be a recognition of reality. Prescriptive standards like the NERC O&P (or “693”) standards require traditional auditors, and they need to maintain their independence by not providing compliance advice before the audit. This is necessary because the O&P standards address threats that are absolutely certain, since they’re based on the laws of physics: If you don’t do X, Y will happen.[iv]

But cyber security is very different. There, everything is based on probabilities. The various cyber threats are always changing and they are only probabilities, not certainties. For that reason, prescriptive requirements don’t work in cyber – or more correctly, they do work but at a big cost. Cyber security standards should be non-prescriptive and objective-based (indeed, almost all other cyber standards in the world are such, although the nuclear power industry’s cyber standards are even more prescriptive than the CIP standards). And non-prescriptive standards require a collaborative approach – not an audit in the traditional sense - in order to avoid exactly the sort of snafu that my friend described to me.

In other words, this snafu is almost inevitable when an organization like NERC tries to enforce non-prescriptive standards using a prescriptive compliance regime. The second post in this series will discuss another example of this problem.

I expect to come back to the second lesson in a post a week or so from now. But before I do that, I need to keep a promise I made when I spoke on CIP-013 as part of a panel at NERC GridSecCon last week; the post I promised should be out later this week.

The views and opinions expressed here are my own, and do not reflect those of any organization I work with. If you would like to comment on what you have read here or ask any questions, I would love to hear from you. Please email me at tom@tomalrich.com.

---

[i] Except in the case of the six items listed in R1.2. Those six items must be incorporated in the entity’s supply chain cyber security risk management plan in R1. If those aren’t so incorporated, then the auditor could issue a Potential Non-Compliance finding. Of course, since R2 requires the entity to implement the plan from R1, the entity could receive a PNC if they haven’t made an effort to implement those six items (which all involve the vendor making a commitment of some sort). However, whether this can actually be enforced is questionable.

[ii] The standard actually uses the word “assessment”; it uses “evaluation” to describe the activity of identifying threats and vulnerabilities in step 3. Since I and most readers are used to thinking of the process of identifying vulnerabilities as an assessment, I have reversed the two words here.

[iii] As I did in the post previously mentioned as well as the first paragraph of this post, I want to point out that I don’t think it’s likely there will be anything more than an Area of Concern issued as a result of a disagreement between the entity and the auditor on an issue like the relative impacts of particular threats. However, I’m sure there will be some in management – and especially in the legal department – who will require more certainty on this point than I’m able to provide.

[iv] This might seem like a big exaggeration – after all, if control room operators don’t communicate properly as required by the NERC COM standards, a cascading outage of the BES won’t necessarily be the result.  If the right set of physical conditions is already in place (hot day in summer, large plant already on outage, etc), there would exist some scenario where if operator 1 does X instead of Y, and operator 2 does A instead of B, then there would be a cascading outage. There is no such certainty anywhere in cyber security.