The SDT Breaks new Ground – Part 3

In my last post, I continued to applaud the CIP Modifications standards drafting team for breaking out of what had seemed to me to be a dead-end program for introducing virtualization into the CIP standards. I pointed out that they accomplished this “breakout” by coming up with two Great Ideas (this is my term. It’s not in the NERC Glossary). But I also pointed out – very nicely, of course – that one of their two Great Ideas was not likely to succeed unless another change was also made.

This particular Great Idea was that the prescriptive requirements in the currently-enforced CIP standards (like CIP-007 R2, CIP-005 R1 and CIP-010 R1) should all be written non-prescriptively, in the manner of CIP-007-6 R3. The question I asked in the post was “How will these requirements be audited?” And I wasn’t alone in asking this question. In the SDT’s webinar on virtualization recently, SDT members said that – or some variation of it – was the most frequently submitted question.

And well it should be. The problem is that NERC’s auditing procedures – like those of almost any other organization that does audits – ultimately come down to determining whether the entity has or hasn’t done specific things that are required. This of course works fine for prescriptive requirements, where the whole point of the audit is determining whether or not the entity has done – or not done, as the case may be – the particular things specified in the requirement.

But this doesn’t work so well for two reasons. First, it doesn’t work well when there is ambiguity in a prescriptive requirement, or in a definition that underlies it.  For example, CIP-010 R1.1.3 requires the entity to include in its baseline any “custom software” that is installed on the system. Of course, a program written for a particular purpose definitely counts here. But how about other pieces of software like scripts? There isn’t a definition of “custom software” that can settle this question, so an auditor can’t issue a PNC (or at least it won’t be upheld) in a case where an entity isn’t including scripts in their baselines.

The other reason is because a requirement is non-prescriptive. Three good examples of this are CIP-014 R1, R4 and R5. In a post last year, I discussed two entities (from the same Region) that both told me the same story: They had been dinged by an auditor for not taking specific steps to protect transformers located in their substations in scope for CIP-014. Their mistake was taking the words of these three requirements literally, since all three only talk about protecting the substation itself, not any equipment located in it.

Unfortunately, if you take away all the prescriptive requirements that have ambiguous language in CIP, as well as all the non-prescriptive requirements, there aren’t many requirements left! What this means is there are very few current CIP requirements for which there isn’t any question on how they will be audited.

Of course, despite this fact, I haven’t heard too many horror stories about an entity and an auditor being completely at loggerheads about a particular issue, to the extent that the entity ends up getting a PNC or even just a very stern warning. Why is this the case?

It certainly isn’t the case because NERC provided lots of guidance for both the entities and the auditors. NERC has tried to provide CIP guidance that would be in some way “binding” for entities and auditors for many years, but in the end it has always had to be removed[i]. And for good reason: There simply is no way, under the NERC Rules of Procedure, for NERC itself to provide any document – other than the standards themselves - that would guide the content of an audit. The only way to fix a hole or an ambiguity in a requirement is to write a Standards Authorization Request to draft a new requirement or revise an existing one[ii]; and that’s part of the CIP Modifications team’s current agenda. 

No, the reason auditing has worked fairly well, despite the many ambiguous and non-prescriptive requirements in the currently-enforced CIP standards, is that the Regions have stepped up and provided the “guidance” that NERC can’t provide. They will almost never do this in writing, but you can usually get questions answered verbally (and one Region takes this a step further, by actively offering “assist visits” to help entities put in place good cybersecurity practices. And you can be quite sure that those practices are compliant!).

But this is by no means an ideal situation. Anytime an auditor speaks to you about a requirement – or speaks in a Regional workshop – they will make it clear that what they say is simply their personal opinion, not that of the Region (and perhaps not that of the other auditors). And they mean this; I’ve heard multiple stories about entities who were told one thing by an auditor at their office, but another thing by an auditor (in one case it was the same one), when they came onsite to do an actual audit. In cases where the auditors themselves are clearly confused by a requirement, I think it’s highly unlikely that an entity will receive a PNC for violating it, but I’m sure this wears on CIP compliance professionals who have to live with this uncertainty.

Why am I bringing all of this up now? Because the SDT is now proposing to make prescriptive requirements like CIP-007 R3 and CIP-005 R1 non-prescriptive. Non-prescriptivity is a good thing, but it needs to be accompanied by a different audit regimen than NERC has now. With non-prescriptive requirements, there’s no getting around the fundamental problem: If a requirement just mandates that you do something like develop or implement a plan, but it doesn’t provide a specific list of topics that you need to include in your plan, there is nothing to audit – except whether or not you developed or implemented the plan at all.

Of course, if you clearly didn’t develop a serious plan, or you barely tried to implement it, you could still be held in violation. But for anything else, like missing a particular element in your plan that the auditor thinks should be there, you can’t receive a violation – although the auditor can still give you a hard time. Again, this situation is wearing on CIP compliance professionals. And to be honest, if the SDT simply changes a few prescriptive requirements like CIP-007 R2 to non-prescriptive ones, I think they’re going to have a hard time getting this passed. I don’t think too many CIP professionals are out looking for another anxiety-producing requirement. They may prefer just to keep the prescriptive ones, simply because by now they know what the auditors want to see, and have put all that in place. A prescriptive requirement may require a lot of work, but at least there isn’t a lot of uncertainty with that work – the old story of sticking with the devil you know, rather than the devil you don’t know.

So what’s the solution to this problem? I’m not at all advocating that the SDT give up the idea of making CIP standards as non-prescriptive as possible, but I also don’t want to see more CIP teams having to order Maalox™ by the case load.

The ideal solution would be if the CIP enforcement process could move from one based on auditing to one based on the Regions and the entities working together to secure the BES. I gave some ideas of what I mean in this post, and you can be sure it will figure prominently in the book I’m writing. But I will be quite honest and say I see just about zero chance of NERC making radical changes in the CIP enforcement procedures anytime soon.

So is there a Plan B? In other words, is there a way that the SDT could develop non-prescriptive versions of current prescriptive requirements, that doesn’t also require changes to the enforcement process? I do have a Plan B, and it’s based on several observations.

First, non-prescriptive requirements are definitely the way to go. However, just being non-prescriptive isn’t enough.

Second, since CIP v5 was implemented, all subsequent requirements that have been developed (plus one requirement that was part of v5 itself) have been what I call “plan-based”. In these, the entity isn’t told to achieve an objective (which as I said above, is always unachievable and unmeasurable, when it comes to cybersecurity), such as “protect your assets against the threats attendant on use of Transient Electronic Devices and Removable Media”. Instead, they are told to develop and implement a plan to manage risks attendant on TEDs and RM. Completely protecting your assets against threats from TEDs and RM isn’t achievable or measurable, but developing and implementing a plan is definitely achievable. Does requiring a plan suffice to make a requirement auditable?

No it doesn’t. Just requiring a plan doesn’t make a plan-based requirement auditable in any meaningful sense. If the requirement just mandates a particular type of plan - say a supply chain cyber security risk management plan - an entity could put together a really minimal plan, but as long as the plan addressed supply chain cyber security risk management in some way, it would have to pass.[iii]

So how can a plan-based requirement be made auditable? The requirement (not just a “guidance” document) needs to provide a list of elements that must be included in the plan – and it’s best if these elements are threats that the plan should mitigate. CIP-010 R4 is my poster child for a good plan-based requirement. R4 reads “Each Responsible Entity…shall implement…one or more documented plan(s) for Transient Cyber Assets and Removable Media that include the sections in Attachment 1.”

Attachment 1 lists three types of devices that must be in scope for the plan, as well as between two and five “topics” (my term) that must be addressed for each of the three types. For example, under the Removable Media device type, there are two topics: Removable Media Authorization and Malicious Code Mitigation. Each topic lists 2-5 mitigations that must be included in the plan. For example, under Removable Media Authorization, the entity is required to authorize use of Removable Media (thumb drives) by both user and location. Under Malicious Code Mitigation, the user is required to scan the RM for malicious code before using it with a Medium or High impact BES Cyber System, as well as “Mitigate the threat of detected malicious code”.

You might ask, “How does CIP-010 R4 differ from a prescriptive requirement? It includes a number of steps you need to take; that sounds pretty prescriptive to me.” I’m glad you asked that question. Here’s how it differs:

1. The individual mitigations aren’t prescriptive. For example, the first mitigation under Malicious Code Mitigation is “3.2.1. Use method(s) to detect malicious code on Removable Media using a Cyber Asset other than a BES Cyber System or Protected Cyber Assets.” The methods aren’t specified, as they would be in a prescriptive requirement.
2. The “preambles” to the mitigations under most topics make clear that the objective in each topic is to mitigate a particular threat. For example, Section 3.2 reads “Malicious Code Mitigation: To achieve the objective of mitigating the threat of introducing malicious code to high impact or medium impact BES Cyber Systems and their associated Protected Cyber Assets, each Responsible Entity shall…”
3. Under most topics, it is made clear that the mitigations listed aren’t the only ones that can be used to address the threat in question. For example, the mitigations under Section 2.1 Software Vulnerabilities Mitigation, read “Review of installed security patch(es); Review of security patching process used by the party; Review of other vulnerability mitigation performed by the party; or Other method(s) to mitigate software vulnerabilities.”

To summarize, I definitely support the SDT’s idea to make certain prescriptive requirements non-prescriptive, in order to facilitate incorporating virtualization into the CIP requirements. However, this is how I would proceed with each requirement that is going to be rewritten (using CIP-007 R2 as an example):

a)      I would state its objective, based on threat mitigation. For CIP 7 R2, I would state the objective as “On a risk-adjusted[iv] basis, mitigate the threat of software vulnerabilities”.

b)      I would then probably refer the entity to an attachment (as CIP-010 R4 does), rather than try to include everything in the requirement itself. But the important thing is that the attachment has to be called out in the requirement; if that doesn’t happen, then the attachment is just another piece of guidance, with no special importance.

c)       The attachment might list a set of “sub-threats” to be addressed. Each sub-threat would be in its own section, followed by one or more suggested mitigations (but always saying in the last bullet point “Or any equally effective mitigation strategy”). In this case, one of the sub-threats might be “The threat of vulnerabilities in commercial software.” Probably the only mitigation listed would be “A patch management program, including patch source identification, patch discovery, patch assessment, patch application or mitigation plan development, mitigation plan implementation and regular review, etc.” Another sub-threat would be “The threat of vulnerabilities found in software developed by the entity itself”; in this case, one mitigation would probably be identification and implementation of a secure SDLC.

Unfortunately, I also have a poster child for a flawed plan-based requirement: CIP-013 R1. I used to think that this was the perfect plan-based requirement, because of its ZEN-like simplicity:

R1. Each Responsible Entity shall develop one or more documented supply chain cyber security risk management plan(s) for high and medium impact BES Cyber Systems. The plan(s) shall include: 

1.1. One or more process(es) used in planning for the procurement of BES Cyber Systems to identify and assess cyber security risk(s) to the Bulk Electric System from vendor products or services resulting from: (i) procuring and installing vendor equipment and software; and (ii) transitions from one vendor(s) to another vendor(s).[v]

Let me critique CIP-013 R1 in terms of the three criteria for a good plan-based CIP requirement, which I listed above:

a)      R1 starts out with a statement of the objective of the requirement, development of a supply chain cyber security risk management plan. Notice this doesn’t use the word “threat”, but since a risk management plan should certainly talk about threat mitigation, this is OK.

b)      R1 doesn’t refer to an attachment, but that’s probably because of c).

c)       While R1.1 does list three threats[vi] that need to be addressed in the plan, these are quite general. Each of these three threats (and especially the first one) will of course have many sub-threats. If each of these threats listed 5-10 sub-threats that must be mitigated in the plan[vii], then CIP-013 R1 would receive “Tom’s Seal of Approval for an Auditable Plan-based Requirement”.

So since CIP-013 R1.1 doesn’t list specific sub-threats that need to be mitigated in the supply chain cyber security risk management plan, it doesn’t get my coveted Seal of Approval – which means it really isn’t auditable, as long as the entity produces a plan that could be credibly called a supply chain cyber security risk management plan. I won’t belabor this point now, since I’m going to have a new post on CIP-013 soon.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC.

If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that if you’re a NERC entity, Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post – especially on compliance with CIP-013. And if you’re a security vendor to the power industry, TALLC can help you by developing marketing materials, delivering webinars, etc. To discuss any of this, you can email me at the same address.         

---

[i] I drafted a short history of NERC’s efforts to provide real guidance for ambiguities in CIP requirements in writing this post, but I’ve decided it doesn’t need to be in the current post, which is already very long. I’ll post it separately soon.

[ii] There is the Request for Interpretation process. In this process, an entity requests an interpretation; it is drafted and balloted (multiple times) by a NERC drafting team; then it is approved (or not, as happened with two CIP Interpretations in 2012) by FERC, and finally it’s implemented. But this takes almost as much time as the SAR route, and in any case an Interpretation can never modify the words of a requirement – just interpret the current wording. The auditing problems we’re talking about come to pass because the current wording of a requirement doesn’t give enough information to an auditor to audit it.

[iii] I’m being a little too absolute here. An auditor who was confronted with a minimal plan like this would still be justified in giving a PNC for this requirement, since they can always use “professional judgment”. If an entity has developed what is clearly an inadequate plan, the auditor can exercise that judgment to point out that it is grossly inadequate, and therefore doesn’t constitute a serious plan at all. However, if the entity has actually developed more than a minimal plan, but has still left out some topic that the auditor thinks should be in the plan, the auditor can’t give a PNC in this case, and if they do, it most likely won’t be upheld.

[iv] By “risk-adjusted”, I mean that the same controls don’t have to apply to all systems or devices in scope, regardless of the degree of BES risk each one poses. For example, a relay that operates the circuit breaker for a 345 kV line poses a higher risk to the BES than one which operates a 138 kV line. For CIP-007 R2, the entity might decide that a 30-day patching cycle is needed for the former, but a 60-day cycle is perfectly adequate for the latter (of course, when I talk about risk here, it has nothing to do with impact level. You could have BCS with three or five different levels of risk, all installed at a single Medium-impact substation, so the BCS would all be Mediums); this is, of course, not allowed in CIP-007 R2 now.

[v] Of course, R1.2 goes on to list six specific things that need to be included in the plan. But the plan doesn’t consist of these six things; the six things are there because they were ordered by FERC in Order 829. The heart of R1 is R1.1.

[vi] They’re called risks here, not threats. Since most people would do the same thing, I’m not going to vehemently object to this, but I’ve developed my own definitions of what threats, vulnerabilities and risks are and how they relate to each other. The book I’m writing now is carefully following these definitions. But since the CIP-013 SDT (and FERC in Order 829) is using a different idea of risk than mine, that’s fine. But I will keep pointing out that I don’t agree with that idea, since it leads to confusion when you start talking about the amount of risk that a threat poses (using the more popular idea of risk, you would have to start talking about the amount of a risk that a risk poses, which of course is very confusing).

[vii] Of course, each of the three threats will have a lot of serious “sub-threats”, perhaps hundreds. How will the SDT come up with a manageable list of around ten? One way to do this is to aggregate them into a higher level, and just list those higher-level sub-threats. Another is to just take the ten most serious sub-threats and list them – on the idea that the ten of them might account for most of the risk posed by the overall threat.