Why the Canary Died

In early January, I wrote a post comparing the two CIP Standards Drafting Teams currently operating: the CIP Modifications SDT and the Supply Chain Security SDT. In that post, I made the argument that one team (the Supply Chain team) is likely to achieve their goals fairly easily and on time, while the other team isn’t likely ever to fully achieve their goals.

It is now time to update that observation; I have good news and bad news. The good news is I no longer see such a disparity between the two teams. The bad news is I am now quite sure that both of these teams won’t fully achieve their goals.

In fact, for the Supply Chain team, it is just about certain they won’t achieve their goal, which was to draft and get approved – by the NERC ballot body, the NERC Board of Trustees, and finally by FERC – probably the first non-defense mandatory supply chain security standard in the history of the universe.[i]

As I explained in a recent post, it is now very possible that CIP-013, the draft Supply Chain Security standard (along with related changes in three other CIP standards), will never be approved by a NERC ballot, and that NERC will for the first time have to invoke the Section 321 process in order to have a CIP-013 standard ready for FERC’s approval in September[ii] (of course, it is almost 100% certain that FERC won’t have a quorum in September, since they don’t have one now and no new Commissioners have even been nominated yet. This means the standard will probably just sit on their desk for many months, if not for a year or more, once it does get delivered to FERC. But that’s another story). How did this situation come to pass?

For the Supply Chain SDT, the post just linked will give you a good summary of why they likely won’t achieve their goal. So the question is: What has changed since January, when I was very optimistic that CIP-013 would quickly cruise to approval (at least on the second ballot. Approval on the first ballot rarely happens in NERC, and has never happened for any of the CIP standards)?

I admit that when I wrote the January post – which was before the first draft of CIP-013 was even posted for comment, let alone balloted – I assumed that everyone would love a non-prescriptive standard. After all, non-prescriptive, objective-based requirements allow the entity to determine for itself what is the best means of achieving the stated objective. Why wouldn’t NERC entities want that?

The two examples I use most often to illustrate the difference between prescriptive and objective-based requirements are CIP-007 R2 and CIP-007 R3. R2 (patch management) is very prescriptive; I know that many entities are spending large amounts of resources in trying to comply with it (and even then they aren’t able to be 100% compliant, as this post shows). On the other hand, R3 (malware protection) is very non-prescriptive, simply saying that the entity should “Deploy method(s) to deter, detect, or prevent malicious code” (R3.1) and “Mitigate the threat of detected malicious code” (R3.2).[iii]

There has been a large amount of grumbling about R2 – and as I’ve said in another post, one entity told me that half of all the compliance documentation they’re producing in their control centers is tied to that one requirement – vs. almost no grumbling (that I’ve heard of) about R3. This is especially true given that R3 replaced the anti-malware requirement in CIP v3, which required entities to take a Technical Feasibility Exception for every switch, router, PLC etc. that wouldn’t take antivirus software. Given that NERC entities seem to prefer non-prescriptive requirements once they’ve started using them, I just assumed people would be quite happy with the first draft of CIP-013, where all of the requirements were much more like CIP-007 R3 than CIP-007 R2.

However, the first draft of CIP-013 when down to ignominious defeat, garnering a whopping 10% positive vote. While the comments revealed many reasons for the overwhelmingly negative vote, one important theme was that many CIP compliance professionals at NERC entities don’t trust the auditors to be able to audit non-prescriptive requirements; they want to have the “safety” of requirements for which there can be no question how they can be audited – i.e. prescriptive requirements. This is usually expressed with a statement to the effect that the auditors shouldn’t be allowed to exercise “discretion”.

I wrote about this phenomenon in another recent post, where I pointed out that CIP auditors are already required to use a lot of discretion in their CIP audits, especially since v5 became enforceable. In effect, I said (although I didn’t explicitly make this point) that it would require absolutely perfectly-written requirements (and definitions) for the auditors not to need to use any discretion at all. Furthermore, I said the auditors currently have to use discretion mostly about things they’re not trained on, such as how to interpret the meaning of ambiguous or missing terms in quasi-legal definitions. Wouldn’t it be better to remove the need for them to make these kinds of judgments (by making the requirements non-prescriptive) and require them to instead exercise judgment in an area they do understand, namely cybersecurity – by only having to audit non-prescriptive CIP requirements?

To be honest, that post essentially said to the people who are worried about auditor discretion “You’re not being rational. Get over it.” I now realize that it would have been much more productive to assume these people are rational (which they of course are, especially considering experiences some entities had with auditors under CIP v3) and ask what could be a rational basis for their fears.

After much thought, I’ve decided the answer to this question is that, even if all of the CIP requirements were made non-prescriptive (objectives-based) tomorrow, the overall compliance regime for CIP – and for the other NERC standards – remains very prescriptive. So even though all the requirements in a new standard like CIP-013 are non-prescriptive, the fact that the overall compliance regime remains the same old prescriptive one will almost inevitably lead to fears – grounded or not - that the auditors will have a tendency to fall back to their old ways and audit non-prescriptive standards as if they were prescriptive ones. There needs to be a different compliance regime.

Before I describe the new compliance regime I’m proposing, let me describe how I view the current NERC compliance regime:

1. NERC standards are designed to protect the Bulk Electric System from cascading outages caused by errors of omission or commission on the part of electric utilities and other participants in the BES.
2. The requirements in those standards dictate certain actions which are either required or prohibited.
3. If an entity is found to have violated one of those requirements – that is, they have either not performed a required act or performed a forbidden act – they are subject to fines and need to implement controls designed to prevent recurrence of the violation.[iv]

I want to say now that I am not criticizing this regime, which has been in place since NERC was founded in 1967.[v] In fact, I’ll certainly stipulate that it’s the only compliance regime that could successfully accomplish NERC’s original purpose. The problem is that cyber security is very different from the reliability issues addressed by the original NERC standards, as well as what are called the NERC 693 standards today. No matter how many times NERC officials may swear on a stack of Bibles that the CIP standards are reliability standards just like all of the other ones, that doesn’t change the fact: cyber security is very different from security from cascading BES outages. The compliance regime for CIP can and should be different from the regime for the other NERC standards.[vi]

What should this new regime look like? In my last post, I listed six principles that any regime of mandatory cyber standards for critical infrastructure should follow. Since that post was nominally about regulation of natural gas pipelines, I will repeat them here:

1. The process being protected needs to be clearly defined (the Bulk Electric System, the interstate gas transmission system, a safe water supply for City X, etc).
2. The compliance regime must be threat-based, meaning there needs to be a list of threats that each entity should address (or else demonstrate why a particular threat doesn’t apply to it).
3. The list of threats needs to be regularly updated, as new threats emerge and perhaps some old ones become less important.
4. While no particular steps will be prescribed to mitigate any threat, the entity will need to be able to show that what they have done to mitigate each threat has been effective[vii].
5. Mitigations need to apply to all assets and cyber assets in the entity’s control, although the degree of mitigation required will depend on the risk that misuse or loss of the particular asset or cyber asset poses to the process being protected.
6. It should be up to the entity to determine how it will prioritize its expenditures (expenditures of money and of time) on these threats, although it will need to document how it determined its prioritization.

I hope you can see why the fact that the three principles of the current NERC regime still apply to enforcement of all CIP standards, including CIP-013, means that NERC entities are rightly going to be suspicious of any standard, no matter how non-prescriptive its wording, that will be enforced under that regime (and if you can’t, well, it’s late at night now and I’m tired from writing this post – almost as tired as you are from reading it! I’m sure I’ll come back to this topic in future posts, and at least in an upcoming book, which – although I have yet to put a single word on paper for it – has already been at least half written in posts like this one).

And this, folks, is the canary in the coal mine, which has just keeled over dead: It’s clear (to me, anyway) that no new compliance area (like supply chain or virtualization) can be incorporated into CIP unless this is done with non-prescriptive requirements. Yet it is also now clear to me that no new non-prescriptive standards (or requirements) will ever be freely accepted by the NERC membership until the CIP compliance regime itself has changed. I will be watching to see what happens.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

---

[i] Well, at least of the solar system. With so many possibly habitable planets outside of the solar system, it’s almost inevitable that on at least one of them such a standard has already been developed, although I will guess that on at least one other planet the effort to develop such a standard has collapsed in confusion and recrimination. And we’re not even talking about the uncountable number of alternative universes, where it’s almost inevitable there is not only at least one planet with a continent called North America and an organization called the North American Electric Reliability Corporation, but further that this organization has a CEO named Gerry Cauley. Such is the power of having an infinite number of instances, with no possible means of verification for your assertions.

[ii] As the post just referenced points out, there is the possibility that a large push by a major industry trade organization will put the next (and last) ballot over the top, despite the fact that so many of the yes votes will be by entities that strongly opposed the first draft and still have substantial misgivings about the second. Since this would technically count as a ballot victory for CIP-013, some might say that this constituted the SDT’s fulfillment of their mission. But I think the mission of the SDT is to develop a standard that will win the outright approval of 68% of the ballot body, not a grudging approval based on the idea of “Well, it’s either this or have the Standards Committee write the standard, and go against the strong wishes of our trade organization and our CEO”.

[iii] There is also a third requirement part (R3.3) that states the entity should “For those methods identified in Part 3.1 that use signatures or patterns, have a process for the update of the signatures or patterns.” Even this part is non-prescriptive, of course, since it also simply states an objective and requires the entity to have a process to address the objective.

[iv] You may wonder where Risk-Based CMEP (formerly known as the Reliability Assurance initiative) fits in with the three principles of NERC compliance I’ve just listed. I haven’t thought about this too closely yet, but I will point out that the risks addressed by RAI are compliance risks – i.e. the risk that an entity will violate a NERC requirement. You can think of RAI as kind of an overlay to the three principles of the NERC compliance regime; it doesn’t modify them, but it’s designed to lower risk of violations, and perhaps of cascading outages caused by a NERC entity’s doing or not doing something. It doesn't at all change the prescriptive nature of the current NERC CIP compliance regime.

[v] Of course, NERC standards weren’t mandatory until 2007, but I believe the principles of compliance were the same before and after that date.

[vi] The question may now occur to you: Is Tom advocating that the CIP standards be removed from NERC’s purview? I’m not necessarily saying that, although I reserve the right to change my mind on that point in the future. It seems to me that NERC has already put in place much of what is needed to have a successful program for regulating the cyber security of the BES, not the least of which is a large team of auditors – some of whom are very competent – that understand both the cyber and electrical issues involved with auditing compliance with mandatory cyber regulation of the BES. What isn’t in place now is a compliance regime that would be very different from the current CMEP, which is described by the six principles listed in this post. I believe that technically speaking this regime could be built within the NERC organizational framework – you might have to have two CMEPs, for example. However, I’m not sure that politically this will be possible within the NERC organization itself; there are few organizations that are flexible enough to undergo such a wrenching change. I certainly hope NERC proves to be the exception to this rule, since I believe there is no alternative to implementing the changes to the CIP compliance regime that I’m proposing. But if NERC can’t do it, some other organization will have to.

[vii] My eyes were opened at the RF CIP workshop this past week, when Lew Folkerth pointed out that the key to being able to audit non-prescriptive requirements is for the entity to have to demonstrate that the measures they took were effective. I will do a post on this soon.