Some Heretical Thoughts

Note: This post originally started out to be part II of II on lessons learned so far from the CIP-014 experience – that post covered the first lesson, and this post was intended to cover the second lesson. I started this post with what I thought would just be a paragraph or two expanding on what I’d written in the first post – and you can see what happened. But have no fear, I have most of the post regarding the second lesson already written, and it will appear very shortly.

Two weeks ago, I wrote a post that was intended to be the first of two, on lessons that can be learned from NERC’s experience so far with CIP-014, the physical security standard that applies to certain key substations. Why am I worried about CIP-014, since I’ve written very little about it previously? Because CIP-014 is an objectives-based (non-prescriptive), risk-based standard like CIP-013, the upcoming supply chain security standard[i]; any lessons that can be learned from the CIP-014 experience so far are very relevant to CIP-013. And I also think the other CIP standards should be replaced by objectives-based, non-prescriptive ones, so these lessons will be relevant when that long-hoped-for (at least by me. I’m not sure if anyone else hopes for them, although I believe I’ll be able to persuade my dog in the near future) day arrives.

My main concern regarding objectives-based, risk-based standards is how they will be enforced by NERC. I realized earlier this year that simply rewriting the CIP standards isn’t enough. Because the whole NERC compliance regime, as embodied in the Compliance Monitoring and Enforcement Plan aka CMEP, is based on auditing very prescriptive standards, to properly enforce objectives-based, risk-based standards will require a huge adjustment in the mindset of the auditors, as well as everybody else involved in enforcement at NERC and the regions. And it will probably require a rewrite of CMEP, or perhaps a separate CMEP for the CIP standards, vs. the other NERC standards.[ii]

The lesson from CIP-014 that I wrote about two weeks ago was that there really needs to be some sort of review by NERC or the regions of the entity’s plan, before they implement it. In the case of CIP-014, this is the plan for improving physical security of a key high-voltage substation. In the case of CIP-013, it is the entity’s Supply Chain Cyber Security Risk Management Plan. In that post, I discussed how a large NERC entity would probably never implement a planned $80MM physical security system at their key substations, because the region refused to say whether this deployment would help (or hurt) their compliance position on CIP-014.

Of course, having NERC (or in this case, the Regional Entity) review the plan before it is implemented goes against everything in CMEP – plus, as an auditor pointed out to me after he read that post, it also goes against GAGAS, the government guidelines for auditors that all NERC auditors follow. According to those two documents, auditors need to maintain their independence, and can’t provide compliance advice in advance to the entities they will audit. And I totally agree with them that this is the case.

Yet this also poses a huge problem for objectives-based, risk-based standards. Think about it: In a world of prescriptive, non-risk-based requirements (as is basically true for CIP v5/v6, although there are a number of requirements that are objectives-based, or at least non-prescriptive. On the other hand, there are no requirements in CIP v5 or v6 that are risk-based), the auditors absolutely shouldn’t be providing compliance advice up front to the entities they audit.[iii]

But objectives-based, risk-based standards don’t lend themselves to the type of audits that prescriptive standards do; in fact, it’s hard to see how the word “audit”, in the normal sense of somebody with a clipboard going through and checking a Yes or a No box after each of a set of questions, of the form “Did the entity do X?” is at all applicable for these standards. In CIP-013 and CIP-014, the entity is simply required to develop and implement a plan of some sort. The only criteria by which they can be judged to have complied or not are the two questions: 1) Did they develop a good plan? and 2) Did they implement the plan well? How would it be possible for an auditor to answer either of those questions except by using a tremendous amount of judgment, no matter how informed that judgment is by a host of guidance emanating from NERC, the regions, and other parties (and right now, of course, there is very little official NERC guidance on CIP-013 and not enough on CIP-014)?

Strictly speaking, if NERC were going to seriously audit CIP-013 and CIP-014 according to CMEP and GAGAS, they would have to make the whole enforcement process a giant gamble. The entity would have to, completely on their own or at least relying on no guidance that comes directly from NERC or the regions, come up with their own ideas on how to develop and implement their plan, and then how to implement it. At the end of this whole process, they would be subject to a single audit, where the auditors would just check Yes or No for each of the two questions in the previous paragraph. The fate of the entity’s entire CIP-013 and CIP-014 compliance programs would rest entirely on whether the auditors checked Yes or No for these two questions. If you think you’re under pressure before your CIP audit now, just imagine what this would be like!

I point this out not because I think this scenario will come to pass, but because I’m sure it won’t. It simply makes no sense to do this. How does it make sense not to provide any guidance to entities as they develop and implement their plans? What possible good would be achieved by not doing this? Just think of the example from the first post in this series: the utility that probably won’t make a particular big physical security investment, since they don’t know whether it might help or possibly even hurt their chances of being determined compliant with CIP-014? And I recently met with a utility that said they were having a hard time getting management at all interested in starting to prepare for CIP-013 compliance (despite the benefits of doing so now) because there is very little real guidance from NERC or the regions about how to develop and implement a supply chain cyber security risk management plan (more on this in a subsequent post).

Face it, objectives-based (and probably risk-based) CIP standards are here to stay. Since CIP v5, all of the new CIP requirements and standards that have been developed (this includes CIP-010-2 R4, CIP-003-6 and CIP-003-7, CIP-014, and CIP-012) have been objectives-based. This is partly because FERC ordered both CIP-013 and CIP-014 to be objectives-based (although FERC’s term for it was different – something like “not one-size-fits-all”), but also because nobody on a NERC CIP standards drafting team nowadays has an appetite for developing prescriptive standards (as I found out when I attended what turned out to be a key SDT meeting in June of 2016). The problem of a mismatch between NERC CIP standards and the NERC auditing regime is only going to get worse as time goes on.

So how will this mismatch be resolved? In the longer run, it will be resolved both by rewriting the CIP standards and by hopefully developing a separate version of CMEP for the CIP standards (since the current CMEP is fine for the O&P standards). That will of course be a huge, wrenching change for NERC and the regions. You might thus dismiss this as a pipe dream, and I would agree with you if I thought NERC had another option; but in the long run I don’t think they do. There is pressure building among the public and in Congress for the government to take more steps to secure the electric grid from cyber attack (and I wrote about how this pressure might come about in this  post)[iv]. I believe that in the next few years NERC is going to be faced with the choice of either developing a sustainable set of mandatory cyber security standards, and a sustainable compliance framework to go with them, or the responsibility for cyber security of the Bulk Electric System will be taken away from NERC (and FERC) and invested in some other agency like the Department of Energy, the Department of Homeland Security, or even a proposed Department of Cyber Security.

But in the short run, it is clear to me that the problem of the current CMEP not allowing CIP-013 and CIP-014 to be properly enforced will lead to the result that….they won’t be “enforced” at all. Auditors will review both the entity’s plan and how the plan was implemented; then they will provide advice on how the entity could improve on what they did. But this advice won’t be in the form of a Potential Non-Compliance finding (which can lead to an actual Violation), but rather in the form of an Area of Concern finding. As you probably know, AoC’s are issued now whenever an auditor finds some security practice that they think is deficient, but which is not actually in violation of any CIP requirement. The entity is under moral pressure to correct the problem, but they can’t be found in violation if they don’t.

Am I complaining about the fact that CIP-013 and CIP-014 aren’t really “enforceable”? No, I think this is much preferable to the doomsday audit scenario I described above.  But I also think there have to be mandatory CIP requirements that are actually enforced; I discussed my reason for saying this in another recent post. In brief, NERC entities will simply not get the same budgets for cyber security spending if their managements begin to perceive that the standards are increasingly being enforced on a “voluntary” basis. There really should be substantial monetary penalties when an entity clearly develops an inferior plan for CIP-013 or CIP-014, and/or doesn’t implement it properly or at all. Until a new compliance regime is in place at NERC, this simply can’t happen.

I wish to close with one of my favorite quotes from Lewis Carroll (from Through the Looking Glass, the companion book to Alice in Wonderland). Humpty Dumpty has just used a word in a very non-normal way. Alice objects that he isn’t using the word in the proper way.

'When I use a word,' Humpty Dumpty said, in rather a scornful tone, 'it means just what I choose it to mean — neither more nor less.'

'The question is,' said Alice, 'whether you can make words mean so many different things.'

'The question is,' said Humpty Dumpty, 'which is to be master — that's all.'

I would rephrase Humpty Dumpty’s response as “The question is, which is to be master – you or the words?” And so I ask you, which is to be master in the case I just described, CMEP and GAGAS, or the need to have an enforcement framework that is appropriate for objectives-based, risk-based standards like all of CIP should be and like CIP-013 and CIP-014 are now? I promise I’ll have a lot more to say on this in the future. You have been warned.

The views and opinions expressed here are my own, and do not reflect those of any organization I work with. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.

---

[i] In general, I think CIP-013 handles risk better than CIP-014, but that may be because the subject matters of the two standards are so different: purchasing policies and procedures vs. physical security of certain high-voltage substations.

[ii] My general idea is that there is a very big difference between the CIP cyber security standards and the other NERC standards, which are usually called the Operations and Planning (O&P) or “693” standards. The latter are ultimately based on the laws of physics: If you do or don’t do X, Y will happen. This type of standard has to be prescriptive, and needs to be audited in a very prescriptive manner.

But cyber security practice is statistical. If you don’t patch one server for one day, it’s unlikely you’ll suffer adverse consequences. If you don’t patch any of your servers for one year, you will be much more likely to be successfully attacked, but even then it isn’t a certainty. However, there would undoubtedly still be some small security benefit to patching your servers every day. Does this mean that CIP-007 R2 (my poster child for a prescriptive, non-risk-based requirement) should be expanded so that NERC entities now will need to patch their servers daily? Most NERC entities – and auditors – would shudder at the mere thought of that, since it would require a massive effort and many more resources. Yet, in the prescriptive framework of NERC standards, with no allowance for risk at all, there really is no way to say what the stopping point should be – that is, on a risk-adjusted basis, the point at which any possible security benefits are outweighed by the cost. The only logical stopping point would be when there were so many prescriptive CIP standards in place that the risk of cyber attack was effectively zero. Even if that point could ever be reached – which it can’t, of course – if we got there, we would have long ago passed the point where electric utilities would have to suspend distribution of electric power and devote their entire staffs to NERC CIP compliance!

Thus, I think cyber security standards in general need to be objectives-based and risk-based. And to be honest, I don’t know of any other mandatory cyber security standards that are prescriptive and non-risk-based, like CIP-002 through CIP-011 (although there are certain individual CIP requirements like CIP-007 R3 that are actually objectives-based. However, until CIP-013 was developed, none of the CIP standards or requirements have been explicitly risk based, meaning that all systems in scope don’t need to be treated exactly the same way, but the controls applicable to them can be based on the risk they pose to the BES).

A few readers may remember that I started saying about a year and a half ago that I was working – with two co-authors – on a book about how the CIP standards can be rewritten to make them much more sustainable than they are now. I confess that we haven’t put many words on paper (or virtual paper) yet. This is mostly because my ideas were still evolving – for example, my realization mentioned above, that the enforcement program itself will need to be rethought, not just the standards themselves.

My ideas are now at a point where I think I have the whole argument in my head, at least in principle (and a lot of the argument can be found in different posts I’ve written. However, it’s in a few sentences or phrases here and there, not in any form where I can copy and paste them into the book); and I have started serious writing. This, coupled with the fact that I will have more free time for the next couple months or so, makes me hope that I will make some serious progress on the book in the near future, although I strongly doubt I’ll be able to finish it that quickly.

[iii] If you read any of my posts from say 2014 through the end of 2016, you may be surprised with this statement. Those posts coincide with the period when the industry (and I) was struggling to figure out what the wording of CIP v5 meant. I frequently suggested that NERC and the regions should go beyond what was strictly allowed by CMEP and the NERC Rules of Procedure and provide true interpretation guidance to entities, simply because there was so much in CIP v5 that was undefined, vague, or – in the case of CIP-002 R1 and Attachment 1 – completely contradictory. Ultimately, both NERC (through the Small Group Advisory Sessions) and the regions (through unofficial advice that is always provided verbally, never written down), ended up doing exactly that, not because I’d advised it but simply because it was the only thing they could do. They couldn’t continually tell the entities “You have to figure it out for yourselves. We can’t tell you anything”, and then expect the entities to meekly submit when they are assessed violations that are due to their misunderstanding what a requirement means (or at least what the region thinks it means).

So my statement that I agree that auditors shouldn’t be providing compliance advice to the entities they audit, in a compliance regime of prescriptive, non-risk-based requirements, only applies in the case where the requirements are very clear and unambiguous. This unfortunately is not the case with CIP v5, although I need to add that I don’t blame this on the drafting team. I blame it on the fact that cyber security is a discipline in which unambiguous prescriptive requirements simply cannot be drafted. A full discussion of why this is the case will probably have to wait for my book.

[iv] And I’m not taking any position on whether or not that pressure is warranted, just stating that it exists and it’s inevitably going to grow.