How will CIP-013 be audited?

I may have been a little premature. Recently, I wrote a post gushing over how great CIP-013 was because at heart it simply requires the entity to develop and implement a supply chain cyber security risk management plan, rather than being just a set of prescriptive requirements. True, CIP-013 does require that the plan include six particular items that are listed in R1.2, but at heart it just requires a plan. Since this is what a NERC entity – with sufficient funding – should do on its own, I think this is a much more efficient way of addressing supply chain security (or any security, for that matter). This is because a much higher percentage of the entity’s total spending on CIP-013 compliance will go to cyber security vs. pure compliance paperwork, than is true for CIP-002 through -011.

However, an auditor emailed me the next day to say that he thinks entities are just going to address the six things required by R1.2 and that’s it. He said he has seen it too often in the past: A CIP requirement will specify what is required “at a minimum”. Immediately, the lawyers will prevent the compliance people from doing anything beyond that minimum.

I was quite surprised to hear the auditor say this. After all, CIP-013 R1.1 states that the entity must develop and implement a supply chain cyber security risk management plan that addresses “cyber security risk(s) to the Bulk Electric System from vendor products or services resulting from: (i) procuring and installing vendor equipment and software; and (ii) transitions from one vendor(s) to another vendor(s).” It doesn’t just require that the plan include the six items in R1.2. So my question became (I’m paraphrasing here) “Yes, I understand why you expect that lawyers will prevent CIP compliance people from doing more than the minimum required. But clearly, R1.1 requires the plan to cover a lot more than just the six items. As an auditor, why would you let them get away with just addressing the six items in their plan?”

The auditor’s reply was “You are presuming that the auditor has greater knowledge of the risks and threats applicable to an entity than the entity does.  You are also assuming that the entity will not argue that the strict language of the Standard does not require anything more.  After all, the language of Part 1.2 says ‘One or more process(es) used in procuring BES Cyber Systems that address the following, as applicable...’  I can see entities arguing that the six procurement elements are the risk areas that they also have to plan for, although I have some (considerable) trouble mapping the six procurement elements into the two planning elements.”

The most important sentence in his response was the first one. In that sentence, I believe he’s saying “A NERC auditor is required to determine whether the entity fulfilled the strict language of the requirement, period. In order for me to determine whether the entity properly developed their plan, I would need to know all the risks and threats applicable to the entity. Moreover, I would have to know them better than the entity itself does. Otherwise, how could I say the plan was a good or bad one? Yet an outside auditor will clearly never know these things better than the entity itself does.”

  

I certainly couldn’t argue with this. And I’ll further paraphrase my paraphrase: As long as the CIP standards must be audited under the existing NERC Compliance Monitoring and Enforcement Plan (CMEP), there is simply no way an auditor will be able to determine whether a risk management plan, such as the one required by CIP-013, is valid or not. In other words, the only part of CIP-013 that is actually auditable under CMEP is R1.2.

In the week following this email exchange, I attended RF’s Fall CIP Compliance Workshop outside of Cleveland. There was a lot of good discussion about CIP-013 in the regular sessions, but the highlight for me was a long conversation with Lew Folkerth of RF after the workshop ended. I brought up my email exchange to him, and he completely agreed with the auditor: There is no way that any part of CIP-013 other than R1.2 can be strictly audited under the current CMEP.

But this time I followed up and asked Lew “Well, if you were an auditor, would you simply ignore all of the rest of CIP-013?” His answer was “No”. The audit team still can review the plan thoroughly, and if the team finds deficiencies it can issue an Area of Concern (as opposed to a Technical Non-Compliance finding, which can ultimately lead to a violation being assessed). The entity will be in no danger of ultimately being found to have violated CIP-013 R1.1 in this case, but they are well advised not to simply ignore the AoC. Instead, they should work diligently to rectify the deficiencies that the auditor found in the plan.

My takeaway – for NERC entities subject to CIP-013 - from these two conversations is “Don’t just focus your plan on the six items required by R1.2. Instead, make sure your plan addresses the three risk areas included in R1.1:

1. Risks from procuring vendor equipment and software;
2. Risks from installing vendor equipment and software; and
3. Transitions from one vendor(s) to another vendor(s).

You may not actually be liable for a violation if you don’t do this, but the auditors aren’t going to be very happy with you, to be sure. Besides, it’s the right thing to do.”

And you know what? I sincerely doubt there’s any NERC entity with Medium or High impact BES Cyber Systems (i.e. an entity that is subject to CIP-013 compliance) that won’t do their best to develop a good supply chain cyber security risk management plan (which I hereby christen a SCCSRMP[i] – remember, you heard it here first!).

At this point, you may wonder why I’m writing this post at all. After all, it seems there really is no problem, right? It doesn’t matter that most of CIP-013 isn’t strictly enforceable, as long as NERC entities act as though it is.

Well, there’s more to it than that. You may have figured out that NERC’s CMEP is the villain in this post. This document sets out a compliance regime based on prescriptive requirements – and that regime is quite adequate for all of the NERC standards except the CIP family. The non-CIP standards (referred to as the 693 or O&P standards) are ultimately based on the laws of physics; for example, if two control centers don’t communicate properly and one misunderstands what the other ordered and does the wrong thing, there can easily be a serious outage. I have no issue with requirements being prescriptive in the case of the 693 standards.

However, cyber security isn’t physics. In cyber, it isn’t a matter of doing or not doing particular things; it’s a matter of statistics. If entity A doesn’t patch one server in its Control Center for one month, it’s highly unlikely there will be a major BES outage the next month, as a result of that omission. On the other hand, if all of the major electric utilities in one region (and their RTO, for good measure) don’t patch any of their servers for a year, the likelihood of some sort of major disturbance now becomes much higher.

This means that trying to base cyber security regulation on prescriptive requirements is IMHO a completely losing cause. The entities lose because their compliance expenditures escalate rapidly (and my poster child for this is CIP-007 R2 patch management. One entity told me that half of their compliance costs in their control centers – for all NERC standards – are due to this one requirement). And the public loses because a large portion of those expenditures (which the public pays for, of course!) doesn’t do anything to advance the security of the power grid. Moreover, because developing new NERC standards has become a hugely cumbersome process, major cyber threats like phishing and ransomware are nowhere addressed by CIP (for a discussion on these topics, see these posts: here, here, and here).

I used to think that rewriting CIP in a non-prescriptive (and risk-based and threat-based) format would fix these problems, but for a while now I’ve realized that something more is required. That something is a revision of CMEP, or even better a separate CMEP that applies just to CIP. If CMEP were rewritten as I would like, CIP-013 would suddenly become completely auditable. That is, it would actually be possible for an auditor to find an entity in violation if their SCCSRMP is hugely deficient, and even more importantly if they won’t take the steps needed to bring it up to snuff. Moreover, were the other CIP standards to be rewritten in a manner that emulates CIP-013, they would be completely auditable as well.

Would you like to know how I would rewrite CMEP (at a high level, of course)? I didn’t think so, but I’m going to tell you anyway. Here’s how I think a CIP-only CMEP should be written:

1. First, the CIP standards need to be rewritten so they are all in the general form of CIP-013: develop a risk management plan, implement it, and review it annually.
2. After the entity develops the plan, they need to submit it to their NERC region for review (this will require a large increase in staff or contractors for each region, I’ll admit. They will have to be true cyber security professionals, and will need to receive a lot of training in the ways of NERC and the electric power industry, assuming they don’t have that experience when they’re hired).
3. The review of the plan isn’t an audit like what happens today. Rather it’s a collaborative effort between the reviewers (who may still have the title Auditor) and the entity, to see if the plan needs improvement - and if so how best to do that. After the review, the reviewers will give the entity a document outlining the steps it needs to take to improve the plan.
4. The entity revises the plan and re-submits it. Assuming they have addressed the problems, the reviewers/auditors give them the green light to implement the plan.
5. The entity implements the plan. Six months (or so) later, the auditors come in to judge how well the plan has been implemented. If they think the entity has done a good job, they issue them a passing grade. The entity then needs to maintain compliance with the plan they implemented.
6. If the auditors think the entity has tried to implement the plan, but has fallen short in one way or another, they will describe what the entity did wrong and schedule another audit for say six months later to see if they’ve corrected the problems.
7. However, if the auditors think the entity hasn’t really taken all of this seriously, and hasn’t really tried to implement the plan in an effective manner, then – and only then – will they consider assessing a violation.

Doing all of this will of course require a big increase in NERC’s budget, as well as a huge drafting effort. Is NERC likely to do this? I really don’t know. However, I do think that in the not-too-distant future NERC will face the question of whether they want to change or whether they want to give up regulation of grid security. I don’t think the ship will wait for them forever. The grid is too important.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

---

[i] The auditor and I have had a good discussion of how to pronounce this acronym, or rather how to write the pronunciation. He says it should be written “Scuzzy-rump”. I harken back to the early days of the PC, when the SCSI interface was ubiquitous, so I prefer to write it “SCSI-rump”. In any case, the pronunciation is the same. And since the auditor has decreed it, he’s likely to issue an Area of Concern if he hears you pronouncing it differently