How about CIP-013? Is that Auditable?

My last two posts have been about plan-based CIP requirements and standards. This isn’t an academic question, since it seems clear that these are the wave of the future. Every important new standard or requirement developed since CIP version 5 has been plan-based (meaning: The entity isn’t required to perform certain actions or to achieve a certain objective, but to develop and implement a plan to achieve the objective). I doubt this will change anytime soon.

My last post was about how, and even whether, a plan-based CIP requirement or standard can be audited. Although I didn’t state this specifically, my answer essentially was that it depends on how the requirement is written. The requirement for a plan needs to include criteria for topics that need to be in the plan; the auditor can examine the plan to see if it has addressed those topics. An entity that doesn’t include one of the topics would potentially be non-compliant.

But the fly in the ointment here is the level of detail that is provided in the criteria. For example, if a criterion just reads “user authentication”, there are lots of ways that could be addressed, including just requiring an unchanging four-character password. So the criterion might be “strong user authentication”, and there might be a separate guidance document providing examples of what this would be. With this, it would be very hard for an entity to argue that a four-character password is passable.

Of course, at the same time you don’t want to provide so much detail in the criteria that they become prescriptive requirements of their own. If you say the plan must cover “strong authentication using passwords that contain letters, numerals and characters and are changed every 30 days”, you have definitely just done that. And as my original post in this series discussed, once you provide that level of specificity in the criteria, you’re virtually inviting the auditors to require that you document every instance of when you have complied with those criteria. And then you lose the whole benefit of having the requirement be plan-based in the first place.

I concluded my last post by looking at CIP-014, which is definitely a plan-based standard. I went through each of the requirements and said whether I thought it was auditable – and they all were. The requirement for the actual plan (called the physical security plan, since CIP-014 is a standard for physical security of certain key substations) is R5, which lists four criteria that must be addressed in the plan:

5.1. Resiliency or security measures designed collectively to deter, detect, delay, assess, communicate, and respond to potential physical threats and vulnerabilities identified during the evaluation conducted in Requirement R4. 

5.2. Law enforcement contact and coordination information.

5.3. A timeline for executing the physical security enhancements and modifications specified in the physical security plan. 

5.4. Provisions to evaluate evolving physical threats, and their corresponding security measures, to the Transmission station(s), Transmission substation(s), or primary control center(s).

I like these criteria because they are quite comprehensive – you could say they’re mini-plans in themselves. 5.1 essentially says that the physical security plan needs to tell how the entity will mitigate all of the threats and vulnerabilities identified in the assessment required by requirement R4. R2 says the plan needs to cover law enforcement coordination. R3 says the plan must include a timeline for executing all this. R4 says there must be provisions in the plan to evaluate new or changing physical security threats, as well as new or changing security measures that are available to address those threats.

I feel these criteria provide enough specificity to be auditable, but certainly not enough so that these would suddenly become four prescriptive requirements that would need to be audited as such – with documentation of every particular instance, available for the auditors on demand.

But this post is really about CIP-013. Is that auditable? Actually, this is a question I asked in a previous post. Now I want to ask the question again, in light of the framework for answering the question that I laid out in my last post.

CIP-013 R1 is the requirement to develop a supply chain cyber security risk management plan. Here is the entire requirement:

R1. Each Responsible Entity shall develop one or more documented supply chain cyber security risk management plan(s) for high and medium impact BES Cyber Systems. The plan(s) shall include: 

1.1. One or more process(es) used in planning for the procurement of BES Cyber Systems to identify and assess cyber security risk(s) to the Bulk Electric System from vendor products or services resulting from: (i) procuring and installing vendor equipment and software; and (ii) transitions from one vendor(s) to another vendor(s).

1.2. One or more process(es) used in procuring BES Cyber Systems that address the following, as applicable:

1.2.1. Notification by the vendor of vendor-identified incidents related to the products or services provided to the Responsible Entity that pose cyber security risk to the Responsible Entity;

1.2.2. Coordination of responses to vendor-identified incidents related to the products or services provided to the Responsible Entity that pose cyber security risk to the Responsible Entity;

1.2.3. Notification by vendors when remote or onsite access should no longer be granted to vendor representatives;

1.2.4. Disclosure by vendors of known vulnerabilities related to the products or services provided to the Responsible Entity; 

1.2.5. Verification of software integrity and authenticity of all software and patches provided by the vendor for use in the BES Cyber System; and

1.2.6. Coordination of controls for (i) vendor-initiated Interactive Remote Access, and (ii) system-to-system remote access with a vendor(s). M1

Let’s look at this. R1 itself simply says the entity has to develop the plan. R1.1 describes the objective of the plan. It is to identify and assess[i] three types of risks:

• Risks from procuring vendor equipment and software;
• Risks from installing vendor equipment and software; and
• Risks from transitions between vendors.

Of course, it’s important to state the objective of the plan, but these aren’t criteria for what should be included in the plan.

R1.2, however, lists six criteria that have to be in the plan. These are exactly the specific items that FERC ordered when they approved Order 829 in 2016. They are certainly specific, and in my opinion they should all be auditable. However, does this mean that R1 itself is auditable? No, I don’t think so. This is because the three objectives of the plan (from R1.1) are much more comprehensive than just these six items. In fact, the six items all relate to just one of the three plan objectives: risks from procuring vendor equipment and software.

So is CIP-013 R1 auditable, meaning that it provides specific criteria that need to be addressed in the plan? Let’s look at each of the three plan objectives in R1:

• For procurement risks, there are six specific criteria. These are auditable, but there are many other procurement risks which also should be addressed[ii] in the plan. For example, there’s the risk that the vendor product, when shipped to the entity, will contain malware right out of the box. There’s the risk that information about the configuration of devices which have been installed on your site will be stolen from the vendor (which has happened in North America, more than once). There’s the risk that a vendor employee will inadvertently disclose information about your equipment, or its configuration, on social media. These should all be at least considered in the entity’s plan.
• For installation risks, there are no criteria.
• For risks of transitions between vendors, there are no criteria.

Now we can ask the question whether CIP-013 R1 is auditable. Out of three risk areas that are supposed to be addressed in the plan, there are specific criteria provided for only one of those areas, and not even comprehensively for that area. Overall, I would say R1 isn’t auditable.

How about CIP-013 R2? That requirement reads (in its entirety) “Each Responsible Entity shall implement its supply chain cyber security risk management plan(s) specified in Requirement R1.” What would be required to make R2 auditable? Let’s go back to the last post, where I discussed CIP-014. Requirement 5 of that standard mandates that the entity “develop and implement” the physical security plan. Moreover, it says that the plan should be “executed according to the timeline specified in the physical security plan(s).” In other words, if in CIP-013 the supply chain cyber security risk management plan from R1 provides a specific timeline, then the entity’s implementation of that plan in R2 should be auditable.

But there is a big difference between CIP-014 and CIP-013 in this regard. One of the criteria for what should be covered in the plan in CIP-014 is that there should be “A timeline for executing the physical security enhancements and modifications specified in the physical security plan.” (CIP-014 R5.3) So it’s almost certain that the entity will have a timeline in their plan. In CIP-013, that criterion isn’t there. One would hope the supply chain plan from R1 would include a timeline, but it might not.

More importantly, since there aren’t criteria covering most of what should be in the supply chain plan, the plan itself for the most part isn’t auditable, as we’ve just said. It’s not at all clear to me that the implementation of the plan would be auditable if the plan itself wasn’t. So I’m going to say I’m undecided on whether CIP-013 R2 is auditable or not.

How about R3? It reads “Each Responsible Entity shall review and obtain CIP Senior Manager or delegate approval of its supply chain cyber security risk management plan(s) specified in Requirement R1 at least once every 15 calendar months.” Is this auditable? Not in any meaningful sense. Yes, the auditor can verify that the right person signed off on the plan (whether revised or not) after 15 months. But that surely isn’t why this requirement is here.

I wrote a post about this requirement this summer. I pointed out there that, while the requirement itself seems close to meaningless (it just requires a review of the plan. It doesn’t provide any criteria for that review, nor does it require the entity to revise the plan if they find something needs to be added to or changed in it), it was actually different in the first draft of CIP-013, where it read (at the time it was R2, not R3):

R2. Each Responsible Entity shall review and update, as necessary, its supply chain cyber security risk management plan(s) specified in Requirement R1 at least once every 15 calendar months, which shall include:

2.1. Evaluation of revisions, if any, to address applicable new supply chain security risks and mitigation measures; and

2.2. Obtaining CIP Senior Manager or delegate approval.”

Here you see there are actual criteria for the review of the plan. Specifically, the entity needs to address new supply chain security risks, as well as new mitigation measures for those risks (in fact, this language is similar to the language in CIP-014 R5.4, which I quoted above. In CIP-014’s case, this language became one of the criteria for what needs to be in the plan, while in CIP-013’s case – at least in the first draft – it appeared as a separate requirement).

But the fact is that this language wasn’t in the final version of CIP-013-1, so it doesn’t officially have any status. This isn’t even mitigated by the fact that the Implementation Guidance prepared by the SDT does provide some good criteria for review of the plan (page 9); but as we all know by now, guidance of any sort isn’t auditable.

So how auditable is CIP-013? Let’s look at the box score:

1. R1 is only partially auditable.
2. It isn’t clear whether R2 is auditable or not.
3. R3 isn’t auditable.

So CIP-013 isn’t very auditable, although it isn’t completely un-auditable (the only part of CIP-013 that is definitely auditable is R1.2). I could stop here and leave you with the impression that I don’t think very much of CIP-013. However, I have said that I think CIP-013 is a good standard. In fact, I’ve said that it comes closest to what I would call the ideal of any of the CIP standards. So what gives? Am I just being inconsistent?

I’m not being inconsistent because I no longer think that auditability is how the CIP standards should be judged – at least, not how the CIP standards and requirements that are based on plans should be judged. And why do I think this? This post has already gotten too long. I’ll discuss this in the next one.

The views and opinions expressed here are my own, and do not reflect those of any organization I work with. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.

---

[i] I just noticed for the first time that R1 just says the plan has to have processes to identify and assess risks, but not to mitigate them! R2 says the entity has to implement the plan, but if the plan is just to identify and assess risks, that means that’s all the entity has to do when they implement it! Obviously, I don’t think the SDT would have bothered to list the risks in R1 but only require the entity to identify and assess those risks, not mitigate them. If anybody has an idea about this – maybe I’m missing something important – I’d appreciate your letting me know.

[ii] When I say addressed, I’m not necessarily saying the entity will have to implement policies, procedures or technologies to mitigate all – or even most of - these risks. Since CIP-013 is risk-based, the entity needs to rank all the risks it faces (to be more exact, all of the threats it faces, ranked by the degree of risk of each), and determine the amount of resources it will devote to each one (with the understanding that the highest risks receive the most resources). The lower-risk threats will receive little or no attention. But the entity is required to at least look at all supply chain risks initially.