When and how can you receive Advice from your Region on your Plan?

I recently wrote a series of posts about “plan-based” requirements (e.g. CIP-010 R4 and CIP-013 R1) and raised two main questions about them. The first was whether they could be strictly audited using the standard NERC auditing framework (which is embodied in the Compliance Monitoring and Enforcement Plan, or CMEP). My answer to that question was that they are auditable to various degrees (depending on how they are written), but none of them are auditable in the strict sense that the prescriptive CIP requirements (like CIP-007 R2) or the Operations and Planning requirements (like FAC-003 R1) are auditable.

The second question was more important; I only delved into this question in the last post  in the series. This question is effectively[i] “Given that the real goal of auditing is promoting the reliability and security of the Bulk Electric System, is it possible that trying to audit plan-based requirements (which, as I’ve pointed out several times, are the wave of the future for NERC. In fact, all of the important new CIP requirements developed since CIP v5 have been plan-based) using the standard NERC auditing framework will actually hinder this goal?”

And my answer to that question was yes. In that post, I referenced a previous post  I had written on CIP-014 enforcement. CIP-014 was the first plan-based standard to be approved by NERC, and the NERC Regional Entities are already auditing it. In this post, I recounted a conversation I’d had with a CIP physical security compliance person at a very large utility, who had been rebuffed by his regional auditors when he asked them a question about whether a particular technology – that this entity proposed to implement in their substations subject to CIP-014 – would likely be determined to be appropriate to include in the Physical Security Plan required by CIP-014 R5.

He was flatly turned down when he asked this question, on the grounds that answering it would constitute a violation of the principle of auditor independence: If the auditors answered it for him now, when they came to audit him later (perhaps years later), they would in effect simply be auditing themselves. On the face of it, this seemed to be the only possible answer that the auditors could have given. But the unfortunate result of this was that the utility he worked for would most likely cancel their plans to implement this technology (which would cost $80 million to deploy to all of their CIP-014 substations).

In my post, I pointed out that this clearly seemed to be a case where the standard NERC audit framework was actually working against the goal of enhancing the physical security of the BES. I further hinted that, given the choice between maintaining that standard framework and securing the BES, I would choose the latter any day. I was thus preparing to suggest that the standard NERC auditing framework – which works very well for the NERC Operations and Planning standards, but not for the CIP standards, and especially the plan-based ones – be replaced with a new auditing program just for the CIP standards.

However, as has happened before, an auditor had written in to me about this issue, and in an email dialogue he pointed out that not only is it not necessary that there be a new auditing framework, but that the elements required to deal with this problem are already in place in at least two of the NERC Regions, and could be implemented in the other Regions as well. In the rest of this post, I won’t usually quote the auditor directly, but I will include his ideas, as well as my interpretation of them, without necessarily saying at every point whether they are his or my ideas. After I initially wrote this post, I sent it to the auditor to review for any mis-interpretations on my part, and he corrected those.

Before I discuss this further, I want to make sure we all understand what the big problem is. It is not that plan-based requirements are not very auditable (if they are auditable at all) under NERC’s CMEP; I’ve already stated that I consider auditability to be a distant second to the main concern. The main concern is the security of the BES, and the problem is that, as exemplified in the case I just discussed, that goal will not be aided if, for plan-based requirements, NERC entities can’t get their NERC Region to review their plan before they implement it. Additionally, when it comes to implementing the plan, the entities would be greatly helped if they could ask their Region to review the implementation while it’s in progress and point out potential problems. The case just discussed is an example of how auditing concerns can prevent NERC entities from getting the advice they need on complying with plan-based requirements. As we have just seen, this problem has already appeared for CIP-014, and it will appear in spades when FERC approves CIP-013 and entities start working seriously on their supply chain cyber security risk management plans.[ii]

In my opinion, here is what is needed to address this problem:

1. NERC entities, when faced with plan-based CIP requirements, will of course first have to develop the plan mandated by the requirement (the Physical Security Plan mandated by CIP-014 R5, the supply chain security plan mandated by CIP-013 R1, the Transient Cyber Asset/Removable Media plan mandated by CIP-010 R4, etc). In the process of developing the plan, they need to be able to ask their Region questions about what should be in the plan, what are best practices for mitigating the threats addressed by the plan, etc.[iii]
2. Once they have developed their plan, the entity needs to be able to take it to their Region and ask them to review it. The review won’t tell the entity whether the plan is “compliant” or not; rather, the reviewer will point out whether the plan doesn’t address any threats that should be addressed in the plan, and whether the mitigations proposed follow best practices as the Region understands them. If the entity can’t get this review, they will have to take a deep breath, hope the plan they’ve developed is one their region will think is good, and then implement the plan. The danger is that they may go a long way down the road to implementation (or even finish it) before their next CIP audit, and that the auditors will then tell them the plan had a lot of problems and needs to be redone. Of course, that could possibly lead to a lot, or even most, of the work the entity has done implementing the plan needing to be re-done as well.
3. If the Region does the review and sees problems with the plan, they will point those out to the entity. At that point, the entity could elect to re-work the plan to fix the problems, or else to dismiss the Region’s advice if they think it isn’t well-founded for some reason.
4. If the entity did re-work their plan, they would be well-advised to ask for a new review from the Region, to make sure they have addressed whatever objections the Region brought up.[iv]
5. Once the entity was satisfied that it had a good plan, it would start implementing it. However, at any point during the implementation, the entity could request of their Region that they review the entity’s implementation work so far, and let them know of any developing problems they see (e.g. that the entity isn’t implementing everything in the plan, or that they are implementing parts of it badly).
6. If the Region does point out problems with the implementation, the entity has the choice either to try to address these problems or to dismiss them if they don’t think they actually are problems that need to be fixed – just as in the case of the plan review.

In reading this, you may have already thought of the objection that first came into my mind when I realized these steps would be required in order for the entity to be sure they had a good plan (and that they were implementing it correctly): How would it be possible for the Region to provide these services to the entity, then turn around and audit them later without having their auditor independence completely compromised?

The key to resolving this question is that the Region will need to have what is known as an Entity Development program in place; currently, one Region does have such a program, and I was told another Region is now putting one in place. The point of this program is for the Region to have a formal way of providing advice, like the above, to entities outside of an actual audit[v]. In general the staff members for this program will be separate from the auditors, although the auditor pointed out to me that it isn’t impossible for the CIP auditors to also provide Entity Development services, assuming the entities trust them not to mix the two functions.

One absolute requirement for the Entity Development staff is that they be knowledgeable in the subject matter of the plans they are providing advice on – for example, if they are providing advice on the Physical Security Plan in CIP-014, they need to understand physical security for substations. Of course, this requirement isn’t something to be taken lightly, since such people – with electric utility experience – may be hard to find. So putting together this staff may be a multi-year process.

While the Entity Development staff will in theory just be providing best practices-type information to the entity, it is likely that they will sometimes, in the process of reviewing an entity’s plan, discover some element of non-compliance. When this happens, they will point this out to the entity, but they won’t be able to provide advice on what the entity should do to remediate this non-compliance; that would probably compromise auditor independence.

Of course, the Entity Development staff wouldn’t report these instances of possible non-compliance to the auditors, and the report wouldn’t become part of the record for the entity’s next audit. However, when receiving a report like this from Entity Development, the entity that requested the plan review will need to decide whether to self-report a violation; and if they do self-report, they need to also “discover the scope and extent of the non-compliance” (to use the auditor’s words), as well as mitigate the non-compliance[vi].

If the entity does self-report, and the issue that was the subject of the report is discovered as part of the next audit, the entity won’t be subject to a Potential Non-Compliance (PNC) finding for the same issue, covering the same time period that was self-reported. Of course, if the entity doesn’t self-report and the potential violation is discovered, the entity would be subject to a PNC, which of course will be more serious since it was discovered at audit.

The auditor did also point out that there is a way that the Region can provide advice on whether a plan is likely to be found compliant or not, but they can only do this during the period before a standard becomes enforceable. If the entity develops their plan and specifically asks the Region for a “readiness assessment” of the plan, then, depending on available time and resources, Regional staff (either auditors themselves or Entity Development staff) can perform, to quote the auditor, “a non-binding, no risk, no consequence ‘audit’” of the requirement for developing the plan.[vii]

For example, suppose your entity wishes to have your Region review your CIP-013 supply chain cyber security risk management plan before the CIP-013 compliance date (which I am currently expecting to be toward the end of 2019, assuming FERC approves CIP-013 this spring). You would of course have to first develop the plan (and this would have to be done well before the compliance date), then request a readiness assessment of your compliance with CIP-013 R1.

Of course, developing your CIP-013 plan in the first place will require having some idea of what should be in the plan. There is the 13-page Implementation Guidance produced by the CIP-013 drafting team; this is a very good document as far as it goes, but it is nowhere near a comprehensive guide to developing the plan. There will also be more guidance coming from other sources (including specifically the North American Transmission Forum, although I’m not sure that will be available to non-members), and NERC is now considering a CIP-013 “Transition Study” similar to the one for CIP v5. In this study, a small number of early adopters will share their experience with NERC, to help them develop Lessons Learned (remember those?) and other guidance.

And there will be another source of guidance: I have been thinking about what should be in the CIP-013 plan and discussing this with an auditor, and I plan to write a series of posts (probably not consecutive posts, of course) about this question. Unlike the NATF, I’m not on NERC’s list of approved guidance providers, so you’ll of course have to take whatever I say with a grain of salt (but in fact the same applies to the approved providers like NATF, since NERC specifically says that no guidance these approved providers turn out they will itself be “approved” by NERC). But I hope you’ll find my posts on this subject to be helpful, and I’ll welcome any feedback you have on them.

The views and opinions expressed here are my own, and do not reflect those of any organization I work with. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.

---

[i] I say “effectively” because I now have a better way of wording the question than I did when I wrote that post.

[ii] This is because the requirements in CIP-013 provide very little “guidance” on what should be in a good supply chain security plan, even less so than the guidance that CIP-014 R5 provides to guide entities in developing their physical security plans.

[iii] Of course there is, and will be, guidance provided on these questions by various industry organizations. However, the guidance NERC provides will be very limited due to NERC’s limited interpretation of what guidance they are allowed to provide. NERC entities will need to weigh all the guidance they find, but in the end what will count most for them is what their Region says. This is the way it has been since CIP version 1, although this trend intensified with CIP v5.

[iv] It bears repeating that the Region’s review of the plan won’t be for the purpose of saying whether it is compliant with the requirement or not, but only a) whether all of the threats that should be mitigated in the plan are in fact addressed; and b) whether the proposed threat mitigations actually follow best practices. It is possible that whoever reviews the plan will notice something in the plan that is non-compliant and will point this out to the entity; it would then be up to the entity to decide whether they want to revise their plan to address this concern, or whether they think the observation is mistaken for some reason. In any case, the observation made by the Region wouldn’t become part of the audit record, and wouldn’t be passed to the auditors when the entity was next audited.

[v] The auditors have always been able to point out Areas of Concern covering cyber security practices that aren’t within the scope of CIP, when they notice something regarding the entity’s practices during the course of an audit. But there has never been an official way for them to provide such advice outside of an audit – and, as I pointed out earlier, an audit will often come a year or two after the entity has started implementing their plan. It will be much better if the plan can be reviewed as soon as it is developed.

[vi] The auditor did explicitly warn against what might be a temptation: agreeing with the opinion that your plan was non-compliant in some way and fixing whatever the problem was, but then still not self-reporting the issue. While it is true that the friendly advice of possible non-compliance that you receive from Entity Development will not in any way be reported to the auditors (and even if it were, they would ignore it), it is still very likely the auditors will discover that at a certain point your documentation changed, from reflecting the old non-compliant wording in the plan to reflecting the new compliant wording. Of course, the penalty for non-compliance discovered in an audit is likely to be much more severe than for a self-report.

[vii] Of course, the readiness assessments are nothing new. The Regions conducted a number of these during the run-up to the CIP v5 enforcement date. They were very helpful, both to the entities that received them and to the auditors that conducted them. However, the auditor did caution that there is no way that the readiness assessment will be able to issue an opinion that the plan seems to be compliant. The team will point out gaps in compliance and recommend steps for remediation; after that, the entity is on its own to determine what it should do with the information.