Roll Your Own, Part VIII: Will there be PVs Issued for CIP-002-5.1 R1?

In a recent post, I expressed the opinion that NERC should declare CIP-002-5.1 R1 an “open” requirement, meaning that entities who make a good faith effort to comply shouldn’t be issued Potential Violations if they get something wrong.  I said this because there are so many ambiguities and contradictions in the requirement – and because NERC has not come across with the guidance that would be needed for R1 to be truly “auditable” (I did try to make clear, however, that this only applies to this one requirement – the other requirements in CIP v5/6/7 are clear enough that this is not needed for them).

I didn’t stop there, though.  I continued to say that R1 would effectively become an open requirement whether or not NERC takes me up on my suggestion to make it so.  This is because I really can’t see auditors wanting to waste their time writing up violations that would never hold up if challenged in a court of law (which, of course, NERC entities can do). 

A respected CIP auditor with one of the NERC regions took issue with this.  His argument runs like this:

1. He points to paragraph 320 of FERC order 706 (which approved CIP Version 1), which says “We will not allow a ‘safe harbor’ for good faith compliance as requested by AMP Ohio.  We do not believe that blanket waivers from an enforcement action are appropriate in this context and have previously denied other requests for safe harbors from enforcement.   Rather, we believe that demonstrable good faith compliance is a legitimate mitigating factor in an enforcement action.”  In other words, even if NERC wanted to make R1 an open requirement, FERC would never allow it.
2. He states that, while he agrees he wouldn’t write PVs in cases where he doesn’t think his region would prevail if there were an appeal, he wouldn’t hesitate to write one in a case where he thought the entity had violated the clear meaning of the requirement.  To support this argument he pointed to paragraph 72 of Order 706, which says that “compliance will in all cases be measured by determining whether a party met or failed to meet the Requirement.”

Regarding the auditor’s first point, I never believed there was a significant probability that NERC would take up my suggestion. I likened the probability to that of the Cubs winning the World Series this year – enough said about that.  This brings us to point 2.  How do we differ on that one?

People who have been reading this blog for a while know that I started a series of posts in September called “Roll Your Own”, of which this is the eighth installment.  These posts discuss the need for NERC entities to come up with their own definitions and interpretations in CIP v5, in the many cases where NERC hasn’t provided adequate guidance.  Does the auditor’s argument undercut my advocacy of rolling your own? 

Not at all.   The fact is that NERC entities can’t keep waiting for NERC to come out with guidance on the CIP v5 standards (if they still are waiting – I hope not, but I suspect many are).  They have to have something to fill in the gaps.  The only option they have is to consider all the guidance on a particular issue that is out there (say, on the definition of “programmable” – which includes a draft NERC Lessons Learned[i] document released last week[ii]) at the time they need it, come up with their best definition or interpretation, then make sure to document it, along with how they came to these conclusions. 

Of course, the entities need to do their best to adhere to the wording of the requirement in question.  But if the requirement or definition isn’t clear enough for compliance in the normal sense (i.e. following the requirement exactly), and if NERC hasn’t produced guidance on this issue or what they have produced is inadequate, the entities have no choice but to roll their own definition or interpretation; in fact, the very auditor who wrote in to me on this issue is the same one who previously agreed there is no other option for NERC entities.[iii]

Does the auditor’s argument negate my prediction that there will be no PVs issued for good faith CIP-002-5.1 R1 violations?  Well, I’ll admit this may have been an exaggeration (not that I ever exaggerate, of course – except in the preceding seven words).  There could well be a few PVs issued for mistakes made in good faith, by an auditor who truly believes certain wording in R1 is crystal clear, even though it is actually ambiguous.  But I continue to believe that entities who make a good faith effort to comply with R1 (including carefully considering any guidance from NERC or the regions), and who roll their own definitions where these are MIA from NERC, have nothing to fear when the auditors come calling to assess their compliance.  After all, what else can they do?  Simply tell the auditors they’re not complying with this requirement because they don’t understand it?[iv] 

The difference between what I and the auditor are saying is really one of degree.  To understand what I mean, I refer you to the great philosopher Donald Rumsfeld, who said “There are known unknowns. That is to say, there are things that we know we don't know. But there are also unknown unknowns. There are things we don't know we don't know.”

I will paraphrase this in reference to CIP-002-5.1 R1.  There are unambiguous ambiguities, meaning things that are definitely ambiguous and on which there is little disagreement as to their ambiguity. And there are ambiguous ambiguities, meaning things that are ambiguous but on which there is disagreement as to their ambiguity (i.e. some people think they’re crystal clear, while others think they're not clear at all, or even worse do not make sense in the English language).

I think my disagreement with the auditor is over the relative proportions of these two types of ambiguities in R1 (and by R1, I mean “R1 and Attachment 1”).  He clearly thinks that most of the ambiguities in R1 are of the first type; I happen to think most of them are of the second type – so they aren’t being officially acknowledged by NERC and thus aren’t going to be addressed in Lessons Learned, FAQs, etc.

I’m sure I and the auditor both agree (and he has reviewed this post beforehand, so I’m not speculating here) that auditors won’t issue PVs for violations of wording that is ambiguous of the first type.  In other words, if it is pretty clear to the auditor that the wording is ambiguous (or that a definition is missing), he/she won’t issue a PV for a violation.  This makes sense; auditors aren’t evil people; they’re professionals who try to be as fair and consistent as possible.  Plus, auditors don’t want to make a bunch of unnecessary work for themselves.  Violations cost a huge amount of time to the auditor, as well as (especially) to the other staff of the region; and that is even before any appeal of the finding.  I’m sure all auditors write PVs very reluctantly, knowing that it will probably mean at least some lost Friday evenings for them in the coming months.

However, since this auditor believes there are few ambiguities of the second type in R1, he thinks that any PVs that are issued will be fairly indisputable - in other words, there aren’t many parts of R1 where there are ambiguous ambiguities.  Given this, auditors won’t hesitate to issue PVs when they think an entity is wrong, since they won’t worry that there are a lot of hidden ambiguities in R1 that may come out and invalidate the PV he/she just issued.

I, on the other hand, think there are many ambiguities of the second type in R1.  This leads me to believe that auditors won’t issue many PVs, since they will always be second-guessing themselves on whether the wording is clear enough for them to do this.[v]  This is why I say that R1 will end up becoming effectively an open requirement, whether or not NERC declares it such.

Of course, we won’t know whether there will be PVs on R1 until v5/6/7 comes into effect and audits start taking place.  But here’s how you’ll be able to tell whether I or the auditor was right: If you hear of a lot of PVs being issued for CIP-002-5.1 R1, he is right.  If you don’t, I’m right.[vi] 

The problem with this little contest is it will take a number of years for you to determine which of us was right.  But I have a better idea.  Why doesn’t NERC make this contest irrelevant and do the three things I’m requesting it do?

1. Postpone the compliance dates for CIP v5/6/7, hopefully by a year;
2. Declare CIP-002-5.1 R1 an open requirement; and
3. Start writing a SAR for a new version of CIP-002 that could actually be interpreted without ambiguity.

 A guy can dream, can’t he?

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Honeywell.

---

[i] The auditor did point out to me that, while they don’t have the standing of true Interpretations, the Lessons Learned documents will have a higher legal standing than just some PowerPoint that a NERC staff member may have put together, since they are produced as part of a process - including comments from the membership - specified in the NERC Rules of Procedure.  For more on that, see this post.

[ii] The auditor also informs me that NERC will come out with their “top 15” Lessons Learned by April 1, 2015.  This will certainly help some entities, but it’s about a year too late for others; plus I’m sure there are more like a couple hundred LL’s actually needed (I identified 20 issues just in CIP-002-5.1 R1 in this recent post, and I have about 5-10 more I could now add, just about that one requirement.  Lew Folkerth of RFC discussed a serious issue in CIP-010 in this post.  The list goes on and on, and will keep growing as entities struggle to comply.  For example, I wouldn’t be surprised if there ended up being over a hundred issues just having to do with the bright-line criteria in Attachment 1of CIP-002-5.1).

[iii] The auditor said, as I have before, that the entity is obligated to carefully consider any guidance NERC has provided.  For example, I just provided a link to the draft Lessons Learned document on “programmable”, posted for comment last week.  Entities can still roll their own definition if they think this document isn’t particularly helpful, but they have to document why they feel this way and be prepared to defend their position with the auditors.

[iv] The auditor points out, “Just bear in mind that the auditor is making his/her evaluation based on the best information available, coupled with the auditor’s technical training and work experience.  That will be true regardless, but in the absence of a formal definition or guidance, the auditor will fall back on training and experience.  For example, ‘Programmable’ is a well-defined term in the IT world.  The issue is its applicability in the generating plant, and that is where the Lessons Learned guidance will come in.”

[v] At this point, the auditor adds, “Actually, there will be little second guessing.  The auditor has to be qualified to audit a requirement, through training and experience, in order for the audit objectives to be met.  The auditor will rely upon his/her training and experience, along with the best information available, in forming an expectation.  The auditor goes into an audit with an expectation of what is necessary to demonstrate compliance.  The challenge will be to be open to an entity’s approach as opposed to only allowing an approach fixed in the auditor’s mind (it is what you do, not how you do it; or as I have heard often, the color of the widget only matters if the requirement prescribes the color).  And the Regional auditors are sufficiently experienced that this should not be a widespread problem.  That is also why we have audit teams and a consensus process, not individuals, making the finding determination.  The entity will have to be able to persuade the audit team that the entity approach comports with the intent of the requirement and with the specific prescriptions of the requirement as may be present.  And don’t forget that consensus is not the same as unanimity.”  This is good clarification, but it rests on the assumption that the wording of the requirement (or definition) in question is fairly clear.  As I’ve said, we differ greatly on our assessments of how much of CIP-002-5.1 R1 is “clear”.

[vi] The auditor points out a third option: Maybe there will be few PVs because the auditors think the entities are doing it right!  I guess there’s always that possibility….