An Auditor Addresses Auditing on Ambiguous Authorities

My most recent post discussed what kind of information the NERC regions are willing to provide regarding how they interpret (little “I”, of course!) areas of ambiguity in the CIP standards, as well as the format they’re willing to provide it in (verbal or written). As almost a sidebar in that post, I made the assertion – which I used to repeat about once a week during the big debates in 2014 about how to handle ambiguity in the runup to CIP v5 compliance – that, in cases where the entity has to make a decision on how to comply with a truly ambiguous requirement, they need to look at all available guidance from NERC, the regions, Tom Alrich’s blog, The Tibetan Book of the Dead, the I Ching, etc. But in the end, it is up to the entity to make – and especially document – how they made their decision.

If the entity gets audited three years later and the auditor doesn’t agree with the decision they made, there’s no way he or she can issue them a PV (and have it upheld). You gotta do what seems best given the available information at the time. I pointed this out because an auditor I was quoting in the post had endorsed this position by email (quoted in the post).

However, in seeing how I had interpreted what he said, the auditor emailed me back. Here is the full text of his email:

“A clarification...

 ”When I said we would audit based on what the entity actually did and using the best information at the time, I was not saying we would forgive the entity if they relied on advice or guidance provided at the time of the inquiry, only to be found non-compliant at the time of audit.  At the time (I) was referring to the time of audit.  And what the entity actually did is very important because we have seen entities misapply our guidance.

“We provide guidance using the very best information available at the time of the inquiry.  But, as we all know, things change.  Standards are revised.  Interpretations, although infrequent, do get approved by FERC.  NERC issues guidance through its Section 11 process.  And, while we try very hard to thoroughly understand not only the nuances of the Standards, but also the question being asked, we are not infallible.  We reserve the right to get smarter.

“We expect the entity to take our guidance into consideration as they would any other.  Moreover, we expect the entity to keep up with changes and additional guidance as they evolve.  We expect the entity to determine their course of action after due consideration; not just to do something because the Region ‘told them to’ unless the direction is in response to a non-compliance issue, such as a violation mitigation or a RAD.

“That said, the Region is still the best resource.  We are closer to NERC, NERC guidance, and the collective wisdom of all eight Regions.  Auditors across all eight Regions and NERC have the ability to collaborate and seek consensus on issues as they arise, whether submitted as a question or encountered at audit.  We collectively communicate and discuss issues of a frequent basis.  We have also had the benefit of seeing numerous and sometimes widely varying approaches to compliance, and know what works and what is problematic.”

The three main lessons I draw from this email are:

1. Suppose your region provided compliance guidance for a requirement or you read about the issue in some official guidance, and you based your compliance approach for that requirement on what you had been told or read. You did this because this was the most recent guidance you could find. This doesn’t preclude you from still being found in violation at audit. NERC or your region may change its mind or decide it made a mistake, the CIP Standards Drafting Team may issue a draft requirement that clarifies the issue, FERC could issue an Order or a NOPR that affects the issue, etc. In other words, contrary to what I wrote in the previous post, just being able to show that your action was in accordance with the best guidance available at the time doesn’t give you a Get Out of Jail Free card for a future violation.[i]
2. Even when your region provides guidance on a requirement, they aren’t expecting you to follow it blindly. If you have documentation of guidance issued by some “official” entity that contradicts the region’s guidance and you want to follow that guidance not the region’s, you should feel free to do so.
3. And it seems the regions aren’t infallible and can change their minds! I was shocked…shocked! to hear this, of course.

At this point, I realized that the idea of the entity having to decide for itself how to comply, based on all the guidance available at the time, made lots of sense in the context where I described it in 2014 (and it wasn’t my idea, but that of a longtime control system/CIP professional at a large generating organization), when entities were staring at a seemingly fast-approaching v5 compliance date and had to make decisions right away if they were going to be in compliance with v5 by the mandated date. But what does the idea mean now, when the compliance date has long passed?

I then replied to the auditor and laid out a set of scenarios where an entity found or received guidance either a long or short time before an audit, the guidance they received was either very quick or very time-consuming to implement, the entity either could or couldn’t implement the changes before the audit, etc. I asked what his region would do if it found potential non-compliance in the audit, in each of the scenarios I laid out.

Fortunately for me (since I now realize my request was fairly foolish), the auditor didn’t take the bait. Here is the entire text of his response:

“The answer to all your questions is...  It depends.  When was guidance originally issued, if any? When was it revised?  What did the entity do before the current guidance?  What did they do with the guidance once it came out?  Essentially, the entity needs to tell its story and explain why it thinks it should not be found non-compliant.  The auditor will listen and evaluate what the entity presents.

“The auditor starts out reading the plain language of the Requirement.  Where there is vagueness and uncertainty, we will be conservative in any response we give during outreach or in response to questions.  Our recommendations with respect to virtualization bears that out.  We have no idea where the SDT will ultimately go.  But our intent with conservative guidance is to give the entity a direction that will likely be compliant with whatever the SDT produces and FERC approves.  If the entity wants to bet against our advice, they are free to do so.  Maybe they will get lucky, maybe they won't.  But the guidance we have been giving today is firmly rooted in the language of the Standards today.  And, my Region, at least, will explain our position couched in the language of the Standards.

“We will be, hopefully, reasonable in both our guidance and also our ultimate finding.  But there is absolutely no way an auditor will declare today how it will find an entity in 2019.  We have to see the facts and circumstances at the time of the audit.  In the end, we take industry guidance under advisement and give it weight.  But, to the extent the guidance includes errors or contradicts the language of the Requirement, we have no choice but to audit to the language of the requirement. The entity can appeal the auditor's finding through the enforcement process.

“Here is an example.  By when does an entity have to first test its Incident Response Plan for Low Impact BCS?  Many entities think they have until 4/1/2020 and base that on the fact that there was a delayed effective date (by 12 months) for the equivalent Requirement applicable to High and Medium Impact BCS.  But, show me where in any Implementation Plan a deferral is specified for Section 4 of Attachment 1 to CIP-003-6. There is none.  The Implementation Plan says 4/1/2017.  Maybe the SDT overlooked this detail and intended to give a delayed start date. Maybe not.  Regardless, all we have to work with is what FERC approved; the specifics in the Implementation Plan.  That and the Excel spreadsheet that NERC published that also shows 4/1/2017.

“Now, if 4/1/2017 comes along and we start writing violations, and then the Implementation Plan is changed to delay the first test, then Enforcement can dismiss the violation.  But the auditor determination is a finding of fact at the time of the audit.

“Here is another example.  The CIP-002-5.1 guidance recently published by NERC was produced by the entities and not the Regions.  It contains errors that the authors declined to address following a Regional Entity review.  So, what happens if an entity follows that guidance and gets the wrong answer?  Possibly a violation for failing to properly identify and categorize their BCS….the entity will likely not receive a violation if they over-categorize their BCS.  But declare something Low when it is Medium, they will likely be found non-compliant, regardless (of) what the guidance says.  Guidance is not approved by either the NERC Board of Trustees nor FERC. It is given some deference, but it is not…(binding from a regulatory point of view).

“So be very careful trying to characterize what an auditor will do in the future….(T)hat does not invalidate the appropriateness of an entity asking for advice and guidance.  They are still better off than asking some of the consultants out there that have not been as closely involved with the CIP Standards.”

I won’t try to summarize this statement; it seems pretty straightforward to me.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte Advisory.

---

[i] Of course, it is still pretty unlikely that you would receive a large penalty – or perhaps any penalty – when you can show that you were acting on the best information available at the time you had to make the decision.