Wednesday, December 28, 2016

What will the Regions Say?

This is the second post inspired by comments I received after my post entitled “A Lesson Still Unlearned

In this recent post, I made the statement “Since the Rules of Procedure say nothing about the regions having any authority to interpret the standards, no region will ever commit an interpretation to writing, even in an email. I have heard from a lot of entities that you have to call up an auditor and ask his or her opinion, if you have an interpretation question. They might not tell you, of course, but if they do they will only do it on the phone. Of course, this means that if, three years from now, a different auditor issues a PV because their interpretation was different from that of the auditor you talked to, there won’t be any documentation of what the original auditor told you.” (note I slightly revised this quotation to clarify it)

I got two sets of comments from two CIP auditors on this statement. One set was from an auditor who has contributed to many of my posts over the years. The other set was from Lew Folkerth, who was formerly a CIP auditor but is now head of CIP outreach for the RelabilityFirst region; he has been the subject of a number of my posts. For both of their regions (although I don’t know whether this is true of any of the other regions), my statement above wasn’t completely accurate. Interestingly enough, I seem to have been wrong in different ways for their two regions.

Let’s start with the current auditor. He wrote “My Region receives and responds to far more email requests than…phone calls.  And we do so by email…That said, entities like to get on a conference call because it is more efficient and comprehensive to have a two-way discussion than to tag back and forth via email.  Entities need to understand that we are giving our best professional opinion, (that) we are not directing an approach or implementation, and that we will audit what the entity actually did in the light of the best understanding of the requirements available at that time.”

In a subsequent email, he elaborated on this: “As far as email responses, it is often the collective opinion of the team and not just one person.  We don't usually preface the response.  Our entities generally know us well enough that an explicit statement each time is not necessary.  We have been doing outreach and responding to questions for 7 years now.

The auditor is saying that, not only do he and the other auditors in his region respond to “interpretation” questions by email, they do this much more often than by just a phone call. At the same time, he says that in their comments they’re not directing a particular approach to compliance, and they will audit entities on ambiguous requirements based on whatever was the best information available at the time the entity had to make the decision.

For example, suppose you have to make a decision on a particular issue like the cloud or virtualization. You investigate the available guidance and implement your decision; yet NERC subsequently comes out with new guidance that calls into question the judgment you made. This region (and I suspect most if not all the other regions) won’t ding you for not following guidance that wasn’t available when you had to decide.[i]

Moving to the auditor’s second paragraph, it is clear that not only do the auditors in his region respond by email, they also don’t insist that anything they say is merely a personal opinion; they discuss many issues as a team, and are willing to stand behind their team’s collective decision. Of course, this doesn’t mean a) they won’t as a team change their opinion later, nor b) that the individual auditor you talk to won’t actually be giving his or her personal opinion, not the collective one (note the auditor says that they don’t usually preface a statement by saying that it is either a collective opinion or an individual one); so this means you still can’t rely on these emails as being the “official” position of that region. But this does seem to be a step further than what most other regions will do.

Let’s move to Lew Folkerth of RF. Lew writes excellent articles on CIP in the bi-monthly RF newsletter; these articles are always called “The Lighthouse”.[ii] They provide compliance guidance on different aspects of CIP; some have even dared to suggest that what he does constitutes “interpretation”! (of course, I would never use that forbidden word in describing Lew). I have written more than one post on these articles[iii]; you can find all of the newsletters on RF’s website. In an email, Lew wrote “at RF we do a lot of ‘Assist Visits’ which an entity can request through the RF web site. Most Assist Visits are phone calls with multiple RF SMEs and entity SMEs. We seldom, if ever, provide a written response to questions as a group. Individually we may respond to emails, but always with the caveat that this is one person’s opinion and is not an official RF response.” Lew goes on to point out that, in his Lighthouse articles, any “interpretation” he does of the Standards is his own opinion, nothing more.

So Lew is saying that RF’s auditors and outreach people will sometimes respond to questions by email, but they will always preface the email by saying this is their personal opinion. And the same goes for his Lighthouse articles. Any collective opinions will only be expressed verbally, not in writing (and Lew doesn’t even say that RF even formulates any collective opinions of the auditors, as the other region just discussed does). RF clearly doesn’t go as far as the other region goes, and I suspect the other regions fall more in RF’s camp – although I will point out that Lew’s “Lighthouse” articles are literally unique among the regions, in actually providing compliance guidance in an article format.

I’m not making any judgments on any of the NERC regions in this post.  There are no official NERC guidelines to the regions for providing unofficial guidance! If anything, the moral is that if you plan to rely heavily on something that a CIP auditor or outreach person in your region tells you, you should find out under what conditions the opinion is provided. Is it an individual opinion? More than that? And you also need to remember that you will never receive an “official” position from any region, even if it is more than the individual auditor’s opinion. There will always be some risk that when you get audited three years from now, the auditor won’t even have heard about what was originally said to you, and in any case will discount it as simply another auditor’s opinion.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte Advisory.

[i] This idea is very similar to what I often said during the time in 2014 and 2015 when entities had to make decisions on ambiguous areas like the meaning of “Programmable”: You can only be held responsible for compliance with the meaning of a requirement as it was understood at the time. This means you have to look at all available guidance, but in the end it is up to the entity to make its decision – although a consultation with its region is definitely also advised.

When I asked the auditor to clarify this point, he wrote back an email that added a further dimension to this particular issue that I hadn’t anticipated. Rather than try to shoehorn that into this post, I will do a new post dedicated to it soon.

[ii] I hereby reveal the meaning of this title. I have been harboring this dark secret for so long, I can no longer continue to do so in good conscience. Lew is a great fan of the lighthouses on the Great Lakes, and always adorns his column with a picture of one of them. I am also a fan of those lighthouses, but I’m sure he has seen far more than I have.

[iii] And my next post will call people’s attention to two recent articles, which I think are really excellent.

No comments:

Post a Comment