Flailing Away

I had the pleasure to attend – by webcast – almost the entire RFC CIP v5 Workshop, which was held in Cleveland last Thursday and Friday morning.  I strongly recommend that anyone working on CIP v5 compliance view as much as they can of the presentations.[i]  They were all good, although my personal favorites were the ones by Scott Mix of NERC (he did three or four, all worth listening to[ii]), Felek Abbas of NERC (who did two, both excellent), and Lew Folkerth of RFC (whose presentation on the “CIP Version 5 Core Requirements” included, for each requirement discussed, a list of “implicit requirements” – ones that aren't specifically stated but which become apparent when you carefully consider what needs to be done to actually comply with the requirement as written.  This is of course a big problem with CIP v5 – the fact that so much of what you need to do to comply with it isn't actually explicitly stated in the requirements.  One implicit requirement I’ve often pointed out, which Lew didn't mention, is a result of the fact that CIP-002-5.1 R1 never tells you to identify BES Cyber Systems in the first place, only to classify them, although R1.1-1.3 use the word “identify” when “classify” should have been used.  This is why I have done several posts on BCS identification).

However, there was one aspect of one of Scott Mix’s presentations that I found rather depressing.  In one of his presentations (I believe the one on “CIP Standards Modifications”), he discussed NERC’s ongoing efforts to address all of the questions about interpretation of the requirements of CIP v5, and mentioned that NERC is unveiling next week (at a webinar on Wednesday - you can register here) a brand spanking new approach to providing guidance, above and beyond the Lessons Learned and FAQs (which are continuing as well).

The basis of this new approach seems to be a fairly recent realization on NERC’s part (I had heard about it and mentioned it in this recent post) that there is a more authoritative trove of guidance already available; this is a 1000-plus-page section of NERC’s original filing of CIP v5 with FERC on January 31, 2013 (I won’t include the link to this, because it’s a huge file).  This section contains the primary comments that were provided to the SDT by NERC entities as v5 was developed, as well as the SDT’s responses to those comments. 

Scott implied that, because the responses to the comments were written by the SDT, and because at least one person at FERC presumably read through these as the staff was deciding whether or not to recommend to the Commissioners that they approve v5, they therefore have a higher “authoritative” status than do the Lessons Learned and FAQs.  Essentially, it seems NERC has decided to “mine” this document for whatever pieces of interpretive wisdom can be gleaned, and publish these as separate documents (not yet named, although see below for more on that subject).

This might sound wonderful – here’s a whole treasure trove of guidance from the SDT that might address lots of v5 problems that have been brought up, both by unscrupulous bloggers only out for personal gain, as well as by NERC entities who have been uncovering them as they struggle unsuccessfully to understand what the v5 requirements mean.  However, while I don’t think this is necessarily a bad thing, I also don’t see that it will provide much benefit – especially given that the effort put into this task would be better spent trying to accelerate the rate of production of Lessons Learned (which, given that only two have been finalized in the time since the LLs were announced last September, can’t be said to be super-fast).  Here are my reasons for saying this:

1. It is a stretch to say that the SDT’s responses to comments were something official from the SDT, at the same level as the Guidance and Technical Basis in each of the v5 standards.  The latter were debated by the SDT before being finalized with the requirements themselves.  While I’m sure the responses to comments were ultimately voted on by the SDT, they were prepared by individuals.  The SDT really had no other choice but to do this.  I remember one of the v5 drafts drew about 2,000 pages of comments – and that was only one of the four official drafts.  The SDT had to respond to every comment, and the only way to do that was to parcel them out among the different members to respond.  I didn't attend a lot of the SDT meetings, but I don’t think the members spent a lot of time debating responses to comments.  How could they possibly do that, given their otherwise huge workload?  So these responses need to be taken as primarily the work of individual SDT members, not the SDT itself.
2. Since FERC Order 791 (which approved v5) didn't specifically refer to these comments or the SDT’s responses, I think it’s a stretch to imply that FERC in some way “approved” the responses – just because they didn't take issue with any of them.  I know Scott didn't state that FERC had approved them, but by even bringing up FERC he was implying something like, “FERC didn't have objections to the responses”.  As it is, the fact that FERC didn't refer to any of the responses in 791 could just as well be taken to mean they didn't think they had any real relevance.
3. The whole idea that, in trying to understand what the v5 requirements mean, it would be beneficial to learn the “intent of the SDT” is fallacious.  I wrote a post on this question last year, so you may want to read that.  The conclusion of the post is that there is no way to definitively discern the “intent of the SDT” on any particular issue having to do with v5; in fact, it’s really a meaningless concept.
4. I haven’t read the section of the NERC v5 filing that's in question (it’s on my reading list, but it’s behind Finnegan’s Wake.  Since I first tried to tackle that in college and I've never gotten even to the end of the first chapter, it’s likely to be a while before I get to the SDT’s document), so in general I can’t say anything about the SDT responses in that document.  However, I recently wrote a post on the meaning of “adversely impact” in the BCA definition; the post took as its starting point one of the sections in NERC’s April 1 FAQ document.  That section repeated the SDT response to the same question, which was included in the v5 filing.  The SDT response was basically that the meaning of “adverse impact” should be obvious and nothing more needs to be said about it.  If this is exemplary of the nuggets of wisdom to be mined from the filing, I recommend those nuggets be left unmined.

During Scott Mix’s presentation, one person raised the question whether these new NERC “guidance” documents were really just another try at the CANs and CARs, previous unsuccessful NERC efforts to provide some sort of mandatory guidance to the auditors on the meaning of particular requirements.  Scott said no, and I agree that isn't the issue I’m concerned about.  As I've said many times over the past year, there is no longer a way NERC can provide any definitive clarification of v5, other than to rewrite the standards or go through the formal RFI process.  Both of these will take years to bear fruit, so they don’t do any good for the run-up to v5 compliance next year.  NERC has already tried to imply that the Lessons Learned will provide mandatory guidance (in some way) for the auditors, but that has run into opposition from a lot of NERC entities and at least one region (at NPCC’s CIP v5 workshop that I attended in Albany in March, it was stated unequivocally that the LLs aren't mandatory, for the auditors or the entities).

So I think it’s a waste of time, although perhaps not pernicious, for NERC to pursue this new type of document, rather than doing what they should be doing – thinking about the different questions on v5 and coming up with well-reasoned Lessons Learned and FAQs, which can provide good non-mandatory guidance to entities.  It's as if NERC has decided that basing their new documents on the SDT filing relieves them of the burden of having to think about what's reasonable and what's not; I'm afraid that's not the case.

In the meantime, I’ll keep writing my Lessons Learned; I've done three so far (in just over one week), although I won’t declare them final for another month, to give people a chance to comment on them.  And I’m more convinced than ever that both I and NERC (as well as anyone else who wants to try their hand at writing Lessons Learned) have our hands full in writing these things.  In the RFC workshop, they kept a running log of all of the v5 questions that were raised, that couldn't be answered on the spot; these will be turned over to NERC to address.  Can you guess how many questions they logged in a day and a half?  64.

To give some perspective on this number, I estimated in February that there are over 500 questions that need to be answered before the 4/1/16 compliance date for v5 (and they need to be answered not just on March 31, 2016, but anywhere from three months to two years before the compliance date.  Of course, there are now only 11 and a half months ‘til that date, so the two year part will be pretty hard to meet without a time machine).  But with 64 questions coming up in a day-and-a-half workshop in just one of the regions – a workshop whose purpose wasn’t even to come up with questions but to try to explain the standards – I’d say my estimate is definitely on the low side.  I’m sure that a full list gathered today would include probably 1,000 questions, and that – since the questions are growing metastatically as entities try in earnest to understand CIP v5 – by next April there will be well over 1,000 questions left unanswered, no matter how many are answered between now and then.

NERC, with efforts like the one just described (and the SGAS, discussed at the end of this post), it seems you’re flailing away, desperately trying to do something – something – to answer all of these questions in time.  However, I said in January that it was already too late.  My opinion hasn't changed since then: The ship has sailed.  There is no longer any chance that CIP version 5 can be made fully enforceable on April 1, 2016.  The only thing you can do now is to admit this, try to pick up the piece, and figure out a course that will get you on a path to having a truly enforceable version in a year or two.  To reiterate (and update) the steps I said you need to take in the January post:

1.       You need to push back the compliance dates for v5 by a year.  So April 1, 2017 will be the date for the Highs and Mediums, and all the other dates will be a year later.  Note this doesn't mean you need to leave v3 in effect until 2017; you can still say v5 will be the law of the land on 4/1/16. However, 4/1/16 to 4/1/17 should be a "free" period during which no PVs will be assessed for any of the v5/v6 standards, provided the entity is making a good faith effort to comply.

2.       You need to really get cracking on the Lessons Learned, etc. – with the goal of having all important questions about CIP v5 answered by April 1, 2016.  This will give entities a year to put their compliance programs in place, with some assurance that they understand what is required of them.

3.       You need to declare CIP-002-5.1 R1 an “open” requirement, meaning there will be no PVs issued (even after 4/1/17) for entities that make a good faith effort to comply with it – reading everything available about it, “rolling their own” definitions where needed, etc.  There are simply too many contradictions and inconsistencies in this requirement (and in Attachment 1) for it to be fixable with Interpretations, Lessons Learned, etc.  It needs to be rewritten from scratch (while trying to preserve what is good about the current version, which is actually a lot).

4.       You (or one of the entities) need to issue a SAR to rewrite R1 to make it consistent and unambiguous.  When that is done – say in three years – this can then become an enforceable requirement.

And what happens if you don’t take my advice (and I don’t think you will)?  Every month you delay taking these steps only increases the embarrassment you will suffer when you finally have to admit that v5 can’t be enforceable on 4/1/16.  The fallout from this will be severe, the closer we come to the compliance date.

I’d like to make another suggestion based on the RFC meeting, NERC.  When Scott Mix was discussing the new guidance documents that you’ll be putting out, he said their name hadn’t been decided on (indeed, that it was changing hourly), but that “Compliance Application Memo” was the leading candidate - at least at the moment he spoke.

Let me suggest that you not use this term.  It seems to me that, if you’re trying to erase the memory of the Compliance Application Notices (CANs) and the Compliance Analysis Reports (CARs) from people’s minds, the last thing you want to do is come out with a new document that has a similar name, and whose acronym (CAM) sounds almost identical to CAN.  But maybe I’m over-thinking this.  What could possibly go wrong?

Postscript: The SGAS

There is another recommendation I made to NERC recently – that they make public the compliance advice they give in the Small Group Advisory Sessions (SGAS), currently being held in Atlanta.   I still stand by every word in that post, but I realize my analysis was too narrow.  Steve Parker of EnergySec did a much better analysis of the problems raised by the SGAS in one of their NERC CIP newsletters in March.  He raised three main issues, which I’d like to elaborate on here.

1. “The possibility (or perhaps likelihood) that NERC will be providing specific, non-public advice to individual entities jeopardizes the independence of the ERO with respect to future audits.”  This means that NERC is essentially tying their own hands on particular issues.  If they tell one entity that the method they've chosen to comply with a particular requirement is correct, how could they later issue any guidance that said anything else?
2. “The non-public nature of the meetings creates doubt that determinations made during such a meeting will be properly vetted and published for other entities to reference. This essentially creates a two-class system in which entities with the ability to attend an SGAS potentially receive compliance information (or determinations) on a preferential basis.”  This is part of my argument in the post referenced above.
3. “It creates a likely scenario in which Regional auditors will be pressured, or at least unduly influenced, to rule one way or another based on the advice given to an entity in such a session.”  This is really important.  The entities are audited by the regions, and an auditor from the entity’s region will usually be in the room for the SGAS.  If NERC says that what the entity is doing to comply with a particular requirement is correct, how could the region possibly find any differently when they go to conduct the audit?  Remember, the regions are part of NERC.  If your boss tells you that something is correct and that it’s a settled matter, how can you possibly go against this?  Of course, this basically destroys auditor independence, one of the principles of GAGAS, the rules that supposedly govern NERC auditors.  

What will be the likely effect of the SGAS?  I don’t think it will be immediate, since it will only be felt when an entity gets a PV they don’t agree with and takes it to court – this is likely to be four or five years from now.  But at that point, I believe all of CIP v5 will be deemed unenforceable (I describe my reasoning for this conclusion in the post referenced above).  And I frankly don’t know what will happen after that.

But until then, don’t worry – the SGAS will be deemed a great success.  It’s like the guy who jumps off the top floor of the Sears Tower.  As he passes the 50^th floor he yells out, “So far, so good!”

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Honeywell.

---

[i] I would include a link if it were available.  I know the presentations were recorded, so I imagine they will be posted on the NERC v5 Curriculum site.

[ii] His very last presentation on Friday really opened my eyes to an aspect of CIP v5 I’d never realized before.  I hope to have a post on this soon.