We Still Haven’t Learned Our Lessons

I’ve heard a lot about a discussion at last week’s CIPC meeting in which Scott Mix and Tobias Whitney of NERC tried to “clarify” what the Lessons Learned were and ended up causing more confusion, not reducing it. 

This shouldn’t be any surprise, since NERC is basically trying to square the circle with the LL’s.  I do have some sympathy for them.  They are really between a rock and a hard place when it comes to CIP v5, namely:

1. There are a lot of problems with CIP v5: missing definitions, ambiguity, outright contradictions, etc.  These are most apparent with CIP-002-5.1, but they aren’t limited to that standard.  NERC entities, the NERC regions, and opportunistic bloggers are clamoring for more guidance, given the looming 4/1/16 compliance date.
2. At the same time, the NERC Rules of Procedure don’t permit any modifications to an approved standard other than through Requests for Interpretation, which require a process almost as involved as that for drafting and approving the standards in the first place.  First, the Interpretation needs to be drafted (there was even an Interpretations Drafting Team during the reign of CIP v3).  Then it needs to be balloted by the membership (with possible multiple drafts and ballots) and approved by the NERC Board.  Finally, it needs to be approved by FERC (although FERC doesn’t have to approve it.  In fact, they remanded the last two v3 Interpretations, setting the process back to zero for both of them).  This is easily a 2 - 3 year process, even when NERC starts accepting RFIs on v5; and I’m told they haven’t accepted any yet.  Obviously, RFI’s aren’t the tool NERC needs to address the problems in CIP v5 by 4/1/16.
3. Yet a total rewrite of CIP v5 is unthinkable.   V5 took close to five years to develop; absolutely nobody has the stomach to start again from scratch.  But even if we do that (and I am of course pushing to rewrite CIP-002), the question becomes: What do we do in the 3-4 years it will take to develop a new version?  Go back to v3? 

So what’s Plan B?  Clearly, NERC has to figure out a way to provide guidance without actually saying they’re doing Interpretations.  And that’s where the Lessons Learned came in.  Originally, they were pitched as a way to provide guidance to NERC entities based on the experience of the five entities that participated in the v5 Transition Study.  Of course, the LLs have moved way beyond that charter, and are now simply a way for a committee of entities and NERC/Regional staff members to develop quasi-interpretations, which are then vetted through a comment process.

I don’t have a problem with the LL’s in principle; in fact, I welcomed them when I first heard about them last fall.  But I do have a problem when NERC staff members try to say that the LLs have some sort of status other than simply being reasoned opinions that have been put through a public comment period.

Some attendees at the CIPC meeting understood Tobias and Scott to be saying that the regions were somehow going to be “bound” to audit based on the LLs.   There was something said to the effect of “The guidance (i.e. the Guidance and Technical Basis included with each standard, as well as the Lessons Learned and FAQs) is our understanding of the requirement.  If you don’t meet our understanding, you can get a PV.”  I also understand that Tobias and Scott went back and forth on this, so I won’t say this is some sort of final pronouncement.  But it does show why there is so much confusion over the LLs.

Folks, take it from me, Trustworthy Tom: The Lessons Learned are simply documents that express the consensus of a number of NERC community members who have examined the issue in question.  You should definitely consider these when you are deciding how you will interpret a particular piece of wording in CIP v5 (for instance, you should consider the Lesson Learned on the meaning of “Programmable” as you identify Cyber Assets).  

However, that doesn’t mean you should consider the LLs to be the word of God.  If you have a valid reason for believing that the definition of Programmable should be different from what is stated in the LL, use your version!  But you do need to clearly document, for future audit,  a) that you considered all the available NERC and Regional Entity documents addressing this issue (especially any Lessons Learned), and b) the reasons why you chose the particular interpretation you used.  As long as you build a solid case for your interpretation, and as long as the requirement itself hasn’t been revised or a contrary Interpretation hasn’t been approved by both NERC and FERC, I find it highly unlikely that you will be issued a Violation (whether you’re issued a Potential Violation is another story, since I can’t speak for what individual auditors may do; but I see no way that a PV could make it to a full violation, as long as your entity has done its best to comply with the requirement in question.  I have written about two auditors that have endorsed this idea: here and here).

However, all this discussion about how much validity the Lessons Learned have is in some way missing the bigger point: there are only two completely finalized LLs available now, and it is very likely there will be no more than two finalized before April 1 (which, of course, marks exactly a year before full compliance with CIP v5 for High and Medium impact BES Cyber Systems is due).  And what are these two LLs?  Why, two blockbusters: Impact Rating of Generation Resource Shared BES Cyber Systems and Impact Rating of Relays (Far-End Relay).   Both of these are interesting, but they can hardly be considered fundamental to the understanding of CIP v5.

What about some of the LLs that actually address topics on which there is a lot of uncertainty – and for which guidance is needed right now?  I can think of no better example of this than the Lesson Learned on the meaning of “programmable”.   If any LL is needed right now, it’s this one.  Without knowing what “programmable” means, you can’t identify your Cyber Assets.  If you can’t identify those, you can’t identify BES Cyber Assets.  If you can’t identify BCAs, you can’t identify BES Cyber Systems.  And if you can’t identify BCS, you might as well consider quitting your job and exploring careers in the circus.  Especially today, one year and twelve days before the High / Medium compliance date.

So what is the state of the LL on “programmable”?  I had thought it was pretty settled and that NERC would be finalizing it any day now.  But in Kevin Perry’s presentation at the SPP meeting last week, he discussed his preferred definition – which seems to me (illiterate blogger that I am) to be quite at odds with the definition found in the current LL draft.  Now, Kevin is on the committee that is in charge of the LLs, so his opinion will definitely carry weight in whatever ends up being finalized.  I of course don’t know what the outcome of this debate will be, but I do know it will probably be 2-3 months (or even more) before the LL on “programmable” is finalized.  And that’s only 9-10 months before the compliance date.  Folks, if you wait until then to start your CIP v5 asset identification process, you’re SOL (a new NERC Functional Model classification that I’m proposing).  You have no choice but to roll your own definition now and see what comes out when the LL is finalized.   You need to get the job done now.

Unfortunately, this argument can be repeated for other LLs.  Interactive Remote Access sounds like it’s a long way from being finalized.  And virtualization?  Well, the LL on that hasn’t even been started yet.  The only question in my mind is whether it will be out this year or next.  So do I recommend you completely put off IRA or virtualization work until these LLs come out? No, I don’t.  You need to try to the best of your ability to understand these issues and come up with your best judgment on what they mean for your own CIP v5 compliance situation.  And you need to document the approach you ultimately decided on, as well as the different alternative viewpoints you considered.  If a LL comes out two months from now when it’s too late to change your process, then too bad for the LL.  You just need to make sure you document the fact that the LL was developed too late for it to do you any good.

And don’t even bring up the subject of the LL on Functional Obligations and Control Centers when you’re around a compliance person from a municipal or a coop utility; you’ll get an earful for the next eight hours.  This is a huge issue for them, which they're fighting in any forum they can.  We’ll be lucky if we ever have a final LL on this topic, let alone by 4/1/16.

There’s a further question: What about all the Lessons Learned that haven’t even been started yet (as far as I know)?  Of these, my favorite – natch – is a LL on Identification of BCS.  I recounted in a recent post how a CIPC subgroup, charged with developing a particular LL on CIP-002, had decided they couldn’t fulfill their mandate (which had to do with a particular case of BCS identification) until the issues with BCS identification in general are addressed.   And there are a number of other potential LLs that I might suggest right now, were it not for the fact that I would be wasting my time: they all depend on having some document[i] clarifying how the BCS identification process should work – which obviously ain’t coming anytime soon.

Let’s sum up here, and ask some important questions:

1. Should NERC continue writing Lessons Learned?  Absolutely, they are important no matter when they come out.
2. Should NERC entities wait for LLs before they kick off their CIP v5 compliance programs in earnest?  Absolutely not.  They need to roll their own definitions or interpretations, using any available LLs as guidance.  In the end, they’ll get a PV if they haven’t even begun to comply with a particular requirement.  They won’t get a violation (or hopefully even a PV) if they have done their best to understand what a requirement or term means, have documented their reasoning, then have implemented based on the solution they just documented.
3. Should NERC try to tell entities they will be audited in some way based on the LLs?  No, because it’s not true.
4. What should an entity do with an auditor who suggests that the Lessons Learned (or the Guidance and Technical Basis in the standards, or the FAQs, or the Bible) provide some sort of mandatory interpretation of the standards?  I suggest defenestration.

  

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Honeywell.

---

[i] In CIP v1-v3, there were documents prepared by subcommittees of the CIPC that provided guidance on identification of Critical Assets, as well as of Critical Cyber Assets.  They were certainly late in coming, of course.  But the need for them had at least been acknowledged early on.  I haven’t seen NERC even come to that realization yet - that comprehensive guidance is needed on the question of BCS identification and classification.