Friday, October 17, 2014

Roll Your Own, Part IV: The News from GridSecCon

I have spent the entire week in San Antonio at GridSecCon.  As usual, it was a really excellent event – I highly recommend it to anyone involved in cyber security in the power industry.  While I knew there was at least one discussion on the CIP v5 transition planned, I didn’t think I would learn anything groundbreaking about CIP – since that is really not what GridSecCon is about.

But it’s hard to stay away from CIP when you’re at a NERC cyber security meeting and the attendees are people who have CIP responsibility at their entities (it’s safe to say that no cyber security professional at a NERC entity is not at least partially involved in CIP compliance), as well as regional auditors.  And I ended up getting surprising confirmation that I’m on the right track in my series of “Roll Your Own” posts. 

Actually, more than that.  When I did the first of these posts in September, I harbored the secret hope that I might start a revolution in the NERC community.  Little did I realize that I was actually only reporting on a revolution that has been in progress for a while.  Not only have the revolutionaries been laying their plans for insurrection, they’ve already stormed the presidential palace and taken photographs of themselves in the president’s office, weapons at their sides and muddy feet on his desk, while smoking his best cigars. 

In other words, the “roll your own” revolution has already happened, at least in a subset of NERC compliance professionals.  They didn’t set out to do this; they’re simply doing what they need to keep their companies from getting huge fines after April 1, 2016.  Here are three different conversations from GridSecCon that have led me to this conclusion:

Conversation 1: Tobias and Steve
On Thursday morning, there was a panel discussion on the CIP v5 transition.  Panelists were Jay Cribb of Southern Companies, Jeff Fuller of AES, and Steve Noess and Tobias Whitney of NERC.  I had to miss Jeff’s and Jay’s presentations, and came in for the beginning of Tobias’.  The highlight of this presentation, at least for me, was his listing of the initial documents that the CIP V5 Transition Stakeholders’ Group (which I wrote about in this post in September) is going to deliver.

These include guidance documents on a) how to demonstrate your systems are appropriately segregated to justify classification of BCS as Low impact under Criterion 2.1; b) the meaning of “programmable”; c) virtualization; d) serially connected BES Cyber Systems at Medium impact assets with external routable connectivity; and e) shared-ownership substations.  These are all important topics on which guidance is sorely needed, and I applaud them for this.

But here’s the deal: These are five problems (admittedly, all big ones) out of 4,786 problems with CIP v5, according to my latest count (and this doesn’t count the 500 or so new problems that I heard about while I was at GridSecCon).  From the sound of what Tobias said, addressing these five might take up the rest of the year for the Cv5TSG (in fact, I believe he said one or two of the documents might not appear until 2015).  Meanwhile, on October 1 we passed the 18-month mark until April 1, 2016, when entities have to be fully compliant.  What do the entities do about the remaining 4,781 problems?  And what if they needed answers to one or more of these five problems months ago, not sometime before the end of the year (as is the case for my friend in the first post in this series, who needed the “programmable” definition a couple months ago)?

I submitted a question, which was read.  It wasn’t one of the above questions, but it was something like “There are a lot of questions about the bright-line criteria.  Will the new team be providing guidance on these?”  And then the smoke machine started operating.  Tobias said something to the effect that, after all, some of the questions the Cv5TSG is addressing are ones that involve CIP-002-5.1 (no dispute on that here.  The “programmable” definition certainly fits that bill.  Also, I believe the team is working on an "official" statement on the "far-end relay" issue, although the whole NERC community already knows how they've "ruled" on that).  And Steve emphasized that the team was focusing on CIP-002-5.1, since they know that’s the most urgent current need (I won’t dispute that either, since until you can be sure you’ve properly identified your cyber assets in scope for CIP v5, you obviously can’t come into compliance with the remaining standards).

But of course, neither of these answers said anything about the BLC.  Steve and Tobias might have saved everybody some time and simply said the answer is “no”.[i]  Don’t look for any guidance on the BLC from the Cv5TSG, at least not anytime soon.  And guess when you need this guidance?  That’s right, you need it now: 3:14PM Central Time on October 17, 2014.

Look, I’m not trying to harass Steve and Tobias, whom I both like personally.  And I’m not saying that NERC or its affiliates, regions, vassals or lackeys has brought us to this difficult position – where entities need to have guidance on the various vague areas and inconsistencies in CIP v5 and it’s not forthcoming – by their own intention.  But the fact is that we’re here, and NERC simply refuses to acknowledge the situation.  Instead, they fill the air with happy talk about how they just have to put a few finishing touches on the v5 masterpiece, and it will be a complete work of art – ready for entities to build their compliance programs on a rock-solid foundation. 

And this is of course why some entities have decided they need to forge ahead with their own definitions and interpretations, where these aren’t now available from NERC or the regions; they have no choice if they’re going to be compliant on 4/1/2016.  I discussed in detail what one entity is doing in the first post in this series.  And I was surprised to hear of another case when Jeff Fuller of AES answered a question (and I honestly forget what the question was).

Conversation 2: Jeff Fuller
Jeff was quite blunt.  He sounded every bit like my friend in the first post.  Entities need to develop their own definitions and interpretations where NERC or their region hasn’t provided them (I believe he referred explicitly to one or more of the documents Tobias is promising – perhaps the “programmable” definition – saying it would be “too late”.  But I can’t remember for sure what he was saying would be “too late”, except that it had something to do with guidance on v5 questions). They need to document what they do, so they can show the auditors they did the best they could with whatever information was available at the time.  And I’m sure Jeff developed these opinions long before I put them in a blog post; he simply had no choice, if he was going to do his job.[ii]

Conversation 3: An Auditor
The third conversation was with an auditor.  Since he’s already agreed with me that “roll your own” is the correct approach to the problems with v5, we didn’t need to rehash that.  But he’s gone beyond that to think of the next problem – at least from his point of view.  He was clearly looking over the horizon much farther than I was, since the question hadn’t even occurred to me, let alone the answer.

His question was, how does an auditor audit in a new world where the compliance people at the entity are going to be themselves responsible for a lot of the definitions and interpretations they need for CIP v5 compliance?  You certainly can’t keep repeating the old bromide about “auditing to the letter of the requirement”.  This was never realistic with the previous CIP versions, and it is very far from realistic with v5. 

Now, I won’t say we had a long conversation on this, so that I can give you a lengthy and accurate rendition of his answer.  In fact, he is currently preparing an article for the November issue of his region’s newsletter, where he will address this question directly; he’s said he’ll provide me the article once it’s published, and I’ll republish it here.  But I can summarize my understanding thus:

Any CIP auditor needs to understand that many of the definitions and interpretations he may consider to be “official” at the time of the audit (say, in 2018) were not available to the entity at the time they needed them – as they were coming into compliance with CIP v5.  He certainly can’t simply ding them for noncompliance with what he or she considers to be the current “letter of the law”.[iii]

So how does the auditor judge whether the entity has correctly followed a requirement?  He or she needs to determine whether the entity[iv]

  1. Applied all the “official” guidance available to them at the time (this includes the actual wording of the requirements and definitions, the Guidance and Technical Basis published with each standard, and any other guidance provided by NERC or the entity’s region – e.g. the Cv5TSG documents); and
  2. Where the available guidance wasn’t enough, whether the entity used a reasonable process to fill in the gaps.  This could include using guidance developed by an industry group like EEI or the NATF or discerning the “intent” of the Standards Drafting Team[v].  Or it might be just drawing up their own definition or interpretation, as long as it’s reasonable and doesn’t seem to be torturing the words of the requirement simply to make the entity’s compliance burden easier.

This might seem simple on the face of it, but think about it: This is 180 degrees from what the auditors are taught from Day 1 of Auditor School, and certainly from everything that NERC and FERC envisioned when they developed (or mandated) NERC reliability standards.  What’s to keep an auditor having a bad hair day, during an audit in 2018, from deciding he or she is going to hold the entity to the current guidance on – say - the meaning of “Transmission Facility”, even if that guidance was just issued a month before the audit?  Or what’s going to keep an entity that really didn’t follow the “reasonableness” rules described above (say, they decided that, because a strict reading of Section 4.2.2 – where it says “All BES Facilities” is what is in scope for CIP-002-5.1 – leads to the conclusion that no control center could ever be in scope, as I wrote about in this post[vi], that their control center that runs the grid for a metropolitan area of 7 million souls isn’t in fact in scope for CIP at all) from appealing a penalty to the courts – where the strict wording of the standard might very well be what decides the outcome?

What indeed?  

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Honeywell.

[i] That was Steve’s answer to me – “No” - when I asked him at the December 2013 CIPC meeting whether NERC would require the new CIP v5 Revisions SDT to address problems with CIP-002-5.  While it wasn’t the answer I wanted to hear, it certainly didn’t waste anybody’s valuable time.

[ii] I do want to point out that both people I know of who have come up with the "roll your own" idea are part of a generation entity.  This makes sense, since these are the ones who need to start earliest on their CIP v5 compliance programs.  Any large plant – especially a coal plant – that hasn’t already started working in earnest on v5 compliance is probably in serious trouble.  My guess is that substations have more time, although that’s offset by the fact that there are so many substations that need to comply with CIP for the first time under v5.  Really, nobody with Medium or High impact assets should be sitting around at this point (although I know some are hamstrung because the money required simply won't be available 'til next year).

[iii] Obviously, he or she can ding the entity if they’re clearly in violation of the letter of a requirement, or they didn’t apply a definition that was available to the entity at the time they needed it for compliance.  Even then, the auditor has to be careful about things like the definition of Cyber Asset.  That is a current NERC definition, but it relies heavily on the definition of “programmable”, for which there is currently no official guidance.  So dinging an entity because they used the wrong definition of Cyber Asset - when the core of that definition wasn't available at the time they needed it, i.e. October 2014 - would be a big injustice.

[iv] At this point, I’m no longer summarizing what the auditor said, since as I said our conversation was brief.  I’m applying my own reasoning to the problem.  I’ll publish the auditor’s own article when it’s available.

[v] This was the auditor’s word.  As I’ve said previously, I don’t think it’s at all possible to discern the intent of the SDT; in fact, I think the term is meaningless in the strict sense.  But that doesn’t mean the entity can’t look at other requirements, other definitions, the Guidance and Technical Basis, etc. – and come up with something like the “intent”.  Hey, you gotta do what you gotta do.

[vi] Look for the italicized paragraphs that say “Note (August 27)”.  I really ought to do a separate post on this, since it’s a huge example of where the strict wording of the standard leads to a completely nonsensical result.  There is lots of other evidence in CIP-002-5.1 that control centers are very much in scope for CIP v5, yet nobody has even tried to show me how 4.2.2 could be read so that they were in scope.  I don’t think it can be done.

No comments:

Post a Comment