NERC CIP in Fantasyland

I attended WECC’s CIPUG – CIP User Group – in Anaheim, Calif. last week.  This was the third CIPUG I’ve attended in this location, at a hotel a couple blocks from the gates of Disneyland.  It was as usual an intimate gathering – just me and 350 of my closest friends.  And as usual, it was a very well-organized and well-programmed event.

The first time I heard about a CIPUG being next to Disneyland, I thought, “How appropriate.  There is such a huge amount of unreality in NERC CIP; we’ll all feel right at home there.”  But after attending the meeting last week, I saw this juxtaposition from an almost opposite perspective. 

I have often put myself in the place of the people working in Disneyland, and especially Fantasyland – playing Mickey and Minnie Mouse, the Seven Dwarfs, etc.  I am sure these people have no illusions that they work in a make-believe world.  When they come off work, they don’t have to adjust to our “real” world –they feel they never left it.  The only people who actually believe in the make-believe world of Fantasyland are of course the very young kids who visit there.

Let’s contrast this with the people attending the CIPUG: staff members of NERC, WECC, and the NERC Responsible Entities, as well as consultants like me.  We are all indulging in fantasies about NERC CIP Version 5 and its path to implementation; those fantasies were on display in the CIPUG presentations as well as the conversations at breakfast, lunch and the breaks.  The difference between us and the people who play the Fantasyland characters is that they know they’re in a make-believe world.  Those of us attending the CIPUG, on the other hand, didn’t have a clue that this is the case. We were in the position of the young kids visiting Fantasyland, not the workers putting in their time there and thinking about anything except Mickey Mouse.

In this post, I will list three fantasies that are quite prevalent in the world of NERC CIP and that were on display at the CIPUG (but, of course, that are certainly not limited to the people who attended the CIPUG).  I do wish to point out that I am not singling out any particular individuals as being more prone to these fantasies than anyone else, although I will illustrate the fantasies through their manifestation in the presentations and discussions at the CIPUG.  These are institutional fantasies that have evolved to enable the whole NERC CIP “world” to live with an increasingly impossible situation, and to justify the fact that thousands of people in that world are plodding dutifully ahead, with no clear idea where they’re actually going or whether in fact they are really getting anywhere at all.[i]

Note: The CIPUG was just one of a total of three days of meetings.  The first two days were a combination of the WECC Compliance User Group (or CUG - i.e. the group that manages compliance with the other NERC and WECC standards, collectively known as the “693” or “O & P” standards) and the Western Interconnect Compliance Forum (WICF) – the association of NERC compliance professionals at WECC entities (whose meetings and forums are off-limits to NERC and WECC staff members).  The presentations from the entire three days can be found at this location; you can find the ones relevant to CIP by looking for titles with “CIP” or “Cyber Security”.[ii]

Fantasy Number One: The Foundation of CIP Version 5 is Strong

It’s no surprise when I tell you that NERC and WECC (and the other seven Regional Entities) are moving forward (perhaps not at full speed) on implementing CIP Version 5.  Given that, it should also be no surprise that the NERC and WECC staff members, in their presentations, didn’t bring up any fundamental issues with v5.  They certainly did mention various issues that needed to be resolved, but none of the “hold the presses” variety.  Indeed, how could they stay in their current jobs unless they truly believed this?  NERC is committed to implementing CIP v5 as written; if you don’t think that can really be done, you should seek employment elsewhere.

If you’ve read any of my posts since April 2013, you know that I think the foundation of CIP v5 is rotten.  The foundation consists primarily of CIP-002-5.1.  I recently documented twenty serious problems with that standard, and I’m sure I could add another 15 – 20 problems to that today without much effort (some of these came out directly or inadvertently in the CIPUG presentations).

But the “foundations” of CIP v5 aren’t just in CIP-002; they’re part of other standards as well, especially CIP-005-5.  And this is why I found the presentation by Morgan King of WECC on CIP-005 Lessons Learned to be so interesting.  Morgan did a very good job of discussing four Lessons Learned (none finalized as of yet, and one or two not even released in draft form) that relate to CIP-005.  His presentation provides some good information, although I really wish there were recordings available, since the Q&A was the really interesting part.

The Q&A for Morgan’s presentation clearly revealed something I’d already suspected: as you[iii] start to probe more deeply into any particular question about CIP v5, you’re almost certain to uncover a number of additional questions.  And so it went with Morgan.  He addressed four particular Lessons Learned, but I’d say there were 3 - 5 new questions raised about each one (and if there weren’t five questions raised, it’s only because discussion had to be shut off to move to the next presentation.  There could have easily been at least a whole half day of Q&A just on Morgan’s presentation; this is true for a few of the other presentations as well, especially those of Dr. Joe Baugh on CIP v5 Pilot Study Lessons Learned and Lisa Wood on Low Impact Assets[iv] in v5.  I highly recommend WECC expand the CIPUG to a day and a half, just like the CUG).

I unfortunately didn’t take notes on the different issues that were raised – they came fast and furious, and as I said Morgan (as well as all of the presenters) was under a lot of pressure to finish up so the next presentation could start.   I do remember there were a lot of questions on the External Routable Connectivity discussion at the beginning of the presentation and the Virtualization discussion at the end.  Morgan handled these new questions in the only way he could be expected to: by saying the NERC Transition Advisory Group (of which he’s an active member) hadn’t addressed them yet.[v]

To illustrate what I said about new questions being raised as soon as you try to address one question in CIP v5, I’ll point you to page 18 in Morgan’s presentation.  There, he lists three criteria for the presence of External Routable Connectivity, including “Would the misuse or disruption of those routable protocols or BES Cyber Assets have an adverse impact on the BES within 15 minutes?”  This isn’t a question that was asked at the meeting, but I’ll ask it now: What does adverse BES impact – within 15 minutes or 15 years – have to do with the question whether a cyber asset has ERC?  Adverse BES impact is certainly important for determining whether a Cyber Asset meets the BES Cyber Asset definition (as I discussed in this post), but it has nothing to do with ERC.

Note (Feb. 2):  Morgan emailed me this morning to point out that his slide 18 (the one discussed above) had been unclear; the third bullet point (quoted above) really had to do with the question whether the protocol converter was a BCS (see slide 12), in which case the question about adverse BES impact does make sense.  He said he mentioned this during the presentation, which I don't doubt - there was so much he had to say and he (like the other presenters) was rushing through it so quickly that I couldn't really absorb even half of the things he said. Besides extending the CIPUG to a day and a half (it's really about 6 hours now), WECC should also make the webinar recording publicly available (they webcast the CUG/CIPUG for WECC members, who have to pay to "attend" that, just as they do to attend the live event).

And this brings up another topic.  In his presentation, Tobias Whitney of NERC said that one way NERC plans to get more information out to entities regarding CIP v5 is the new "CIP University". No, CU won't consist of nice Gothic buildings with ivy on them.  It will make CIP meetings hosted by the different regions available to all NERC attendees.  This is nice, but it isn't going to make a huge difference, since I don't believe any of the other regions make their meetings available by webcast like WECC does; requiring people to attend in person, with limited travel budgets, isn't going to greatly increase the learning opportunities.

More importantly, what WECC is doing for CIP v5 education is far beyond what any of the other regions have done.  WECC has had three two-day v5 workshops, and will be having two workshops on Low impact assets (one is this week, although it's sold out).  This is in addition to the three-times-a-year CIPUGs.  I don't know any other region that has had more than 2-3 days worth of CIP v5 workshops so far (a couple have had zero that I know of); this isn't surprising, since WECC is far more than twice the size of any other region.  I have always recommended that people from other regions attend WECC meetings.  This is allowed (although I don't know if the webinars are made available to non-WECC entities), and is encouraged because there really isn't a lot of WECC-specific content in the meetings (there was virtually none in the CIPUG); everyone can benefit from them. Maybe WECC can offer to expand their meetings and make them all available to NERC entities online.

Another issue I had with Morgan’s presentation was at the end, when he said – probably as part of his response to a question – that a network switch would be a BES Cyber Asset.  I recently wrote a post pointing out that another NERC auditor (different region) strongly believes switches should not be BCAs.  I won’t say who is right on this matter (although I lean toward the other auditor’s position).  However, this shows there are some fundamental questions that are being seriously debated now (or should be debated, if they’re not) within NERC – exactly 14 months before the High/Medium compliance date.  Anybody else see a problem with this?

Speaking of getting on to the next topic, it’s time for me to get on to the next fantasy that was revealed at the CIPUG.  Suffice it to say that the primary “lesson learned” I took away from Morgan’s presentation (as well as a couple others) was that there can be no end to the questions raised about CIP v5, at least within a finite time period such as, say, the 14 months between now and April 1, 2016.

Fantasy Number Two: The Interpretation Issues with CIP v5 are Manageable

The previous paragraph is a great lead-in to this fantasy.  I state again that I’m not pointing a finger at any particular individuals as subject to the fantasies discussed in this post, but I will use the presentation by Tobias Whitney of NERC – and his response to a question I asked him in the meeting – as an illustration of this fantasy.

Tobias’ presentation was titled “Version 5 Pilot, RAI Initiative and Transition Guidance”.  It was good, and especially important because he –as the person in charge of all of this – was the one delivering it.  A highlight was his list of 15 (or so) Lessons Learned that he promised would be addressed by April 1 of this year.  He also mentioned that entities should submit any new questions to their regions, who will then submit them to NERC.

My question to Tobias was in essence the following, but it was much shorter: “Tobias, it’s wonderful that you’re addressing 15 questions by April 1.  By the way, you didn’t mention that you’re using FAQs to address some other questions, but you have addressed maybe 30 additional questions that way, and will undoubtedly do more FAQs as well.  However , as we saw in the presentations earlier today – especially Morgan’s – the questions keep metastasizing, so that as you probe deeper into almost any one of them you find a number of further questions, and so on perhaps ad infinitum.[vi]  I’m sure that NERC entities could today come up collectively with over 500 serious questions on CIP v5, with more appearing all the time.

“I’m not asking you to tell me when every v5 question will be addressed.  You obviously can’t tell me that until you have a list of all v5 questions.  But since these questions are clearly growing daily, what can NERC do to at least develop and maintain a comprehensive online list of v5 questions that have been asked to date?  These would be questions that don’t have an easy answer by referring to the wording of the standards, or one of the guidance documents like the Lessons Learned (although those have to be answered as well).  I think this list would provide a big benefit just by itself, even though it wouldn’t actually answer any questions.  Even though NERC entities wouldn’t have answers to most of these questions, they would at least have a rough idea of the size of the elephant as they take bites out of it.”

Tobias’ answer to me was quite interesting, and not at all what I expected.  He didn’t dispute the idea that there were many questions on v5 that NERC hasn’t even thought of yet, or that NERC ultimately will struggle to address every question that comes up[vii].  What he said was that a comprehensive list of questions might be a bad idea because it could cause entities to get discouraged and slow down or even stop their current efforts to come into v5 compliance!  When I expressed my surprise at this answer, he backed away from it, but later seemed to come back to it when he mentioned the danger of “paralysis by analysis”.

Think of what this means.  He seems to be saying NERC entities need to at some point simply charge ahead and do their best to come into compliance with v5, even though they may have all sorts of questions (both officially acknowledged and unacknowledged questions, as well as ones that are unknown at the moment) that could call into question whether portions of their effort are actually in error and need to be re-done or simply abandoned.[viii]

This might be good advice for adventurers heading off to explore new territory.  For them, it’s obviously impossible to know in advance all the obstacles that may lie ahead (otherwise, it wouldn’t be new territory).  But it doesn’t exactly strike me as wonderful advice for organizations that are moving to comply with a new set of standards, where there can potentially be huge fines for non-compliance.  What if Entity A takes his advice, puts aside any questions they have about whether they’re properly identifying BES Cyber Systems, and proceeds to develop an entire v5 compliance program based on a set of “BCS” that, as it turns out, weren’t properly identified in the first place?  Is Tobias saying they won’t get assessed a PV when an auditor – maybe four years from now – realizes they have completely missed the boat?

Maybe he is saying this.  I’ve already said regarding CIP-002-5.1 R1 that, not only should it be declared an “open” requirement with no PV’s assessed for good-faith efforts to comply, but that it will be an open requirement even if not actually “declared” so.  This is because no NERC auditor is going to assess a violation for a requirement that is ambiguously worded, and which the entity has tried their best to understand and comply with.  Maybe this idea really applies to all of the CIP v5 standards, not just CIP-002.[ix]  In other words, maybe the entire set of v5 standards should be declared open; and even if not declared so, they will be anyway because the auditors won’t assess PVs.

If Tobias really means to declare all of the v5 standards to be open ones, he of course needs to first get NERC and FERC on board with that idea; at the moment I think that would be quite a challenge.  So maybe his idea is to either postpone the v5 compliance dates (as I advocated in this post), or to declare just the first year of compliance to be an “open” one (which I didn’t advocate, although I’m not against this as long as it’s stated explicitly by NERC – not just left up to the discretion of the auditors.  There’s far too much auditor discretion already, and nobody is unhappier about that situation than the auditors themselves.  They want clear guidance).[x]

In any case, it should be clear that I’m not satisfied with Tobias’ answer to me.  I certainly agree that entities shouldn’t stop their CIP v5 compliance efforts at this point.  But I don’t see any way they can go ahead with an untroubled mind – as Tobias wants them to do – without the compliance date being moved back or the whole of CIP v5 being declared “open” for a year or so.[xi]

Fantasy No. 3: “We’ve Got it Under Control”

The third fantasy I identified at the CIPUG (and have identified before) is one indulged in by some compliance personnel at Registered Entities.  They believe they have CIP v5 pretty well figured out, and just need to fill in the blanks in order to complete their compliance implementation.

Let me say that I don’t think these people are lying when they say this.  They honestly believe that – while there are admittedly some less-important questions that need to be resolved – the fundamental concepts in CIP v5 are clear.  And why shouldn’t they believe that?  Every NERC presentation, webinar, bulletin, etc. says or implies the same thing.  I have yet to see a presentation by a NERC staff member who says, “Yes, for this important v5 issue we haven’t a clue about what we’re going to do to address it.”  Yet I could name a number of very fundamental issues[xii] for which that is exactly the case (or at least NERC hasn't announced they're addressing them).  

I quoted the physicist Richard Feynman in a recent post, who famously said “If you think you understand quantum mechanics, you don't understand quantum mechanics.”  I’ll paraphrase that: “If you think you understand how NERC CIP v5 works, you don’t understand how NERC CIP v5 works.”  As someone who has spent close to two years trying to understand how CIP v5 works and isn’t much further toward that goal than when I started, I absolutely believe this to be true.  No matter how smart you are, how many people you’ve talked to, or how many conferences you’ve attended, if you think you have CIP v5 down pat conceptually and you can just unthinkingly forge ahead to compliance, you’re living in Fantasyland.  The utmost humility is called for when CIP Version 5 is concerned.

As an example of this, one consultant and I were discussing at CIPUG the entities who say they understand CIP v5, then say that they really just have to identify all of their Critical Cyber Assets from CIP v3 as BES Cyber Systems, and they’ll be done with their BCS identification (one entity said that to me at the CIPUG).  If you’re one of these, I won’t say you’re wrong when you think that your lists of BCS (or BCAs) and CCAs will be the same; you may well be right about this.  However, you are wrong if you think the auditor will be satisfied if you tell him/her you just took your CCA list and made it your BCS list.  CIP-002-5.1 R1 requires you to develop and document a methodology for identifying BES Cyber Systems, and to apply that methodology to identify BCS.  You need to develop the methodology, and then run every Cyber Asset you have through it to develop your BCS list.  If it turns out the result identifies the exact same cyber assets that were on your CCA list, great.  But you can never assume that at the start.

So this is the last of the three fantasies I saw running rampant at the WECC CIPUG last week.  Like the measles, these things seem to spread at Disneyland.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Honeywell.

Feb. 9: I've written a sequel of sorts to this post, which you can find here.

---

[i] Of course, institutional fantasies are nothing unique to the NERC world.  In fact, there is probably no institution that doesn’t indulge in its own fantasies, since not to do so would make it impossible for the employees to do their work.  Examples (and much more serious ones) include:

·         I’m sure employees of tobacco companies – in the years before the companies finally admitted the documented serious health effects were in fact real – were trained in how to respond when people brought up these effects to them: by denying that there were such effects (and probably believing that what they said was true).  What else could they possibly do?

·         I’m also sure that most US Congressmen/Congresswomen and Senators don’t believe that the vast majority of the work they’ve been doing in recent years has been completely futile, if not outright counterproductive.  How can you possibly stand before the voters and ask to be re-elected, with any other belief in your mind?

·         There are people to this day that assert that the Vietnam War produced some positive good for the people of Vietnam.  Again, if you were closely involved with that effort, how could you say otherwise?

[ii] A few of the CUG presentations will be of interest to CIP compliance professionals, even if they don’t deal with any other NERC standards.  This includes one presentation on the BES definition and about five on RAI – two with “RAI” in the title, as well as presentations labeled “Internal Controls” and “Risk-Based Framework”.

[iii] And by “you” I mean a cyber security professional.  I am not that, but there were of course many of them in the room at the CIPUG.

[iv] I notice now that Lisa’s presentation is shown as being about “Low Impact BES Cyber Systems” on the WECC page, but the actual title on the slides says “Low Impact Assets”.  Of course, the politically correct way to say this is “Assets Containing a Low Impact BCS” (and Lisa did catch herself and use that phrase a few times), which is a nonsensical attempt to bridge the gap between the two completely different points of view from which CIP-002-5.1 was written (without any clear reconciliation).  I’ve written about this sorry mess a number of times, including in this post under the heading “Have an Apple, Adam?”

[v] I much prefer Morgan’s approach to answering hard questions to that of a NERC manager who often addresses industry meetings, and who seems to feel compelled to answer every question that gets raised.  This has led him multiple times to say things that later have to be retracted or reworded by NERC.  Unless you’re going to say that the opinions you’re expressing are entirely your own (as I do in this blog), you shouldn’t be making statements without confirming them with the organization under whose auspices you’re making those statements.

[vi] I have wondered why the situation is so much different with CIP v5 than it was with v1-v3, where there were certainly some questions but they seemed to be much fewer, and much more contained.  I believe the problem is that v5 was much more ambitious, and requires the entity to make judgments about a number of areas that weren’t relevant in CIP v1-3.  The bright-line criteria are one example of this, but the biggest example is probably the concepts of BES Cyber Asset and BES Cyber System.  Just look at my recent post on “methodology” for BCS identification and classification to see how fiendishly complicated – and ill-defined – the concepts of BCA and BCS really are.  I hope to have a post on this topic in the future.

[vii] In fact, I will state unequivocally that there is no way NERC will ever be able to address all of the questions with v5, no matter what time frame you look at.

[viii] In fact, there is a real danger that some entities’ CIP v5 compliance efforts may be entirely for naught if it turns out they guessed the answer to a particular question wrong.  Let’s say your entity has only one Medium impact substation, and that the Attachment 1 criterion it falls under is ambiguous.  You go ahead and spend a million or so (which isn’t a lot for a large entity’s compliance program, but is huge for a small entity’s) implementing compliance with all the requirements, for that substation.  Four years later, you get audited and the auditor casually mentions, “Oh, that criterion was recently clarified by (someone at NERC), and your substation would now be considered Low impact.”  Wouldn’t that make you feel wonderful?

[ix] Of course, by CIP v5 I really mean the combination of v5, v6 and v7 standards that entities will actually have to comply with, which I have otherwise called CIP v6.3940.

[x] Even if CIP v5 is declared “open” for a year or so (I’m sure the idea of making it permanently open wouldn’t fly with FERC or Congress), NERC also needs to write a Standards Authorization Request for a complete rewrite of CIP-002-5.1.  While the other CIP v5 standards can probably be salvaged with enough interpretation effort, CIP-002 is beyond salvation.  It needs to be condemned to the eternal fires and be reborn in a completely new standard. That will take a few years, but at the end CIP v5 will be on a solid foundation.  Without that, there will always be questions about whether an entity has properly identified and classified its BES Cyber Systems in the first place, even if there are no remaining questions about the other v5 standards.  I called for rewriting CIP-002 in this post.

[xi] At one point in his presentation, Tobias asked how many entities had completed their CIP v5 compliance implementation.  He seemed genuinely surprised when nobody – of the 350 people in the room – raised their hand.  I would have been astounded if anyone had.

[xii] Here are five examples of fundamental issues that NERC doesn’t even plan to resolve, as far as I know: a) the use of Facilities vs. assets in the bright-line criteria; b) the meaning of “impact the BES”; c) the status of the term “Group of Facilities” (discussed in the CIP-002 Guidance) as it relates to the Criteria; d) whether connectivity has anything to do with whether a Cyber Asset is a BCA; and e) whether entities are advised to group BCAs into different BCSs depending on the requirement (this is implied to be desirable in the Lessons Learned document on “Grouping BES Cyber Assets”, but strikes me – and others – as a recipe for utter chaos).  These are all quite fundamental questions that NERC hasn’t even said they’ll produce guidance on (I intend to do posts on all of them, not just the first two, which I have already addressed).  I could definitely list a few more, except I’ve already been working on this post all day (fortunately, there’s a huge blizzard going on in Chicago as I write this, and I don’t feel bad that I’m not doing something outside the house).