Smart Grid Regulation - Coming to a State Near You?

All opinions expressed herein are mine, not necessarily those of Honeywell International, Inc.

Much ink (including mine) has been spilled regarding cyber security regulations for the Bulk Electric System (BES).  Whatever your opinion of them, the NERC CIP standards are in place. But the distribution system is another story.  CIP doesn’t apply to Distribution, because it’s not part of the BES.  This wouldn’t have been a problem even five years ago, since the distribution system was fairly ‘dumb’.  There weren’t a lot of intelligent devices to attack – certainly not the meters, and very little of the distribution equipment in the substations.  It was very hard to imagine how someone could cause a large-scale distribution failure, and almost impossible to imagine how such an attack could cause problems beyond the immediate areas affected.

But as we all know, Distribution is rapidly changing.  There are millions of smart meters already deployed, and almost every utility has a substation automation project either in process or about to start; yet this is all just a down payment on what’s to come.  Not only are there many more intelligent devices waiting to be attacked in the distribution system, but the consequences of those attacks could potentially be widespread – not just in one neighborhood or town.

This isn’t to say that there is a huge cyber security problem in the Smart Grid – in fact, I doubt there is.  However, what I or you think is irrelevant.  The fact is that wide deployment of the Smart Grid depends on the public’s acceptance of the fact that it will improve their lives (and remember, they have to be willing to pay for it).  Were the idea to become rooted that the Smart Grid is insecure, that could very well mark the beginning of the end (Pacific Gas and Electric has already run into cyber security concerns regarding their smart meter rollout).

Which then raises the question: How can we prevent this from happening?  Waiting until a substantial portion of the public has become convinced that the Smart Grid is insecure, then unleashing a fusillade of assurances from cyber security experts, is clearly not the answer.  We all know who will win that one.  I think the only thing that will assure the vast majority of utility customers is regulation.  If regulations are in place that require a certain level of cyber security practices on the part of the utilities and the vendors, this will allow Smart Grid deployments to go forward despite the cyber security scares that will regularly show up, justified or not.[1]

So who should do the regulating, the Feds or the states?  I think the answer is fairly clear: On the Federal level, FERC and NERC don’t currently have authority to regulate the Smart Grid (or power distribution in general), and they have no desire to do so.  The only other likely Federal regulator would be the Department of Energy, but they don’t have that authority and have made no attempt to obtain it.  NIST developed – with much industry assistance – the comprehensive set of cyber security guidelines contained in NISTIR 7628.  While this is a very useful document, it does not at all pretend to be regulations or even guidelines.

On the state level, the story is quite different.  The state Public Utility Commissions already have extensive authority to regulate electricity distribution.  And they are stepping up to the table to meet the challenge of assuring the public that the Smart Grid is “cyber safe”.

There are two documents that are particularly relevant to this.  The first was published this June by Miles Keogh and Christina Cody of the National Association of Regulatory Utility Commissioners, entitled “Cybersecurity for State Regulators”.[2]  It is a very well written document that describes the cyber security and regulatory landscape as it relates to electric power, and lays out several steps that state regulators can take to help address the issue of Distribution-level (and especially Smart Grid) cyber security.  The most important of these steps is to ask questions of their utilities regarding their cyber security policies and procedures.  These questions are listed in Appendix A, and I recommend them as a great cyber security “pop quiz” for any electric utility (in fact, they would be very relevant to a lot of other organizations, such as gas and water utilities).

However, you won’t see a recommendation for actual cyber security regulations in this document.  The authors don’t rule that out – and they discuss the relative advantages of “risk-based” and “compliance-based” approaches to cyber security – but they don’t make any recommendation for or against regulations.

The second document was published on September 19 by Elizaveta Malashenko, Chris Villareal, and J. David Erickson of the California Public Utilities Commission (CPUC).  It is entitled “Cybersecurity and the Evolving Role of State Regulation”.  Like the NARUC document, it is very well written, and includes a good overview of cyber security as it relates to electric (and gas) utilities, as well as an excellent review of government initiatives to address this – on both the Federal and state levels.

Unlike the NARUC document, this document (written by CPUC staff members) does call on the CPUC commissioners (page 22) to consider various options for regulation of cyber security for California electric power distribution in general and Smart Grid deployment in particular.  But the document clearly doesn’t favor a prescriptive approach as in NERC CIP.  Rather, the authors believe that a risk-based approach, in which each utility (with active guidance from the CPUC) analyzes its risks and decides how to address them, is best. 

You may say, “OK, so California may regulate Smart Grid cyber security.  I’m not in California – why should I care about that?”  The point is that California has been the leader in many areas of regulation (I think about California every time I make a right turn on red, since they were the first to allow that).  This is true in information security, where California SB 1386 (which came into effect in 2003) was the first law requiring organizations to notify individuals when their personal information was compromised in a security breach; there are now similar laws in effect in 46 states. 

Indeed, the authors state (page 21), “If the CPUC takes action, it can not only potentially protect Californians from safety and reliability threats, but also provide an example for other State regulatory agencies.”  So if the California commissioners take up their staff’s recommendation, Smart Grid cyber security regulation may truly be “coming to a state near you”!

---

[1] An analogous example from another era is what’s now called the Food and Drug Administration, which was put in place in the face of revelations of awful conditions in meat packing plants.  Had that not happened, the US now might well be a vegetarian nation!

[2]NARUC has come out with Version 2 of this document.  It is available here.  I hope to update this post soon with a discussion of that.

Early approval of CIP V5 is very unlikely

All opinions expressed below are mine, not necessarily those of Honeywell International, Inc.

(May 8: Well, I guess this is my "Dewey Beats Truman" post.  Of course, I could delete it and spare myself some embarrassment.  But as Richard Nixon famously said on one of his tapes, "That would be wrong."  If you'd like to see my analysis of where I went wrong, see this post - Tom)

There is a general expectation – including among many at NERC – that CIP Version 5 will be approved by FERC in time for CIP Version 4 not to come into effect.   While I certainly would like to see this happen, this seems to me to be extremely wishful thinking.  Here’s why:

Just think about the timeframe.  It really doesn't do any good for V5 to be approved the day before V4 comes into effect on 4/1/2014 - everyone will have already done the V4 compliance work by that time.  FERC really needs to approve V5 at least 6-9 months before that date, meaning by July or October of this year. 

And to say the Commission will approve V5 means they have to have made up their minds.  For the Commissioners to make up their minds, the staff needs to first complete their analysis, then the Commissioners need to take some time to decide whether there’s a good enough chance that they will approve V5 that they should issue a NOPR.  Then they have to issue the NOPR, get comments and analyze them.  I don’t think any of these steps can be avoided.

In fact, if you just think about the NOPR, that almost seals the argument.  FERC issued the V4 NOPR in mid-September 2011 and approved V4 in mid-April 2012; there were seven months between those dates.  This means that, for FERC to approve V4 by October 1 of this year (as I’ve said they really have to do), they have to issue the NOPR in three weeks (if they’re going to take the same seven months).  I’m sure their staff won’t even have their analysis done for them for a couple more months (a FERC staff member speaking at MRO’s compliance meeting last December said it takes a month just to assign a case number).

 However, there are other things I think FERC may need to do as well, including:

• Holding workshops, as they did with CIP Version 1.  V5 is a huge change in CIP, and they’re likely to seek a lot of discussion.  These workshops alone should add 2-3 months to the approval process.
• Ordering NERC to conduct a new survey of assets, as they did in 2010.  There are a lot of questions about how many facilities are covered in V5, and at what level; in order to judge the bright line criteria in V5, I think FERC will require a survey.  The survey took three months in 2010.
• Considering any serious objections from their staff, such as Stephen Flanagan’s (and I am sure there are other staff objections as well).
• I also think it is likely FERC will send V5 back to NERC and require changes (such as the ones discussed in this post); FERC can’t simply change the standard on its own.  This will be at least a 90-day process, and more if they require significant changes in V5. 

Remember, FERC approved V4 fourteen months after NERC submitted it to them.  There are exactly 14 months from when NERC submitted V5 (end of January) through April 1, 2014, when V4 goes into effect.  V4 was exactly the same as V3 except for CIP-002; V5 is a radical revision of everything in CIP.  Does it stand to reason that FERC would even approve V5 in the same amount of time as they did V4, let alone shave 5 or 6 months off of that?

I’m not saying FERC couldn’t move heaven and earth and get V5 approved early if they really wanted to.  And they could decide to move the V4 compliance date back, if – say – this summer they decided that approval did look likely enough that they didn’t want to see the industry spend any more time or money on V4.  But should entities bet on this?  I’m told the regional auditors are gearing up to audit full compliance with Version 4 starting next April; if an entity waits ‘til next fall to start their V4 compliance work, is it really likely they’ll be compliant by that date? 

 I’ve separately recommended that the compliance date for V4 be pushed back simply because of all this uncertainty: http://tomalrichblog.blogspot.com/2013/01/why-cip-version-4-compliance-date-needs.html  But for some reason, NERC and FERC don’t just jump whenever I suggest they do something.  So I’m certainly not betting that this will happen either. 

The moral of the story: If you’re not already working on CIP V4 compliance, you should start right away – at least assess what you will need to do to come into compliance.

CIP Version 5: The Order 761 Problem

All opinions expressed herein are mine, not necessarily those of Honeywell International, Inc.

NERC’s recent filing of CIP Version 5 had to accomplish a number of objectives.  One of the most important was to explain to FERC that NERC has complied with the directives for Version 5 that FERC gave in Order 761, issued last April.  Indeed, the filing devotes ten pages to explaining why Version 5 does meet these directives.[i]  Is this really the case?

I bring this up because NERC is obviously counting very heavily on FERC’s approving Version 5 very quickly (given their repeated appeals to FERC in the filing to do so).  A lot of NERC entities are putting off Version 4 compliance activities - which need to be finished by April 1, 2014 - in the hope that Version 5 will be approved before that date and thus supersede Version 4.  If there is a substantial question whether NERC has in fact satisfied FERC’s directives in V5, this doesn’t bode well for speedy FERC approval, and therefore for the idea that Version 4 will be bypassed.

Here are the specific FERC directives that NERC discusses in the filing (the page references are from the filing), as well as my opinion on whether or not NERC has addressed each directive:

“Application of NIST Risk Management Framework” (Section V part b, pp. 31-34) – In NERC’s words, Order 761 “urged NERC to review relevant NIST standards for guidance in developing effective cybersecurity standards for the electric industry.”  NERC’s discussion of how they have done this seems to be primarily a justification of the “identify, assess and correct” approach used in 17 CIP standards (which may or may not have anything to do with FERC’s directive).  I discussed that issue in a separate post.  However, my guess is FERC is going to give NERC a pass on this one; I agree they have probably addressed this directive as much as reasonably possible.

“Regional Perspective” (Section V part c, pp 35-37) - In paragraphs 101 to 104 of Order 761, FERC points out that in Order 706 they said there was a need for NERC or the Regional Entities to be able to designate Critical Assets, when for whatever reason a NERC entity didn’t do that in the case of a clearly critical asset.  They go on to agree with commenters that there is less of a need for this because of implementation of the bright-line criteria in Version 5 (versus the RBAM in Versions 1-3, which gave the Registered Entity substantial discretion in designating Critical Assets). 

However, they state that they still see the need for a “limited” capability for NERC or the Region to designate a facility as critical when the criteria for whatever reason don’t make it so (of course, since we’re talking about CIP Version 5, you should substitute “Medium or High impact BES Facility” for “Critical Asset” in this whole discussion.  FERC had to say “Critical Asset” since Version 5 hadn’t been presented to them last April).  This is a pretty clear directive to NERC.

In the filing, how does NERC say they have met this directive?  They at first act as if only the first two sentences of the above paragraph were applicable: i.e. they “agree” with FERC that the bright-line criteria in CIP Version 5 obviate the need for any external review of Critical Asset designation.   Is this true?  I certainly don’t think so.  As I have argued elsewhere, the so-called bright lines are hardly bright; they will need a lot of interpretation (and this isn’t because of the particular criteria included in Version 5.  In an industry as diverse and fragmented as electric power, any criteria would require a lot of interpretation).  I think both FERC and NERC are being naïve here.

However, FERC goes on to say (paragraph 103 of Order 761) that, even with bright-line criteria, there will still be the need for regional or NERC review.  How does NERC address this?  They point out that the Rules of Procedure allow them, if need be, to issue Recommendations and Essential Actions.  They say, “NERC can use Level 2 Recommendations and Level 3 Essential Actions to address assets that NERC and Regional Entities later determine should be treated as a higher impact level than would otherwise be categorized under the CIP Version 5 impact criteria.”

Will this statement satisfy FERC that there is no need for any other review of designation of High and Medium impact BES facilities?  I really don’t think so.  The point of Recommendations and Essential Actions is that they apply to all NERC entities.  They would have to be worded as clarifications to the bright-line criteria, saying something like “Notwithstanding anything in Attachment 1 of CIP-002-4, if you have a facility that meets… (a certain description), it needs to be a High (or Medium) impact BES Facility.” 

Why is this not enough?  Because FERC wants NERC and the Regional Entities to be able to designate particular facilities (plant X, substation Y) as High or Medium impact.  But NERC is saying they have the ability to essentially require that all facilities that meet a particular description should be High or Medium; isn’t that better than designating one particular facility?  In my opinion, the answer to that is no.  There are a lot of particular reasons why one – say – generating station might be deemed Medium impact (of course, no gen station can be a High) while a seemingly similar station in a different location might not be considered such.  If a general rule is issued by NERC, the entities will always disagree with whether it applies to their particular facility or not (after all, we’re talking about a lot of money here!).

There is another reason why I think FERC won’t be satisfied with NERC’s answer.  They want both NERC and the Regional Entities to be able to designate High and Medium impact facilities.  As far as I know, the Regional Entities can’t issue Recommendations or Essential Actions.  So they would be precluded from doing this, if NERC has its way.  Bottom line: I don’t think FERC is going to agree that NERC has met their Order 761 directive to have NERC or the Regions be able to designate High or Medium impact BES Facilities (“Critical Assets”).

“Connectivity (1)” (Section V part d, pp 37-39) – In the section entitled “Connectivity” in the Version 5 filing, NERC actually addresses three separate issues that had been raised by FERC in Order 761.  I will discuss these separately, since I think FERC will judge NERC’s response differently in the three cases.

The first of the three connectivity issues is that of impact of the cyber system.  Specifically, FERC says (in paragraphs 52 and 88) that connectivity of the facility, at which the cyber system is located, with other BES facilities should be considered in classifying the cyber system.  What this means for Version 5 is that connectivity of the BES facility (and therefore of the BES Cyber Systems associated with it) should be used to determine whether it is High, Medium or Low impact.

In practice, what FERC is talking about here are control centers – they’re saying they should all be classified as High or Medium impact.  CIP Version 5 does make an effort to include more control centers than Version 4 did – that is, include them as High or Medium impact.   So I think NERC has addressed this particular concern of FERC’s.[ii]

“Connectivity (2)” (Section V part d, pp 39-40) – The second connectivity issue that FERC raised has to do with Mutual Distrust (no, this doesn’t refer to the relationship between FERC and NERC.  FERC says Mutual Distrust denotes “how ‘outside world’ systems are treated by those inside the control system[iii].”).  While I don’t agree with NERC’s reasoning in this case[iv], I do agree that Version 5 has incorporated the principle of Mutual Distrust.

“Connectivity (3)” (Section V part d, pp 40-41) – In paragraph 87 of Order 761, FERC said “we support the concept of applying electronic security perimeter protections ‘of some form’ to all bulk electric system cyber systems.”  NERC does cite this quotation (although not all of it), but then somehow connects it to Mutual Distrust.

It really has nothing to do with Mutual Distrust.  FERC is saying that they want to see every BES cyber system enclosed within an ESP – whether it’s High, Medium or Low impact.  Does Version 5 do this?  NERC says yes, citing CIP-003-5 R2, which states that, for Low impact BES Cyber Systems, the entity must implement “cyber security policies that collectively address the following topics…”  Those topics include “Electronic access controls for external routable protocol connections and Dial-up Connectivity.”    

Does this amount to a requirement that Low impact BES Cyber Systems be enclosed in an ESP?  I don’t believe it does, but let’s concede the point for the moment.  How will NERC audit to ensure that all BES Cyber Systems are within an ESP?  FERC is going to want this “requirement” to be auditable.

The problem is that CIP-003-5 R2 concludes by saying “An inventory, list, or discrete identification of low impact BES Cyber Systems or their BES Cyber Assets is not required.”  When the auditor comes around and wants proof that all of the Low impact BES Cyber Systems are in an ESP, how will the entity ever be able to provide that?  They may point to a few systems and show they’re in the ESP.  But they can never prove that all of them are in the ESP unless they have a list of what they all are. 

I will point out that I had a couple long arguments with the Standards Drafting Team about exactly this point – that of auditability - last year, to no avail.  They had their good reasons of course, but the fact is that, had V5 required NERC entities to inventory their Low impact BES Cyber Systems, it would never have passed the NERC ballot (many entities believe that the task of inventorying all of their BES cyber assets would be a monumental one.  I’m sure this would be the case for at least some of them).  The SDT was between the proverbial rock and a hard place, and since they had to get Version 5 out the door, they in essence kicked the problem upstairs to FERC.  They in effect said, “This is too political a problem for us to solve on this level.  If you decide you need to impose this on your own, go ahead[v].”  (Of course, this is all my interpretation.  Nobody on the SDT would agree with this)  So I think this is the second case in which FERC will conclude that NERC has not met a directive in Order 761. 

Now to my conclusion: There are at least two cases in which it seems likely FERC will conclude that NERC has not met its directives for CIP Version 5[vi].  This doesn’t mean they’ll simply disapprove it – most likely, they’ll conditionally approve it and also require a compliance filing (in something like 90 days) to correct the deficiencies (as was done when FERC approved CIP Version 2 but required a new filing in 90 days to include a new requirement). 

However, this does mean that any hope that FERC will approve Version 5 before April 1, 2014 (and thus bypass Version 4) is forlorn at best[vii].  All of this will require a lot of time for FERC to address.  You had better not build your CIP compliance strategy on the hope that Version 4 will be bypassed by Version 5, Mr./Ms. NERC Entity.

  

---

[i] In Order 761, FERC was clear that any directives they gave for Version 5 were merely restatements of directives in Order 706 (issued in January 2008).  Of course, there were a lot of directives in 706!  Order 761 presumably repeated the ones that FERC was most concerned about.

[ii] An Interested Party has added this note: “The question is, are there any ICCP or other cyber systems connected to a control center from other than another control center that fall through the cracks?  These interconnected systems communicate over trusted paths that can be exploited.  The trusted path generally defeats the mutual distrust that otherwise is protecting the BES Cyber Systems and is one of the reasons the jump host for interactive access has to sit in a DMZ outside of the ESP.”  I agree this is a legitimate concern, but I still think NERC has addressed FERC’s directive in this case.

[iii] The use of “control system” here is unfortunate.  I think FERC means something like “inside the control network”.

[iv] In explaining why they have incorporated Mutual Distrust in CIP Version 5, NERC essentially assumes that their argument in what I call “Connectivity (3)” is correct.  As I say in that section, I don’t believe their argument is correct. 

[v] I believe this is also the reason why the SDT didn’t address the FERC directive to allow NERC or Regional designation of High or Medium BES Facilities.  They knew it would kill the chances of passing Version 5.

[vi] There are other reasons why I believe FERC will want to amend Version 5.  See this post for more on that.

[vii] And there are other reasons – besides the likelihood that FERC will require changes – that make it very unlikely Version 5 will sail through approval.  See this post for more on that.

Is CIP Version 5 "Un-Auditable"?

All opinions expressed herein are mine, not necessarily those of Honeywell International, Inc.

NESCO recently posted a paper written by Stephen Flanagan of FERC entitled “Self‐Correcting Cyber Policies: Pathway to Convergence of Compliance and Security?”  Stephen’s primary point is this: NERC CIP Version 5 contains a number of requirements that would be impossible to audit.  He is presumably recommending that the FERC commissioners either reject CIP Version 5 entirely or require NERC to make substantial changes.

Stephen is a wonderful writer, but his prose is quite dense – he never uses five sentences when one well-crafted sentence will do (I have a vision of him toiling over each sentence like a jeweler over a diamond ring).  So I’ll warn you: you’ll need a couple readings to understand this well.  I won’t even pretend to follow everything he says, but here is a statement of at least the problem he addresses:

The language he objects to is used in seventeen requirements in CIP Version 5.  It reads “Each Responsible Entity…shall implement, in a manner that identifies, assesses, and corrects deficiencies…” (followed by one or more programs or policies for whatever the requirement addresses).  This is one of the big innovations in Version 5: the idea that, for these 17 requirements, the entity doesn’t have to report every single mistake they make (for example, not getting senior manager approval by exactly the date it is required), but rather has to have a program or policy to identify all shortfalls and correct whatever caused them to happen.  This has been much touted in the SDT’s webinars and written discussions of V5.[i]

I’ll let you read his arguments, but one of his main points is very clear: He doesn’t see how these seventeen requirements can be audited.  Is he right?  I have no idea.  Will the FERC Commissioners agree with him when they weigh all of their staff’s comments on Version 5?  Only God knows. 

Stephen doesn’t think this “identify, assess and correct” approach is a bad idea, though.  He does say it would be a good way to extend NERC oversight into areas not currently addressed by the CIP standards.  In other words, this would be the right approach for NERC to start addressing areas like application security that are simply not addressed by CIP now.  However, he clearly believes that the security domains currently included in CIP are much better addressed in the “traditional” manner of CIP Versions 1-4.[ii]

So what’s the point of this post, given that I admit I don’t know whether the Commissioners will listen to Stephen or not?  It is that this is a very clear and substantial objection to CIP Version 5, by a senior FERC staff member.  To think – as many seem to, including many at NERC - that the FERC commissioners will simply ignore this objection (and others raised by other FERC staff members) and quickly approve V5 is to engage in very wishful thinking (See my other post from today about another big issue with V5).  What is the moral of our story?  Don’t expect V5 approval anytime soon, as NERC is clearly hoping for in the filing for V5).  If you haven’t started already, move at full speed toward full compliance with CIP Version 4 (which is already approved by FERC) on April 1, 2014.

Extra Credit Section

This next part is speculative, but I’d like to ask, “What will FERC do if the Commissioners agree with Stephen?”  I see two possibilities:

1. They’ll conditionally approve Version 5, but also send it back to NERC and require changes in a short time, such as 90 days (I think they’ll require other changes as well – see this post).
2. They’ll simply disapprove V5 and the whole standards development process will have to start again.

 The first possibility is much preferable, from NERC’s point of view.  Of course, it will be a stretch to make all the changes that will be required (and hopefully get them approved by the membership, although if need be the NERC Board of Trustees will be able to submit a revised CIP V5 to FERC without membership approval.  See Section 321 paragraph 5 of the NERC Rules of Procedure).  But the NERC staff is used to working hard.

The second possibility is a lot less desirable, (again, from NERC’s point of view).  My reason for saying so is that I don’t think FERC is going to give NERC much discretion if it tells them to start over.  FERC is likely to simply dictate the standards it wants.  I really don’t think FERC has the stomach to wait while NERC develops the SAR, forms a new SDT, drafts the standards, goes through 3 or 4 ballots (as with V5), etc.   That will take 3-4 years, and FERC wants something much sooner than that (plus assurance that what comes out of the process will be something it likes).

FERC has authority to dictate the new standards to NERC, and stated so in paragraph 417 of Order 672 (which established the rules for their dealings with NERC).  And they certainly have the capability to write their own CIP standards.  Last fall, they formed their Office of Energy Infrastructure Security, which in my opinion is waiting in the wings for exactly this situation. I know the office is staffed by some very good and very experienced electric sector cyber security professionals.

To summarize, do I want FERC to change Version 5 in the way implied by Stephen’s paper?  Certainly not – I agree with NERC that the “identify, assess and correct” approach in V5 is a much better way to write cyber security standards.  But the only people whose opinions matter are the five FERC commissioners.  And they’re not going to be rushed to a decision by NERC’s desire to have one quickly.

  

---

[i] NERC justifies this approach on pages 31-34 of the Version 5 filing, already linked above.

[ii] I do want to note that these seventeen requirements were only changed to reflect the new approach after the July 2012 SDT meeting in Minneapolis.  Before that, they were like the current CIP requirements, where every lapse is a new violation.  You can see this by looking at the first and second drafts of CIP Version 5, which were written before July.

Not-so-Bright Lines?

All opinions expressed herein are mine, not necessarily those of Honeywell International, Inc.

Note: This is a post I originally made early last September.  It is a call to action for NERC to develop a guidance document for application of the Bright-Line Criteria in CIP-002 Attachment 1 in CIP Version 4.  Sadly, five months later (and five months closer to the April 1, 2014 deadline for CIP V4 compliance), we are no closer to having this.  Meanwhile, literally every conversation I have had with a NERC entity since then has brought up yet another problem (or two) with applying the criteria in a particular area, or with their particular Transmission Operator, etc.  I am more convinced than ever that this is needed. 

I had my eyes opened at the last WECC CIP User Group meeting in June, when auditor Joe Baugh gave a presentation on “Migrating to CIP-002-4”.  He set out to do something which at first sounded fairly pedestrian – discuss how WECC might audit Version 4 of CIP-002, and especially audit the entity’s application of the “bright-line” criteria for Critical Assets in Attachment 1.

This sounded almost like a wasted exercise.  The whole idea of the bright-line criteria is that they are supposed to make designating critical assets a mindless process: you go down each of your assets and apply each of the criteria in Attachment 1 to it in turn.  If one of the criteria fits, it’s critical.  If none do, it’s not.  What could be simpler?

However, the reaction to Joe’s presentation was anything but boring.  Speaker after speaker lined up at the floor microphones to point out how his rules for auditing this or that criterion in Attachment 1 would never work because of x,y,z etc.  And the reasons given were all over the place – not in any way pointing to some sort of common theme.  I believe the discussion would have gone on all day had it not been limited by the need to go on to the next agenda item.

My first reaction to this was, “Well, Joe really blew it.  He really didn’t think this through before he put his slides together.”  But later I realized that there’s no way you could put together bulletproof auditing standards for a lot of the criteria.  There are just so many ways they could be interpreted.  Joe was the pioneer who was met by a hail of arrows as he walked into new territory.  He is to be commended for doing what he did, and bringing the problem into the open.  Because – as will be evident below – I think this is a problem that needs to be addressed very soon by NERC.

So is the problem with the criteria themselves?  Did the Standards Drafting Team not take enough time to craft those carefully, to make sure there wouldn’t be any interpretation problems?  Having attended some of the SDT meetings where they spent hours discussing a single criterion – and then reopened the discussion in the following meeting – I really don’t think this is the case.

I finally realized that the very idea of bright line criteria is flawed.  I really don’t think you could ever have a set of comprehensive criteria that would be unambiguous, and for which the auditing procedures would be self-evident.  Not in this industry, anyway.

I want to illustrate the problem by discussing some of these interpretation problems.  I unfortunately wasn’t taking notes during the discussion at the WECC meeting, but I have gone back over Joe’s slides, and Attachment 1 of CIP-002-4, to identify what seem to me to be real interpretation problems with some criteria (although I probably should say “evidence problems” – since the issue is that there’s no way to produce unambiguous evidence for some of the criteria).  And I’m not an electric industry guy by any means; I’m sure those of you who are can find many more holes than I can.  Here are some of the problems I see:

Criterion 1.3 reads “Each generation Facility that the Planning Coordinator or Transmission Planner designates and informs the Generator Owner or Generator Operator as necessary to avoid BES Adverse Reliability Impacts in the long-term planning horizon.”  Joe’s slide 13 calls for the entity to produce “Studies related to generation facilities and/or other rationale for including/excluding generation Facilities under this clause”.

The problem with the auditing requirement is that it doesn’t seem to relate at all to what Criterion 1.3 calls for.  You would think that 1.3 would require the GO/GOP being audited to produce evidence that the PC or TP had designated the generation facility as “necessary to avoid BES Adverse Reliability Impacts in the long-term planning horizon”.  But Joe is saying that the entity has to show “studies” demonstrating whether or not the facility is “necessary”.(1)  Who produces those studies?  The entity, the RC/TP, another party like the Regional Entity, all of the above?  And if there are multiple studies with differing conclusions, which one do the auditors accept as definitive?  Do you see a problem here?

Criterion 1.5 reads “The Facilities comprising the Cranking Paths and meeting the initial switching requirements from the Blackstart Resource to the first interconnection point of the generation unit(s) to be started, or up to the point on the Cranking Path where two or more path options exist, as identified in the Transmission Operator’s restoration plan.”  This wording alone raises the red flag: Is it really likely that there will be an unequivocal way this can be audited – and that no disputes will arise between the entity being audited and the auditor?

Joe’s presentation says that, for Criterion 1.5, the entity will need to produce “A one-line diagram of the entity’s Transmission system and the TOP restoration plan” (the same one-line diagram is required for Criterion 1.7).  If the entity and the auditor dispute whether a facility (usually a substation in this case) meets this criterion and the entity then produces their one-line diagram, is that going to resolve the issue?

One would think that the entity wouldn’t take a position that they didn’t believe was supported by their diagram.  They will bring out their best engineers, who will fill the air with a lot of very technical discussion to support that position.  Are the auditors (many of whom do not have an electrical engineering background) going to be able to argue with them?  And who can they call on to resolve the technical dispute?  Presumably someone from the 693 side at the Regional Entity.  But what authority would these engineers have to make a determination regarding CIP compliance?

Criterion 1.8 reads: “Transmission Facilities at a single station or substation location that are identified by the Reliability Coordinator, Planning Authority or Transmission Planner as critical to the derivation of Interconnection Reliability Operating Limits (IROLs) and their associated contingencies”.(2)  For this criterion, Joe wants the entity to produce “A list of all Transmission Facilities and evidence related to the identification of Transmission Facilities by the RC, PA, or TP as critical to the WECC interconnection”.

There is a striking difference between Criterion 1.8 and Joe’s requirement: The Criterion seems to be written with the idea that there is some sort of formal document that will always be produced when an RC, PA or TP identifies a facility (again, usually a substation) as critical to derivation of IROLs.  Joe seems to admit (and I imagine he’s right) that there could be a number of different pieces of “evidence” that might constitute this act of identification.  What if they contradict each other?  Or what if the identification (or most likely the non-identification) was done verbally by the RC/PA/TP?  What individual does the auditor need to contact at these organizations to verify this?  What if the RC and the TP have different opinions on this, or even different people within the RC or TP do?

Criterion 1.12 reads: “Each Special Protection System (SPS), Remedial Action Scheme (RAS) or automated switching system that operates BES Elements that, if destroyed, degraded, misused or otherwise rendered unavailable, would cause one or more Interconnection Reliability Operating Limits (IROLs) violations for failure to operate as designed”.  Joe requires “A list of SPS, RAS, and automated switching systems that would violate IROLs”.

I confess I only have a vague idea what SPS and RAS are, but it seems to me that the list the entity produces for this criterion isn’t likely to be accepted unchallenged by the auditor.  Both the auditor and the entity will then have to bring out their EE’s to discuss any differences of opinion.  And who decides which side won that discussion?

I think you get the idea now. There are going to be a lot of disputes about at least some of the Criteria in Appendix 1 of CIP-002-4.(3)  How are those disputes going to be resolved?  I see two mechanisms currently available in NERC.

The first is an audit.  Of course, an entity can dispute any audit finding; but this is a tremendously inefficient and expensive way to resolve Attachment 1 differences.  Say you’re a Registered Entity that believes that one of your generating plants does not meet any of the Criteria in Appendix 1; when Version 4 becomes effective on April 1, 2014, you document that you have applied the criteria and have not found it to be a Critical Asset.

In 2016 (say), you get audited and your auditor believes that Criterion 1.3 does in fact apply to this plant.  You fight it but ultimately you lose.  What are your potential fines?  Well, you’ve not only violated CIP-002 R1, but just about every other requirement in CIP-002-4 through CIP-009-4 (since you presumably didn’t take the steps required to comply with the other requirements).  Say you’ve violated 40 requirements.  Your maximum fine is $40 million a day times two years…that could be fairly expensive.(4)

The only other way to resolve a dispute like this – that I know of – is to make a Request for Interpretation.  There are at least three problems with that.  First, RFIs take over a year to resolve and require at least one vote of the NERC membership.  Second, RFI’s can’t be requested (I believe) until the standards are in effect; this doesn’t help entities that are trying to decide whether or not their assets are critical in advance of April 1, 2014.  Third, the volume of RFI’s that would result from Attachment 1 would immediately completely overwhelm the system.  You can be sure that virtually every entity that has any question at all whether an asset is critical or not will file probably not one but multiple RFI’s (for different criteria) for that asset.

Of course, the entity could always ask their Regional Entity for an interpretation.  They might or might not get one, but there is a high probability the interpretations will be different across the regions – an unfortunate result, given that one of the big reasons for having the bright line criteria was to have uniformity of Critical Asset designation across the regions.

If you want to suggest CANs as a dispute resolution mechanism (and as a way to ensure uniformity across regions), forget those.  The same thing applies to them as to RFIs: the system would be completely overwhelmed.  Plus there’s no mechanism I know of by which a Registered Entity can request a CAN.

So what’s the solution?  To be honest, I don’t know of a real solution, although I know this problem can’t wait to be solved until after Version 4 comes into effect and disputes start appearing in audits.  I did at first think that a kind of “Attachment 1 Supreme Court” might be formed – perhaps from NERC Registered Entity representatives and Regional Entity auditors – to make decisions on particular cases and publish them for the entire NERC community.  But I think even that group would be completely overwhelmed by the volume of cases.

The best I can think of – and I would be very interested in hearing others’ opinions – is that NERC put out a definitive guide to interpreting Attachment 1, much like the Critical Asset and Critical Cyber Asset identification guides that were put out a couple of years ago.(5)  This would unfortunately have to be quite lengthy, since there are probably lots of different ways each criterion can be interpreted, for each asset to which it might apply.  But the size of the endeavor is more than justified by the size of the mess that will result otherwise.

I think it is quite important that this be done soon.  We’re already coming close to the minimum amount of time required to bring a sizable asset like a generating station into CIP Version 4 compliance by April 1, 2014.  Entities that are wrongly interpreting a criterion are either a) risking sizable fines if they are found to have not classified an asset as critical that should be or b) potentially spending millions of dollars (and lots of staff time) putting in place a CIP compliance program and technologies for an asset that turns out not to be critical.

And while I’m at it, I think there should be a similar document produced for CIP Version 5, since many of the bright line criteria in that version now differ significantly from the Version 4 criteria.  Of course, there’s no point in developing such a document until the V5 criteria are set in stone.  And who knows when that will be?  But that’s another blog post.

---

(1) The fact that Joe considers “studies” to be necessary seems to be an admission – probably true – that there isn’t some sort of definitive document from the PC/TP that informs the GO/GOP whether or not the generating station is “necessary to avoid BES Adverse Reliability Impacts in the long-term planning horizon”.  In this way, the argument I have with this criterion is similar to the one I have with Criterion 1.8 below.

(2) There is more to it than that, but those do not affect the point of my argument.

(3) I will admit that some of the other Criteria are fairly straightforward in their interpretation, but even having just one criterion that was ambiguous could be a big problem.   I also want to emphasize again that this discussion is in no way an attack on Joe Baugh.  In fact, I greatly admire him for having the guts to actually tackle this question, which as I said is a thankless exercise.  It certainly woke me up to this whole issue, and perhaps others as well.

(4) Obviously, the actual fine will be nowhere near this.  But even if it were one ten-thousandth of that amount, it would still be over $2 million.  This isn’t to say that the fine would even be that much, or that a fine wouldn’t even be waived because this was a good faith misunderstanding of the criteria.  But having the audit process be the only way to resolve a dispute over the bright line criteria is pretty scary.

(5) FERC Order 761, Paragraph 41 (p.24), concludes “To address the concerns of uniform implementation, the Commission believes that responsible entities would benefit from the ERO’s guidance. “  FERC is referring here to Criterion 1.3 in Attachment 1, but I would like to generalize that recommendation to all criteria.   The SDT did publish a CIP Version 4 Rationale and Implementation Reference Document in 2010 that discusses the Attachment 1 criteria, but only from the point of view of how they were derived.  It won’t furnish much help in deciding how they should be interpreted.

Will CIP Version 4 Ever Be Enforced?

All opinions expressed herein are mine, not necessarily those of Honeywell International, Inc.

Note: This article was originally posted in early December, 2012.  All of the points made in it remain valid as of February 16, 2012.  I have just put up a new post that addresses most of what is in this post in a much shorter fashion - for those pressed for time.

 

I attended the MRO compliance meetings recently in St. Paul, and was struck by two things: 1) the degree to which concerns about CIP now predominate over concerns about almost all of the other NERC standards put together, and 2) the high level of interest in CIP Versions 4 and 5 – both in their content and in the possible scenarios for their implementation.

Crowning all of the Versions 4 and 5 concerns is this one: Will the industry have to comply with CIP Version 4 – now approved by FERC and scheduled to come into effect April 1, 2014 – or will Version 4 be bypassed in favor of Version 5, which now has NERC Board of Trustees approval and will soon be submitted to FERC?

This is literally the (multi-) million dollar question for many NERC entities.  Many are desperate to avoid having to comply with V4 and then two or three years later with V5.  A V4 compliance program will be much different from a V5 program – documents, processes and procedures will mostly have to be redone.  And there are some controls required by V4 that aren’t required by V5, such as the infamous six-wall boundary of CIP-006. 

Because the two versions are applicable to differing sets of assets, an entity could literally spend millions putting in place Version 4 controls and programs for a facility that will no longer be needed under V5 because it isn’t in scope (as a Medium or High impact facility).  Conversely, entities could expose themselves to huge penalties if they don’t put in place controls and programs for a facility that is in scope for Version 4, if Version 4 is in fact enforced on 4/1/2014.

I will be honest at the outset: I don’t know the answer to the question whether V4 will be enforced.  The only ones who possibly could know are the five FERC commissioners, and I suspect they have not made up their minds.  What I will try to do in this post is at least parameterize the different areas of uncertainty, and suggest developments that might occur next year which will indicate whether this event is more or less likely.  You are hereby warned: this will be a long post.  Like all things NERC CIP, this is a very complicated issue.

To start out, I would like to try to identify the groups that will and won’t be affected by this issue.  It is certainly true that this isn’t a problem for many NERC entities, while for others it is a huge problem.

But first I want to clarify one point: When I say an entity “has to comply” with CIP Versions 3 or 4, I mean they will have at least one Critical Asset with at least one Critical Cyber Asset.

And when I say an entity “has to comply” with CIP Version 5, I mean that they will have BES Cyber Systems associated with facilities that are listed as Medium or High impact in CIP-002-5 Attachment 1.  There are of course many more entities that will have BES Cyber Systems at facilities that are Low impact in Attachment 1 (or more accurately, BES facilities that aren’t listed in Attachment 1, since all others become Low by default).  Since the requirements for the Lows are so much less than those for the Mediums or Highs (Lows only need to develop and implement four policies, and cyber assets don’t have to be inventoried), I don’t consider the question whether or not an entity has to comply with Version 5 as a Low to be one that carries a high dollar impact.[i] 

Let me first list some types of entities for which the question whether CIP Version 4 will be implemented isn’t really an issue.  They include:

• Entities that don’t have to comply with CIP V3 now and won’t have to comply under V4 or V5. 
• Entities that currently have to comply with CIP Version 3 and will continue to have to comply for V4 and V5, for the same assets.  They will have to make the transition from V4 to V5 like everybody else, but since they already have a V3 program in place for the asset(s), they won’t have to change anything they’re now doing when V4 is implemented (since CIP-003 through -009 are unchanged from V3 to V4).
• Entities that a) currently have to comply with V3, b) will continue to comply with V4 for the same asset(s), and c) will not have to comply with V5.  An example of this would be an entity that currently has declared a blackstart plant as critical under V3.  The plant will continue as critical under V4, but will not be Medium or High impact under V5.[ii]  So the question whether V4 will be implemented or not is just a question of when they can discontinue their CIP compliance program for that asset (hopefully, they’ll still leave most of the important security controls in place, but they certainly won’t have to continue to file TFE’s, for example).  This is a budgetary question, but not one that requires them currently to change what they are doing.
• Entities that don’t have to comply with V3 now and won’t have to comply with V4, but will with V5.  I don’t think there are a lot of these, but one example might be some substations (since the V5 bright-line criteria for substations differ from the V4 ones).  In any case, these entities will ultimately have to comply with V5 regardless of whether V4 is implemented or not, so the question of V4’s implementation isn’t a big deal for them.
• I’m sure this isn’t all the cases, but you hopefully get the idea – there are a lot of entities that don’t have to worry about whether V4 will be implemented or not.  I’m sure it’s by far the majority of NERC entities.

 So who are the entities that are worrying?  Well, probably you – otherwise, why are you reading this post?  The two main categories include:

• Entities that don’t have to comply with V3, but will with V4 and with V5.  Their problem isn’t that they’re not sure they’ll have to comply sometime in the future; it is certain they will.  However, since they don’t have a CIP compliance program in place now, they will have to implement one.  Because the V4 and V5 compliance programs are quite different, they are running the risk of putting in place a full V4 program, then having to scrap most of that and implement a V5 program. 
• Entities that don’t have to comply with V3 and will have to comply with V4, but then won’t have to comply with V5 when it comes out.  To comply with V4, these entities will have to put in place both a V4 compliance program and a lot of security controls (both technical and procedural).  Since this won’t be needed when V5 comes into effect, a lot of the compliance program will be wasted money and effort (although presumably most of the controls themselves are a good investment regardless of CIP compliance).  The biggest example of this case is owners and operators of blackstart generating stations and substations in the blackstart cranking path.  These are Critical Assets under V4, but are Low impact facilities under V5.  I’m sure a lot of these people are agonizing over their decisions now (although see endnote 2 for more nuance on blackstart generators).

 The question we began with is really two questions:

1. Under the normal process for NERC/FERC interaction, what is the likelihood that Version 4 will be bypassed? 
2. What extraordinary actions could be taken (by NERC and FERC) to prevent Version 4 from coming into effect?  There seems to be at least one scenario under which this could happen, which I’ll discuss.  I won’t even guess at its probability, though. 

To address the first question, let’s start by asking what are the events (or non-events) we need to watch for over the next year and a half in order to know whether V4 will come into effect?

The most important date will be April 1, 2014.  If FERC lets V4 come into effect on that date, they will never go back and rescind it after that.[iii]   It will only be replaced when V5 comes into effect (which will probably be at least two years later), not before that.

And practically speaking, FERC definitely has to make the decision to approve V5 well before 4/1/2014, say six months earlier at least.[iv]  Can you imagine what would happen if the Regional Entities and Registered Entities had spent a lot of time and money getting ready for V4, and on - say - March 1, 2014 FERC pulled the plug on it?  It wouldn’t be pretty.

What this means effectively is that FERC needs to finally approve CIP Version 5 by October 1, 2013 at the very latest, in order for Version 4 to be superseded, following the Version 5 implementation plan.

What is the probability of FERC’s approving V5 by October 1, 2013?  I’ll be honest; I think it’s extremely remote, perhaps not significantly different from zero.  Consider these facts:

• The NERC Board of Trustees approved V5 on Nov. 26.  V5 now has to be filed with FERC.  The filing has to include just about everything that went on in developing V5 – all of the comments, all of the different drafts, meeting minutes, etc.  I’m sure the filing will be over 5,000 pages, probably more like 7 or 8,000.  Most of this is just cut and paste, of course, but there is a lot of writing that has to be done as well.  Let’s assume that the filing is the end of December, 2012 – I’d say that’s the earliest it could be, and it will likely be later than that.[v]
• A FERC representative addressed the MRO meeting this week.  While his subject was virtualization and compliance (and the presentation was quite good), he was asked about the prospects for quick approval of V5.  He said that the staff had been preparing for the V5 submittal for many months now and would be as expeditious as possible in pushing it through to the Commissioners, but he pointed out that just getting a case number assigned takes more than a month.  And the big question is how long the Commissioners will take to decide, once the staff presents V5 to them – I doubt they themselves know at this point.
• FERC took 14 months from the date NERC submitted V4 to the date they approved it.  V4 revised CIP-002 but left the other 8 CIP standards exactly the same.  V5, on the other hand, is a radical revision of all of CIP (more like Version 1, which took FERC 17 months to approve).  If V5 is submitted by NERC by Jan. 1 (again, not very likely), is it really possible that FERC will take just nine months to approve V5?
• I have argued elsewhere that it is likely FERC will send V5 back to NERC and require specific changes in perhaps 90 days.    It is also very possible, if they think the problems with V5 are more fundamental, that they will do what they did in Order 706: tell NERC they just want them to do better, and here are the principles we want you to follow.[vi]  In either case, this will clearly make it impossible to meet my stipulated October 1, 2013 deadline for V5 approval. 
• I think it is also likely that FERC will order NERC to do a new survey of asset identification, just like they did for V4 in 2010.  This is because I think FERC will be concerned about the number of generation assets that will be High or Medium impact under V5 (see the blog link above for more on this).  The V4 survey took three months, so I’m sure this one will as well.
• FERC always issues a NOPR (Notice of Proposed Rulemaking) before actually issuing an Order approving regulations; the purpose is to provide a forum for concerned parties to submit comments.  The V4 NOPR was issued in September 2011, about seven months before Order 761 approved V4.  Moving back seven months from the 10/1/2013 date leaves March 1, 2013 as the date by which FERC would have to be certain enough that they wanted to approve V5 that they issued the NOPR.  Given that the Commissioners won’t have even received their staff report by then, I’d say that this consideration alone dooms the idea that V4 can be bypassed just through normal NERC and FERC actions. 

So the answer to the first question looks decidedly negative.  We now arrive at the second question: What extraordinary actions could be taken by NERC and FERC to keep V4 from coming into effect?  Fortunately, I don’t have to guess in this.  NERC has been soliciting comments from the trade organizations on two possible alternative actions they can take.  One doesn’t seem very realistic, but the other would possibly work if FERC agrees.

In this scenario, NERC would submit a request to FERC (possibly with the V5 filing but not as an actual part of it) to push the implementation date for V4 out a certain amount of time, say one or two years.  This way, entities wouldn’t have to spend big bucks making assets compliant with V4 that then wouldn’t have to comply with V5 (there’s more to it than that, but this is essentially what NERC is suggesting).  There are a number of questions with this option, the main one being (as my Knowledgeable Person points out) that this just prolongs the uncertainty for entities.

The Knowledgeable Person has added another option: file V5 as intended, but simultaneously petition FERC to completely rescind Order 761, meaning that V3 would remain the law of the land until V5 came into effect.  This strikes me as the best, since it removes all uncertainty about whether V4 will be implemented or not.   The question then is, “Will FERC be interested in this?”  And my answer: I haven’t a clue.  Ask the Commissioners.[vii]

---

[i] I’m sure some will disagree with me in this regard, and say there could be a substantial effort required by entities to comply with V5 for Low impact facilities.  I agree a number of technologies like firewalls and locks on the doors have to be put in place (and I know that even now there are facilities without those), but the biggest burden of CIP is all the compliance procedures and paperwork,.  Those are almost entirely absent for Lows in V5.

 [ii] A Knowledgeable Person has pointed out that many if not most blackstart plants were not designated Critical Assets under V3 (or if they were most didn’t have Critical Cyber Assets due to the routable protocol exemption), so there probably aren’t many in this category.  As you can see below, this may make them entities that do have to worry about whether V4 will be implemented (although they can still avail themselves of the “non-routable protocol exemption” in V4).  However, this person also believes that a lot of them have altered their EOP-005 blackstart plans so that the plants won’t be required as blackstart in the future.  In that case, these entities are in the first category: they don’t have to comply with any of the CIP versions.

 [iii] CIP V5 junkies may raise the point that the V5 Implementation Plan currently states that V5 will supersede V4 no matter when it is approved by FERC – i.e. even after 4/1/2014.  But FERC doesn’t have to live with that – they can send the plan back to NERC to change it.  It is simply unimaginable that they would let a new version (V4) go into effect, then pull it back later.

 [iv] My Knowledgeable Person says that even six months isn’t enough time, because of the limited vendor resources (both security and ICS vendors) available to help everyone come into compliance; if entities wait until then to start implementing V4 compliance in earnest, they will collectively never be able to do it).  He is probably right, but I am trying to see in this essay if there is any basis at all for a belief that V4 won’t be implemented.  So I’m trying to give that belief all the benefit of any doubt.

 [v] The filing was in fact on January 31, 2013.

[vi] The astute observer might note that Order 706 approved CIP Version 1, even while asking for something better.  However, when FERC did that, there was no mandatory cyber security standard already in place, so they undoubtedly felt something was better than nothing.  That is obviously not the case now.

[vii] NERC didn’t include any sort of extraordinary request to FERC in the 1/31/2013 V5 filing.  Does this mean they’ve given up on the idea of trying to get V4 bypassed?  It could, although they could still make a separate filing with the new request.   However, this certainly doesn’t increase the very low probability that Version 4 won’t come into effect, and probably pushes it even closer to zero.  See this post for more on what NERC should be doing, IMHO.