Big News: FERC will start Auditing CIP v5 Compliance

Nov. 4: On Nov. 2, at least one NERC region sent an email to their members confirming what is reported in the first paragraph below. Also, EnergyWire has an article today stating that a FERC spokesperson has confirmed this.

Kevin Perry, the Director of Critical Infrastructure Protection (and Chief CIP Auditor) of SPP Regional Entity, provides a CIP update at each of the SPP RE Board of Trustees meetings. At the meeting on October 26, 2015, Kevin’s final slide included the following statement: “FERC expected to conduct CIP V5 and CIP-014-2 audits beginning in 2016.” This was followed by two bullet points: “SPP RE will cancel planned audit activities for a registered entity if FERC steps in” and “Regions and NERC may observe FERC-led audits”.

Translation: FERC is going to start choosing entities it wants to audit for CIP. A FERC audit may supersede a planned audit by one of the Regions. The Regions and NERC can only observe FERC audits, not participate in them. Kevin’s slides are in the single file with all of the meeting slides, accessible here (Kevin’s presentation starts on slide 28; the slide in question is his slide number 13. I haven’t had a chance to ask Kevin if he deliberately chose 13 to be the number of the slide that delivers this news – or that he chose to reveal this so close to Halloween).

I am told that Kevin also said that FERC would have an official announcement of this policy change soon, and that - at the moment - nothing is known for sure on what entities will be targeted for FERC audits. The above are the only facts I have. But I’m going to speculate on what they may mean:

• It’s safe to say that FERC’s audit approach will be very different from NERC’s, probably much tougher. It’s also important to remember that FERC can audit whenever it wants; it doesn’t have to follow the three- or six-year schedule in the NERC CMEP (Compliance Monitoring Enforcement Plan). You could have had a CIP audit this year, and FERC will audit you next year.
• Also, there is no time limit on FERC audits. I know of at least one audit (of all the NERC standards, including CIP) that took multiple years. Hopefully, that won’t be the norm.
• It’s also probably safe to say that the lucky entities chosen for FERC audits will be among the larger ones.
• Can anything be said about how FERC might address the various areas of controversy in CIP v5? Normally, of course, FERC doesn’t have anything to say about NERC standards once they’ve approved them – unless they also ask for specific changes, which then get incorporated in new standards (as happened with the changes FERC ordered when they approved CIP v5 in 2013; these were incorporated into the CIP v6 standards).
• However, FERC did surprise me and others by weighing in quite forcefully on the meaning of External Routable Connectivity in their NOPR (Notice of Proposed Rulemaking) for CIP v6, issued in July. In the NOPR, they discussed NERC’s new concept of Low-Impact External Routable Connectivity (LERC), and disagreed quite forcefully with NERC’s idea that a “protocol break”, somehow involved with the transition from routable to serial communications, would terminate LERC (they didn’t rule out that something else might break LERC, but they also didn’t make any suggestions on what that something else might be – see this post for some ideas on that topic).
• Of course, in their NOPR, FERC was talking about LERC, not ERC – so technically their comments have no bearing on ERC. But as I pointed out in this post, FERC’s arguments against the idea of a protocol break can just as easily be applied to ERC as to LERC. With the news that FERC will start auditing v5, these arguments suddenly take on new significance.

I must admit that the first question that came to my mind when I read Kevin’s slide was what impact FERC’s move would have on the idea of CIP v5 “enforceability”, which I had conveniently just addressed in my most recent post. In that post, I said that the CIP v5 standards won’t be “effectively enforceable” (i.e., PVs assessed) until the regions are comfortable that they understand them[i]; I said that in general this won’t happen for at least six to 12 months after April 1, 2016.

I also said that the effective enforcement dates for CIP-002-5.1 R1 and the concept of ERC are currently “never”; that is, until R1 and the definition of ERC are rewritten to address the many ambiguities and contradictions in the current language, it is unlikely any auditor is going to issue a PV if his/her opinion on an issue differs from the entity’s opinion – as long as the entity has made a good faith effort to understand the issue in question and document why they arrived at that conclusion. For example, if the auditor and the entity have different opinions on the meaning of the words “affect the Bulk Electric System” in the definition of BES Cyber Asset – and the entity has documented how they researched that issue and came up with their interpretation – I don't believe any auditor will issue a PV because in her/his opinion the entity has not properly identified their BES Cyber Assets.

There may well be a few other “inherently ambiguous” requirements that will require rewriting (a 2-3 year process, of course) before they can be effectively enforced. Let’s designate as “ambiguous requirements” the following: CIP-002 R1, requirements where ERC comes into play, and perhaps one or two other requirements. All the other requirements will be “non-ambiguous” requirements, although of course there’s always some ambiguity in any requirement that could ever be written in the English language.

So how will FERC’s coming into the audit picture change the effective enforcement dates? In the case of the non-ambiguous requirements where I’m saying there won’t be PVs issued for at least six months for good-faith compliance efforts, I don’t think FERC auditors are going to be much harder than NERC Regional auditors. They will understand that there has been so much confusion about what the CIP v5 standards mean in general – confusion which will unfortunately not be cleared up in any meaningful sense right up until 4/1/16, and beyond that as well – that entities need more time to be held compliant to the letter of the requirement (they also need time to develop their evidence record, as Tobias Whitney acknowledged at GridSecCon two weeks ago).[ii]

What about the ambiguous requirements: CIP-002 R1 and those that apply to assets with ERC? I also believe the same rule will apply for FERC as for NERC auditors: No PVs will ever be issued until there is definitive clarification of the current requirements. Since it appears that NERC will not provide such definitive clarification, I’ve been saying they need to be rewritten.

But, while FERC can’t officially issue new requirements or revise existing ones, FERC can issue its own clarifications of areas of ambiguity in CIP v5. In particular, as I said above, I think FERC’s discussion of LERC in their CIP v6 NOPR can easily be read as applying directly to ERC. And here the impact is profound: I believe entities need to review their own “definition” of ERC[iii] (and every entity with serially-connected devices at an asset that has a routable connection to the outside world needs to figure out for themselves how they will determine whether or not there is ERC in any particular case) to make sure it conforms with what FERC wrote in their NOPR. If it doesn’t conform, they need to make it conform – and they will need to rethink (and probably re-do) their identification of BES Cyber Systems with ERC if there is a discrepancy. If FERC is going to audit you, it's safe to assume that FERC will apply the reasoning in the NOPR to determine whether or not your cyber assets have ERC.

The same consideration will apply for any other guidance that FERC may issue. Unlike guidance issued by NERC, such as the Lessons Learned and FAQs, you need to consider FERC’s guidance to be written in stone.[iv] So if FERC decides to clarify the various issues in CIP-002 (which I’ve been writing about in my series of posts on “Rewriting CIP-002”, starting with this one), you will need to pay close attention to this guidance, and re-do your original BES Cyber System identification and classification methodology to conform to whatever they say it should be (and then re-apply that methodology wherever needed). On the other hand, and in my opinion, the problems with CIP-002 R1 and Attachment 1 are so profound that FERC may not want to wade into the problem directly[v]; they may simply issue an order that CIP-002-5.1 needs to be rewritten (at least I hope they will!).[vi]

To summarize: There’s a new sheriff in town. Every NERC entity needs to rethink a lot of what they may have considered settled decisions on CIP v5 compliance; and I think they’ll probably determine they have to do a lot of things differently given that FERC could be their auditor. Of course, FERC’s official announcement may clarify what they intend to do. And it also may not.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte Advisory.

---

[i] I also said that, for an entity that hasn’t made a good-faith effort to understand or comply with a particular requirement, the effective enforcement date won’t be delayed beyond 4/1/16. The delayed effective date only applies to good-faith efforts to comply, for which there is a disagreement between the auditor and the entity about the meaning of the requirement in question. It doesn’t apply when the entity hasn’t made an effort to determine the proper interpretation, or even worse, has chosen an interpretation solely because it will make its compliance effort easier (for example, an entity that decided that only processors of 500 mhz and higher are “programmable”).

[ii] On the other hand, I think it is likely that FERC will be less tolerant in general of violations of the “little stuff”, which the NERC Regions are less likely to be concerned about – a missed signature, etc. This could actually turn out to be the biggest difference between FERC’s and NERC’s auditing of the CIP standards.

[iii] Of course, this isn’t a dictionary-type definition. Rather, it’s a procedure for determining whether or not there is ERC, in the case of a communications stream that contains both routable and non-routable elements.

[iv] Technically, of course, even FERC can’t issue “binding” interpretations. But since they have to approve all enforcement actions including fines, your only recourse if you disagree with FERC’s interpretation of an issue is to take it to court. I don’t believe any such appeal has ever been adjudicated.

[v] Except perhaps to provide a “definition” of Programmable. This wouldn’t in itself make CIP-002 enforceable, but it would be a good interim step during the 2-3 years it will take for the standard to be rewritten.

[vi] I think this would be the right thing to do; in fact, I thought it was the right thing to do in 2013, when I very helpfully “rewrote” CIP-002 to make it clearer. I’m very glad FERC didn’t order NERC to follow what I wrote back then, because I didn’t understand a lot of the problems in CIP-002-5.1. And even now I’m not saying that CIP-002 needs to be rewritten along the lines I’m suggesting. My primary motive has always been for CIP-002 to be rewritten so it is unambiguous and can be consistently followed; I’m not so worried about the particular concepts that would be in the rewritten version.

The CIP v5 Enforcement Date

I spent all last week (Oct. 13-16, 2015) at NERC GridSecCon; as has been the case with the previous two events I’ve attended, this was an excellent program. Every year I say it can’t get any better, yet the next year it does; I’m already looking forward to the 2016 meeting in Quebec. My compliments to Bill Lawrence and his team for putting this together so efficiently and creatively. As well as for choosing as venue the city where I was born (and in whose suburbs I grew up), Philadelphia.

One of the tenets of GridSecCon is that it’s about security, not compliance. That being said, there was one discussion titled “CIP V5 – the Home Stretch”; this was led by Tobias Whitney, Felek Abbas and Tom Hofstetter of NERC. One of the questions asked in that discussion was whether the “enforcement date” for v5 would be pushed back a year. Tobias answered that, while nobody expected strict enforcement to begin on April 2, 2016, enforcement would not be officially pushed back. This was certainly the answer I expected from Tobias, and I don’t disagree with it. However – as with all things NERC – there are a lot of nuances to this topic, at least some of which I’ll discuss in this post. I’ve already stated everything you’ll read below in previous posts, but this will bring all those statements together.

On the next break, a couple people asked me if I had asked that question (questions were submitted on cards and read by Bill Lawrence, but he didn’t mention the questioner this time – perhaps intentionally since the subject was compliance). I was asked this because I had this summer advocated the approach of leaving the main compliance date for v5 as 4/1/16, but setting an enforcement date a year later – 4/1/17. This was what was done in CIP version 1. The practical effect of this would be that, while entities would need to be compliant with v5 to the best of their ability on 4/1/16, they would not have to self-report violations, and they would not be assessed Potential Violations at audits, until 4/1/17.

When I made this proposal, I didn’t wait by my phone for Tobias to call excitedly to say this was a wonderful idea (of course, since my cell phone is always by my side except in the shower, this is an antiquated expression); I realized when I made the proposal that it wasn’t likely to be accepted. So in this post in August, I came up with the concept of the “effective enforcement date” (although I didn’t use that term). The idea of effective enforcement date is fairly simple: A standard will be enforced only if the entities that are in charge of enforcement – that is, the eight NERC regions[i] – feel comfortable enforcing it. The effective enforcement date of a standard or requirement in your region is simply the date your Regional Entity feels comfortable issuing PVs for non-compliance.

So why wouldn’t the regions feel comfortable enforcing CIP v5 on 4/1/16? Tobias mentioned one reason in his answer: Enforcement of many of the v5 requirements depends on a record of having performed a particular operation like patch management on a regular basis. These requirements obviously can’t be properly audited for 3-6 months after the compliant date of 4/1/16, so that entities can build a record of compliance.

However, this isn’t the main reason why I believe the regions won’t feel comfortable enforcing v5 on 4/1/16. The main reason is – you guessed it – the huge amount of uncertainty over the standards. I can certainly verify this uncertainty from my discussions with a lot of NERC entities, but also from a recent Bridge Energy Group Utility Industry Survey[ii] that found that 68 percent of utilities believe their organization is “not well prepared” for CIP v5 compliance.

So I’m saying I believe the effective enforcement date for the CIP v5 standards will definitely be much later than 4/1/16. How much later? That will vary by standard and even requirement, as well as region. And to be frank, it’s likely to vary by auditor, since an individual auditor is only going to feel comfortable enforcing a v5 requirement when the auditor believes he or she understands what constitutes compliance with that requirement, in the context of the entity he/she is auditing.

However, just because the effective enforcement dates will vary a lot doesn’t mean I can’t give you some idea of what those dates might be. You may already know that I don’t often let lack of complete knowledge hold me back from making definite statements; my motto is “Often wrong, but never in doubt.” As usual, I will preface my statements with some “your mileage may vary” language.

1. First, there isn’t one compliance date for CIP v5/v6, but lots of dates. This post describes the set of official compliance dates for the different standards, requirements, and even requirement parts. The fact that the effective enforcement date for most of the standards will be later than 4/1/16 will affect a lot, but perhaps not all of, the other compliance dates.
2. The above set of dates is dependent on FERC’s approving the CIP v6 standards on time, as discussed in this post. To summarize that discussion, if FERC doesn’t approve v6 by the end of November (not December as some might think. This is because FERC’s approval won’t become official until the Order is published in the Federal Register, and that takes about 30 days), the v6 dates will be delayed. And NERC may decide to delay the v5 dates to match the v6 delays (at least, I hope they would do that – again, this is because in this case the regions aren’t going to effectively enforce v5 anyway).
3. When I say compliance will be effectively postponed, I am specifically talking about Potential Violation citations not being issued. Entities will still need to self-report any potential violations they know of; I just find it very hard to believe these will turn into actual citations, and even less into Violations. But this is the biggest difference between the idea of NERC’s officially moving the actual enforcement date back, and the idea of the enforcement date being effectively moved back without any official action: in the former scenario, entities won’t have to do self-reports, while in the latter they will have to.
4. The biggest caveat in all of this is that PVs and violations will still be issued for entities that simply blow off their responsibility for complying with all or part of CIP v5. There has to be a good faith effort to comply. And this includes the entity doing its best to research any areas of ambiguity; for example, even though there is no definition of “programmable” and no definitive guidance from NERC on this important issue, it is still up to the entity to read the different documents NERC has put out on this (two of these were later rescinded, but they still describe valid alternatives for this definition). It is also important to get whatever guidance is available from your region, either in public meetings or in private conversations. Finally, it is vitally important to document all of this – both the different alternatives you considered, as well as how you came to your final conclusion on the particular issue. Of course, this will be a lot of work[iii]; but there really is no alternative, given it’s been clear since at least July that there will never be any fairly definitive guidance from NERC on the great majority of areas of ambiguity in CIP v5, at least not before 4/1/16 (and probably a good while after that).

So here are the details:

1. I believe CIP-002-5.1 R1 will never be enforceable until it is rewritten in full; this includes the definitions that are missing from it, including the word “programmable” in the Cyber Asset definition and “affect the reliable operation of the BES” in the BES Cyber Asset definition. Of course, rewriting this standard will be a massive job and will take a minimum of three years starting when a SAR is accepted by NERC (and none has yet even been written, of course). I should point out that Tobias Whitney said at GridSecCon that there would probably be a SAR for a definition of “programmable”. Just addressing this one part of the CIP-002 problem, while leaving the rest untouched (and I’m writing a whole series of posts on “Rewriting CIP-002” now), will frankly be useless. This is because a team drafting a new CIP-002 R1 might well decide that having a separate definition of Cyber Asset isn’t needed; they might want to fold this into the definition of BES Cyber Asset. They might even decide that the whole concept of BCA can be eliminated, so that BES Cyber Systems will be the first thing identified in a rewritten R1. Even with “programmable” finally defined, R1 will be no more enforceable than it is now, if the other issues aren’t addressed. And all of the R1 issues need to be addressed as a whole, not individually.
2. The concept of External Routable Connectivity has turned out to be a black hole, meaning that in my opinion there will never be an end to the arguments on what constitutes ERC (as I concluded in this recent post). In fact, as I pointed out in this other recent post, an RF auditor announced at their recent CIP v5 workshop that RF won’t issue PVs for improper identification of BCS with or without ERC; and I’ve heard this may soon be an ERO-wide provision.[iv] Fixing the problem will require at a minimum rewriting the definition of ERC (and very likely the new “definition” will really be a set of procedures for determining when there is ERC, rather than a definition like one found in a dictionary). So the effective enforcement date for the concept of ERC (which of course affects a lot of requirements) is also “never”, until the definition is rewritten.
3. I am certain there are at least a couple other requirements or definitions that won’t be enforceable until they are rewritten. I’ll just have to keep you informed as I discover them.
4. What about all of the other requirements (in a few cases, complete standards) that aren’t affected by these various black holes – i.e. the requirements that to this day seem fairly unambiguous? Even though these may be unambiguous, I’m reasonably sure that no PVs will be issued for any CIP v5 requirement for a minimum of four to six months after 4/1/16, provided of course that the entity has made a good faith effort to comply with the requirement. The exact effective enforcement date will of course depend on how well the region feels it understands the requirement, as well as how the individual auditor feels; in many cases, the EED could be well beyond 10/1/16.[v] I’m told that some at NERC were concerned about a “bow wave” of violations of v5 starting 4/1/16. I’d say their concern is misplaced; the bigger question is whether CIP v5 will ever be enforceable in any meaningful sense.

I’m not kidding with that last sentence. If you think about it, CIP-002 R1 (and Attachment 1) and the definitions of Cyber Asset, BES Cyber Asset and ERC are the complete set of components of the asset identification process in v5; however, it is precisely these items that are black holes. Since all of the other CIP v5 and v6 requirements assume the entity has properly identified and classified its BES Cyber Systems, and since that assumption can never be proven or contradicted given the current wording of the standards, how can the CIP v5 and v6 standards ever be meaningfully enforced?

How indeed? That’s the $64,000 question.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte Advisory.

---

[i] I realize it isn’t technically correct to say the NERC Regions, or even NERC itself, are in charge of enforcement of CIP or the other NERC standards (i.e. assessing fines). In the US, FERC is technically the enforcer; in Canada it’s the appropriate provincial authority.

[ii] The results had previously been released, but were reported by Richard Jones of Bridge at GridSecCon.

[iii] I estimate the total compliance paperwork burden for v5 is at least three to five times that for CIP v3, and perhaps more than that. And that’s holding the number of assets in scope constant – which of course isn’t the case for most entities that had to comply with v3. I’d guess that 80-90% of such entities have more assets and cyber assets in scope for v5, often significantly more.

[iv] Of course, the “good faith” rule applies here. If you have an all-routable connection between a device in a substation and your EMS, I don’t recommend you claim there is no ERC. The confusion only applies in cases where there is at least some serial communications in the stream.

[v] I have previously said there needs to be a gap of a year between the date when it can be said that NERC and the regions have provided sufficient guidance on the CIP v5 and v6 standards (so that they can be described as “well-understood”) and the date they will effectively be enforceable. If you want to go by that rule, then it’s hard to see when the standards will ever be enforceable in any meaningful sense. It’s just about certain that there will not be significant guidance from NERC on most areas of uncertainty before 4/1/16, so by my rule the effective enforceable date for all of v5 and v6 will be 4/1/17, and probably much later than that.  This is why it is truer than ever that a NERC entity needs to run any questions it has by its region. The only “interpretation” of a requirement that you can bank on to be followed when you’re audited is one that has been provided to you by your region. And BTW, if you got an “interpretation” from your region say six months or more ago, it greatly behooves you to get it reconfirmed. Given all the changes that have happened – including the various Lessons Learned and Memoranda that NERC has issued and then retracted – it would be very surprising if your region hasn’t changed their opinions on the meaning of “programmable”, ERC, etc.

An Excellent Presentation on Lows

At the risk of further strengthening the impression that this has become the Lew Folkerth Blog, I want to mention that I just read an excellent presentation by Lew (who is with RF, of course) on complying with CIP v5 for Low impact assets. He gave it in August, but I just got around to reading it yesterday.

If you have Lows - and especially if you have only Low impact assets - I highly recommend this. It takes great pains to gather into one place everything you need to be considering and doing.

You can find the presentation on this page. Under "Seminars / Workshops", click on "Open Compliance Call 8/17/2015" and click on the second PDF file.

I found Attachment C!

In my last post, on an interesting discussion on CIP-002 R1 at the recent RF meeting, I described a spreadsheet they showed. I thought this document (officially, Attachment C to the CIP-002-5.1 RSAW) would be very helpful for NERC entities who have to identify and classify BES Cyber Systems for CIP v5. When I first put up the post, I stated I couldn't find a link to it on the NERC web site.

It turns out I should have looked on RF's site. Here is the page where you can find it, in the section titled CIP Document Library.

Rewriting CIP-002, Part III: An Interesting Conversation with RF

My previous post discussed various interesting things I’d learned at RF’s CIP v5 workshop in Cleveland on October 1 and 2, 2015. In that post, I mentioned there had been a good conversation at the meeting that I would discuss in my next post. Well, now it’s the next post, and here’s the story of that conversation.

On the second day of the meeting, Ron Ross and Frank Kapuscinski of RF gave a presentation entitled “CIP-002-5.1 Audit Approach”. The presentation was very similar to presentations on this topic by other NERC Regional Entities; I doubt any CIP compliance person from any region would have thought there was anything wrong with it. The bulk of the presentation was dedicated to displaying and discussing a very large and interesting spreadsheet that constitutes Attachment C for the CIP-002-5.1 RSAW (this file is available on this page, in the CIP Document Library section). It allows the entity to collect a wealth of detail on its assets and cyber assets – everything that’s needed to comply with v5.[i]

After the presentation, I asked Ray Sefchik, who leads RF’s CIP auditors (his title is Manager, CIP Compliance Monitoring), whether RF would be putting out any document to clarify the wording in CIP-002-5.1. Ray answered as I thought he would: “No”. He implied (I can’t remember his exact wording) that it really wasn’t RF’s job to try to clarify the wording in any NERC standard– and I totally agree with this. The Regional Entity’s job is to audit compliance with the wording of the requirement.

However, if you have happened to read any of the 70-80 (perhaps more) posts I’ve written on issues with the wording in CIP-002-5.1 over the past two and a half years, or in particular the first two posts (Part I and Part II) in this series on Rewriting CIP-002, you will know I think there are a huge number of ambiguities and contradictions in this standard (especially in R1 and Attachment 1). So the question arises, is it actually possible to audit to the strict wording of this requirement? And if so, what would “the strict wording of the requirement” mean?

Fortunately, I don’t need to spell out now what it would mean to audit to the strict wording of R1; I did that in Part II of this series. There, I started by laying out the fairly simple five-step process that is how almost all of the regional auditors, and close to 100% of the compliance people at NERC entities, understand R1 to work. It was certainly the methodology that lay behind the RF presentation. But there’s just one problem with this methodology: It isn’t the actual wording of the requirement.

Further on in that post, I listed a very complicated 15-step process for compliance with R1 that is as close to the actual wording as I can get. And I pointed out that this was just the down payment; to be correct, there is no finite set of steps that could be written down that would be compliant with the actual wording. You could write 15 steps or 15,000, and still not have addressed all of the ambiguities and lacunae in R1 (including its missing definitions, like “programmable”). But if RF, or any entity, is going to audit to the "strict wording" of CIP-002-5.1 R1, they need to audit based on this complicated methodology, not the simple five-step one. Of course, no NERC entity is going to follow the complicated methodology as they comply with CIP v5, and certainly none of the regions are going to audit based on it; if you don’t believe me, just read the post.

Since the discussion so far has been fairly abstract, let me point out a few areas where the presentation and the spreadsheet differ from the strict wording of CIP-002-5.1 R1:

1. First, Todd talked about “assets” being classified as High, Medium and Low impact. Of course, this is how virtually all NERC entities started their R1 compliance process, and it’s how all of the Regions understand the process as well.[ii] However, nowhere in R1 (or Attachment 1) is the entity told to do this. What the entity is told to classify are BES Cyber Systems, not assets. R1 sends the user to Attachment 1 to determine the classification of each BCS, depending on whether the BCS meets the High, Medium or Low impact criteria. There is nothing about classifying High or Medium impact assets themselves, only Lows (and even that is indirect – see the next item). Of course, virtually all entities and all regions but one (or maybe two) are assuming the first step in R1 compliance is to classify “assets” as High, Medium or Low impact; but this implicit requirement isn’t found anywhere in the actual wording.
2. R1 does tell the user to identify “assets containing Low impact” BCS – of course, Todd’s presentation used the same wording. Looking just at that wording, it seems quite clear there is only one way to do this: a) inventory cyber assets at every asset (High, Medium and Low impact) owned by the entity; b) identify Cyber Assets among those; c) identify Cyber Assets that meet the BCA definition; and d) finally aggregate these into BCS. At that point, the entity will be able to determine whether an asset – that has not already been identified as High or Medium impact – “contains” Low BCS. But there’s a problem with this, since both R1 and Attachment 1 state that no list of BCS is required for Low assets. So how is the entity going to identify assets containing Low BCS without identifying BCS? Of course, virtually all entities and regions are interpreting “assets containing Low BCS” to mean “BES assets not already identified as High or Medium impact”; with this adjustment, the meaning is perfectly understandable and doesn’t require a list of Low BCS. This is fine – but it’s not what the requirement says, or Attachment 1 either.
3. Just about all of the regions are telling their members that they need to identify all of their assets that correspond to one of the list of six asset types in R1, then “run these through” the Attachment 1 criteria to determine which are High, Medium and Low impact; Todd’s methodology also started that way. Again, this is a fine idea. But what do the actual criteria apply to? Let’s see…Criterion 2 applies to “reactive resources”. I don’t see that on the list of six assets. Criterion 2.3 applies to “Generation Facilities”. Since Facilities is a defined term, this must be a single unit in a generating plant, not the plant itself; again, not on the list. Criteria 2.4, 2.5, 2.7 and 2.8 apply to “Transmission Facilities”; these are lines, buses, transformers, etc. – but not substations, so the list of six doesn’t apply to these criteria either. I could go on…The point is that very few of the criteria actually apply to one of the six asset types – in fact, the only criteria that do actually apply to one of those six asset types are those that deal with control centers. Yet NERC entities are being told by their regions that the criteria do apply to these six asset types, directly contradicting what’s actually in black and white in CIP-002-5.1. Once again, this isn’t a problem in the short term, since the entities and the regions are in agreement on this point. Also once again, the problem is this is not what the requirement says, and having this disconnect will lead to longer-term problems (see below).

There are lots of other ways in which the widely-accepted methodology for complying with CIP-002-5.1 R1 directly contradicts the wording of R1 and Attachment 1; in fact, of the five steps listed in the previous post in this series, all but the first are either not stated in R1 or are directly contradicted by its wording (I’ll have more to say on all of these issues as I put out more posts in this series on Rewriting CIP-002). Again, I’m not criticizing RF or any of the other regions, or any NERC entities, for using this simple interpretation of R1; indeed, I think it’s the only way that one can make sense of that requirement.  But it is important that everyone throw away the idea that this interpretation is based on the requirement itself; it just isn’t. It is literally true that the only practical methodology for complying with CIP-002-5.1 R1 requires the entity to "rewrite" the wording of the requirement itself.

So what’s to be done? Should RF and the other regions stop whatever they’re doing and bring all the auditors together for a month or more, so they can try to figure out what the wording of R1 actually means and come up with a common audit methodology – one that conforms as closely as possible to the actual wording of R1 and Attachment 1? Or should they continue following the fairly simple methodology that all the entities and regions (and NERC itself) currently understand to be the correct methodology for R1 – even though this doesn’t conform to the wording of the requirement?

Obviously, the latter is the only course to take – the former is completely unworkable. However, at the same time, I believe someone needs to draft a SAR to rewrite CIP-002, and the process of rewriting it must begin. But that process will take probably three years. In the meantime, there needs to be a consensus among NERC, the regions, and the entities that the simple methodology is the one to follow (or something close to it – there would actually be a few more steps required, but they’re all also being followed today).

This also means that no region, including RF, can truly say it’s auditing to the strict wording of CIP-002-5.1 R1. There is simply no way to audit R1, other than by coming up with a workable “interpretation” of how to comply with the requirement. As I said, the NERC community has actually done that without knowing it. They just need to acknowledge that this is an interpretation, not the strict wording of the requirement. And the community needs to start working on the only real fix to this problem – rewriting CIP-002.

In the meantime, I hope it’s clear why I’ve been saying for a while that CIP-002-5.1 R1 will never be enforceable until it is rewritten (other than in the case of an entity that more or less blows it off and doesn’t even try to comply - I'm sure they will still be issued PVs. I don’t know any like this, but I’m sure a few exist.). How would any auditor write up an entity for not following a compliance procedure that isn’t even in the requirement? The only question in my mind is whether this non-enforceability of R1 (which is after all the fundamental requirement in CIP v5 and v6) will spread to all of the other requirements.

For the moment, I’m thinking it won’t – that is, that the other requirements can be enforced, as long as the stipulation is made that whatever list of BCS the entity came up with in CIP-002 R1 is valid. But I’m not sure even that will work in the long run, especially once an entity challenges a large fine by going to court. And this is the worst part: as long as CIP-002-5.1 R1 remains the current version of the standard, it is very possible that four or five years from now, the entire v5/v6 edifice will be invalidated by some legal ruling. And what does the industry do then? Go back to v3? Decide the NERC CIP experiment has failed and turn cyber regulation over to DoE or DHS? Just give up on CIP and hope Congress doesn’t notice? Lots of great choices!

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte Advisory.

---

[i] Of course, there is the slight problem that this is probably a year too late to help most entities. With about five months remaining before the compliance date, it’s a little late for entities to be identifying cyber assets in scope for the first time. When available, it will of course help in getting evidence together for an audit.

[ii] There are one or two regions that will most likely call the three asset types “assets likely to contain a High BCS”, “assets likely to contain a Medium BCS” and “assets likely to contain a Low BCS”. This fits a little better with the wording of R1, but the fact remains that R1 (or Attachment 1) never tells the entity to classify assets or “assets that contain High or Medium BCS” (it does require identification of assets containing Low BCS). The entity is told to “consider” six asset types, but those are possible locations where BCS can be found, not a list of assets to be classified H/M/L. It is the BCS that are classified H/M/L, not the assets – according to the wording of R1 and Attachment 1. This is because the criteria apply to BCS, not assets. Of course, the fact that it is impossible to classify BCS without basing that classification on the asset it’s associated with was overlooked when R1 was drafted. This is the primary problem in CIP-002 R1, although certainly not the only one.

The News from RF

I have a confession to make: Even though Reliability First Corporation (RFC) changed their name to ReliabilityFirst (RF) a couple years ago (perhaps more than that), I have continued to refer to them as RFC in my posts. This was a grievous sin, and I have decided to come clean on it in front of all the world. I will refer to them simply as RF from now on.

On October 1 and 2, 2015 I attended RF’s CIP v5 Workshop outside of Cleveland. As with the other two RF CIP v5 workshops I’ve attended (one remotely), I found this an excellent event. Below are some of the interesting things I learned (you can find all of the presentations here). I want to spend some time describing in depth one important discussion that occurred during the meeting, but I’ll address that in a separate post.

The points below may be helpful to you, regardless of your region. However, as I say very regularly now, for any questions regarding the CIP v5 requirements or enforcement, you need to address them directly with your region. Your mileage may vary.

Low Impact Update (Mike Ketchens and Scott Pelfrey)

Mike and Scott gave a good presentation on Lows. In a question, I pointed out that, since a Low impact asset is defined as one “with Low impact BCS”, this implies that an asset that isn’t High or Medium impact - but that doesn’t have any cyber assets at all - won’t be Low impact either. It will simply be “no impact” and will be out of scope for CIP v5. I asked if this was RF’s understanding; the answer was yes.

I followed up this question by asking whether, if there are cyber assets in an otherwise Low impact asset and the entity wants to go through the exercise of demonstrating there are no BES Cyber Systems among those cyber assets, this asset would also end up being “no impact” and out of scope for v5. The answer to that was also yes.

Of course, if you are claiming “no impact” assets, you need to be prepared to prove they are so in an audit.

2016 CIP Monitoring Plan (Ray Sefchik)

RF will not require an annual self-assessment for CIP next year, as they normally do. This is because the self-assessment is usually done in Q1. Since CIP v5 won’t come into effect until Q2, it doesn’t make sense to have entities assess their compliance in Q1. That assessment would presumably cover v3 compliance, and that won’t be relevant after 4/1/16. The attendees were quite happy to hear this. My guess is the other regions may adopt this policy as well, but be sure to check before you take this off your calendar.

CIP v5 RSAWs (Lew Folkerth)

Lew has been very active in the ERO program to develop the v5 RSAWs, and provided some great insights on the new RSAWs (for the ten CIP standards you’ll be audited on, three v5 and seven v6). You can find these by going here, and then picking out the CIP standards with “-5.1”, “-5”, “-6” and “-2” in front of them (no one can accuse NERC of spending unnecessary resources making it easy to find things on their web site. I’ve often said I know of only one surefire way to protect Critical Infrastructure Information: post it on the NERC site. Al Quaeda will never find it there!).

These were some of Lew’s more interesting points (interesting for me, that is. If you don’t find these interesting, get your own blog).

• The RSAWs are now organized down to the Requirement Part. This will allow auditing of compliance with a particular part, rather than the entire requirement.
• The Compliance Narrative is the most important part of the RSAW. It’s there so the entity can tell what they did to comply, and why. Lew suggested a narrative of 1-2 pages was appropriate for each Requirement Part. You might look at this and this post for some guidance from Lew on what might be required in the narrative - if the requirement in question is ambiguous or depends on a missing definition.
• NERC and RF are interested in outcomes, not documentation of what the entity did to comply. The example Lew provided was the case where a patch won’t install and the entity needs to take another action to mitigate the threat – close a firewall port, put in an IDS signature, etc. The outcome being measured will be whether or not the threat was mitigated; what is done to mitigate it is less important.[i]
• Lew pointed out that there are a number of “implicit requirements” in CIP v5. These are things that the entity needs to do to be in compliance, which are not specifically identified in the actual Requirements. Lew gave the example of the implicit requirement to “verify the identification of any associated PCAs” (this requirement never appears in v5, but the entity obviously needs to identify PCAs). RF isn’t just looking for a list of the PCAs, but wants to know how the entity verified that every Cyber Asset in the ESP had been identified. Another example is identification of EACMS and PACS. The entity is never explicitly required to identify these, but they obviously have to do so to be in compliance - and they need to be able to show that they haven't missed any EACMS or PACS systems.
• Of course, you can’t find out about these implicit requirements by reading the RSAWs, since they only deal with the explicitly-stated requirements. A questioner asked Lew if RF would publish a list of the implicit requirements. Lew said he’d look into doing that. I certainly hope he does – it is greatly needed (and not just in RF).

CIP-005 Audit Approach (Bob Yates)

The highlight of this presentation for me was when Bob said that no PVs will be issued for “improper” identification of BCS as having or not having External Routable Connectivity (ERC). Of course, given the huge amount of controversy over what ERC means (and for a great example, see my last post. When I first wrote the post, I reported what two auditors were saying about ERC, but I had to rewrite that section two days later, before I published it. This was because both of the auditors had refined their positions as we discussed the subject in emails. I’m sure I could do posts on ERC from now ‘til Doomsday and I still wouldn't exhaust that subject) and the lack of any definitive guidance from NERC on the issue, it’s hard to see how they could have any other policy. All the same, it’s good to hear RF at least explicitly acknowledge this – and it sounds like this may be the case ERO-wide; as usual, consult with your Regional Entity. I am sure the same policy will apply to CIP-002 R1 as well, since the ambiguities in that requirement are even more intractable than those in ERC.

A questioner asked whether an entity needs network diagrams to document ESPs. Bob’s answer was that they are not required, but they are probably the best way to communicate what the ESP covers to the audit team. In other words, the audit team won’t appreciate it if they’re not given diagrams, but have to rely on some verbal description of the ESP.

CIP-007 Audit Approach (Todd Thompson)

Todd Thompson addressed the monster requirement, CIP-007. After Todd’s presentation, a questioner asked if an entity must upgrade their firmware if the upgrade will assist with compliance – for example, it will enable longer passwords – given that firmware upgrades can sometimes lead to unanticipated issues. Todd answered that the entity needs to test to make sure the upgrade won’t cause a problem. If it does, they don’t need to apply the upgrade, but they do need to document why they couldn’t apply it.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte Advisory.

---

[i] An emphasis on outcomes, not the process by which they were achieved, sounds wonderful and is a big hot button for NERC – mainly because in theory it should reduce the documentation requirements. However, as the previous bullet point alludes (and as I’ve discussed in many blog posts, including the two linked in that bullet point), the many ambiguities in CIP v5 require a lot of documentation just to show why the entity chose the approach it did for that particular requirement. In fact, the amount of documentation required for CIP v5 will be at least several times – and perhaps much more than that – what was required by v3. My own guess is CIP v5, if done right, requires somewhere between 5 and 25 times the documentation required for v3; and this isn’t even considering that most entities have more systems in scope than under v3, which will of course lead to the need for even more documentation.

An Auditor Comments on My Last Post

My last post discussed interesting things I’d learned at WECC’s recent Advanced CIP Training. An auditor (not from WECC) made a few good comments on the post, which I’d like to share with you – along with comments from Morgan King of WECC and a well-known relay vendor. None of this will make sense unless you read the last post.

“Protocol Break”

First, let’s go to the topic of ERC, which Steve Parker of EnergySec has referred to (in one of EnergySec’s newsletters) as “the gift that keeps on giving”.[i] In the last post, I paraphrased Morgan King of WECC, who stated at the training that, “if a device that “translates” routable to serial communications requires authentication of the user, this still does break both ERC and LERC.” In addition, he acknowledged there could be other ways in which ERC and LERC are broken.

Since the Schweitzer SEL-3620 is a widely-used device for substation communications, I approached Schweitzer and got a technician’s view on what this device does. The technician said, “The (3620’s) user session…is terminated on the 3620.  The user is not communicating directly with the end device, and the IP address used to talk to the device is that of the 3620, not the relay.  In addition to requiring authentication of the user, the 3620 enforces restrictions on user roles based upon group membership, reformulating commands to the end device as required, acting as the user's proxy.”

When I sent this description to the auditor, he replied “The key information…is the reformulation of the user’s commands and the acting as a proxy for the user.  Yes, the IP address is that of the gateway and the port number identifies which serially-connected device is to be acted upon.  That is how a terminal server works regardless of any other functionality. At audit, the technical information your source provided is exactly the type of information the entity will need to provide to demonstrate whether there is a protocol break.” In a subsequent email, the auditor said, “A terminal server may or may not include session authentication, but unless there are two clearly independent conversations like is seen in the 3620 or in an RTU, there is no protocol break[ii].”

To make his point clear, I’ll paraphrase the auditor by saying that just requiring authentication and substituting the 3620’s IP address don’t in themselves constitute a protocol break, since a pure terminal server could do the same things. What does break ERC in this case is the “reformulation of the user’s commands and the acting as a proxy for the user,” as well as creating “two clearly independent conversations.”

This might on the surface look like a contradiction of what Morgan King said at the WECC meeting, but I now realize this ERC discussion just keeps jumping up to a new level every time it comes up – so both auditors are right for the level at which the discussion stood when they made their statements. The fact that devices like the 3620 require authentication is necessary but not sufficient for them to break ERC.  The auditor is pointing out that more is required for ERC to be truly broken, and I’m sure Morgan would agree with that (of course, it seems funny for me to refer to “the auditor” here, since Morgan is an auditor, too. I really mean “the other auditor”).

When I first wrote this post, I concluded this discussion by saying, “I would love to say that I think this settles the question of ERC with serially-connected devices, if I hadn’t thought the same thing at least four times before this.” Lo and behold, I’ve now rewritten this section just a few days later, even before it is posted – and I’m absolutely certain this isn’t the final word on this topic. ERC is a black hole. No amount of Lessons Learned and blog posts will ever be able to settle this question. What does this mean for folks who don’t have the luxury of endless debate, since they have a 4/1/16 compliance deadline looming? I really don’t know.

You might point out that NERC did release a Lesson Learned on ERC in September. However, this issue isn’t clarified by that document. There is a section on pages 2 and 3 entitled “System-to-system process controls”. This has a diagram showing a setup like we’ve been discussing, with a port server and serially-connected BCAs. The LL states that “Serially connected BES Cyber Assets that can be accessed via a protocol converter (identified as a port server in Figure 1) were not considered to be BES Cyber Assets with External Routable Connectivity as defined below.” This seems to say that a port server or terminal server would actually break ERC. As you can see from the above discussion, two very influential auditors don’t believe this will normally be the case - and NERC themselves didn’t think this was the case in their April Memorandum on this topic, which was subsequently withdrawn. So this LL hardly settles the matter.

LERC

In the next section of my post, I pointed out that Dr. Joe Baugh of WECC had confirmed (in answer to my question) that an entity asserting there was no LERC at a Low impact asset which had one or more routable connections to the outside world would most likely be required to inventory their cyber assets at the location and show none of them actually had LERC. The reason Dr. Joe said this was because the entity would have to be able to prove that the ERC coming to the asset was not actually shared by any of the cyber assets there.

However, the auditor commented “The key to demonstrating there is no LERC to Low Impact BCS is documentation of the network/communications design to and within the asset.  I can envision this being demonstrated without having to have a detailed inventory of Low Impact BCS.  Show me what is connected to the routable network (e.g., detailed network diagrams, router/switch/firewall rules) and you can demonstrate no BCS are attached without enumerating the BCS themselves.  Now, if you have a terminal server device on your routable network, I will be asking questions.  But I have seen plenty of substations where the BCS are all serially connected within the asset and communicate with the Control Center with a serial link while there is a routable network for non-BCS stuff including the station PC, substation automation, and digital fault recorders.”

The auditor is stating that an inventory of individual cyber assets may not be required to prove there is no LERC at a Low asset with an external routable connection. You just have to be able to show there are no BES Cyber Assets (whether serially or routably connected locally) that are connected to that link in a way that would lead to one or more of them having ERC (e.g. you would need to show that the device that connects them to that link is doing something similar to what the SEL-3620 does in the above discussion).

Intermediate Systems

In a discussion of Morgan King’s presentation at the WECC meeting, I stated that he said “an Intermediate System (IS) should be in the DMZ, but an overriding consideration is that it must be in a PSP; this is because an IS must be declared an EACMS (per the definition of IS). So if your DMZ is outside of the PSP, you need to include the IS within the ESP/PSP.”

However, Morgan pointed out to me that he didn’t say the IS should be within the “ESP/PSP”, but just within the PSP. In fact, he and the (other) auditor both pointed out that the IS can never be in an ESP. As the auditor said, “… per the very definition of Intermediate System, it absolutely cannot be within the ESP.  Period.  It is an EACMS and must be in a PSP.  Putting it in a DMZ is a best practice, not explicitly required by the standard.” So I apologize to Morgan for misquoting him.

Physical Ports

Discussing a presentation on CIP-007, I paraphrased the presenter as saying “For CIP-007 R1.2 compliance (physical ports), it’s not OK to just put signs warning against use of USB devices on the PSP. They need to be on the device itself (or its cabinet if locked).”

The auditor commented, “Not exactly.  The standard allows signage.  The standard does not prescribe placement.  We will evaluate the placement and its proximity to the protected Cyber Asset to determine if it has a deterrence capability.”  Morgan said he agreed with the auditor, but added “The important thing is simply this: If an SME has physical access to a cabinet that contains Cyber Assets in scope for CIP v5 and others that are not in scope, the signage should a bit first…  how about this ……”nk before plugging anthing into a Cyber Asset t need further training shoting for noveber 2015 tremind the SME with physical access of this directive control. This will hopefully make him/her think before plugging anything into a Cyber Asset in scope without being authorizeda Cyber 5 in Cyber Assetsean it up a bit first...s already been made.. ures but need further training shoting for noveber 2015 t. If every asset in the cabinet is an Applicable System, then it may be a reasonable approach to place signage on the cabinet itself.”

Real-Time Alerting

From the same presentation on CIP-007, I quoted the presenter as saying “For R4.2, the Guidance implies that real-time alerting can be accomplished with technical means only. But the actual requirement allows for procedural means as well.” The auditor pointed out “I cannot envision a procedural control that provides for “real-time” alerting.  Procedural controls can detect an alertable issue, but it will not be in real time.” Good point, Auditor!

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte Advisory.

---

[i] I certainly can’t argue with that statement, since I’ve done at least ten posts on ERC and LERC in various permutations. This is second of course only to the (probably) 60-80 posts I’ve written on problems with CIP-002 R1 and Attachment 1 - and associated topics like the meanings of “programmable” and “affect the BES”. It isn’t surprising that all of these topics should be what Steve Parker calls a “gift that keeps on giving” – and what I call black holes – since they all are really parts of the general issue of identifying and classifying cyber assets that are in scope for CIP v5.  I believe CIP v5 is an excellent set of standards, once you get beyond the first step of identifying what they apply to. But with the current wording of the standards, there is nothing but confusion and contradiction in that first step.

This is why I’m advocating that CIP-002 needs to be rewritten if it’s ever going to be enforceable, even though this will take several years. It’s the foundation for the other standards, and no matter how good they are, I don't believe they can survive in the long run, being built on top of a rotten foundation. However, I’m not sure what to do about ERC. ERC is defined, and I don’t think there’s a problem with the definition as far as it goes. But coming up with a rule for how that definition applies in the case of BCS connected serially to an externally routably-connected device like an RTU has so far proved completely elusive.

It’s possible that a definitive guidance document (probably pretty long, covering every possible case of ERC) would suffice to make ERC an enforceable concept. But I do know that all of the ten or so times I’ve addressed this issue I thought I had the problems solved – and then it turns out I didn’t consider something else, which I then address in a subsequent post. Until that situation changes, I will continue to call ERC a black hole along with CIP-002 R1. If NERC comes to agree with me that ERC is a black hole – meaning that no document could possibly solve its problems – then I think all of the v5 and v6 requirements that apply to BCS with ERC will need to be rewritten. This is a daunting task and will probably have to wait for the next major revision of CIP. Until that time, there will need to be agreement that no violations will be issued for honest differences of opinion on what constitutes ERC. My next post will discuss one region that has already stated this is their policy

[ii] The auditor went on to say “Another, maybe better example, is a standard firewall that performs AAA authentication at the initiation of the session and then allows communication into the protected network.  Yes, the firewall is inspecting each message, but once authenticated, the only decision by the firewall is whether the subsequent packets in the authenticated session are permitted or denied by the applied Access Control List rules.  There is no further interception, reformulation, and proxying of the traffic.  It is just passed along if the ACL permits it.  If the firewall is fronting a relatively unsophisticated terminal server, there is no apparent protocol break anywhere in the path between the end user and the target Cyber Asset serially connected to the terminal server.  And I said “maybe” because there are proxying firewalls out there.  That is not the type of firewall I am talking about.”