The News from SPP and WECC: Back to the Far End

June 29 Note: It was recently pointed out to me that the post below is based on the original version of the “Far-End Relay” Lesson Learned (posted for comment last September), not the finalized version that I linked below (which I admit I hadn’t downloaded or read when I originally wrote the post, since I didn’t realize there had been a big change in the LL). When I realized my error, I at first thought it would be easy to just update the post based on the final version. However, when I read the final version it became immediately clear to me that, while NERC’s conclusion didn’t change, their rationale did change –and very substantially. Based on the wording of criterion 2.5, I definitely think the original version had the right rationale, not the final version.

The conclusion of both versions of the LL is straightforward: A “far-end” relay, located in a Low impact substation, that is associated with a 200-499kV line that terminates at a substation that falls under criterion 2.5, is Low impact – not Medium. The original purpose of this post was first to state that I agree with the LL (not a big surprise there), but more importantly to warn against drawing from the LL a conclusion that I believe would be unwarranted – namely, that all BES Cyber Systems at Low impact assets are now automatically Low impact.

Fortunately, both versions of the LL support my warning, even though I don’t agree with the rationale in the final version of the LL; so the conclusions of my original post don’t need to change. I’m therefore leaving the original post as it is below, even though it’s discussing the September draft of the LL, not the final version.  But I do discuss the final version of the LL in the last footnote of this post (BTW, the September version of the LL has been taken down from the NERC website. If you want to see it, you can email me at talrich@deloitte.com and I’ll send it to you).

I must say, this is all quite disappointing to me. Here I thought I could for once give a clean endorsement to an important NERC document without any ifs, ands or buts; but now I have to go through an elaborate dance of saying I agree with the conclusion but not the reasoning of the final document, and that the document I do agree with has been officially superseded. I continue to hold out hope that someday I’ll find a NERC document I can agree with entirely. If that ever happens, you’ll be the first to know.

This is the third post in my series on things I learned at the SPP and WECC CIP conferences the first week of June. I would subtitle the series “What I Did on my Summer Vacation”, if I could convince you that a week with more than two days of travel and three days of meetings was a vacation.

I have expressed my displeasure with NERC for its slowness in coming out with guidance on the many issues with CIP v5, as well as in many cases for the content of the guidance it has produced.  But there is at least one guidance document that I consider spot on, in terms of saying exactly what needed to be said about its subject and not causing any “collateral damage” by saying more than it should; this document is the Lesson Learned (LL) on Impact Rating of Relays, aka the “Far-End Relay” LL (it also happens to be one of only two LLs that have been finalized).

So why am I bringing this up? I almost always deal with problems with the rollout of CIP v5, not with things that aren’t problems. The “problem” with this LL isn’t due to its content, but to the fact that almost nobody seems to understand what it means – and this includes people from NERC entities, the regions, and NERC itself. This lack of understanding can and will likely lead to problems with implementing and auditing compliance.

The second paragraph of the LL summarizes the complete argument of the document.  It reads:

“As discussed further below, the language of CIP-002-5 and its support documents limits the application of the medium impact rating to the BES Cyber Systems associated with Transmission Facilities operating between 200kV and 499kV at a single station or substation. The Transmission Facilities must be located ‘at a single station or substation’ that meets certain connection criteria in order for the associated BES Cyber Systems to receive a medium impact rating.”

Of course, this paragraph – indeed the whole LL – refers to criterion 2.5 and only that criterion. To unpack the content of the paragraph, it says the following:

1. The subject of the criterion – i.e. what gets classified as Medium impact – is Transmission Facilities between 200 and 499kV. This includes lines operated in that voltage range that terminate in the substation. It does not include the substation itself; in fact there is technically no such thing in CIP v5 as a “Medium substation” – all of the criteria that apply to substations actually classify the Facilities at the substations, not the substations themselves (of course, in practice it’s almost impossible to avoid using this language, as I’m about to demonstrate).
2. Because the “preamble” to Section 2 of Attachment 1 states that BES Cyber Systems are Medium impact if they are “associated with” the subject of one of the Medium criteria, this would normally lead one to conclude that all BCS that are associated with a Medium line at a substation that has Facilities meeting criterion 2.5 (it would be much easier to say “a criterion 2.5 substation”, of course) will themselves be Medium impact.  And this would include “far-end” relays in a transfer-trip scheme, even if these are located at a substation that is otherwise Low impact (and yes, a substation can itself be Low impact. In fact, no Facilities are Lows, just assets are. The wording of CIP-002-5.1 is contradictory on this point, as on others).
3. When this implication became widely known, there was a great hue and cry that this would lead to huge costs for transmission entities, as they would have to spend lots of money to protect these Medium BCS at Low substations. However, the Lesson Learned (released last September) made it clear this won’t happen. To see NERC’s reasoning, just look at the paragraph quoted above: It points out that in criterion 2.5 the word “Facilities” is modified[i] with the words “at a single station or substation”.
4. This means that, for this particular criterion[ii], all lines are excluded from being Medium impact Facilities, since they are inherently not limited to a single station or substation. Because the line isn’t a Medium Facility, the far-end relay can’t be considered a Medium BCS, since it isn’t associated with a Medium Facility. About three months before this Lesson Learned was released in its first draft, exactly the same argument had appeared in my blog, contributed by an Interested Party who has often contributed to my posts.[iii]

So what’s the problem? The problem is that many people in the NERC community – I’m willing to bet it’s the majority, although I haven’t conducted a survey – believe that what the LL really says is something like “Location does matter”;[iv] in other words, that all BES Cyber Systems that happen to be located at Low impact assets are therefore Low impact simply because of that fact. This is absolutely not the case; the Lesson Learned only applies to BES Cyber Systems (probably always relays) associated with lines that terminate at a substation that “meets” criterion 2.5. It doesn’t apply to anything else.

Does this have a real-world impact? Yes, it does. Here are examples of two systems, located at a Low asset that might actually be Medium BCS:

1. Suppose you have a centralized system – located at a Low impact substation - providing access control for cyber assets at substations, including some Medium BCS. Would the access control system be a Medium BCS? I believe it would, since it would presumably be associated with the Medium Facilities (lines, etc) that the Medium BCS it controls are associated with (in other words, “guilt by association”).
2. Or suppose the Automatic Generation Control (AGC) system for a Medium plant is located at a Low impact plant, substation or control center. Since it’s associated with a Medium plant (perhaps meeting criterion 2.3), it will itself be Medium impact.[v]

Note: An Interested Party pointed out that both of the examples I just gave are fairly unlikely to occur in practice. He pointed out that one very real example is an SPS/RAS system that meets criterion 2.9. The different components of the SPS - each a BES Cyber Asset in its own right - could be located at a number of different substations and/or generating plants that are Low impact. However, since the SPS (now officially called RAS, I believe) is an asset (one of the "magic six") that is Medium impact by 2.9, all of its component BCS will be Mediums as well - regardless of whether they're located at a Low or "Medium" impact asset.

Here's a note on my note: As I wrote the note, I was trying to figure out the implications of calling SPS an "asset", when it is actually really a system, with components spread out among multiple assets. It would be nice to figure out exactly how SPS/RAS fits into the admittedly shaky "system" of asset identification and classification in CIP-002-5.1. I'll put that on my list of posts to work on. If anybody has any particular thoughts on this matter, let me know.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte & Touche LLP.

---

[i] And if you don’t remember what “modified” means here, please dig up your sixth-grade textbook on diagramming sentences.

[ii] The same phrase appears in Criterion 2.6, where it would most likely also have the effect of removing Transmission lines from consideration.

[iii] You may wonder what happens to the “near-end” relay – i.e. the one that resides in the “Medium” substation and is associated with the 200-499kV line; is that now also Low impact? This would be true if it were only associated with the line. However, it is also associated (and more closely, too) with the circuit breaker that can trip that line. Since the circuit breaker would be a Facility operated at 200-499kV at a substation that has 3,000 points, then the relay is a Medium BCS. And BTW, if the far-end relay directly controlled that “near-end” breaker, I would say that relay would then be a Medium BCS, in spite of being located at a Low substation. Fortunately, I don’t think this is generally the case with transfer-trip relay schemes.

[iv] These were Tobias Whitney’s words when he “explained” NERC’s position on this issue at the June 2014 CIPC meeting, as described in this post. Those words seem to have taken on a life of their own, even though the Lesson Learned uses a very different argument – both the draft and final versions.

[v] Here’s my footnote on the final version of the Lesson Learned. I must say, this version is an odd document. It seems to make two slightly different arguments, both leading to the same conclusion.  I don’t agree with either argument, but as I said in my note at the top of the post, the good news is that the overall conclusion of the final version of the Lesson Learned is the correct one.  This is also the conclusion of the September draft of the LL – and I agree with that document 100%.

The first argument is in the first paragraph of the LL, which states that relays “located at Transmission stations or substitutions (sic – I’m guessing NERC meant to say ‘substations’ here) described in criterion 2.5” should be Mediums, while those located at substations that don’t meet 2.5 (or any of the other Medium criteria) should be Lows.  My problem with this is it seems to completely ignore the fact that criteria 2.3 – 2.8 apply to Facilities, not to assets. I’ll say this for probably the 15^th or 20^th time (and the second time in this post): the SDT didn’t put the word “Facilities” in those criteria just because they wanted to break up the monotony – they did it because they wanted Facilities to be what those criteria apply to, not assets. Facilities are lines, breakers, transformers, etc. Assets (with a little “a”) are the substations, generating plants, etc. So the substations don’t technically “meet” criterion 2.5 or any other criterion. This can have consequences for the amount of work the entity has to do to comply, but I’ve also discussed that issue at length, such as in this post.

More specifically, reading this first paragraph of the final version of the LL will lead one to conclude that the entire determinant of whether a BCS is Medium or Low impact is the substation it’s located at. This is simply not true. For example, in a criterion 2.5 substation, a relay associated with a circuit breaker operated at less than 200kV will be Low impact, not Medium.

The last paragraph seems to bring up a different argument, although it also leads to the same wrong conclusion as the first paragraph. I quote that paragraph in full:

“The Guidelines and Technical Basis (Guidelines) section of the Reliability Standard also discusses Transmission Facilities described in Attachment 1 which states: ‘In most cases, the criteria refer to a group of Facilities in a given location that supports the reliable operation of the BES. For example, for Transmission assets, the substation may be designated as the group of Facilities.’ According to the Guidelines, ‘The Transmission Facilities at the station or substation must meet both qualifications [i.e., the connection specifications described above] to be considered as qualified under criterion 2.5.’”

I actually agreed with the SDT when they wrote in the Guidance and Technical Basis that a substation could be considered a “group of Facilities” – that’s about the best definition of “substation” you could come up with (my usual definition is “a bunch of expensive equipment with a fence around it”). It’s hard to say what the above paragraph is saying, but it seems that, instead of moving from this observation to the conclusion that the near-end relays are Medium if and only if they’re associated with a Medium Facility, whoever wrote this LL seems to be falling back on the idea that all the BCS at a “Medium” substation should be classified as Medium impact, regardless of whether or not they’re associated with a Medium Facility. By implication, they’re also implying that all BCS at Low impact assets will be Lows. Neither of these statements is true.

The ironic part is that whoever wrote the draft LL from last September seemed to understand quite clearly that it is the Facility that determines the impact level in criterion 2.5 (and by implication in criteria 2.3 – 2.8). I’d love to know why this understanding has been lost to NERC. It’s kind of like if Apple had suddenly forgotten how to make smart phones.