ERC

Almost anybody who is involved with NERC CIP v5 compliance will tell you that one of the most vexing issues they face is determining exactly which BES Cyber Systems in a substation[i] can be described as having “external routable connectivity”. I wrote four posts on this topic last year (the first of them is here; note that I christened these posts as being about “serial”, but ERC is the real issue). However, I believe I have this issue finally figured out[ii], so I’d like to spread the word about this.

Unlike issues like “programmable” and “adversely impact”, the problem here isn’t that there is no NERC definition of ERC. It is defined as “The ability to access a BES Cyber System from a Cyber Asset that is outside of its associated ESP via a bi-directional routable protocol.” I must admit that I never thought there was a problem with this definition until last year, when people working on CIP v5 compliance for substations started telling me they were having lots of issues figuring out how this definition applies in a very typical substation scenario.

In that scenario, you have an EMS connected via a routable protocol[iii] to an RTU or terminal server (or similar device) in a substation; the RTU is connected by a serial protocol like Modbus to one or more devices (usually relays) that are BES Cyber Systems. Given this scenario, can the relays be said to have ERC? More to the point, what conditions would need to be in place for the relays to be said to have ERC?

In the series of posts last year, I identified two conditions that could be said to be required for there to be ERC in this situation. I identified the first condition at the end of the post linked above, where it seemed to me that the important part of the definition was the “ability to access” the cyber asset from outside of the ESP by a routable protocol.[iv] I identified the second condition in the fourth post in the series. Taking my cue from a webinar by the CIP v6 SDT, I said the relay needed to be “directly addressable from outside the asset” in order for it to have ERC.[v]

I also mentioned in these posts that NERC had promised a Lesson Learned on ERC. As it turns out, NERC instead produced a Memorandum in April, that addressed this and three other topics under the heading of “Network and Externally Accessible Devices”. This document made a different argument, which caused a lot of consternation among NERC Transmission entities. It seemed to be saying that, if there is a routable protocol anywhere within the communication stream between the EMS and the relay, the relay has ERC.[vi]

In the webinar I recently did with Steve Parker and Karl Perman of EnergySec (you can access – using a routable protocol, please – the recording and slides here), we discussed the entire Memorandum, including this issue. I stated that “NERC focuses on devices that just convert routable communications to serial, saying that a serial-only device has thereby been transformed into one that is routably accessible. I don’t argue with this position.”

However, I went on to discuss a presentation that Morgan King of WECC had given at their January CIP User Group meeting in Anaheim. In it, Morgan distinguished between devices that merely convert routable communications to serial and vice versa (i.e., the devices contemplated in NERC’s Memorandum) and devices that actually “break” the routable protocol and initiate serial communications with the relay.  In Morgan’s opinion (and he recently confirmed to me that it is still his opinion), the former devices have ERC, while the latter do not[vii].

What we concluded in the webinar was that both NERC and Morgan were right. NERC confined their analysis to the single case of a device that simply converts between serial and routable communications protocols; both NERC and Morgan agree that, if such a device is in place between a serially-connected relay and an EMS communicating routably, there will still be ERC. However, Morgan went beyond that to look at a device that “breaks” the routable protocol session and initiates a serial session with the relay. In that case, we agreed with Morgan that there is no ERC.[viii]

At this point, I must point out that my original intention was to end the post here, sending you away with the happy thought that you merely have to determine whether the intermediate device breaks the routable protocol, in order to know whether or not the relay has ERC. But I asked myself, “How do you know whether or not the routable protocol session has been broken? Do you find a bunch of pieces lying on the floor?”

I can’t give you a definitive answer[ix] – i.e., a definition of “protocol break” – but I can point you to two examples. In his January presentation, Morgan suggested that if the intermediate device required the user (or machine) coming from the control center (via a routable connection) to authenticate before being able to connect to the (serially-connected) relay, the device can definitely be said to break the routable protocol.

And I also point you back to the example I gave in footnote iv about the RTU that simply polls the relays and reports the data back to the EMS on a periodic basis. I assume that, in such a case, there would be no way for an outside user to access the relay. This is also a protocol break.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte & Touche LLP.

---

[i] I’ve never heard of this being an issue in a generating plant, but I certainly wouldn’t rule it out.

[ii] I tremble when I use this phrase - “finally figured out” – with regard to CIP v5. If there’s anything I’ve learned, it’s that any statement you come up with regarding CIP v5 is likely to need to change in the near future. Just look at how NERC has gone back and forth on their pronouncements about the meaning of “programmable”.

[iii] Or a serial protocol running over IP, which is the same thing as far as CIP is concerned.

[iv] The specific example I gave of the absence of this condition was an RTU that just polls the relays and forwards the data to the EMS. Presumably, there is no way for the EMS to itself initiate communication with the relay.

[v] I opined that I thought the two conditions amounted to the same thing, but that this question was above my pay grade. That’s still the case. BTW, since the topic was Low impact assets, the point was made about LERC (Low-impact ERC, as used in CIP-003-6 R2), not ERC. In this case, I don’t think there’s any difference between ERC and LERC, although I know that isn’t so in all cases.

[vi] In fact, I heard an auditor for one of the Regional Entities describe what the Memorandum said in almost exactly those terms.

[vii] It is important that, when trying to decide whether or not a device breaks the routable protocol, you look at how it actually works. RTUs and similar devices can be configured in different ways. In some cases, they may simply convert routable to serial and vice versa. In others, they may make a clean break between the two.

[viii] As I mentioned earlier, at least one Regional Entity is interpreting the NERC Memorandum to mean that, if there is any routable communications within the entire “stream” between the relay and the EMS, there is ERC. That interpretation would make Morgan’s distinction irrelevant; that is, the fact that the routable protocol is “broken” at the intermediate device wouldn’t matter – there would still be ERC. I don’t believe Morgan would agree with this interpretation, and I don’t either.

[ix] I can’t give you a definitive answer, but Morgan can. In a recent email, he reminded me of the excellent Reference Models that were included in the Guidance and Technical Basis section of CIP-003-6. These models show different ways in which external connectivity can occur. Reference Model 6, found on page 36, describes a case in which the end device is connected serially to a cyber asset that then connects routably outside the asset.  The description says “There is a Layer 7 (application layer) break or the Cyber Asset requires authentication and then establishes a new connection to the low impact BCS. There is no LERC in this example.” LERC is discussed in footnote v above, but for our purposes is equivalent to ERC.