The News from RF, Part V: Felek Tackles Jungle Muxes

I’ve already mentioned that the highlight of RF’s CIP workshop in April was the joint presentation by Felek Abbas of NERC and Lew Folkerth of RF, which addressed various current topics in CIP. Felek gave a very good presentation on CIP-003-7, which includes the revised requirement for electronic access control at Low impact assets that have external routable connectivity (CIP-003-7 was approved by the NERC ballot body and Board of Trustees, and is now awaiting FERC approval).

Felek spent a lot of time on the Reference Models that provide examples of different ways to comply with this new requirement; these are found in the Guidance and Technical Basis for CIP-003-7. When he discussed Reference Model 5 (found on page 39), he pointed out - and I agree with him - that this model addresses FERC’s concern that led them to order NERC to clarify the meaning of the word “Direct” in the LERC definition in CIP v6 (as you probably know, the current CIP Modifications drafting team decided to eliminate the defined terms LERC and LEAP altogether, instead incorporating them into the requirement itself. For a description of what they did, read this post, although if that isn’t punishment enough, you could read this one). I’d like to clarify why Reference Model 5 does actually address what FERC asked for, although this isn’t the point of this post. You can skip the next two paragraphs if this historical item isn’t of interest to you.

Reference Model 5 shows a routable communication stream entering a Low asset, but then being converted to a non-routable protocol for connection to a Low impact BES Cyber System. In 2014 and 2015, as NERC entities realized the clock was ticking on coming into compliance with CIP v5, there was a lot of discussion about whether or not just converting a routable to a serial connection (e.g. with a protocol converter) was enough to “break” External Routable Connectivity for Medium BCS. The regions and NERC made it fairly clear that merely converting the protocol wouldn’t break ERC, but it was also clear that the ERC definition itself didn’t provide an answer to this question.

Of course, at the time there was no requirement regarding electronic access control at Low assets (and also no LERC), since the CIP v6 drafting team was still working on this. CIP v6 was approved in January 2016, and by then FERC was worried enough about this issue that they explicitly included it in their order approving v6; that is, they required NERC to clarify the meaning of “Direct” in the LERC definition. Reference Model 5 shows (using the green dotted line) that the protocol conversion by itself doesn’t change the fact that there is external routable connectivity to a Low BCS. However, the fact that the “non-BES Cyber Asset” in Reference Model 5 also performs electronic access control is deemed sufficient mitigation of the risk caused by the presence of external routable connectivity to one or more Low BCS – which is the objective of the revised requirement. So Reference Model 5 shows one way to comply with the revised requirement in CIP v7 (indeed, the other Reference Models all show alternative ways to comply).

At this point, somebody in the audience raised the question of “jungle muxes”. These are devices – often found in substations – that accept a routable external connection, but then convert that into a number of serial connections to a set of serial devices (often relays). The question was whether, since the mux does communicate routably, it has to have an Electronic Security Perimeter around it.

Felek answered this question by going back to a Lesson Learned that dealt with communications between two Medium and/or High assets, at least one of which doesn’t have an ESP (i.e. it doesn’t have any internal routable network). The Lesson Learned was occasioned by the fact that Section 4.2.3.2 of all the CIP v5 and v6 standards (it’s also found in v7, at least in CIP-003-7) says that cyber assets involved in communications between “discrete” ESPs are exempt from CIP; at the same time, under v5 and v6 (unlike in v3) there can be assets subject to CIP that don’t contain any routably connected devices. The Lesson Learned addressed the question whether the cyber assets that are associated with the communications network between two Medium and/or High assets, one or both of which doesn’t have an ESP, are also exempt from CIP, and if so where the “inter-asset” communications stream begins in an asset without an ESP.

That Lesson Learned introduced the idea of the Demarcation Point (which is an old idea from the days when dinosaurs roamed the Earth and data communications was mostly over phone lines), as the device where the external communications carrier “takes over” from the building’s internal communications network. The LL said that an entity complying for an asset without an ESP should designate the device that is best described as the demarc point. All cyber assets that are “inside” of this device are potentially in scope for CIP; all cyber assets that are “outside” of this device are not.

Felek said that, even though the jungle mux does have a routable connection, the fact that this connection is used purely for external communications – and that there are no other devices at the Low asset that communicate routably – means the mux itself can be considered the demarcation point. Thus, it doesn’t need an ESP.[i]

I thought Felek’s answer was very good, but I was even more impressed by the fact that he was willing to state it in a meeting with 200 or so representatives from NERC entities in attendance. There are some who might consider what he said to be an “interpretation” of the wording of a standard, rather than simply implementation guidance. NERC and regional employees are permitted to provide the latter but not the former; in practice, this has meant that a lot of questions about CIP v5 and v6 have only been answered in private conversations (primarily in the private Small Group Advisory Sessions that were conducted in 2015 and 2016) or not at all. Felek wasn’t afraid to publicly state his opinion on a disputed issue like this; I wish it would happen more often.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

---

[i] Of course, this isn’t to say that the serial devices that connect to the mux don’t have external routable connectivity. Whether they do or don’t depends on what happens at the jungle mux. The definition of ERC is “The ability to access a BES Cyber System from a Cyber Asset that is outside of its associated Electronic Security Perimeter via a bi-directional routable protocol connection.” If all the jungle mux does is convert the routable communications stream to serial and send it to the proper device, then there is ERC. If the mux performs some sort of authentication, then this probably removes that ability, and there is no ERC.

However, I’ll admit this is still an ambiguous area, and that points to an interesting irony. That is because FERC’s qualms about the ERC definition (discussed above in this post) led to a change; but the change was in the LERC definition, not ERC. When FERC ordered NERC to clarify the meaning of “Direct” in the LERC definition – in the order approving CIP v6 in Order 822 in January 2016 – they weren’t reacting to problems with LERC since it wasn’t in effect yet. They were really responding to the discussions about ERC in 2014 and 2015 that I described above (I linked above to just one of the posts I wrote on this issue. Others include this one, this one, this one (in the order I wrote them). I believe FERC was especially concerned about the idea that mere protocol conversion “breaks” ERC.

If you want to plow through these three posts, you will see that, by the third one, I had concluded (in conjunction with Morgan King, a WECC auditor who has weighed in a lot on technical issues like this) that mere protocol conversion didn’t break ERC, but requiring re-authentication would. I thought when I wrote the third post that this was the last word on the subject. However, in this post, which was written a few days after the third one, I reported that another auditor said this wasn’t enough to break ERC – that there had to also be “reformulation of the user’s commands and the acting as a proxy for the user”.

At this point, I threw up my hands and decided ERC was a black hole – I could write 100 posts and never get to the bottom of the problem, which was that the definition needed to be rewritten. I believe this is one of the tasks on the CIP Modifications SDT’s agenda, and I hope they do this. I think they should build on the very good work they did with the LERC issue last year, and base their new definition of ERC on use cases similar to the ones they developed for CIP-003-7 (although some of those cases won’t apply because the new Section 3.1 “defines” what used to be LERC in terms of the asset’s boundary, in order to avoid requiring a list of BCS at Lows). I think any attempt to write a dictionary-style definition will fail. They just need to say “In these cases, there is ERC. In these cases, there isn’t ERC.”