For what seems the 50th time, I’m starting a post by referring to an article by Lew Folkerth of Reliability First, found in the March-April issue of RF’s newsletter (as always, under the heading “The Lighthouse”). As has been the case on every previous occasion, Lew has put his finger on an important issue in the current NERC CIP compliance regime.
The article[i] specifically addresses the question whether the use of virtualization is permitted by the current CIP standards, and if so what is permitted and how it should be implemented in a compliant manner. Lew answers the first question by saying virtualization is neither permitted nor prohibited. This is of course not earth-shattering news, but Lew quickly moves on to something that is quite interesting: the concept (which he invented, at least as far as CIP goes) of “zones of authority”.
Lew brings this concept up for the following reason: Even though there are no actual CIP requirements that apply to virtualized environments, this hasn’t stopped NERC and the regions from coming up with a few “guiding principles” for virtualization; probably the most important of these is that there should not be “mixed-trust” virtualized environments. Briefly, this implies that a host server for multiple VMs shouldn’t host both ESP and non-ESP devices, and a switch with multiple VLANs shouldn’t host both ESP and non-ESP networks.
Lew says (or implies) that auditors are having differences of opinion with entities on the security of mixed-trust switches. It seems these entities have switches that implement both ESP and non-ESP VLANs. When the auditors tell them this isn’t secure, the entities point out that the non-ESP VLANs have just as good security as the ESPs do[ii]. So why aren’t they safe?
Lew doesn’t dispute that it is very likely that, from a pure security point of view, these entities are right: mixed-trust switches are just as safe as switches that only implement ESPs. But this is where zones of authority come in. Lew points out that the auditor only can look at CIP assets, like ESPs. Anything else, such as a VLAN that isn’t an ESP, is completely out of his or her purview; the auditor just has to assume these are completely insecure, and thus shouldn’t be found on the same switch as ESP VLANs.
Of course, Lew is completely right about this: The NERC CIP standards only take account of BES Cyber Systems and their associated ESPs, PCAs, EACMS, etc. Anything that isn’t one of these is completely out of scope. This is a double-edged sword, of course. On the one hand, the auditors can’t look at what isn’t in an ESP, which limits the kinds of evidence an entity can show them. On the other hand, the entity has no compliance obligations for something that isn’t in an ESP or that protects ESP devices, like an EACMS or PACS. My guess is most entities, if told they could force CIP auditors to consider security controls they have implemented for non-ESP networks, but only in return for having at least some of the cyber assets on those non-ESP networks fall into scope for CIP, would say “Thanks but no thanks. We’ll leave things as they are.”
And this is exactly the issue: It would make a lot of sense from a security point of view to have all cyber assets and networks that are in the entity’s control be in scope for CIP[iii]. Just look at the Ukraine attacks, which started with phishing attacks on the IT networks. It was only after completely taking over those networks that the attackers were able to find a way into the relays that were their ultimate targets, even though they were on the OT network.[iv] But the idea of having all or even just a few of their IT cyber assets suddenly become BES Cyber Systems would cause a lot of CIP professionals to bash their head against the nearest hard object, or seriously consider making a career change to fast food service.
So are we stuck here? While it would certainly be a good idea to have all networks under the NERC entity’s control be in scope for CIP, I would agree with my hypothetical CIP professional that the burden of making these networks ESPs, and making even a few of their cyber assets be BES Cyber Systems, would be too great to justify the benefit: the enhanced ability to demonstrate their security posture to an auditor.
But we’re not stuck here. Were we to rewrite the CIP requirements so that they weren’t prescriptive (i.e., so they were objectives-based), and enclose them in an appropriate “wrapper” so that the entity was simply required to address a certain set of threats (and this list of threats were regularly updated, outside of the standards development process), I think we could safely expand CIP to include all networks under an entity’s control. But there would still have to be an appropriate classification of cyber assets in scope, depending on how “near” they are to the BES. Those that are directly involved in control of the BES (as BCS are in the current standards) would have to have stronger controls than those whose involvement is only indirect (such as the IT network assets at the Ukrainian utilities that were attacked).
Of course, I’ve made this point about the need to rewrite CIP previously. And I will continue to make it, including in my next two posts.
The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.
[i] I recommend you read the article yourself, of course, even though I will summarize parts of it here. Don’t worry – it’s just a page. If I had written the article, it would have at least been the length of one chapter in a book, if not an entire book.
[ii] I’m speculating here. I’m not sure if auditors are actually running into this problem already, or if Lew just anticipates that they will. Either way, Lew’s discussion provides an excellent window into a real problem.
[iii] I recently was introduced to another CIP blogger, Larry G, who publishes a blog called NERD CIP; while I haven’t had time to spend a lot of quality time with his posts yet, they look quite insightful. In particular, he recently wrote a post that starts off with a consideration of the same Lew Folkerth column that this post started with (although he has a different take on it). In that post, Larry points to the PCI definition of Trusted Network, which reads “Network of an organization that is within the organization’s ability to control or manage.” Since all Trusted Networks are potentially in scope for PCI, they will all need to be considered – either because they themselves hold payment card data or because their compromise could lead to compromise of the networks that do hold that data. In fact, I will venture a guess that CIP is the only set of cyber security standards that limits itself to a subset of the networks actually controlled by the entity.
[iv] And please don’t tell me that it was only because the three Ukrainian utilities had poor separation between their IT and OT networks (lack of two-factor authentication, lack of intermediate server, etc) that the latter were penetrated. When attackers have complete control of the IT network for months, they will one way or the other be able to find their way into the OT network, no matter how good the security may be. How about this: One day all of the relay engineers get an email from their boss (when he is on vacation) that due to some need for emergency maintenance they need to open all of the circuit breakers at a certain time that day. Some might not follow this instruction; others might do it. In any case, there would be a coordinated outage. If this happened at a number of utilities, it could cause a serious BES problem.