Friday, June 15, 2018

Our very own Maginot Line!

I have been saying for the past 2 ½ years that I am writing a book about how to fix the problems in the NERC CIP standards (and the compliance regime that accompanies them). This year, I’m actually making progress toward that goal (with one or two co-authors), and I hope to have the book published this year (self-published, to be sure!).

As part of this effort, I have been thinking a lot about what should be in scope for CIP. And I just reread a post I wrote in July 2016, very soon after FERC issued Order 829, which mandated development of a supply chain security standard. In this post (which I wrote after the initial post describing what was in Order 829), I pointed out that in one part of the Order, FERC seemed to be considering having more in scope for the new standard than just BES Cyber Systems.

In fact, FERC laid out four objectives for the new standard(s) they were ordering. Three of the four objectives applied to BES Cyber Systems, but the third objective was called “Information System Planning and Procurement” (paragraphs 56-58, pages 37-38). I found this title very interesting, because I’m sure FERC understands that control systems aren’t “information systems”. Indeed, in the entire discussion of the third objective, FERC never once mentions BCS.

Yet the first sentence of this section reads “The new or modified Reliability Standard must address how a responsible entity will include security considerations as part of its information system planning and system development lifecycle processes.” And the first sentence of the next paragraph (paragraph 57) reads “This third objective addresses the risk that responsible entities could unintentionally plan to procure and install unsecure equipment or software within their information systems, or could unintentionally fail to anticipate security issues that may arise due to their network architecture or during technology and vendor transitions (my emphasis).”

Clearly, FERC isn’t being sloppy here; they are talking about controls on the procurement of information systems (which are also called IT systems, of course). This is underlined in the next sentence of paragraph 57, where they bring up BlackEnergy. As you know, this is the malware that allowed the attackers in the first Ukraine attack in 2015 to take control of the IT network at several utilities. The attackers had free run of the IT network for more than half a year, before they finally figured out a way to take control of key relays on the OT network – which was their ultimate objective, of course.

BlackEnergy did all of its direct damage on the IT network; it never itself penetrated the OT network (and probably couldn’t have, since all the OT connections were most likely serial). But FERC used it as their poster child for what they were trying to prevent, in articulating their third objective in Order 829. Paragraph 56 states that this third objective includes “identification and documentation of the risks of proposed information system planning and system development actions (my emphasis).”

So it seems clear to me that FERC was asking NERC to start looking at some controls on IT systems as well as OT systems, at least as far as procurement and installation are concerned. They understood that the Ukraine attacks wouldn’t have happened if the attackers had to penetrate the OT networks first, rather than starting with the soft underbelly of the IT network.

Of course, the CIP-013 drafting team didn’t take FERC up on their implicit suggestion to include systems deployed on IT networks in the scope of the new standard. And I certainly can’t blame the SDT for not doing that, because:

1.       The decision to include IT systems within the scope of a CIP standard would have to be made at a higher level than an SDT; in fact, it would likely require some vote of the NERC ballot body.
2.       More importantly, FERC only gave NERC one year to develop the new standard, put it through four ballots (with changes between each one), get it approved by the NERC Board of Trustees, and finally put it on FERC’s desk to approve. Debating a big change like including IT systems would have made it impossible to meet that deadline.[i]

But I don’t think the fact that the SDT didn’t take up FERC’s suggestion of including IT systems in the scope of CIP-013 in any way settles the question whether IT systems should ever be included, in any way, in the scope of any CIP standard. My contention is that the Ukraine attacks show that ignoring the IT network altogether can make it more likely that a cyber attack could impact the North American BES at some point. I am certainly not saying that IT systems need to be included in the scope for all of the current CIP standards, or even for any of them. It may be that the main risk from IT systems is when they are deployed, as FERC implied in the quotations above, meaning that they should be included in the scope of CIP-013 at some point - although even then probably not in the same way as BES Cyber Systems are now.

But when I’ve said something about including IT systems in scope for CIP to people knowledgeable about NERC and NERC CIP, they have always disagreed with me, for two reasons. The first reason they bring up is that NERC has “no jurisdiction” over cyber assets on the IT network. I simply don’t believe this. Anything the utility does that can have an impact on the Bulk Electric System is in scope for NERC standards in general (and the distinction between IT and OT networks first appeared in the NERC standards with CIP, even though I don’t think the term “operational technology” had been coined then).

For example, there are a number of NERC standards (like FAC-003, the standard requiring tree trimming) that require records that are certainly kept on the IT network. In fact, OT networks don’t normally hold records at all, except records of the operations or configurations of the network devices themselves. If NERC wanted to create a new type of Cyber Asset in scope for CIP, called something like “Protected IT Cyber Asset”, I doubt this would violate anything in NERC’s Rules of Procedure, let alone Section 215 of the Energy Policy Act of 2005 (which set the foundation for mandatory reliability standards for the industry).

However, it is the second reason they bring up that I find most interesting. When I point out (as FERC did) that a compromise of IT networks at Ukrainian utilities led to the attacks on their OT networks, I inevitably hear, “Oh, that would never happen in North America. Even if a utility doesn’t have to comply with CIP, they all have well-configured firewalls in place to protect their OT networks (which of course the Ukrainian utilities didn’t have). And any utility subject to CIP has very good protections in place, beyond a doubt.”

Let’s stipulate that the point about all utilities having good firewalls is correct (and I have no evidence to suggest otherwise, although the problem with firewalls is they can always be made “wide open” through one mistake by an administrator, let alone a skilled attacker). And let’s go beyond that to stipulate that all utilities have great remote access control with two-factor authentication (as is required for Medium and High impact assets by CIP-005 R2). What these people are saying is that, if these two protections are in place (as well as other protections required by CIP-005 R1), there is virtually no possibility that a compromise of the IT network (even a thorough one like in the Ukraine, where it seems the attackers had free run of the entire network for six months or more, after initially getting a foothold through a phishing email) could lead to a successful attack on the OT network.

Of course, this is nonsense. It is equivalent to the French belief after World War I that an impenetrable line of forts along their border with Germany would prevent the Germans from ever invading France again. These forts were actually constructed and were called the Maginot Line. Of course, at the beginning of World War II, the Germans simply bypassed the line and invaded France through Belgium[ii].

So I really don’t believe there is any way someone can assert that the IT network can never have an impact on the BES, and therefore never needs to be in scope for the CIP standards. If they control the IT network, anyone with enough resources and time (both of which were in abundant supply for the Ukrainian attackers) will be able to find a way into the OT network, no matter what controls are in place. Here’s one example: Suppose an engineer gets an email from his or her boss’s account (which of course has been taken over by the attackers, probably through a keystroke logger that recorded his or her password), saying that at 2 PM the next day, he needs to open a particular set of circuit breakers as part of a test they are doing. Hopefully, the engineer will be suspicious of that request, but I think all of us can attest to times when we have come close to believing a phishing email, despite our thorough understanding of the dangers.[iii]

Again, I’m not saying that I want the scope of the existing CIP standards to suddenly be expanded to include IT systems. The proposal I am making in my book is to completely rewrite the standards (or more exactly, to replace them with new standards, or really just one new standard), so that – to make a long story very short - they are objectives-based and risk-based. IT systems will never pose the same level of risk to the BES as OT systems do, and therefore the entity will never need to apply the same level of controls to IT systems as they do to OT systems[iv]. But I also don’t want IT systems to be left out of CIP altogether. NERC and NERC entities have to give up the idea that their OT networks are safe from anything that could come through the IT network, behind their impenetrable Maginot line.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC.

If you would like to comment on what you have read here, I would love to hear from you. Please email me at Please keep in mind that if you’re a NERC entity, Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post. And if you’re a vendor to the power industry, TALLC can help you by developing marketing materials, delivering webinars, etc. To discuss this, you can email me at the same address.    

[i] As it is, I think the enforceability of CIP-013 R1.1 – the key part of the key requirement in CIP-013 – was reduced close to zero, due to the fact that the drafting team had to get something – anything – developed and passed by the deadline. I say this because R1.1 provides no list of threats that the entity needs to at least consider in developing their supply chain cyber security risk management plan. I discussed the issue in this post – where I called this a “near-fatal flaw” in CIP-013. Again, I can’t blame the SDT for leaving a list of threats out, since their short deadline didn’t allow for the discussion that would have been needed to include a list, as well as perhaps a few additional ballots as NERC entities second-guessed whatever list the SDT would come up with.

[ii] The French had of course anticipated the Germans would do this, and had a sizable force (with the British) in Belgium to counter them. But they were outmaneuvered because of another mistaken assumption they’d made; see the Wikipedia article referenced above.

[iii] Last week, I got an email supposedly from FedEx, about a package that was going to be delivered to me that day. Since I did have a package scheduled for delivery, I almost clicked on the link in the email before I looked at the email address and realized it was a phish. And three or four years ago, I heard a gentleman who was in charge of a big cyber security group at DHS mention that, in a test phishing email, a large percentage of the employees that reported to him – and probably warned people of the dangers of phishing every day – clicked on the link.

[iv] Of course, I realize that most utility IT systems are very well protected anyway, even though CIP doesn’t apply to them. But an important part of the “new CIP” proposal I am making in my book is that the utility needs to be able to look at all of the cyber threats to the BES at once, compare the risk each threat poses to the BES, and direct their efforts (and funds) toward mitigating the most important risks. Saying the IT network is completely out of scope, without even considering whether there are any serious threats to the BES that could come from the IT network, obviously defeats this process. If the utility truly believes they have already completely mitigated any IT threats to the BES from IT, they would certainly be able to assert this, with appropriate documentation. In the CIP compliance regime I’m proposing, NERC entities would be in charge of the decision as to which threats to mitigate, and to what degree they should do so. But they would have to document the reasons for their decisions.

1 comment:

  1. Nice blog... I really appreciate your work which you have done about the Securing OT Networks, many thanks and keep it up.