Thursday, September 6, 2018

What if it really was control centers?

As I described in my last post, until last Friday I believed that a) despite whatever misleading statements were made in the DHS briefings on the Russian attacks (and duly broadcast in the press), the Russian attacks didn't penetrate any utility control center (i.e. EMS) networks. I also believed that b) while it was unfortunate that the two walk backs by DHS were different (one said that only a single plant was compromised, but the second said that just two wind turbines were compromised), it was certainly conceivable that the first one was mistaken and the second was simply a correction of it.

However, last Friday a longtime industry observer pointed out to me that the single piece of concrete evidence of compromise of a control system network that was presented in the DHS briefings was an HMI screen shot showing a diagram of a gas combustion turbine, which had been taken by the attackers and uploaded . It seems that both a gas CT plant and a wind farm were compromised. This means that either a) Christopher Krebs, the person who said that only one facility - and at that facility only two wind turbines - was compromised was wrong; or b) Leslie Fulop, the earlier spokesperson who said that a single plant was compromised, was wrong.

However, I still believe that this confusion was just that – not intentional. But for me to continue to believe this much longer, I (and presumably others) would like DHS to, for once and for all, say exactly how many control system networks were compromised and what kinds of assets they were associated with (gas CT plants, wind farms, coal plants, nuclear plants - God forbid! - or anything else). Since this will be DHS’ fourth story, I sure hope this doesn’t have holes as well.

But now I want to ask, what if a) is wrong above? That is, what if there were actually one or more utility control centers penetrated – meaning the actual OT network, not the IT network that’s physically contained in the control center? Why am I asking this? Does it mean I've begun to think that really happened? No it doesn’t, mainly because the implications of that, if true, would range from serious to truly horrific. But there are obviously a lot of people outside of the utility industry (including in the technology press) who are quite ready to believe this, which is why the story rocketed around the world a month ago that the Russians had penetrated “hundreds” of US utilities and were poised to throw the entire US into darkness at a single word from Mr. P. I contend these people wouldn’t hold this view so easily if they knew the real implications – which I’ll outline shortly.

But why do these people believe the story? Of course, it’s because it came from major press reports. And where did those reports come from? Was it just the fertile imaginations of some reporters? Unfortunately not. There were a number of statements from DHS during the briefings that a reasonable person would assume meant that multiple utility control centers had been compromised. Here are a few of them:

  1. Jonathan Homer of DHS said in the first briefing that “They got to the point where they could have thrown switches” and disrupted power flows[i]. Of course, we all know that he was talking about software “switches”, but by talking about disrupting power flows he could only be referring to software running in utility control centers, not control rooms of individual generating plants. An attacker that penetrated a plant control room wouldn’t be able to do anything more than shut down the plant. And since DHS has admitted that the “single plant” that was compromised was very small and couldn’t affect the grid if it was lost, there is simply no way that this is what Mr. Homer could have meant when he talked about disrupting power flows (assuming the walk back is correct).
  2. The second WSJ article on this topic, dated August 7 (which I wrote about in this post), starts by saying “Top administration officials are..(discussing striking back) deter attacks such as the successful penetration of U.S. utilities by Russian agents last year.” One paragraph later, the article says “Hackers..claimed ‘hundreds of victims’ in a campaign against the energy sector that ultimately put them inside the control rooms of U.S. electric utilities where they could have caused blackouts..” Again, you can't cause a blackout by shutting down a single small generating plant (and you usually can't cause one by shutting down a big plant, given the redundancy built into the grid). Admittedly, neither of these is a direct quote from someone at DHS, but I sincerely doubt this reporter was just making the stuff up.
  3. Here’s a real smoking gun. Later in the same article, you find this quote “In March, Homeland Security and the FBI pinned responsibility on…Energetic Bear, for intrusions into utilities that gave attackers remote access to critical industrial-control systems, called SCADA.” SCADA is found in utility control centers, not plant control rooms. Again, this isn’t a direct quote from DHS, but I'm also sure the reporter didn’t dream that they said this.
  4. In the next paragraph, the article continues “In April, Russian hackers were using…internet another way to…maintain a hidden presence in control networks…” This is of course a different Russian attack campaign (those guys have been really busy!) that attacked routers. I hadn’t heard that this resulted in penetration of control networks at utilities, but they’re saying it did.

What’s most disturbing about this is not so much that these statements were reported, but that DHS has done nothing to set the record straight, except for the two walk backs which themselves need to be walked back. So when they do their third walk back, I would also like them to explicitly state whether they know of any penetration of U.S. utility control networks (meaning EMS or transmission SCADA) by the Russians, Chinese, Nigerians[ii], Maldive Islanders…whoever.

Nevertheless, I continue to believe that no utility control centers were penetrated. Why do I say this? There are two reasons. The first is that there would definitely have been a lot of very noticeable activity if that had happened. When the Ukraine attacks happened, there was a big inquiry and there were lots of briefings, both classified (initially) and unclassified; for an attack in the US, there would have been much more than this. I have heard nothing about any of these things happening.

The second reason is that, if it’s true that just a single small EMS system was penetrated, the two DHS people who said that only a small plant or two wind turbines were compromised should obviously be immediately fired, both because they lied and because they presumably inhibited DHS from taking the needed steps to notify the industry and the public of the danger. And if - let’s say - even one large utility control center (controlling a major urban area) was compromised, not only should these people be fired, but there should be a full investigation of how that happened and whether higher-ups were involved. If we’re really talking about a city being threatened with a major blackout (which will very likely result in deaths, especially if it’s more than a few hours) due to deliberate actions or inaction by people at DHS, we are now talking about treason, not just dereliction of duty.

And this is why I don’t believe any utility control centers were compromised, despite all the DHS statements implying (or stating) otherwise.

However, we seem to be forgetting something very important: If the Russians have compromised one or more major utility control networks and could be poised to cause a major outage (as most of the news stories on the attacks indicated), this constitutes a true national emergency. I am mystified that this hasn’t been done already, but somebody needs to get on the red phone to Mr. P and tell him very clearly that he needs to immediately cease all cyber attacks against US critical infrastructure (which now includes voting systems, of course), and make sure any malware that has been planted has been removed. He will have 48 hours to get this done, at which point a set of rapidly escalating sanctions will be put in place.

This time the sanctions won’t just consist of putting even more financial pressure on some of his oligarch cronies or exposing to all the world where he’s stashed the approximately $35 billion he’s reported to have amassed for himself (all on his modest salary, I’m sure).  One of the progression points might be banning Russian aircraft from all airspace worldwide (of course, this would require coordination with our allies, which seems to be a lost art in Washington these days. Hopefully, somebody still remembers how to coordinate with, rather than bash, our allies) until full compensation is paid to the families of all victims of the shooting down of Malaysian Airlines flight 17 in 2014, as well as to their governments for the direct expenses and general grief their countries have suffered because of this event.

Someone suggested to me that the fact that we hadn’t come down harder on the Russians so far for their vigorous attempts to penetrate utility control networks was because we have been doing the same with their utility control networks, and – unlike the Russians – we may have actually succeeded in penetrating them. In other words, everyone does it, so we’d be self-righteous to make a big deal about it – especially since the Russians didn’t succeed.

Here’s a story about another national emergency: the Cuban missile crisis of 1962, when President Kennedy found out the Soviets had installed nuclear-armed missiles in Cuba, aimed of course at the US. Khrushchev had installed them in response to a) the US’ attempted invasion of Cuba at the Bay of Pigs in 1961, and b) NATO’s recent installation of Jupiter nuclear missiles in Italy and Turkey aimed at, naturally, Russia.

Kennedy didn’t tell people to calm down, since the Soviets were quite justified in being a little upset about these two events. Rather, he did what was necessary to defend the US. If in fact the US has penetrated Russian utility networks, great! We all know that the US isn’t going to launch a cyber “first strike” on a foreign power grid. And we also know that Russia has already launched one of these in the Ukraine. Let’s not wait around for them to launch one here. 

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC.

If you would like to comment on what you have read here, I would love to hear from you. Please email me at Please keep in mind that if you’re a NERC entity, Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post – especially on compliance with CIP-013. And if you’re a security vendor to the power industry, TALLC can help you by developing marketing materials, delivering webinars, etc. To discuss any of this, you can email me at the same address.                        

[i] This was quoted in the Wall Street Journal article of July 23. The quotation marks didn’t include the phrase ‘and disrupted power flows’. But I seriously doubt the reporter just inserted that phrase on her own – she was probably paraphrasing Mr. Homer.

[ii] Perhaps the Nigerian prince who has been emailing me to send him money has given up trying to get blood from a stone and has turned to hacking SCADA for a living.

1 comment:

  1. "HMI screen shot showing a diagram of a gas combustion turbine" - this evidence alone doesn't mean a CC was compromised. Almost all entities have read-only stations connected to a server which has a read-only historical feed from Production (typically via a data-diode). Often times, the same exact "client" interface is used, and other than a lack of control access, it appears identical. Further, both the gas turbine HMI and wind farm could be monitored by a single entity with views into each system as I described, and no compromise on any control networks. This all could be just one CxO's laptop that was hacked who had read-only access to view both.