The Russian attacks: A new WSJ article puts them in a whole new light

This week, I intended to write the second part of my last post on Lew Folkerth’s great article on CIP-013. However, I believe this topic has more urgency. I will write a second post on this topic, then get back to Lew's article (I hope) next week.

Last Friday morning, I opened my subscription copy of the Wall Street Journal to see a front-page article entitled “Russian Hack Exposes Weakness in U.S. Power Grid”.[i] Then I read the article, very carefully. What was my first reaction? It was “Well, there goes my weekend.” I realized that this article is very important, for two reasons. First, it points the way to an important cyber attack vector that the industry, and especially the NERC CIP standards, hasn’t paid too much attention to. And yet it turns out that this was the primary vector the Russians are using, not the one I thought they were, based on the first WSJ article and DHS briefings last July. That is the subject of this post and the one to follow.

The second reason why this article is important is that it makes me (and I’m sure it will others as well) far less certain that the DHS briefings in July constituted a gross exaggeration of the success that the Russians had. Those briefings implied that the Russians had penetrated a number of utility control centers, where they would have had the opportunity to plant malware that they might call into action at a later date. I expressed great skepticism about this conclusion, and two days later DHS put out a completely different story, in which they said that only one “insignificant” generating plant (presumably gas-fired, going by a diagram that was shown) had actually been penetrated (i.e. at the control system level). Yet this was followed up a week later by a different story: that in fact just two wind turbines had been impacted, not a whole plant.

In a post in early September (which was preceded by others, and followed by one more), after describing the timeline that produced these three mutually contradictory explanations from DHS, I stated that I continued to believe that statements made at the initial briefings were wildly exaggerated – if not actually factually wrong, since the wording seemed to be very carefully chosen. I also emphasized that I really wished DHS would come out with a straight story on what really happened. However, last Friday’s article makes me question that conclusion, so that I now think it’s possible that the initial briefings were correct, and the Russians did penetrate a number of utility control centers. My third (and probably fourth) posts will discuss how Friday’s WSJ article caused me to rethink my conclusion, and will go on to address some of the huge implications, if it’s actually true that utility control centers were penetrated. These implications aren’t so much cyber implications as political ones.

Before I get on with the discussion of the cyber implications of Friday’s story, I want to point out that this is a great reporting job, by Rebecca Smith and Rob Barry. Ms. Smith is a veteran WSJ writer on the electric power industry and cyber security, and is the author of the article last July that caused a firestorm in the US and elsewhere, with its implications that the Russians had used the supply chain to penetrate a number of U.S. utilities and plant malware in their control centers.  The big difference between Friday’s article and the one in July is that the latter was primarily based on the first DHS briefing. Ms. Smith published it the day after the briefing, and there was certainly no time to follow up with other industry sources, try to verify some of the statements made by DHS, etc.

By contrast, Friday’s article is based on a lot of really dogged reporting (which has probably been going on since soon after the briefings), tracing in great detail, with lots of quotations from victims, how the Russian attacks actually proceeded through a number of small vendors to actual utilities (the article names five utilities that were attacked). In the article, Ms. Smith provides evidence that convinces me that my original scenario for how the attacks unfolded is incorrect.

The July briefings and WSJ article didn’t directly provide a scenario for the attacks, but I made a few assumptions in developing my own implicit scenario. I never wrote it down, but it was behind all of the articles I wrote on the Russian attacks last year. This scenario was:

1. The attackers were aiming for the Big Prize of cyberattacks on the US power grid: causing a cascading outage in the Bulk Electric System (this is obviously the way to cause the greatest total damage to the US economy). This means they would necessarily attack only transmission-level assets (i.e. BES assets), not distribution-only ones. You can’t cause a cascading outage by just attacking the latter.
2. Because of this, the best way to proceed is to try to obtain direct access to the control systems that control or power the transmission grid – i.e. control systems located at control centers, generating plants over 75 megawatts (including larger wind farms), and substations connected to the grid at greater than 100 kilovolts. In NERC CIP terms, these are High, Medium and Low-impact BES Cyber Systems, located at High-Medium and Low-impact assets (control centers, substations and generating plants).
3. Getting access to these systems is a formidable challenge. High- and Medium-impact assets (i.e. the more important control centers and substations, along with a small number of large or otherwise strategic generating plants) are almost all protected by two strong defenses (both required by NERC CIP).
4. The first of these defenses is well-managed firewalls, which make it very hard to make a direct frontal attack on the network in the asset. Largely due to NERC CIP compliance, these firewalls will have very few, if any, open and unprotected ports that a hacker could exploit.
5. The second defense at these assets is a well-protected system for Interactive Remote Access (IRA), including an Intermediate Server and two-factor authentication. This means that an attacker attempting remote access out of the blue will probably never get through the IRA system, unless they have found a way to break two-factor authentication – and I know of no verified cases to date in which an attacker has done that.
6. Low impact assets don’t necessarily have these two strong protections (some do), so they are easier to penetrate. On the other hand, they’re classified as Low impact because if compromised their loss will cause a much less severe impact on the grid than the loss of a Medium or High-impact asset. So the poor Russians won’t even come close to causing a cascading outage if they bring down a single Low-impact asset (they could perhaps do it if they attacked a lot of Low-impact assets simultaneously, but that is hard to do).
7. This means that no Transmission-level assets (BES assets) would be fruitful targets for Russian hackers. I assumed the attackers had tried to compromise these assets, not knowing how hard it would be to accomplish this goal. And I was for the same reason very skeptical of the initial DHS briefings and the WSJ article last July, which strongly implied (if they didn’t state it outright) that some Transmission-level assets (probably utility control centers) had been penetrated.
8. When DHS came out with their new story (and a week later, a second story) that said only a very small generating plant had been compromised (far below the 75 MW threshold for being a part of the Bulk Electric System), I took this as confirmation that I was right, and the Russians had essentially wasted a lot of time and money trying to break into something that was pretty much impenetrable.

However, the Friday WSJ article implicitly describes a very different scenario for the attacks:

1. The biggest difference between the new scenario and the one I was assuming is that the attackers weren’t obsessed with a cascading BES outage as their be-all and end-all. They were looking to cause whatever damage they could (or more specifically to position themselves to do so in the future if called upon), and they were fine with attacking the distribution system. In particular, they were looking at cutting off power distribution to military installations, which of course is a very understandable strategic purpose (and I assume the US is doing the same sort of reconnaissance and probing in the Russian grid).
2. This means that the attackers weren’t going to be stymied by the fact that they couldn’t penetrate any Medium- or High-impact assets. A single military base could in most cases easily be attacked by disrupting a single Low-impact generating plant or substation, or even a distribution-level plant or substation. Because of this, the Russians’ universe of possible targets was much larger than I was assuming last summer – so I was wrong last week in pointing out to the large spike in Russian readers of my post (among whom I assumed were at least some of the people involved in attacking the US grid) that their attacks so far had been a “dismal failure”. Instead, they might well believe them to be at least moderately successful, and Friday’s WSJ article provides some documentation for why they would be justified in this belief (of course, I’m not trying to lift the spirits of the Russian attackers by saying that! In any case, my spike of Russian readers quickly dissipated after that story, and now Russia is number four in my readership list, after the US (once again firmly in first place), Canada and the Ukraine (where I seem to have a steady readership, unlike the fickle Russians).
3. Another big difference between my original scenario and the one from Friday’s article is that I was assuming that the Russians would want to attack US power entities through vendors of control systems, by compromising the remote-access channels they already had set up with their customers. But the vendors discussed in the Friday article are quite different. They are all fairly small firms, including two excavating companies, an office-renovation firm, individual engineers (attacked through a watering-hole attack on a publisher of magazines read by power engineers), and others. So I was entirely wrong in my idea of the vendor entities that served as the intermediaries for the Russian attacks.
4. There’s no way that an attack on any of these vendor targets could ever get the Russians into the utility assets they needed to compromise in order to cause a cascading BES outage. But what could it do? It could get them into the IT networks of utilities. After all, every vendor interacts probably every day with utility staff using workstations attached to the IT network.
5. And the Russians didn’t have to compromise a remote access system to get to these workstations. All they had to do was to follow the same path used in the Ukraine attacks, as well as just about every other successful cyberattack worldwide in recent years: use phishing emails (or watering-hole attacks) to load malware onto workstations on the IT network. And once they were on one or a few workstations, it was much easier to compromise almost any other workstation on the IT network, since most IT network assets are much better protected from external attacks than they are from internal ones. The WSJ article provides great detail on how some of these phishing attacks proceeded.

Of course, the goal of the attacks wasn’t to compromise the IT network, but somehow to reach the control systems (i.e. the “OT” network, meaning operational technology), where they could drop malware that will allow them to come back later to turn that into actual destruction. And here we need to ask “Did the attackers reach any control systems?”  The article answers this question in the affirmative – and the systems weren’t in just two wind turbines or one small natural gas-fired power plant, as DHS stated this summer. Here are four paragraphs from the last part of the article:

Federal officials say the attackers looked for ways to bridge the divide between the utilities’ corporate networks, which are connected to the internet, and their critical-control networks, which are walled off from the web for security purposes.

The bridges sometimes come in the form of “jump boxes,” computers that give technicians a way to move between the two systems. If not well defended, these junctions could allow operatives to tunnel under the moat and pop up inside the castle walls.

In briefings to utilities last summer, Jonathan Homer, industrial-control systems cybersecurity chief for Homeland Security, said the Russians had penetrated the control-system area of utilities through poorly protected jump boxes. The attackers had “legitimate access, the same as a technician,” he said in one briefing, and were positioned to take actions that could have temporarily knocked out power.

           ……..

Vikram Thakur, technical director of security response for Symantec Corp., a California-based cybersecurity firm, says his company knows firsthand that at least 60 utilities were targeted, including some outside the U.S., and about two dozen were breached. He says hackers penetrated far enough to reach the industrial-control systems at eight or more utilities. He declined to name them.

To make a long story short, it seems the Russian attackers had a much broader goal than just causing a cascading BES outage, which made it perfectly acceptable for them to attack Low impact Transmission-level assets, as well as distribution-level assets not part of the Bulk Electric System at all – since both of these types of assets are much less well-defended than BES assets. Because of this broader goal, they weren’t confined to attacking utilities by commandeering vendor access to their remote access systems; they were perfectly happy using the tried-and-true phishing route to get into the IT networks of utilities. And from there, they were able to penetrate the control system networks of at least eight utilities, where they might have been able to deposit malware.

My second post in this series will discuss the implications of this finding for cyber regulation of the electric power industry, including the NERC CIP standards.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC.

If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that if you’re a NERC entity, Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post – especially on compliance with CIP-013; we also work with security product or service vendors that need help articulating their message to the power industry. To discuss this, you can email me at the same address.

---

[i] The WSJ web site is behind a paywall, so you can’t read the article there. I requested that the site provide a free link to this article, since I think it is of very high importance to the North American power industry. In the meantime, I found this online reproduction of the article.

I think all of you should seriously consider subscribing to the Journal, either in print or online. It has the best coverage of cyber issues of any major American newspaper. It also has the best coverage of economic issues, which I’m also very interested in. I don’t agree with the majority of the editorials or op-eds, but even then they’re all very well-written and informed, so you can’t just dismiss them unread like you can in some other publications.