Do the NERC CIP standards drive grid investment?

Blake Sobczak of E&E News on Friday published the retrospective below (part of a series of articles on major energy events of the last decade), which put the two Russian cyberattacks on Ukraine’s power grid in their larger context. It’s a very good article, although I don’t think there’s anything in it that will be terribly surprising to anybody reading this post (and Blake said as much when he sent me the text). But I do want to point out one thing:

Midway through the article, Blake says “NERC and the Federal Energy Regulatory Commission took lessons learned from the 2015 and 2016 Ukraine attacks and incorporated them into new cybersecurity rules for the bulk power sector. Changes to the so-called Critical Infrastructure Protection standards brought about hundreds of millions of dollars in new cybersecurity investments across the U.S. grid.”

Actually, the Ukraine attacks haven’t led to any changes in the NERC CIP standards that are currently in effect. One change that did result from them was CIP-013, since in Order 829 FERC pointed to the first Ukraine attack (which had occurred about seven months previously) as one of their reasons for ordering NERC to develop a supply chain security standard.

CIP-013 will go into effect next July, but even then I doubt it will lead to “hundreds of millions” in new cybersecurity investments. As I wrote earlier, any entity that is spending large amounts of money on CIP-013 compliance is probably doing something very wrong. I’ve been working on almost nothing but CIP-013 compliance for a year, and I fail to see any reason for even large utilities to spend huge amounts of money on compliance (that is, anything close to the scale of what they spent coming into compliance with CIP version 5).

Literally all of the risk mitigation activities that I and my clients have identified for CIP-013 compliance are policies and procedures – either on the part of the utility or the vendor. Once you put in place the different parts of your mitigation program – RFPs, contract language, vendor questionnaires, procurement risk assessments, etc. – there is just about zero additional cost to add more mitigations. For example, if you’re already requiring vendors to answer a questionnaire with 10 security questions as part of their response to an RFP, asking them to answer 50 questions doesn’t add much more cost.

Of course, this is a good thing, since the CIP v5 rollout was just the opposite – it was hugely expensive, especially for the biggest NERC entities. However, I wouldn’t call that an investment in grid cybersecurity. A lot of people think that CIP compliance is mostly about buying and implementing software and hardware to enhance grid security. While there is certainly a portion of that, much more than 50% of CIP compliance spending goes to implementing processes and procedures.

The difference between spending on CIP v5 and CIP-013 is that v5 required huge investments in implementing some very prescriptive requirements like CIP-007 R2 (patch management) and CIP-010 R1 (configuration management), while CIP-013 – since it’s entirely risk-based – allows NERC entities to target whatever funds they have available toward mitigating the maximum possible amount of supply chain risk. In other words, the utility doesn’t have to go to the poor house in order to make a significant dent in the supply chain cyber risks it faces.[i]

However, I won’t deny that the power industry does need to make significant investments in grid security, mostly because of all the things that aren’t now required by the CIP standards (and probably never will be, absent a complete rewrite of the standards as risk-based). These include the need for much better network monitoring, the need to make much greater investments in preventing ransomware, the need to address new cloud security risks so NERC entities can start making much more use of the cloud for OT systems, and more. But probably the most significant is the need to start paying much more attention to securing the distribution grid, since that now seems to be the focus of the Russian attacks.

But here’s the rub: This spending would be on top of what utilities are now spending for grid security and CIP compliance. How deep is this well, anyway?

I think we’ve reached the point where we need to acknowledge that grid security is a national responsibility, and should be funded on a national basis. Of course, NERC entities will still have to spend lots of money out of their own pockets (which in most cases are ultimately the ratepayers’ pockets, but in many cases – e.g. the IPPs – every dollar spent comes straight from their bottom line). But these additional investments – and especially the investment in distribution security – need to be funded nationally. After all, the military bases and dams that the Russians (and Chinese) are probing have national importance.

However, at the same time we need to reform the CIP standards and compliance regime, so they are much more efficient and effective than they are now; if you’d like an overview of how I would do this (which doesn’t mention the national funding, but does include the other elements), you can listen to my recent webinar on this topic, or email me to see the slides from that webinar. Here’s the article:

The cursor slid across the Ukrainian grid operator's screen and clicked circuit breakers open, knocking out the lights to thousands of people outside Kyiv. Someone outside the country was controlling part of its power grid. Before that night, Dec. 23, 2015, hackers had never managed to douse the lights anywhere in the world. The first-of-its-kind cyberattack redefined the threats facing electric utilities and contributed to billions of dollars in spending on improving U.S. defenses. The unprecedented cyberattack — later traced to suspected Russian hackers — blacked out about 250,000 people in western Ukraine. Grid operators at the three victim utilities were able to flip breakers by hand and restore power within a few hours, but it would be many months before they could trust computers in any of their control rooms. The event was a shot across the bow for power utilities globally amid a rapid shift to so-called smart grid technology and internet connectivity. While these digital tools offer power providers the means to improve efficiency and gather reams of valuable data, they have also opened new pathways for hackers to break into critical infrastructure networks. Impact A month after the attack, a group of U.S. experts from the departments of Energy and Homeland Security, the FBI, and other agencies traveled to Ukraine to gather more information about what happened. They were joined by representatives from the private sector and the North American Electric Reliability Corp., which sets and enforces mandatory cybersecurity standards for the bulk U.S. power system. Details of their visit trickled out several months later, when cybersecurity firms started to share a few public takeaways from the investigation. The findings set off alarm bells in U.S. homeland security circles: a Russia-linked hacking group had deployed a variant of the "BlackEnergy" malware to take control of Ukrainian computers and stage a systematic attack on the distribution grid around Kyiv. A year later, the same hacking crew struck again, this time using highly specialized attack code dubbed "CrashOverride" to temporarily bring down a bigger target: a transmission-level substation north of Kyiv. Ukraine's grid operator was again able to restore electricity within a matter of hours, but the episode drove home the potential real-world consequences of new dangers posed by connected technology. NERC and the Federal Energy Regulatory Commission took lessons learned from the 2015 and 2016 Ukraine attacks and incorporated them into new cybersecurity rules for the bulk power sector. Changes to the so-called Critical Infrastructure Protection standards brought about hundreds of millions of dollars in new cybersecurity investments across the U.S. grid. The attacks also provided a stark backdrop for the establishment of several new cybersecurity agencies in the intervening years, including DOE's Office of Cybersecurity, Energy Security and Emergency Response; DHS's Cybersecurity and Infrastructure Security Agency; and most recently a reorganization of cybersecurity functions at FERC this year. Crystal ball A cyberattack is not known to have cut out power to any part of the North American grid. The only documented U.S. grid cyber disruption to have occurred was in March, but the attack was relatively unsophisticated and limited in scope. That "denial of service" incident caused a series of five-minute communications outages at several wind and solar farms in Utah, Wyoming and California, but stopped well short of causing any blackouts. Despite the lack of any grid hacking disasters, top American intelligence officials continue to warn of dire consequences from cybersecurity complacency. Then-U.S. Director of National Intelligence Dan Coats said Russia can cause temporary damage to critical infrastructure networks, "such as disrupting an electrical distribution network for at least a few hours — similar to [abilities] demonstrated in Ukraine in 2015 and 2016." "Moscow is mapping our critical infrastructure with the long-term goal of being able to cause substantial damage," Coats said in his office's Worldwide Threat Assessment. A successful takedown of even a small part of the U.S. grid would have far-reaching impacts for security policy, utilities' cybersecurity practices and — depending on who launched the attack — global statecraft. But the combined efforts of U.S. power companies, intelligence and homeland security professionals are likely to offer enough of a bulwark against the most catastrophic lights-out scenarios.

---

[i] The CIP-013 methodology I’ve worked out with my clients this year is designed to achieve close to the maximum amount of supply chain cyber risk reduction, given whatever resources the utility has available for the effort. If you’d like to learn more about this, drop me an email.

Opinions expressed in this post are not necessarily those of the clients of Tom Alrich LLC.

If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that if you’re a NERC entity, Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post – especially on compliance with CIP-013. My offer of a free webinar on CIP-013, specifically for your organization, remains open to NERC entities and vendors of hardware or software components for BES Cyber Systems. To discuss this, you can email me at the same address.