The cost of regulatory uncertainty

This isn’t news to most electric power industry participants, but I’ll say it anyway: Uncertainty about whether an entity will be held for NERC CIP violations if they put information on BES Cyber Systems (BCSI) in the cloud imposes a big cost.

I received a good example of this when Mike Prescher of Black & Veatch, whom I know from the Supply Chain Working Group, wrote in about a problem they have. As you probably know, B&V executes a lot of large projects for utilities and other industries. To manage these projects, they have a number of tools that they’ve built in the cloud. For example, on telecom modernization projects – which always include a big enhancement of the customer’s telecom security – a project can take days instead of weeks, and weeks instead of months, when they manage the projects with these tools.

But some of these projects are for the power industry, and I’m sure you can anticipate what I’ll say next: In those cases, they don’t currently use those tools because they’re worried about their clients being cited for CIP violations. Mike asked me to clarify what their options were, and I replied (in line with this post):

1. There is nothing in the current CIP requirements that prohibits keeping BCSI in the cloud. The Information Protection requirement, CIP-011 R1, requires a program for protecting BCSI in “storage, transit and use”. As far as this Requirement is concerned, it doesn’t matter where the BCSI is, as long as it’s protected.
2. The problem comes with four requirement parts in CIP-004, which govern controls on people who access electronic or physical locations of BCSI. And even then, the problem isn’t so much with the requirement parts themselves, but with the evidence required. There is simply no way a cloud provider could provide that evidence (which, as with most CIP requirements, needs to include documentation that the requirement part was followed in every instance, for every person), without abandoning the cloud business model altogether, and becoming something like an outsourced data center, where electric utilities store OT servers for convenience, but maintain full management of them.
3. So in order for BCSI to be officially allowed in the cloud, there will need to be changes to some of the Requirements, or Measures, or both in CIP-004. NERC is in the process of putting together a new drafting team to draft whatever modifications are required, but of course those are now years away from coming into effect. So the problem remains for the foreseeable future: Since the NERC entity can’t provide acceptable evidence of CIP-004 compliance when BCSI is stored in the cloud, any entity that has BCSI in the cloud will be open to CIP-004 violations. I told Mike that any entity who is looking for regulatory certainty should keep their BCSI out of the cloud.

There are two good reasons why more than a few NERC entities are now storing BCSI in the cloud. First, until recently NERC was planning on moving a huge trove of CIP compliance data (mostly BCSI, of course) to the cloud, in their Align project. That project’s on hold now for other reasons (it seems the company that develops the GRC tool they were going to use for it was bought by a Chinese entity – what could possibly go wrong with that?), but if they had gone ahead with the project, it’s hard to see how every NERC entity wouldn’t have felt completely free to move all of their BCSI to the cloud. How could they ever have been cited for a violation when NERC itself was the biggest violator? But even with that project on hold, it will certainly be very hard for NERC to ever come down hard on anyone storing BCSI in the cloud in the future, when they were very happy to do it themselves.

Second, there’s the example of virtualization. It’s no more “legal” to utilize VMs, VLANs, or storage arrays within an ESP than it is to store BCSI in the cloud, yet I doubt there’s any NERC entity today with High or Medium impact BES Cyber Assets, who would hesitate to use virtualization because of fears of getting cited for non-compliance with CIP – in fact, all of the NERC Regions talk freely about how to do virtualization properly in an ESP, and NERC itself has put out at least one document that discusses that subject. NERC entities have been virtualizing in ESPs for a long time; I know at least one entity that passed a CIP audit (probably the “first 13” spot check) in 2010 for their virtualized Control Center.

Does this mean that NERC entities need to just be patient and wait ten years before BCSI in the cloud is widely accepted by NERC auditors and entities? I certainly hope it’s not that long, but it’s certainly going to be 2-3 years before doing this is officially “legal”. Until then, NERC entities have the choice of crossing their fingers and putting BCSI in the cloud, or avoiding any uncertainty and not doing that. But as Mike points out, there are lots of costs to not putting BCSI in the cloud. And there will inevitably be more costs as time goes on.

Will we ever reach the point at which NERC entities snap and demand of NERC and FERC that they fix this problem? I doubt it, simply because it’s too easy to simply allow your BCSI to be put in the cloud, especially when you know other entities in your Region are doing it (and I’m sure this is being done in all Regions – in fact, I attended a forum on using a certain cloud-based workflow tool for CIP compliance last week. There were about ten entities there, from three Regions. There are a good number of other entities using the same tool, in these and other Regions). If nothing is done to change CIP-004, I think BCSI-in-the-cloud will become like virtualization – close to ubiquitous, but technically still not allowed by CIP.

At the beginning of my first post in this series on the cloud, I pointed out that there are two cloud questions for CIP: putting BCSI in the cloud and putting BES Cyber Systems themselves in the cloud (e.g. outsourced SCADA). I ended up doing five posts on the first question; I’m concluding with the statement that the BCSI problem is essentially solved, since NERC entities are doing it now, and they’re passing audits – even if it may be years before this is totally “legal”.

My next post (I doubt it will be the last) in this series will be on the second question. That is much harder, and the outlook is much darker for that question. Currently, there’s no way to put BCS in the cloud and be anywhere close to 100%, or even 50%, compliant with CIP. And I don’t see that changing until the CIP standards are almost entirely rewritten.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC.

If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that if you’re a NERC entity, Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post – especially on compliance with CIP-013. And Tom continues to offer a free two-hour webinar on CIP-013 to your organization; the content is now substantially updated based on Tom’s nine months of experience working with NERC entities to design and implement their CIP-013 programs. To discuss this, you can email me at the same address.