NERC CIP: Time to make your voice heard!

Since last summer, I have been participating in a small, informal discussion group called the Cloud Technical Advisory Group (CTAG). It is composed of NERC and regional staff members (including a few current or past CIP auditors), staff members from NERC entities, representatives of two major cloud service providers, a few consultants like me, and one longtime staff member (and NERC CIP expert) from a four-letter federal commission.

As its name implies, the group’s “charter” is to discuss the problems that are preventing NERC entities with high and/or medium impact BES Cyber Systems (BCS) from fully utilizing the cloud, and to do our best to move the ball forward to finally address these problems. One positive step was the approval in December by the NERC Standards Committee, of a Standards Authorization Request (SAR) intended to lead to a complete “normalization” of cloud use by entities subject to the NERC CIP Reliability Standards.

I’ll be honest: This issue has been around since the cloud first became important, but it is only within the last 3-4 years that it’s received wide attention in the NERC community. I believe this was probably because not being able to make full use of the cloud was initially seen as primarily a missed opportunity to save time and money: “Gee, wouldn’t it be great if we could move all these systems to the cloud and not have to install and maintain them ourselves?” However, it seems that both NERC ERO staff and NERC entities were reluctant even to think too hard about the big changes to the CIP standards, and perhaps the NERC Rules of Procedure, that might be required for this to happen.

One important example of this, which has been discussed for years, is the fact that NERC entities with high and medium impact BCS are not currently able to utilize cloud-based network security services - i.e., services that operate a big SOC that monitors the entity’s networks and internet activities. These services become more and more valuable as they grow their customer base over time, since they can “see” so much traffic worldwide that is not visible through individual networks; thus, they can identify new threats much more quickly.

However, a NERC entity with medium and high BCS can’t utilize these services now, because by doing so, the cloud-based server would then become an Electronic Access Control or Monitoring System (EACMS). This means – among other things – that the server would have to be enclosed in a Physical Security Perimeter (PSP) operated by the entity.

That would have huge consequences; for example, the cloud service provider (CSP) would probably have to install card readers at all entrances to any one of their data centers that held any part of a BCS owned by the entity. All employees would have to badge in and out to that card reader, and they would have to do the same to separate card readers for every other NERC entity with medium or high impact BCS housed, in whole or in part, at any of those data centers. You get the idea: this is impossible for any CSP (and I could go on and on about the other impossible things the CSP would have to do).

One longtime NERC CIP auditor, now retired, told me about six years ago that an entity with high impact BCS in his Region had started using the services of one of the original cloud-based security monitoring services (which is one of the leaders in that field today) to monitor its networks, including its Electronic Security Perimeters (ESPs).

The auditor had to tell the entity to rip out everything they had put in place to use that service and instead install EACMS to do network access monitoring locally, in a PSP the entity could control. He said it “broke his heart” to have to do that, since he knew the entity’s level of security would decline because of this – and they would have to spend a lot of time maintaining on-premises devices that wouldn’t be needed if they could use the monitoring service. Of course, to this day, the entity is still using the on-premises “solution”.

I must admit that six years ago, I wasn’t particularly bothered by what the auditor told me, since I knew the changes to the CIP standards that would be required to allow this entity (and all similar NERC entities) to fully use the cloud were simply out of the question. I blamed the entity for their problems, since they should have known better than even to try such an outrageous stunt.

However, in the last year or so, the discussion has changed. Now, it’s much less about missed opportunities to save time and money and more about actual damage being done both to operations and to security of NERC entities with medium and high impact CIP environments. And now it isn’t just one or two entities that are complaining about this; more and more are complaining all the time.

But there’s an even more serious consequence of this problem, beyond diminished security. The big problem now is that NERC entities are hearing more and more from their software suppliers (including software for real-time operations in medium and high impact CIP environments) that they are moving to the cloud (i.e., becoming SaaS). The supplier might commit to continued support for their on-premises version for a few more years (and not always even that), but they usually make clear that their development dollars are going to the cloud. From now on, if the NERC entity wants to have all the new bells and whistles, they will have to use the cloud version.

When I joined the CTAG last summer, this problem was growing, with of course no end in sight. But even given that, there still wasn’t a sense that this was now not just a nice thing for the to-do list, but an urgent problem that needed to be solved soon. That is, there wasn’t until…last week’s meeting.

At that meeting, it was clear that the complaints about the cloud issue are now pouring, not just trickling in. However, since the SAR was approved in December, that means the clock is now ticking for a solution to the cloud problems to be in place. Of course, BCSI in the cloud was always one part of that solution, and it became reality on January 1. Unfortunately, the changes required for BCS, EACMS, and PACS (Physical Access Control Systems) to be in the cloud require much more thoroughgoing changes to the CIP standards, and perhaps even to the NERC Rules of Procedure (which I don’t believe has been the case for any previous change to the CIP standards) than did the BCSI changes.

So, at last week’s CTAG meeting, we looked at the question of how much time would be required between today and when a full solution to the cloud problem would be drafted, approved by NERC and FERC, and ready to be implemented by NERC entities. Here is my timeline:

1.      When the Standards Committee approved the SAR, they assigned it medium priority. They did that because there are over 20 other standards development projects (across all the NERC standards, not just the CIP standards) already in progress. Therefore, nothing at all will happen to the project before this July.

2.      In July, there will likely be a call for drafting team members. The first task of this team will be to put the SAR, as approved by the Standards Committee in December, up for comment. The team will use these comments to revise the SAR, and submit that to the Standards Committee for a new approval.

3.      When that approval is received, the drafting team will begin to work on drafting new standards required to allow full use of the cloud by NERC entities with medium and/or high impact BCS. Of course, they will then have to approve multiple successive drafts for comments (to which the drafting team will need to respond). Most previous changes to CIP standards have gone through at least four draft/comments/ballot cycles, before the new or revised standards are finally approved.

4.      The next step is approval by FERC, which can take over a year by itself. Given the major changes that will be required for this project, that is likely to be the case.

5.      How long will that process take? The CIP version 5 standards were a fundamental rewrite of all of CIP; this was the only previous set of changes to the CIP standards that is at all comparable to the huge changes that will be required to enable “Cloud CIP” (my term).

6.      The CSO 706 SDT, which drafted and passed CIP v5, had previously drafted (and passed) CIP versions 2, 3 and 4. They started work on CIP v5 in January 2011; FERC approved v5 in November 2013, close to three years later. However, CIP v5 included the “bright line criteria” for identifying BCS. These criteria were originally developed for CIP version 4 during 2010 (v4 was approved by NERC and FERC, but was never implemented. Long story). That effort took at least six months, so let’s say the whole drafting and approval process for CIP v5 took 3 ½ years. Starting from this July, adding 3 ½ years means that FERC approval of Cloud CIP

7.      Given that “Cloud CIP” will constitute an equally fundamental change in the CIP standards as CIP v5 did, it’s safe to say that 3 ½ years is a good estimate for the time the process will take, starting with seating of an SDT in early 2025. Thus, we can expect FERC approval of Cloud CIP by the end of 2027.

8.      After FERC approves the new standard(s), there will be an implementation period of probably one to two years. But, given that there are many NERC entities that want to be able to use the cloud as soon as possible, there will undoubtedly be some provision for them to start complying with the new standards earlier than that. So, let’s say that, six months after FERC approves Cloud CIP, NERC entities will be able to implement BCS and EACMS in the cloud. That means NERC entities will be able to start taking advantage of Cloud CIP by the middle of 2028.

However, there’s an elephant in this room. Since it's very possible that changes will need to be made to the NERC Rules of Procedure (RoP), we have to allow at least a year for that, since there will inevitably be a lot that needs to be discussed before the changes can happen. It would be nice to think that the RoP changes could be drafted and approved in parallel with the new standards, but it’s hard to see how the RoP changes could even be drafted until the new standards are finalized.

So, if we get lucky and there are no major glitches along this path, you can expect to be “allowed” to deploy medium and high impact BCS, EACMS and PACS in the cloud by the end of 2029. Mark your calendar!

Perhaps you may think that six years is a little long to have to wait for the cloud to be completely “legal” for NERC entities. I can assure you that the CTAG members thought the same thing last week.  How this will be fixed needs to be decided. But there should be no doubt about the need to accomplish this in some way. It’s time to make your opinion known!

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.

I lead the OWASP SBOM Forum. If you would like to join or contribute to our group, please go here, or email me with any questions.