Encrypting BCSI in the Cloud

In my most recent post I stated that I thought cloud storage of BES Cyber System Information was permitted by NERC CIP v5 and v6, and quoted a CIP auditor on what NERC entities (with High and/or Medium impact assets) needed to do to remain compliant with CIP if they do this.

The next day I received an email from Judy Koski of Tucson Electric Power, a NERC compliance professional I have known for many years. She pointed out “You have left out any mention of encrypted BCSI in the cloud.  If the information is encrypted in storage, the third party supplier does not have access, except to very limited personnel.  Does this not solve the problem?”

I immediately sent this question to the auditor who contributed to the previous post, and he quickly replied “I would argue that it is BCSI[i] and that the CIP-011-2 requirement to protect that information is achieved, in part, by encryption of the data at rest, what P1.2 refers to as in storage.  The fact that it is encrypted does not change the fact that the data is information about BCS.  So, yes, the other Requirements/Parts still apply.”

Since this auditor won’t ever use two words where one will suffice, I will “decrypt” his statement. He first points out that CIP-011-2 R1.2 requires the entity’s Information Protection Plan to include “Procedure(s) for protecting and securely handling BES Cyber System Information, including storage, transit, and use.” He agrees that encryption of BCSI while at rest at the cloud provider (or other third party) addresses the “storage” side of this, but that entities must also protect BCSI in transit and in use (of course, not necessarily with encryption, since there are many other ways to do this).

In addition to addressing the “transit” and “use” aspects of the above requirement, the auditor also pointed out that the three other requirement parts, included in a numbered list in my last post, still need to be complied with. Encryption won’t help with any of these, so you still have to address each of them.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

---

[i] In my email to the auditor, I had speculated that perhaps the encrypted data wouldn’t be BSCI at all, since the definition of BCSI includes the statement “BES Cyber System Information does not include individual pieces of information that by themselves do not pose a threat or could not be used to allow unauthorized access to BES Cyber Systems…” I reasoned that encryption meant the information couldn’t be used to allow unauthorized access to BCS. The auditor rightly pointed out that it’s still BCSI even though it’s encrypted. The encryption is one control that can be used to block unauthorized access.

A Break in the Cloud(s)? – Part I

At the beginning of January, I wrote a post stating that entities that utilize the cloud to store BES Cyber System Information are on shaky ground, and need to talk to their region before doing this (this problem only applies to NERC entities with High or Medium BES Cyber Systems, since they are the only ones that have to keep track of BCSI; I’ll say something about entities with only Low BCS in a future post). Since then I have come to realize that a number of entities are already storing BCSI in the cloud.

A typical scenario for how this situation comes to pass seems to be that a) the entity has engaged a cloud provider of a service like configuration management for all of their OT cyber systems (and maybe IT ones as well); b) either the decision to engage this provider was made without considering the CIP compliance implications, or those implications were simply ignored since the cost savings and improved ease of ownership were so huge; and c) whenever the CIP compliance people weighed in on this issue, they had to admit that CIP v5/v6 do not mention the cloud, so CIP’s position on storing BCSI in the cloud is very uncertain – if there even is any position at all. The result of all this is that the cost and ease of ownership reasons for using a cloud provider of a service like configuration management can far outweigh any reasons that may be adduced for not using a cloud-based service.

I must admit I have been in a kind of intellectual slumber on this issue, primarily because in CIP v3 there was no question that Critical Cyber Asset information could never be stored in the cloud. This is because CIP-003-3 R5 – which covered access to CCAI – required that the entity explicitly authorize all access to CCAI, and review all access annually to make sure it was still appropriate.

This requirement all but prohibited storage of CCAI in the cloud. Cloud providers usually have massive data centers with many thousands of servers (mostly virtual); hundreds of people walk into and out of these data centers every day or even every hour. Since any one of these people could potentially have at least physical access to the server where a particular piece of CCAI resides, it would require a huge effort to try to track who has access to that CCAI, let alone to authorize it and validate that authorization annually – plus, of course, removing access when the person leaves the organization. It is very unlikely that any cloud provider would agree with that.

But CIP v5 is quite different.   CIP-011-2 R1.2, Information Protection, simply requires the entity to document and implement an Information Protection Program that includes “Procedure(s) for protecting and securely handling BES Cyber System Information, including storage, transit, and use.”[i]

The Guidance for this requirement explains what the Standards Drafting Team had in mind when they drafted it. They made it clear that they weren’t ruling out the possibility that BCSI could be “shared with or used by” a third party (page 14 of CIP-011-2). Specifically, “The entity’s Information Protection Program should specify circumstances for sharing of BES Cyber System Information with and use by third parties, for example, use of a non-disclosure agreement.”

Since a cloud provider is definitely a “third party”, this quote seems to be saying that all a NERC entity needs to do, in order to store BCSI with a cloud provider, is to have an NDA in place. But most of the NDAs that I have seen are really focused on preventing employees of one organization, or the organization itself, from disclosing data belonging to another organization; this is important, but this is really just a small part of what we think of when we talk about protecting data. Is this really all that is required?

As I often do when I have questions like this, I emailed an auditor who understands CIP very well, and who has taught me a lot of what I claim to know about these standards. It turns out that CIP v5/v6 does have more to say about storing BCSI at third parties than what is found in CIP-011-2 R1. I will quote what he said almost verbatim:

“The crux of the matter is CIP-011-2, Requirement R1, Part 1.2, which requires the Registered Entity to have “Procedure(s) for protecting and securely handling BES Cyber System Information, including storage, transit, and use” for BES Cyber System Information associated with High and Medium Impact BCS and associated EACMS and PACS (it is interesting that PCA were left out of the applicability section.  I would be interested to know how often information about a PCA could be leveraged to craft an attack against the “protected” systems).

“So, I think there needs to be more than just an NDA.  If the Registered Entity is going to entrust their BCSI to a third party, I would expect the Registered Entity to have obtained, reviewed, and accepted the third-party’s protection controls, not just a signed boilerplate agreement to protect the information.  The Registered Entity cannot assign the risk and liability to the third party providing the information storage service.

You are correct, the Standard does not mandate anyone who has access shall undergo a PRA and training.  But there must be an “authorization for access” process.  Look at CIP-004-6, Requirement R4, Part 4.1.3 (“Process to authorize based on need, as determined by the Responsible Entity, except for CIP Exceptional Circumstances:  Access to designated storage locations, whether physical or electronic, for BES Cyber System Information”) and Part 4.4 (“Verify at least once every 15 calendar months that access to the designated storage locations for BES Cyber System Information, whether physical or electronic, are correct and are those that the Responsible Entity determines are necessary for performing assigned work functions”). 

“Additionally, look at CIP-004-6, Requirement R5, Part 5.3 (“For termination actions, revoke the individual’s access to the designated storage locations for BES Cyber System Information, whether physical or electronic (unless already revoked according to Requirement R5.1), by the end of the next calendar day following the effective date of the termination action”).  Just because the information is in the cloud does not relieve the Registered Entity of the responsibility to demonstrate compliance with these three Requirement Parts.  The third party will need to implement these controls and submit evidence of compliance to the Registered Entity (either direct evidence or an acceptable audit of the controls by a third party other than the service provider).”

To summarize, an entity with Medium and/or High impact BES Cyber Systems can store BCSI with a cloud provider. Per CIP-011-2 R1.2, the entity must have an Information Protection Plan that includes discussion of how BCSI stored at third parties will be protected. Since CIP-011 R1 is a non-prescriptive requirement, there are no prescribed contents for the IPP; it will be up to the entity and their auditor to determine whether the plan is reasonable or not. However, at a minimum:

1. The provisions for third parties in the plan cannot simply require an NDA; there must be some provision for reviewing the controls put in place by the cloud provider (which isn’t the same as requiring the NERC entity to “audit” the cloud provider!);
2. Per CIP-004-6 R4.1.3, the cloud provider will need to demonstrate to the entity that it has a process for authorizing access – perhaps not to the entity’s BCSI itself, but at least to the servers and physical locations where the BCSI is stored;
3. Per CIP-004-6 R4.4, the provider must demonstrate that at least every 15 months it reviews this access to make sure it is appropriate; and
4. Per CIP-004-6 R5.3, the provider must demonstrate that, for termination actions, access to the servers and physical locations storing the BCSI is removed by the end of the next calendar day.

So it seems I was far too pessimistic about NERC entities being able to store BCSI in the cloud when I wrote the post on this topic at the beginning of January. In fact, unlike almost all of the other questions of interpretation that I write about in this blog, I consider this to be actually a settled question – in this case, the CIP standards as currently written do provide an answer to the question whether NERC entities can store BCSI in the cloud. The answer is yes, as long as the above requirement parts are satisfied.

If you have a different opinion on this question, I’d like to hear it. I’d especially like to hear if there is something about cloud providers that makes them different from other third parties – meaning that taking the four steps just listed wouldn’t be sufficient to ensure CIP compliance.

However (there’s always a “however” in my posts, in case you never noticed that), this isn’t the full story regarding the cloud and CIP. There are at least four other questions that I know of:

First, there’s the question of outsourcing actual BES Cyber Systems to the cloud, as in the case of cloud-based SCADA. My opinion on this issue hasn’t changed since January: While I can’t say with absolute certainty that this will never be allowed under any circumstances for NERC entities with High and/or Medium BCS, I will say that any entity that wants to try this should without fail talk with their Region before moving ahead with the project.

Second, there’s the question of what constitutes BCSI. Is it possible that the NERC entity can store some information about its BCS, without storing actual BCSI? Third, there’s the question about assets containing Low impact BCS (and there are some scurrilous individuals that meet in dark corners and actually have the effrontery to refer to these as “Low impact assets”. Can you believe their chutzpah?). Since entities aren’t required to have a list of Low BCS, it seems they can’t identify Low BCSI either. Does that mean they’re off the hook, and they can take any steps they want in storing BCSI in the cloud?

Lastly, just because I’m a mean-spirited individual and don’t want to end a post on a positive note (after all, I have a reputation to uphold!), I do want to point out that there is an interesting loophole in the way CIP-002-5.1a R1 is written, that technically could mean that entities could outsource High and/or Medium impact BCS (i.e. the Cyber Assets themselves, not BCSI) to any third party they want – ISIS, al Qaeda, etc. – and not have to take any provision at all to protect them. I don’t particularly recommend you try this at home, kids, but I think it does show how the wording of R1 can lead to some strange situations.

I will try to address the second through last points in separate blog posts in the near future. Stay tuned to this channel! 

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

---

[i] This is a great example of a non-prescriptive requirement (CIP-007 R3 and CIP-010 R4 are two others). It would be wonderful if all of the CIP v5 and v6 requirements were written this way, although I believe more than that is needed to fix CIP.

Are You Attending RSA?

Are you attending the RSA Security Conference in San Francisco next week (i.e. the week of Feb. 13)? Deloitte will have a large booth (no. 427 in the South Expo Hall) to display some of the many services our 3,000 US-based cyber security consultants provide. We will also be presenting at some of the conference sessions. For more information on our presence at the show, go here.

I will be there all week, although I won’t be in the booth some of the time. If you would like to get together, drop me an email at talrich@deloitte.com. See you there!

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte Advisory.

Taking Your Position(s)

In the fall of 2014, I wrote a series of posts entitled “Roll Your Own”.[i] I wrote them because I had been despairing earlier that year due to my fear that NERC would never come up with guidance on the many ambiguities, inconsistencies and missing definitions in CIP version 5. That guidance never came (or if it did come, it was withdrawn), but of course NERC entities still had to press on with their v5 compliance programs, since the compliance date was (at that time) April 1, 2016.

At the time, the NERC entities who were most impacted by this uncertainty were owners of generation stations, especially large plants subject to Criterion 2.1 (i.e. greater than 1500 MW). Those plants can have literally thousands of devices that might or might not be determined to be Cyber Assets, and potentially BES Cyber Assets. They couldn’t wait until sometime in 2015[ii] for guidance. Their biggest concern was the meaning of the word “Programmable” in the definition of Cyber Asset. This is nowhere defined by NERC, and it turns out that different ideas for what this word meant could cause huge differences in what would be in scope for CIP v5 in large plants.

The NERC CIP compliance manager for a generation entity with several of these large plants – who is a personal friend of mine, and is now retired – wrote in to tell me what he was doing about this problem. Since he couldn’t wait any longer to identify Cyber Assets in these plants (which is of course the first step in the CIP v5 compliance process, along with identifying Medium and High impact assets in scope under the criteria of Attachment 1 of CIP-002-5.1), he had simply “rolled his own”  definition[iii] of Programmable.

When I asked his justification for doing this, he asked what choice he had. If he waited for NERC to come up with their own definition[iv], he would probably miss the compliance deadline. What was important was that he documented a) the definition itself, b) his reasoning for arriving at that definition, and c) the fact that he had looked through whatever guidance was available from NERC and his region on his issue, and his reasons for following or not following that guidance.

Of course, this approach to ambiguity in CIP v5 – which was independently “discovered” by others in the industry – was ultimately adopted by many entities, for many different interpretation issues: the meaning of “affect the BES” in the BES Cyber Asset definition, the meaning of “Facilities” in some of the Attachment 1 criteria, the entire methodology  for identifying BES Cyber Systems, etc. For all of these issues and many more, the only good approach I know of[v] is to take the three steps listed above. As Lew Folkerth of RF, along with a CIP auditor with another region, confirmed  a couple of months after that post, it would be very difficult for a future auditor to argue that you had done something wrong in this process (e.g. used the “wrong” definition of Programmable), even if they don’t agree with the interpretation you made. As long as your definition is reasonable and well documented, it is very hard to understand how you could fall into any compliance trap with this approach.

However, what is critical is that your position be documented. If you haven’t done that, and a future auditor disagrees with how you interpreted an ambiguity or an undefined term, you won’t be able to show that you used good reasoning to arrive at your position, and that you had considered any other official guidance that might be available. The moral of this short history lesson is: Whenever you encounter an ambiguous definition or requirement or a missing definition, you need to “roll your own” interpretation or definition, following the three documentation steps listed above.

You may ask what relevance this has now, for entities with High and/or Medium impact assets. After all, the compliance date for those was last July 1. What good will it do you if you can’t show that you had developed these “position papers” before that date? And the answer to that is simple: Position papers are nowhere required in CIP v5 (or v6), so you aren’t out of compliance if you didn’t develop them by last July. When you do have an audit, you will be in a much better position regarding these issues if you have these papers available than if you don’t. So you should develop them now.

But suppose you didn’t necessarily use a particular position, even undocumented, as you came into compliance with v5? What if your substation people, your Control Center people, and your generation people all used slightly different definitions of Programmable, or worse didn’t use any particular definition at all, just relying on “gut feel”?

Even if this was the case, I don’t believe you’re out of compliance since – once again – CIP is either ambiguous or silent on this and a host of other issues. So it’s not too late to develop positions. What you also have to do, though, is rerun the processes – especially for BES Cyber System and ESP identification – that are dependent on the interpretation of these issues, once you have developed your position on them.

For example, if you decide one group of people (say, your relay group) used a definition of Programmable that is at odds with the one you have now documented in your position paper on that topic, you will need to go back through all the devices that they reviewed to determine if some of those should now be identified as meeting the definition of Cyber Asset (based on your new definition of Programmable). You will then have to document that you have done this (you will also have to document that the devices whose classification hasn’t changed were also determined to meet this new definition). And if you have just now developed a definition of Programmable that was never used by anybody in your organization before this, you will have to go through every electronic device to see if this definition applies to it; those that do meet the definition will be Cyber Assets. Of course, this is a lot of work, but it’s worth doing it now rather than a year from now when you’re ordered to because you’re found to be in massive violation of CIP-002 R1.

Of course, for any newly-identified Cyber Assets, you will next have to run them through the BES Cyber Asset definition to determine whether they meet that[vi]; if they do, you then have to incorporate them into one or more BES Cyber Systems and apply all the protections of all the CIP v5 and v6 requirements that apply to them.

But what if your organization did develop and document positions regarding every significant area of ambiguity or missing definitions already, and you can prove they were applied wherever applicable? Does that mean this whole discussion doesn’t impact you? No, it doesn’t. You still have to document positions whenever you are going beyond the current wording of the CIP v5 and v6 standards and definitions.

For example, suppose you have virtualization – even just a little – within any of your ESPs. You will need to document how you are protecting virtual assets (e.g. VMs within an ESP, an ESP implemented as a VLAN on a physical switch that incorporates non-ESP VLANs as well, a SAN that stores data both from inside and outside an ESP). Since CIP doesn’t say anything about these topics, you have to develop your own guidelines on how to protect these virtual assets. NERC tried to develop guidelines two years ago, and ended up withdrawing whatever they did develop. The issue of virtualization is now on the table for the CIP v7 Standards Drafting Team, but it will be at best a very long time before the CIP standards address virtualization. If you’re able to wait years, great. But a lot of companies feel the savings from virtualization are too large to wait any longer to implement it. If you are one of those, you need to start writing your positions.

And then there’s the cloud. CIP is silent on that, and I know of no written guidance from NERC or the regions[vii]. Yet I have recently come to realize that some large and medium-sized entities are using cloud-based asset management software for both ESP and non-ESP devices. I will write a post soon on this issue, but if you are one of these entities, I strongly suggest you document how the information from within your ESP that is being stored in the cloud is protected.

  

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte Advisory.

---

[i] The first was this one. There were six more after that, which you can find by dropping down the month listings at the side of this page for September through December, 2014.

[ii] NERC’s self-imposed deadline for providing fairly comprehensive guidance (Lessons Learned and FAQ responses) kept getting pushed back, but I know that in January of 2015 the date of April 1, 2015 was stated at a WECC meeting. Of course, that date came and went without comprehensive guidance showing up. A few months after that date, NERC abandoned  the whole idea of providing comprehensive guidance and withdrew a number of the Lessons Learned. Some of these had actually clarified a few important issues, like how virtualization should be addressed and the meaning of “Programmable” in the Cyber Asset definition. But they all faced the same problem: Any “interpretation” of an ambiguous requirement or a missing definition requires either a Request for Interpretation or a SAR. And both of these take years to produce results.

[iii] You wouldn’t call what he did a definition, but rather a procedure for determining whether a device was programmable or not. I know some other entities also adopted that procedure.

[iv] NERC actually put out a Lesson Learned in early 2015 that provided what seemed to be a fairly reasonable definition of Programmable. But that was later withdrawn, along with all of the other Lessons Learned that tried to fill in gaps or clarify ambiguities in the CIP v5 requirements or definitions. NERC finally admitted that there was no way to fix these problems short of redrafting requirements or definitions, and they started the drafting process by writing a Standards Authorization Request and constituting a new CIP drafting team. That team is currently working on the items in their SAR; however, I am now very skeptical that they will ever be able to complete work on those items.

And as it turned out, the list of items in the SAR fell far short of the number that would be needed to address all of the very serious interpretation issues in CIP v5.  But the SAR would have been even more unachievable than it is, had all those items been included. I now believe that we’ve reached the limit for meaningful changes that can be accomplished given the current prescriptive format of NERC CIP. Only a non-prescriptive format will allow CIP to be updated in a timely manner, to address new threats and technologies.

[v] Short of asking your region for a definitive “ruling”, which you will never get, at least not in writing.

[vi] Of course, the BCA definition itself has at least one hole in it: the meaning of “adversely impact the BES”. You need to document how you determine whether the loss, misuse, etc. of a Cyber Asset will adversely impact the BES. Then you have to identify those Cyber Assets whose loss, misuse, etc. would have that adverse impact within 15 minutes.

[vii] Tobias Whitney of NERC said at the last CIPC meeting that NERC was going to come out with some sort of guidance document on the cloud. If I were you, I wouldn’t hold my breath waiting for that to happen. NERC simply can’t do this while still following the Rules of Procedure, unless they convene a new Standards Drafting Team to address the issue by modifying the CIP standards (i.e. CIP version 8). And just as with virtualization, the time required to do this – within the current prescriptive CIP framework - will be not much less than the lifetime of the observable universe, if that.