My CIP-013 Presentation at GridSecCon

As usual, NERC’s annual GridSecCon conference, which was held last week in beautiful St. Paul, Minnesota, was a great success. Every year I say this was the best one ever (I’ve only missed two since it began seven years ago), and this was no exception, both for the great programming and for the very competent organization. This year, attendance passed 500 for the first time. Congratulations to Bill Lawrence and the great team that organized the conference.

I had been asked to be part of a panel on supply chain security. Of course, this being a NERC conference (and the panel being led by Howard Gugel, who is in charge of standards development for NERC), most of the focus of the panel was CIP-013. I was certainly no exception to that rule! In starting my presentation (which I tried to hold to seven minutes – I think I succeeded), I admitted that I would be covering a lot of ground in a short time, and rather than forcing people to take a lot of notes I said I’d write up my presentation (and perhaps embellish it) in a post this week. So here it is!

I didn’t have the time or the inclination to do some sort of general disquisition on CIP-013: how it came about, what’s in it, etc. – I assumed the audience would have some idea of this. I also didn’t go into why I like the standard so much and why it’s so important that it is risk-based, since I’ve already discussed these topics. Rather, I just titled the presentation “Important Observations on CIP-013”, and I discussed ten observations that I thought would be of interest to the audience.  Here are the observations (in boldface), and my discussion of them (which definitely goes far beyond what I had time to say at GridSecCon).

CIP-013 is very simple. R1: You develop a plan. R2: You implement it. R3: You review it every 15 months. So the big question is: What should be in your plan? I believe all cyber security requirements should be objectives-based, and the requirements in CIP-013 are certainly that. So the whole question becomes, what is a good plan? Of course, this is where the real work comes in. Because there’s nothing in the requirements themselves – except for the six mandatory items listed in R1.2 – that tells you what needs to be in the plan, or even how to put the plan together in the first place. There’s the excellent – but far too short – Implementation Guidance developed by the drafting team, and I’m sure there will be a lot of other documents written that provide some form of guidance (including my posts, of course), but in the end your organization is going to have to decide for itself how you will put together your plan.

You need a supply chain cyber security risk management plan. It must address risks (threats) from a) procuring hardware and software; b) installing hardware and software; and c) transitions between vendors. Note there are three areas of risk that must be addressed in CIP-013, not just one; FERC specified that the standard needs to address all three. Just about everything I’ve read about the standard (including most of what I’ve written) focuses exclusively on the first area, vendor risk. But the other two areas are just as important, and they are focused much more on what the asset owner does, not the vendor. Since the requirement says all three areas must be addressed, you probably could receive a PNC if you simply omit either b) or c).

Even though R1.2 lists six particular items that need to be in your plan, you will make a big mistake if you confine your plan to those items. I wrote in a post recently that it’s very possible the only way you can get a Potential Non-Compliance finding from a CIP-013 auditor – other than simply blowing the whole standard off as unimportant – is if you omit one of these six items from your plan.

But this doesn’t mean the auditors won’t look at the rest of the plan at all. They will take a comprehensive look at your plan, and if they believe you’ve missed something, they’ll issue an Area of Concern. You would be well advised to address that AoC, even though strictly speaking you can’t be found in violation if you don’t. There are two reasons for this: 1) It’s The Right Thing to Do; and 2) You don’t want to get on the bad side of your auditor. Remember, even though your auditor’s baby is ugly, it’s not a wonderful idea to say that to him or her. The same goes for ignoring an AoC.

CIP-013 is risk-based: You should classify vendors and systems by risk, and apply controls based on the risk level. This is very different from the other CIP standards. The emphasis here is of course on risk management. If you define risk as threat times impact, you can see the difference between the CIP-013 requirements and most (although thankfully not all) of the existing requirements in CIP-002 through CIP-011, which I’ll call prescriptive requirements.

Like non-prescriptive requirements, prescriptive requirements are designed to address a particular threat (or set of threats). However, prescriptive requirements assume the impact of the threat is equal to 1 – the maximum. They also assume that the impact doesn’t vary according to the importance of the cyber asset(s) being protected. With the impact always being one, this means the risk corresponding to each threat is always the same. It also means that no BES Cyber System can be treated any differently from any other BCS in the facility, since each one is assumed to carry the same level of risk (and please don’t tell me that this is what the High/Medium/Low impact rating does in the other CIP standards. Even though CIP-002 R1 and Attachment 1 are written - although not consistently - to make you believe that the impact rating is an attribute of the BES Cyber System, in fact the rating is treated literally 100% of the time as an attribute of the asset itself. Once the asset is designated High, Medium or Low impact, every BES Cyber System within it is treated exactly the same as every other BCS within it, both by NERC entities and by auditors).

Unlike NIST and other cyber standards, “risk” in CIP-013 doesn’t mean risk to the organization. It means risk to the Bulk Electric System. But you also shouldn’t use the H/M/L classifications from CIP-002. You won’t be helping yourself if you do. Some organizations – including all owned by the federal government – are used to ranking risks posed by computer systems, for compliance with FISMA, NIST CSF, and other frameworks and standards. But the risk considered in these other frameworks is almost always risk to the organization itself.

NERC CIP – indeed, all the NERC standards – treats of risks to the Bulk Electric System; that is what NERC standards are designed to protect. So a payroll server for a utility could be hacked and they couldn’t pay their employees for months, causing many of them to quit. This would of course be a terrible thing for the organization, but as long as there isn’t any direct BES impact, it doesn’t mean anything to NERC CIP. This is why CIP only applies to OT systems, and not all of those, either.

And you shouldn’t classify your BCS using the impact level from CIP-002 R1. There’s a good reason for this. Since CIP-013 only applies to High or Medium BCS (although your plan can cover Lows as well. In fact, I can think of reasons why it would require very little additional work to include them in your plan now. I’ll have a post on this sooner or later), you will then have all high or medium risk BCS for CIP-013! So if your plan says you need to do X for high risk BCS, Y for medium risk, and Z for low risk, you will have to follow that and treat every Medium impact BCS from CIP-002 as a medium risk CIP-013 BCS. You won’t be able to put the set of controls Z on any BCS, since you’re not allowing for there to be any lows!

Of course, you don’t have to rank your BCS for CIP-013 purposes as high, medium and low risk. You can rank them as 1,2,3,4 or high/low risk, for example. It’s probably better if you adopt a different nomenclature for risk in CIP-013 than High/Medium/Low, to avoid the inevitable confusion with CIP-002 R1.

CIP-013 is also threat-based: You should make sure that your plan in principle addresses every threat, based on its level of impact (i.e. the risk). You should rate each threat according to its risk, then determine the appropriate level of controls – if any – based on the risk.

You might well ask, “How can I possibly address every threat to a BES Cyber System? There is literally an infinite number of them. There’s probably a small risk that a truck carrying a BCS to our facility will be overrun by a herd of elephants and the BCS will be crushed. How can I even identify all the threats, let alone mitigate them?” And this is where risk comes in; remember, risk is your friend in CIP-013 compliance! You need to consider all the threats (in each of the three risk areas discussed in the previous slide) and determine both the likelihood and impact of each threat; this yields the risk, of course.

Then you need to rank the threats by the risk that corresponds to each one (of course, you don’t have to rank them highest to lowest; you just need to group them in whatever number of risk levels you’ve decided is appropriate). This gives you your prioritized list of threats you need to address. And you can not only prioritize the threats, you can draw a line somewhere in the list and say “Below this line, the risk is so small that I don’t have to do anything more to mitigate it.” You then don’t have to even consider those small risks, although you have to document you are doing this consistently (not arbitrarily saying “Hey, I really don’t think this threat poses much of a risk. I won’t do anything about it). For example, I think it’s safe to say that no NERC auditor will every find you in violation of CIP-013 because you didn’t even consider the threat that a hurricane would pose in Nebraska!

And not only does the risk level help you prioritize the threats you face, it helps you determine the level of controls required. The riskiest BCS and vendors need to have the most stringent controls; the least risky should have the least stringent controls. Of course, this is very different from the other CIP standards!

The above has covered the six points in my first two slides. My final slide listed four points, and was titled “Some Lessons from Healthcare…” This was because I had been talking with a group of cyber security consultants at Deloitte who had been working for four years in the area of medical device security – working both with the vendors and with the hospitals (which are of course the equivalent of electric utilities in “our” space).

The medical device industry has been complying for four years with mandatory cyber security regulations from the Food and Drug Administration. These apply to the vendors, not to the hospitals. However, the hospitals are just as concerned about the vendors’ compliance as are the vendors themselves, so you could say that the hospitals are also subject to the regulations. Since this is a lot like the CIP-013 situation, it is informative to hear about this industry’s experience with mandatory cyber standards.

Utilities and vendors need to partner to secure purchased BCS. Dealing in a hands-off way doesn’t do much to advance security. One way to “cooperate” with a vendor is to throw some contract language over the wall to them and say “Here, Mr. Vendor. Put this language in your contract or else!” The vendor’s lawyers will then review the language, edit it to their taste and throw it back over the wall to you. Your lawyers will edit that document to their taste and throw it back…etc, etc. Of course, this is all great fun and ensures full employment for the lawyers. But it has nothing to do with making the BES more secure.

How about this approach: Your cyber and supply chain people sit down with the vendor’s cyber and sales people and say “How are we going to solve this problem – i.e. the product security and CIP-013 compliance problem – together? Let’s explore the different options….” You look at how the vendor could change their processes, how they could get you better information about their security and the security of their products, how they handle post-implementation support, etc. Sure, you can also discuss contract language and agree on language you both can live with. But that shouldn’t be the main focus of the discussion. It should be cyber security, and there are much better ways to achieve that besides contract language.

Of these two approaches, which do you think will actually increase cyber security? I’m going to go out on a limb and say number two. And by the way, I’m not saying you need to do this with every vendor. Remember the thing about risk? You will want to do this with the riskiest vendors – i.e. those who sell the systems whose loss would have the biggest impact on the BES; for other vendors, maybe contract language and a questionnaire would be enough. Or maybe just contract language, for the least-risky vendors. But if you’re dealing with your EMS vendor in the paper-over-the-wall manner I just described, I think you should seriously rethink how you’re treating supply chain security.

Legacy devices also need to be secured, even though CIP-013 is silent about this. Again, utilities and vendors need to work together to secure legacy devices. CIP-013 only covers BCS purchased after the standard comes into effect. But there are lots of legacy devices in the field, and it’s almost axiomatic that they don’t have as good a level of security as new systems. What should you do about those?

Fortunately, your discussions with vendors on how you’ll cooperate for CIP-013 compliance are the perfect opportunity to discuss legacy systems. What are ways that both of you can cooperate to improve security in those systems? Don’t wait for the next serious vulnerability to make the headlines.

Incident response is the biggest test of cooperation. There’s nothing like a good ol’ hair-on-fire cyber incident to test how well you and your vendor can work together. But guess what? It’s much better not to wait for an incident to find out how well you cooperate, or whether you can cooperate at all. When you have the meeting with your vendor to discuss CIP-013 compliance, bring this up as well.

Procurement needs to lead the conversation; vendors are much more likely to listen to the people who write the PO’s! I realize this may shock you, but vendors tend to pay the most attention to the people who actually issue the PO’s. If you (Mr/Ms cyber/CIP person) want your vendor to do something important, don’t just bring this up to them yourself. Get your procurement people to make the actual call.

The views and opinions expressed here are my own, and do not reflect those of any organization I work with. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com. 

Two Lessons from CIP-014: First Lesson

I have recently been wondering how CIP-013 will be enforced, since this is a non-prescriptive, objectives-based standard. I recently concluded that it in effect wouldn’t be “audited” at all, since there will be no way to find an entity in non-compliance with the requirements[i]. Of course, the auditor will still review the plan, and its implementation, from a general supply chain security perspective. If he or she feels that there are parts that need to be improved, they will issue an Area of Concern – which the entity would be well-advised to take to heart. However, I don’t believe there will be any Potential Non-Compliance (PNC) findings issued for R1 or R2 as a whole, unless the entity has simply done nothing or very little to comply with these requirements – and I find it impossible to believe that a NERC entity with Medium or High impact assets would do that.

However, CIP-013 isn’t the first CIP standard that is non-prescriptive and objectives-based. CIP-014 (the physical security standard that applies to certain important substations) is in principle the same (although I would say that CIP-013 goes further in that direction, but not by much). While CIP-014 certainly isn’t being audited yet, there has been a lot more opportunity for entities to talk with their regions about auditing and other compliance questions. What have they found?

I haven’t done any sort of scientific survey, but I did have a long conversation with a NERC physical security compliance person at one of the largest utilities in the US, about his experience so far with CIP-014. He had two stories to tell, which illustrate the challenges ahead for both CIP-014 and CIP-013 compliance enforcement. They also relate very directly to the larger question of how, if all of the CIP standards were re-cast in a non-prescriptive, objectives-based format, they would be complied with and enforced.

The First Lesson

This utility is putting a lot of money into CIP-014 compliance. There was one particular investment of $80 million that was being strongly considered. However, before the powers that be would commit to this investment, they asked my friend to find out whether this investment would enhance their chances of being found compliant with the requirements of CIP-014.

Since some of you may not be familiar with CIP-014, the standard requires the entity to (among other things):

1. Conduct a risk evaluation[ii] to identify which of its facilities (control centers and transmission substations) meet the criteria for inclusion in this standard;
2. Have a qualified third party validate that evaluation;
3. For the substations and control centers that are in scope, conduct an assessment of the facilities’ “potential threats and vulnerabilities” to physical attack;
4. For each facility in scope, develop and implement a physical security plan that will, among other things, address the threats and vulnerabilities identified in the assessment; and
5. Have a qualified third party validate both the assessment in step 3 and the plan developed in step 4. The third party may recommend changes in either document; the entity must change the plan to reflect those recommendations, or document why it did not. And since the plan has to be implemented, these changes will also need to be implemented.

Now that you know how CIP-014 works, you’ll be able to understand my friend’s problem. The plan in step 4 has identified the $80 million investment in question as being required to address one or more threats and vulnerabilities identified in the assessment in step 3. However, since no NERC entity has unlimited funds to address each threat and fix each vulnerability, there need to be trade-offs. This $80 million investment undoubtedly came at the expense of spending an equivalent amount of money to address some threats and vulnerabilities that were not considered to have such a high impact. But the entity – and the third party that reviewed both the assessment and the plan – determined that the impact of the threats addressed by the $80 million investment was sufficiently greater than that of the other threats, that this was the proper way to spend the money.

But management’s concern is this: NERC will give the final “assessment” of the plan when they come for an audit. What if they make the investment, then in a later audit NERC decides that they had their priorities wrong? In other words, that they should have spent the $80 million addressing some of what they thought were lower-impact threats, meaning NERC disagrees with them on their assessment of the impact of the threats in question. Will NERC then order them to spend an additional $80 million addressing these other threats?

It’s certainly a reasonable question, and my friend was tasked with asking it of their Regional Entity; in effect, he was going to ask the region whether they could review their assessment and plan, at least as they pertained to this particular issue. What do you think was their answer? There really was only one thing the region could say: For us to review your plan before you implement it would be a compromise of the time-honored principle of auditor independence. If we tell you how to comply up front, then when we come back to audit we will simply be auditing ourselves.

I don’t think any final decision has been made on the $80 million investment, but my friend thought it very possible he wouldn’t be allowed to proceed with it without some sort of nod from NERC or the region. So the threats and vulnerabilities addressed by that investment will likely remain unaddressed, until NERC audits them and decides they need to make the investment; hopefully, this finding won’t come with a Potential Non-Compliance finding, but just an Area of Concern.

I hope you understand that I’m not in any way saying the region had a choice in how they responded to this entity. Under the NERC Rules of Procedure and Compliance Monitoring and Enforcement Plan, the auditors must maintain strict independence. But that independence comes with a cost. In this case, the cost is a set of substations that are probably not going to be as physically secure as they might be, if the entity had gone ahead with the investment.

How does this relate to CIP-013? CIP-013 asks the entity to develop and implement a supply chain cyber security risk management plan (SCCSRMP). There is a 10-page Implementation Guidance that addresses what should be in the plan, which will – according to NERC’s current views on guidance - carry weight with the auditors (unlike any other guidance that may come out). It’s a very good document, but it could be 1,000 pages and still not cover everything needed to develop and implement a good plan.  

As with CIP-014, the entity will have to develop the plan and implement it, without any official guidance from NERC on whether it’s a good plan or not. As with CIP-014, the entity could go for years believing their plan is good, only to have all of this contradicted years later by a NERC auditor. Since the CIP-013 plan also must address security threats (although this time it’s a question of cyber threats to the supply chain, vs. physical security threats in CIP-014), it’s very possible that a putative future CIP-013 auditor will also disagree with the entity’s assessment of the relative impact of the threats they face. Finally, it’s possible that the CIP-013 auditor will issue a PNC due to this disagreement.[iii]

So it’s very possible that the same thing will happen to you, if you’re involved with CIP-013 compliance, as happened to my friend who was involved with CIP-014 compliance: an important project (or section of a project) will be cancelled or greatly delayed because there is no way that NERC auditors can provide the sort of pre-implementation assurance of compliance that would allow management to feel completely assured in making their investments.

How could management feel assured? What if, before an entity starts implementing a plan (either a physical security plan in CIP-014 or a SCCSRMP in CIP-013), they had to submit it to their NERC region? The region would review it thoroughly, identify any problems they find with it, then point these out to the entity; the entity would then need to change their plan. The entity would then have the comfort of knowing, when they approve a large investment, that it will improve their chances of being compliant.

As you can guess, this wouldn’t be possible in the current NERC environment. First, as I’ve said before, NERC’s CMEP (and probably the NERC Rules of Procedure) needs to be revised to allow for this. But there is another, even more fundamental, change that would need to happen: The NERC auditors would have to turn into something like cyber security consultants. They would review the entity’s assessment and plan, then come back later to review how the entity implemented the plan. They simply couldn’t be called auditors any more (maybe “assessors”, as in PCI).

This might seem like a radical change, but really it’s not. It would simply be a recognition of reality. Prescriptive standards like the NERC O&P (or “693”) standards require traditional auditors, and they need to maintain their independence by not providing compliance advice before the audit. This is necessary because the O&P standards address threats that are absolutely certain, since they’re based on the laws of physics: If you don’t do X, Y will happen.[iv]

But cyber security is very different. There, everything is based on probabilities. The various cyber threats are always changing and they are only probabilities, not certainties. For that reason, prescriptive requirements don’t work in cyber – or more correctly, they do work but at a big cost. Cyber security standards should be non-prescriptive and objective-based (indeed, almost all other cyber standards in the world are such, although the nuclear power industry’s cyber standards are even more prescriptive than the CIP standards). And non-prescriptive standards require a collaborative approach – not an audit in the traditional sense - in order to avoid exactly the sort of snafu that my friend described to me.

In other words, this snafu is almost inevitable when an organization like NERC tries to enforce non-prescriptive standards using a prescriptive compliance regime. The second post in this series will discuss another example of this problem.

I expect to come back to the second lesson in a post a week or so from now. But before I do that, I need to keep a promise I made when I spoke on CIP-013 as part of a panel at NERC GridSecCon last week; the post I promised should be out later this week.

The views and opinions expressed here are my own, and do not reflect those of any organization I work with. If you would like to comment on what you have read here or ask any questions, I would love to hear from you. Please email me at tom@tomalrich.com.

---

[i] Except in the case of the six items listed in R1.2. Those six items must be incorporated in the entity’s supply chain cyber security risk management plan in R1. If those aren’t so incorporated, then the auditor could issue a Potential Non-Compliance finding. Of course, since R2 requires the entity to implement the plan from R1, the entity could receive a PNC if they haven’t made an effort to implement those six items (which all involve the vendor making a commitment of some sort). However, whether this can actually be enforced is questionable.

[ii] The standard actually uses the word “assessment”; it uses “evaluation” to describe the activity of identifying threats and vulnerabilities in step 3. Since I and most readers are used to thinking of the process of identifying vulnerabilities as an assessment, I have reversed the two words here.

[iii] As I did in the post previously mentioned as well as the first paragraph of this post, I want to point out that I don’t think it’s likely there will be anything more than an Area of Concern issued as a result of a disagreement between the entity and the auditor on an issue like the relative impacts of particular threats. However, I’m sure there will be some in management – and especially in the legal department – who will require more certainty on this point than I’m able to provide.

[iv] This might seem like a big exaggeration – after all, if control room operators don’t communicate properly as required by the NERC COM standards, a cascading outage of the BES won’t necessarily be the result.  If the right set of physical conditions is already in place (hot day in summer, large plant already on outage, etc), there would exist some scenario where if operator 1 does X instead of Y, and operator 2 does A instead of B, then there would be a cascading outage. There is no such certainty anywhere in cyber security. 

Lew Folkerth discusses the "Dark Auditor"

Once again, Lew Folkerth of RF has written a very good article about CIP. This time, it's about the meaning of the phrase (which I hadn't heard before) "Defending against the dark auditor". You can find the article by downloading the latest RF Newsletter and finding "The  Lighthouse".

Why do we need Mandatory Cyber Regulations?

I have more than a few times talked with someone from a large utility who assures me “We really believe in cyber security. We don’t need NERC CIP because we would do all of that anyway.” Both of these sentences have always struck me as strange.

To take the first sentence, when I hear it I always think (but never say!) “I’m really glad you believe in cyber security; that’s good news. Let me go out on a limb and guess that you also really believe in motherhood, apple pie and the Fourth of July.” In other words, talk is cheap. I’ve never heard anyone say “We don’t believe in cyber security at all.” Although some organizations have said so with their actions.

Now let’s look at the second sentence. When someone says that, I usually think (and sometimes say) “Oh, really? You mean that, if you didn’t have to comply with NERC CIP, you would still take great pains to document that – as required by CIP-007 R2.2 - every 35 days you have checked with the patch source for every piece of software installed on a component of a Medium or High impact BES Cyber System or a Protected Cyber Asset to see if there is a new security patch available – even if that vendor has never released a security patch and probably never will? Would you really do this if you weren’t obligated to by CIP?”

Of course they wouldn’t. While some documentation is obviously required as a good security practice, this – and a number of other documentation requirements of CIP – does very little to advance security. Indeed it detracts from it, since if you’re spending your time documenting something like this, you’re taking away time you could spend actually improving security in a way that isn’t required by CIP, such as combating phishing or ransomware (although I’m sure all NERC entities are putting resources into these two threats as well. But since these are IMHO the two biggest cyber threats today, it’s not an exaggeration to say that almost any amount spent combating them isn’t enough).

But let’s leave the question of compliance documentation aside; I’ve already discussed it in another post. Is the second sentence really true? Are there electric utilities or IPPs that would spend as much on cyber security in the absence of NERC CIP as they do in its presence? I’m sure there are a few that would, but for the majority of NERC entities I’m also sure the answer to this question is no. Mandatory requirements (especially if accompanied by potentially huge penalties for non-compliance) are always going to be funded first, even if other non-required areas of cyber security might in some cases deserve more funding, ahead of at least some CIP requirements. And strict logic indicates that it is inevitable that the level of funding for such non-required threats has to be less than it would be in the absence of mandatory cyber requirements.

At the September NERC CIPC meeting in Quebec City, a good discussion broke out (sparked by a presentation by Tobias Whitney of NERC) about getting money for cyber security projects. A couple participants said that they have more than once advocated for a request for funding for security projects not strictly required by CIP by saying that the expenditure would “help CIP compliance”. And lo and behold, the doors to the bank vault were flung open. However, if those three magic words hadn’t been uttered (like sprinkling pixie dust), those projects probably wouldn’t have been approved. So this is the main reason why the electric power industry needs mandatory cyber regulations: Without them, there would be a much lower overall level of funding for cyber security.

You may now ask “So why are you always complaining about NERC CIP? If it’s getting the industry much more funding for cyber projects, it must be making it more secure.” I agree that CIP has made the industry much more secure than it would be otherwise (see this post for more on this topic). However, I think the cost of compliance with the current CIP standards regime outweighs the benefits by a good margin – and that cost is increasing as new standards like CIP-012 and CIP-013 come online. So the question is how to write a standard that a) is mandatory, but b) doesn’t force the entity to invest a lot of time and effort in activities that don’t benefit cyber security very much (and indeed, what is needed is a standard that “forces” the entity to do what they would otherwise do with adequate funding for cyber security, but no mandatory standards).

I believe that these two criteria could be met by a new set of CIP standards - and a compliance regime to go with them - that would 1) be objectives-based, 2) be threat-based, 3) be risk-based and 4) provide a list of threats to address that would be subject to frequent updates by some central group of representatives of NERC entities. CIP-013 actually comes close to this (actually just the first three, but 3 of 4 ain’t bad!), although since I don’t think it can be audited under the current NERC CMEP and RoP, it can’t really be called a “mandatory” standard – except for the six items listed in R1.2. But even not being mandatory, I think there will be lots of pressure on NERC entities to do the right thing and do their best to comply with CIP-013.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

“Associated With”

In early 2014, soon after FERC had approved CIP version 5, a lot of people in the industry (including me) started taking a serious look at the v5 standards, since they were now on their way to becoming the law of the land. One thing we noticed was this important discrepancy in CIP-002 Attachment 1: In Section 1, which discusses classification of High impact BES Cyber Systems, the criterion for deciding whether a system is High impact is if it is “used by and located at” one of the four types of control centers listed in that section. This makes it clear that no system that isn’t physically located at one of those four types of control centers can be High impact. So the BES Cyber Systems located at Medium or Low impact substations and generating stations that are controlled by the High Control Center will be Medium or Low impact respectively, not High impact.

However, in Section 2, which discusses classification of Medium impact BCS, the criterion for deciding whether a system is Medium impact is whether it is “associated with” one of the 13 criteria for assets listed in that section; two of these criteria (2.11 and 2.12) are for Control Centers. This means that, in the case of Medium Control Centers, a system doesn’t have to be physically located at the Control Center in order to be designated Medium impact.

Why is this a problem? Because Control Centers control lots of Cyber Assets (e.g. relays) “in the field” – i.e. located at transmission substations and generating stations. It would be very hard to argue that these field assets aren’t “associated with” the Control Center that controls them, meaning that any system that is located at one of these assets, that meets the definition of BES Cyber System[1], will be Medium impact. And as anyone who has had to comply with CIP v5/v6 for High or Medium assets knows, even if there is only one Medium Cyber Asset located at the asset, there are a number of CIP requirements that automatically become applicable to the asset itself, or to other Cyber Assets located there.

Of course, if the auditors actually interpreted the wording this way (which clearly seems to me to be the right interpretation), there would have been a huge hue and cry from NERC entities with Low impact assets and a Medium impact Control Center, since  a large number (perhaps all) of those Low assets would now be effectively Medium assets. However, they haven’t been interpreting it that way, and the main reason they haven’t has probably been the fact that in the Guidance and Technical Basis for CIP-002-5.1 (page 16), there is the following wording: “Criterion 2.12 categorizes as medium impact those BES Cyber Systems used by and at Control Centers and associated data centers performing the functional obligations of a Transmission Operator and that have not already been categorized as high impact. (my emphasis)”

Of course, this sentence seems to indicate that remote BCS controlled by a Control Center that meets Criterion 2.12 (these will most often be relays in substations) will not take the Medium impact rating of the Control Center. This certainly seems to contradict the language of Attachment 1, but I haven’t heard of any entity being dinged for not declaring those BCS to be Mediums (see this post from early 2014, which provides a different set of reasoning for why Criterion 2.12 should be interpreted to mean that the remote BCS aren’t Mediums. It is a pretty subtle argument, and I haven’t heard it put forth anywhere else).

However, things are changing. As you may know, NERC has decided that the Guidance and Technical Basis in each of the CIP v5 and v6 standards goes beyond what is permitted for NERC guidance. Some of it becomes an “interpretation” of the CIP requirements (and the passage I just quoted seems to be a good example of that. It not only interprets the wording of Attachment 1, it seems to contradict it). Therefore, NERC will remove the G&TB from the standards.

In theory, this shouldn’t make a difference. It has always been said that the G&TB’s aren’t auditable. However, in practice the auditors have paid a lot of attention to them. What will happen when the G&TB for CIP-002 is removed?

My guess is nothing will happen, although I know some people in NERC-land are very fearful of this. Whatever the strict wording of the standard is, this is a case where the consequences of following that wording would be too harmful. I can’t imagine any of the regions would want to do this, especially since they’ve so far all allowed the entities to follow the “interpretation” in the Guidance and Technical Basis; there would be an uprising if they tried to take that away.

So why am I bringing this up? Because this is just one example of the fact that there is a lot of “interpretation” of the CIP standards going on, both by the entities who have to comply and the auditors who have to audit. It’s the grease that allows the wheels to turn, in the creaky engine of NERC CIP.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

---

[1] Of course, a BCS is just a set of BCAs that are grouped together, so BES Cyber Asset is really the operational definition.

When Should we Start Working on CIP-013 Compliance?

The CIP-013 compliance date – which I’m currently estimating to be the second half of 2019 - may seem like it’s still pretty far off, but it’s really not. There are two components of the delay.

First, FERC has to approve CIP-013, and – as we all know – FERC hasn’t had a quorum for around six months. This recently changed, but they still have a lot of things on their plate to deal with that came in ahead of CIP-013 (such as CIP-003-7, approved by NERC and submitted in early February). I think it’s almost certain FERC won’t approve CIP-013[i] for at least six months and maybe a year; let’s say one year to be safe.

And once FERC has approved CIP-013, the Implementation Plan kicks in. That calls for an implementation date 18 months after FERC approval (of course, this is different for Canada, where each province will have its own implementation date, and some may not implement CIP-013 at all). So it’s very possible that the CIP-013 implementation date will be in late 2019 or early 2020, adding the 18 months to the 12-month estimate for FERC approval.  

Of course, this all assumes FERC will approve CIP-013. They have three options:

• Approve the standard unchanged, in which case it will come into effect as just discussed.
• Approve the standard, but at the same time direct NERC to make some further changes. CIP-013-1 would still come into effect according to the Implementation Plan, but at the same time NERC would constitute a new Standards Drafting Team to develop a new version, that will address whatever FERC directed.
• Remand the standard, which kills it.

Obviously, in both of the first two options, CIP-013 is likely to become effective by late 2919. But were FERC to remand the standard, it would never come into effect.  How likely is FERC to remand it? Normally, I would say the answer is a no-brainer: FERC ordered the standard to be developed last year. Since CIP-013, in my opinion, is very close to what they ordered, why would they not approve it?

The reason why remand isn’t a zero possibility is the turnover at FERC. Only one Commissioner from last July, Cheryl LaFleur, remains on the board – and she dissented from the Order. But her dissent was over the fact that a) the Commission hadn’t gone through their usual Notice of Proposed Rulemaking procedure in this case; and b) the amount of time they were giving NERC to develop the standard was way too short (an opinion I agreed with, in the post linked above). Both of these objections are now water under the bridge, so I believe she is likely to approve CIP-013.

As for the new Commissioners (there are now three in total including LaFleur. There will most likely be five when CIP-013 is approved), all of them come from fairly conventional backgrounds. I – as well as everyone I’ve talked to in the industry about this question – believe they are likely to approve CIP-013.

This has been a long way of saying I expect the implementation date for CIP-013 will be in late 2019. I’ll admit that’s a long time away, although it’s almost exactly the same amount of time as NERC entities had to comply with CIP v5, once FERC had approved it. And as you remember, when the compliance date was extended by three months, the industry was quite glad to have the extra time!

However, the main reason why you shouldn’t put CIP-013 planning off any longer – especially larger NERC entities – is dictated by the following logic:

1. An important CIP-013 compliance tool is contract language. You will need to try to get vendors to commit – in one way or other – to doing a number of things, including (but not limited to) the six items listed in R1.2. The best way to do this is through language in their contracts.
2. You aren’t required to renegotiate existing contracts, but contracts come up for renewal all the time. This means you should start trying to include the appropriate language for CIP-013 in every contract that applies to BES Cyber Systems or the software that runs on them. If you don’t, when CIP-013 is finally implemented, you will need to scramble to try to get some other assurance from each vendor that they will in effect follow the contract language you want; if the language is already in the contract, you won’t have to do this.
3. But how do you find the appropriate contract language? You definitely don’t want to require the same language of every vendor, regardless of risk.  Ideally, you want to tailor each vendor’s language to the vendor’s risk level, as well as to the risk level of the systems or software they sell to you. This means you have to make decisions regarding: a) What will be your criteria for classifying vendors by risk level? b) What will be your criteria for classifying BES Cyber Systems by risk level? For classifying software by risk level? c) Since in principle you have to address all supply chain threats (risks) in your plan, which are the threats that you will try to address through contract language (of course, you do have to include the six threats that form the rationale for R1.2. But what other threats should you also try to address through contract language, as opposed to getting another type of commitment from the vendor)? d) For each risk level, what will be the language you first try to get into the contract, and what will be the language you will finally accept if necessary? Since you aren’t required to address every threat through contract language, at what point will you walk away from the negotiations if the vendor hasn’t met the minimum level of language that you want? In what cases will you simply shut up and sign the contract, even though it doesn’t include any of the language you want? e) etc, etc.

The decisions listed above – and more – are the ones you need to make at the beginning of your CIP-013 planning process. If you don’t make them now, you won’t know what contract language to ask for and contracts will be renewed now that aren’t “CIP-013-ready”. By making these decisions now and asking for appropriate contract language in renewal negotiations, you will be making life much easier for yourself two years from now. This is why you need to start working on CIP-013 compliance very soon.

One more point: I hope you can see that it would be dangerous to wait to start your CIP-013 program until FERC approves the standard, even though there is still some small uncertainty about whether they will. Whenever FERC does approve CIP-013, there will then be only 18 months before you have to have the entire apparatus of CIP-013 up and running. For any NERC entity of a certain size, waiting this long would be very dangerous. And as the man said recently, the nice thing about CIP-013 is that it doesn’t require you to do anything beyond what you should be doing anyway: assessing and classifying risks to the security of your supply chain, then taking risk-prioritized steps to mitigate those risks. So even if FERC remands CIP-013, your money and time are still well spent!

I am currently providing on-site workshops to help your organization prepare for CIP-013 compliance. If you would like to discuss this, please email me at talrich@deloitte.com.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

---

[i] Throughout this post, when I refer to CIP-013 I really mean “CIP-013-1, CIP-005-6 and CIP-010-3”, since the latter two contain new requirement parts that were approved along with CIP-013 itself.

How will CIP-013 be audited?

I may have been a little premature. Recently, I wrote a post gushing over how great CIP-013 was because at heart it simply requires the entity to develop and implement a supply chain cyber security risk management plan, rather than being just a set of prescriptive requirements. True, CIP-013 does require that the plan include six particular items that are listed in R1.2, but at heart it just requires a plan. Since this is what a NERC entity – with sufficient funding – should do on its own, I think this is a much more efficient way of addressing supply chain security (or any security, for that matter). This is because a much higher percentage of the entity’s total spending on CIP-013 compliance will go to cyber security vs. pure compliance paperwork, than is true for CIP-002 through -011.

However, an auditor emailed me the next day to say that he thinks entities are just going to address the six things required by R1.2 and that’s it. He said he has seen it too often in the past: A CIP requirement will specify what is required “at a minimum”. Immediately, the lawyers will prevent the compliance people from doing anything beyond that minimum.

I was quite surprised to hear the auditor say this. After all, CIP-013 R1.1 states that the entity must develop and implement a supply chain cyber security risk management plan that addresses “cyber security risk(s) to the Bulk Electric System from vendor products or services resulting from: (i) procuring and installing vendor equipment and software; and (ii) transitions from one vendor(s) to another vendor(s).” It doesn’t just require that the plan include the six items in R1.2. So my question became (I’m paraphrasing here) “Yes, I understand why you expect that lawyers will prevent CIP compliance people from doing more than the minimum required. But clearly, R1.1 requires the plan to cover a lot more than just the six items. As an auditor, why would you let them get away with just addressing the six items in their plan?”

The auditor’s reply was “You are presuming that the auditor has greater knowledge of the risks and threats applicable to an entity than the entity does.  You are also assuming that the entity will not argue that the strict language of the Standard does not require anything more.  After all, the language of Part 1.2 says ‘One or more process(es) used in procuring BES Cyber Systems that address the following, as applicable...’  I can see entities arguing that the six procurement elements are the risk areas that they also have to plan for, although I have some (considerable) trouble mapping the six procurement elements into the two planning elements.”

The most important sentence in his response was the first one. In that sentence, I believe he’s saying “A NERC auditor is required to determine whether the entity fulfilled the strict language of the requirement, period. In order for me to determine whether the entity properly developed their plan, I would need to know all the risks and threats applicable to the entity. Moreover, I would have to know them better than the entity itself does. Otherwise, how could I say the plan was a good or bad one? Yet an outside auditor will clearly never know these things better than the entity itself does.”

  

I certainly couldn’t argue with this. And I’ll further paraphrase my paraphrase: As long as the CIP standards must be audited under the existing NERC Compliance Monitoring and Enforcement Plan (CMEP), there is simply no way an auditor will be able to determine whether a risk management plan, such as the one required by CIP-013, is valid or not. In other words, the only part of CIP-013 that is actually auditable under CMEP is R1.2.

In the week following this email exchange, I attended RF’s Fall CIP Compliance Workshop outside of Cleveland. There was a lot of good discussion about CIP-013 in the regular sessions, but the highlight for me was a long conversation with Lew Folkerth of RF after the workshop ended. I brought up my email exchange to him, and he completely agreed with the auditor: There is no way that any part of CIP-013 other than R1.2 can be strictly audited under the current CMEP.

But this time I followed up and asked Lew “Well, if you were an auditor, would you simply ignore all of the rest of CIP-013?” His answer was “No”. The audit team still can review the plan thoroughly, and if the team finds deficiencies it can issue an Area of Concern (as opposed to a Technical Non-Compliance finding, which can ultimately lead to a violation being assessed). The entity will be in no danger of ultimately being found to have violated CIP-013 R1.1 in this case, but they are well advised not to simply ignore the AoC. Instead, they should work diligently to rectify the deficiencies that the auditor found in the plan.

My takeaway – for NERC entities subject to CIP-013 - from these two conversations is “Don’t just focus your plan on the six items required by R1.2. Instead, make sure your plan addresses the three risk areas included in R1.1:

1. Risks from procuring vendor equipment and software;
2. Risks from installing vendor equipment and software; and
3. Transitions from one vendor(s) to another vendor(s).

You may not actually be liable for a violation if you don’t do this, but the auditors aren’t going to be very happy with you, to be sure. Besides, it’s the right thing to do.”

And you know what? I sincerely doubt there’s any NERC entity with Medium or High impact BES Cyber Systems (i.e. an entity that is subject to CIP-013 compliance) that won’t do their best to develop a good supply chain cyber security risk management plan (which I hereby christen a SCCSRMP[i] – remember, you heard it here first!).

At this point, you may wonder why I’m writing this post at all. After all, it seems there really is no problem, right? It doesn’t matter that most of CIP-013 isn’t strictly enforceable, as long as NERC entities act as though it is.

Well, there’s more to it than that. You may have figured out that NERC’s CMEP is the villain in this post. This document sets out a compliance regime based on prescriptive requirements – and that regime is quite adequate for all of the NERC standards except the CIP family. The non-CIP standards (referred to as the 693 or O&P standards) are ultimately based on the laws of physics; for example, if two control centers don’t communicate properly and one misunderstands what the other ordered and does the wrong thing, there can easily be a serious outage. I have no issue with requirements being prescriptive in the case of the 693 standards.

However, cyber security isn’t physics. In cyber, it isn’t a matter of doing or not doing particular things; it’s a matter of statistics. If entity A doesn’t patch one server in its Control Center for one month, it’s highly unlikely there will be a major BES outage the next month, as a result of that omission. On the other hand, if all of the major electric utilities in one region (and their RTO, for good measure) don’t patch any of their servers for a year, the likelihood of some sort of major disturbance now becomes much higher.

This means that trying to base cyber security regulation on prescriptive requirements is IMHO a completely losing cause. The entities lose because their compliance expenditures escalate rapidly (and my poster child for this is CIP-007 R2 patch management. One entity told me that half of their compliance costs in their control centers – for all NERC standards – are due to this one requirement). And the public loses because a large portion of those expenditures (which the public pays for, of course!) doesn’t do anything to advance the security of the power grid. Moreover, because developing new NERC standards has become a hugely cumbersome process, major cyber threats like phishing and ransomware are nowhere addressed by CIP (for a discussion on these topics, see these posts: here, here, and here).

I used to think that rewriting CIP in a non-prescriptive (and risk-based and threat-based) format would fix these problems, but for a while now I’ve realized that something more is required. That something is a revision of CMEP, or even better a separate CMEP that applies just to CIP. If CMEP were rewritten as I would like, CIP-013 would suddenly become completely auditable. That is, it would actually be possible for an auditor to find an entity in violation if their SCCSRMP is hugely deficient, and even more importantly if they won’t take the steps needed to bring it up to snuff. Moreover, were the other CIP standards to be rewritten in a manner that emulates CIP-013, they would be completely auditable as well.

Would you like to know how I would rewrite CMEP (at a high level, of course)? I didn’t think so, but I’m going to tell you anyway. Here’s how I think a CIP-only CMEP should be written:

1. First, the CIP standards need to be rewritten so they are all in the general form of CIP-013: develop a risk management plan, implement it, and review it annually.
2. After the entity develops the plan, they need to submit it to their NERC region for review (this will require a large increase in staff or contractors for each region, I’ll admit. They will have to be true cyber security professionals, and will need to receive a lot of training in the ways of NERC and the electric power industry, assuming they don’t have that experience when they’re hired).
3. The review of the plan isn’t an audit like what happens today. Rather it’s a collaborative effort between the reviewers (who may still have the title Auditor) and the entity, to see if the plan needs improvement - and if so how best to do that. After the review, the reviewers will give the entity a document outlining the steps it needs to take to improve the plan.
4. The entity revises the plan and re-submits it. Assuming they have addressed the problems, the reviewers/auditors give them the green light to implement the plan.
5. The entity implements the plan. Six months (or so) later, the auditors come in to judge how well the plan has been implemented. If they think the entity has done a good job, they issue them a passing grade. The entity then needs to maintain compliance with the plan they implemented.
6. If the auditors think the entity has tried to implement the plan, but has fallen short in one way or another, they will describe what the entity did wrong and schedule another audit for say six months later to see if they’ve corrected the problems.
7. However, if the auditors think the entity hasn’t really taken all of this seriously, and hasn’t really tried to implement the plan in an effective manner, then – and only then – will they consider assessing a violation.

Doing all of this will of course require a big increase in NERC’s budget, as well as a huge drafting effort. Is NERC likely to do this? I really don’t know. However, I do think that in the not-too-distant future NERC will face the question of whether they want to change or whether they want to give up regulation of grid security. I don’t think the ship will wait for them forever. The grid is too important.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

---

[i] The auditor and I have had a good discussion of how to pronounce this acronym, or rather how to write the pronunciation. He says it should be written “Scuzzy-rump”. I harken back to the early days of the PC, when the SCSI interface was ubiquitous, so I prefer to write it “SCSI-rump”. In any case, the pronunciation is the same. And since the auditor has decreed it, he’s likely to issue an Area of Concern if he hears you pronouncing it differently