Monday, October 16, 2017

Lew Folkerth discusses the "Dark Auditor"

Once again, Lew Folkerth of RF has written a very good article about CIP. This time, it's about the meaning of the phrase (which I hadn't heard before) "Defending against the dark auditor". You can find the article by downloading the latest RF Newsletter and finding "The  Lighthouse".

Sunday, October 15, 2017

Why do we need Mandatory Cyber Regulations?

I have more than a few times talked with someone from a large utility who assures me “We really believe in cyber security. We don’t need NERC CIP because we would do all of that anyway.” Both of these sentences have always struck me as strange.

To take the first sentence, when I hear it I always think (but never say!) “I’m really glad you believe in cyber security; that’s good news. Let me go out on a limb and guess that you also really believe in motherhood, apple pie and the Fourth of July.” In other words, talk is cheap. I’ve never heard anyone say “We don’t believe in cyber security at all.” Although some organizations have said so with their actions.

Now let’s look at the second sentence. When someone says that, I usually think (and sometimes say) “Oh, really? You mean that, if you didn’t have to comply with NERC CIP, you would still take great pains to document that – as required by CIP-007 R2.2 - every 35 days you have checked with the patch source for every piece of software installed on a component of a Medium or High impact BES Cyber System or a Protected Cyber Asset to see if there is a new security patch available – even if that vendor has never released a security patch and probably never will? Would you really do this if you weren’t obligated to by CIP?”

Of course they wouldn’t. While some documentation is obviously required as a good security practice, this – and a number of other documentation requirements of CIP – does very little to advance security. Indeed it detracts from it, since if you’re spending your time documenting something like this, you’re taking away time you could spend actually improving security in a way that isn’t required by CIP, such as combating phishing or ransomware (although I’m sure all NERC entities are putting resources into these two threats as well. But since these are IMHO the two biggest cyber threats today, it’s not an exaggeration to say that almost any amount spent combating them isn’t enough).

But let’s leave the question of compliance documentation aside; I’ve already discussed it in another post. Is the second sentence really true? Are there electric utilities or IPPs that would spend as much on cyber security in the absence of NERC CIP as they do in its presence? I’m sure there are a few that would, but for the majority of NERC entities I’m also sure the answer to this question is no. Mandatory requirements (especially if accompanied by potentially huge penalties for non-compliance) are always going to be funded first, even if other non-required areas of cyber security might in some cases deserve more funding, ahead of at least some CIP requirements. And strict logic indicates that it is inevitable that the level of funding for such non-required threats has to be less than it would be in the absence of mandatory cyber requirements.

At the September NERC CIPC meeting in Quebec City, a good discussion broke out (sparked by a presentation by Tobias Whitney of NERC) about getting money for cyber security projects. A couple participants said that they have more than once advocated for a request for funding for security projects not strictly required by CIP by saying that the expenditure would “help CIP compliance”. And lo and behold, the doors to the bank vault were flung open. However, if those three magic words hadn’t been uttered (like sprinkling pixie dust), those projects probably wouldn’t have been approved. So this is the main reason why the electric power industry needs mandatory cyber regulations: Without them, there would be a much lower overall level of funding for cyber security.

You may now ask “So why are you always complaining about NERC CIP? If it’s getting the industry much more funding for cyber projects, it must be making it more secure.” I agree that CIP has made the industry much more secure than it would be otherwise (see this post for more on this topic). However, I think the cost of compliance with the current CIP standards regime outweighs the benefits by a good margin – and that cost is increasing as new standards like CIP-012 and CIP-013 come online. So the question is how to write a standard that a) is mandatory, but b) doesn’t force the entity to invest a lot of time and effort in activities that don’t benefit cyber security very much (and indeed, what is needed is a standard that “forces” the entity to do what they would otherwise do with adequate funding for cyber security, but no mandatory standards).

I believe that these two criteria could be met by a new set of CIP standards - and a compliance regime to go with them - that would 1) be objectives-based, 2) be threat-based, 3) be risk-based and 4) provide a list of threats to address that would be subject to frequent updates by some central group of representatives of NERC entities. CIP-013 actually comes close to this (actually just the first three, but 3 of 4 ain’t bad!), although since I don’t think it can be audited under the current NERC CMEP and RoP, it can’t really be called a “mandatory” standard – except for the six items listed in R1.2. But even not being mandatory, I think there will be lots of pressure on NERC entities to do the right thing and do their best to comply with CIP-013.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

Friday, October 13, 2017

“Associated With”

In early 2014, soon after FERC had approved CIP version 5, a lot of people in the industry (including me) started taking a serious look at the v5 standards, since they were now on their way to becoming the law of the land. One thing we noticed was this important discrepancy in CIP-002 Attachment 1: In Section 1, which discusses classification of High impact BES Cyber Systems, the criterion for deciding whether a system is High impact is if it is “used by and located at” one of the four types of control centers listed in that section. This makes it clear that no system that isn’t physically located at one of those four types of control centers can be High impact. So the BES Cyber Systems located at Medium or Low impact substations and generating stations that are controlled by the High Control Center will be Medium or Low impact respectively, not High impact.

However, in Section 2, which discusses classification of Medium impact BCS, the criterion for deciding whether a system is Medium impact is whether it is “associated with” one of the 13 criteria for assets listed in that section; two of these criteria (2.11 and 2.12) are for Control Centers. This means that, in the case of Medium Control Centers, a system doesn’t have to be physically located at the Control Center in order to be designated Medium impact.

Why is this a problem? Because Control Centers control lots of Cyber Assets (e.g. relays) “in the field” – i.e. located at transmission substations and generating stations. It would be very hard to argue that these field assets aren’t “associated with” the Control Center that controls them, meaning that any system that is located at one of these assets, that meets the definition of BES Cyber System[1], will be Medium impact. And as anyone who has had to comply with CIP v5/v6 for High or Medium assets knows, even if there is only one Medium Cyber Asset located at the asset, there are a number of CIP requirements that automatically become applicable to the asset itself, or to other Cyber Assets located there.

Of course, if the auditors actually interpreted the wording this way (which clearly seems to me to be the right interpretation), there would have been a huge hue and cry from NERC entities with Low impact assets and a Medium impact Control Center, since  a large number (perhaps all) of those Low assets would now be effectively Medium assets. However, they haven’t been interpreting it that way, and the main reason they haven’t has probably been the fact that in the Guidance and Technical Basis for CIP-002-5.1 (page 16), there is the following wording: “Criterion 2.12 categorizes as medium impact those BES Cyber Systems used by and at Control Centers and associated data centers performing the functional obligations of a Transmission Operator and that have not already been categorized as high impact. (my emphasis)”

Of course, this sentence seems to indicate that remote BCS controlled by a Control Center that meets Criterion 2.12 (these will most often be relays in substations) will not take the Medium impact rating of the Control Center. This certainly seems to contradict the language of Attachment 1, but I haven’t heard of any entity being dinged for not declaring those BCS to be Mediums (see this post from early 2014, which provides a different set of reasoning for why Criterion 2.12 should be interpreted to mean that the remote BCS aren’t Mediums. It is a pretty subtle argument, and I haven’t heard it put forth anywhere else).

However, things are changing. As you may know, NERC has decided that the Guidance and Technical Basis in each of the CIP v5 and v6 standards goes beyond what is permitted for NERC guidance. Some of it becomes an “interpretation” of the CIP requirements (and the passage I just quoted seems to be a good example of that. It not only interprets the wording of Attachment 1, it seems to contradict it). Therefore, NERC will remove the G&TB from the standards.

In theory, this shouldn’t make a difference. It has always been said that the G&TB’s aren’t auditable. However, in practice the auditors have paid a lot of attention to them. What will happen when the G&TB for CIP-002 is removed?

My guess is nothing will happen, although I know some people in NERC-land are very fearful of this. Whatever the strict wording of the standard is, this is a case where the consequences of following that wording would be too harmful. I can’t imagine any of the regions would want to do this, especially since they’ve so far all allowed the entities to follow the “interpretation” in the Guidance and Technical Basis; there would be an uprising if they tried to take that away.

So why am I bringing this up? Because this is just one example of the fact that there is a lot of “interpretation” of the CIP standards going on, both by the entities who have to comply and the auditors who have to audit. It’s the grease that allows the wheels to turn, in the creaky engine of NERC CIP.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

[1] Of course, a BCS is just a set of BCAs that are grouped together, so BES Cyber Asset is really the operational definition.

Monday, October 9, 2017

When Should we Start Working on CIP-013 Compliance?

The CIP-013 compliance date – which I’m currently estimating to be late 2019 or early 2020 - may seem like it’s still pretty far off, but it’s really not. There are two components of the delay.

First, FERC has to approve CIP-013, and – as we all know – FERC hasn’t had a quorum for around six months. This recently changed, but they still have a lot of things on their plate to deal with that came in ahead of CIP-013 (such as CIP-003-7, approved by NERC and submitted in early February). I think it’s almost certain FERC won’t approve CIP-013[i] for at least six months and maybe a year; let’s say one year to be safe.

And once FERC has approved CIP-013, the Implementation Plan kicks in. That calls for an implementation date 18 months after FERC approval (of course, this is different for Canada, where each province will have its own implementation date, and some may not implement CIP-013 at all). So it’s very possible that the CIP-013 implementation date will be in late 2019 or early 2020, adding the 18 months to the 12-month estimate for FERC approval.  

Of course, this all assumes FERC will approve CIP-013. They have three options:

  • Approve the standard unchanged, in which case it will come into effect as just discussed.
  • Approve the standard, but at the same time direct NERC to make some further changes. CIP-013-1 would still come into effect according to the Implementation Plan, but at the same time NERC would constitute a new Standards Drafting Team to develop a new version, that will address whatever FERC directed.
  • Remand the standard, which kills it.

Obviously, in both of the first two options, CIP-013 is likely to become effective by early 2020. But were FERC to remand the standard, it would never come into effect.  How likely is FERC to remand it? Normally, I would say the answer is a no-brainer: FERC ordered the standard to be developed last year. Since CIP-013, in my opinion, is very close to what they ordered, why would they not approve it?

The reason why remand isn’t a zero possibility is the turnover at FERC. Only one Commissioner from last July, Cheryl LaFleur, remains on the board – and she dissented from the Order. But her dissent was over the fact that a) the Commission hadn’t gone through their usual Notice of Proposed Rulemaking procedure in this case; and b) the amount of time they were giving NERC to develop the standard was way too short (an opinion I agreed with, in the post linked above). Both of these objections are now water under the bridge, so I believe she is likely to approve CIP-013.

As for the new Commissioners (there are now three in total including LaFleur. There will most likely be five when CIP-013 is approved), all of them come from fairly conventional backgrounds. I – as well as everyone I’ve talked to in the industry about this question – believe they are likely to approve CIP-013.

This has been a long way of saying I expect the implementation date for CIP-013 will be in late 2019 or early 2020. I’ll admit that’s a long time away, although it’s almost exactly the same amount of time as NERC entities had to comply with CIP v5, once FERC had approved it. And as you remember, when the compliance date was extended by three months, the industry was quite glad to have the extra time!

However, the main reason why you shouldn’t put CIP-013 planning off any longer – especially larger NERC entities – is dictated by the following logic:

  1. An important CIP-013 compliance tool is contract language. You will need to try to get vendors to commit – in one way or other – to doing a number of things, including (but not limited to) the six items listed in R1.2. The best way to do this is through language in their contracts.
  2. You aren’t required to renegotiate existing contracts, but contracts come up for renewal all the time. This means you should start trying to include the appropriate language for CIP-013 in every contract that applies to BES Cyber Systems or the software that runs on them. If you don’t, when CIP-013 is finally implemented, you will need to scramble to try to get some other assurance from each vendor that they will in effect follow the contract language you want; if the language is already in the contract, you won’t have to do this.
  3. But how do you find the appropriate contract language? You definitely don’t want to require the same language of every vendor, regardless of risk.  Ideally, you want to tailor each vendor’s language to the vendor’s risk level, as well as to the risk level of the systems or software they sell to you. This means you have to make decisions regarding: a) What will be your criteria for classifying vendors by risk level? b) What will be your criteria for classifying BES Cyber Systems by risk level? For classifying software by risk level? c) Since in principle you have to address all supply chain threats (risks) in your plan, which are the threats that you will try to address through contract language (of course, you do have to include the six threats that form the rationale for R1.2. But what other threats should you also try to address through contract language, as opposed to getting another type of commitment from the vendor)? d) For each risk level, what will be the language you first try to get into the contract, and what will be the language you will finally accept if necessary? Since you aren’t required to address every threat through contract language, at what point will you walk away from the negotiations if the vendor hasn’t met the minimum level of language that you want? In what cases will you simply shut up and sign the contract, even though it doesn’t include any of the language you want? e) etc, etc.

The decisions listed above – and more – are the ones you need to make at the beginning of your CIP-013 planning process. If you don’t make them now, you won’t know what contract language to ask for and contracts will be renewed now that aren’t “CIP-013-ready”. By making these decisions now and asking for appropriate contract language in renewal negotiations, you will be making life much easier for yourself two years from now. This is why you need to start working on CIP-013 compliance very soon.

One more point: I hope you can see that it would be dangerous to wait to start your CIP-013 program until FERC approves the standard, even though there is still some small uncertainty about whether they will. Whenever FERC does approve CIP-013, there will then be only 18 months before you have to have the entire apparatus of CIP-013 up and running. For any NERC entity of a certain size, waiting this long would be very dangerous. And as the man said recently, the nice thing about CIP-013 is that it doesn’t require you to do anything beyond what you should be doing anyway: assessing and classifying risks to the security of your supply chain, then taking risk-prioritized steps to mitigate those risks. So even if FERC remands CIP-013, your money and time are still well spent!

I am currently providing on-site workshops to help your organization prepare for CIP-013 compliance. If you would like to discuss this, please email me at

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

[i] Throughout this post, when I refer to CIP-013 I really mean “CIP-013-1, CIP-005-6 and CIP-010-3”, since the latter two contain new requirement parts that were approved along with CIP-013 itself.

Thursday, October 5, 2017

How will CIP-013 be audited?

I may have been a little premature. Recently, I wrote a post gushing over how great CIP-013 was because at heart it simply requires the entity to develop and implement a supply chain cyber security risk management plan, rather than being just a set of prescriptive requirements. True, CIP-013 does require that the plan include six particular items that are listed in R1.2, but at heart it just requires a plan. Since this is what a NERC entity – with sufficient funding – should do on its own, I think this is a much more efficient way of addressing supply chain security (or any security, for that matter). This is because a much higher percentage of the entity’s total spending on CIP-013 compliance will go to cyber security vs. pure compliance paperwork, than is true for CIP-002 through -011.

However, an auditor emailed me the next day to say that he thinks entities are just going to address the six things required by R1.2 and that’s it. He said he has seen it too often in the past: A CIP requirement will specify what is required “at a minimum”. Immediately, the lawyers will prevent the compliance people from doing anything beyond that minimum.

I was quite surprised to hear the auditor say this. After all, CIP-013 R1.1 states that the entity must develop and implement a supply chain cyber security risk management plan that addresses “cyber security risk(s) to the Bulk Electric System from vendor products or services resulting from: (i) procuring and installing vendor equipment and software; and (ii) transitions from one vendor(s) to another vendor(s).” It doesn’t just require that the plan include the six items in R1.2. So my question became (I’m paraphrasing here) “Yes, I understand why you expect that lawyers will prevent CIP compliance people from doing more than the minimum required. But clearly, R1.1 requires the plan to cover a lot more than just the six items. As an auditor, why would you let them get away with just addressing the six items in their plan?”

The auditor’s reply was “You are presuming that the auditor has greater knowledge of the risks and threats applicable to an entity than the entity does.  You are also assuming that the entity will not argue that the strict language of the Standard does not require anything more.  After all, the language of Part 1.2 says ‘One or more process(es) used in procuring BES Cyber Systems that address the following, as applicable...’  I can see entities arguing that the six procurement elements are the risk areas that they also have to plan for, although I have some (considerable) trouble mapping the six procurement elements into the two planning elements.”

The most important sentence in his response was the first one. In that sentence, I believe he’s saying “A NERC auditor is required to determine whether the entity fulfilled the strict language of the requirement, period. In order for me to determine whether the entity properly developed their plan, I would need to know all the risks and threats applicable to the entity. Moreover, I would have to know them better than the entity itself does. Otherwise, how could I say the plan was a good or bad one? Yet an outside auditor will clearly never know these things better than the entity itself does.”
I certainly couldn’t argue with this. And I’ll further paraphrase my paraphrase: As long as the CIP standards must be audited under the existing NERC Compliance Monitoring and Enforcement Plan (CMEP), there is simply no way an auditor will be able to determine whether a risk management plan, such as the one required by CIP-013, is valid or not. In other words, the only part of CIP-013 that is actually auditable under CMEP is R1.2.

In the week following this email exchange, I attended RF’s Fall CIP Compliance Workshop outside of Cleveland. There was a lot of good discussion about CIP-013 in the regular sessions, but the highlight for me was a long conversation with Lew Folkerth of RF after the workshop ended. I brought up my email exchange to him, and he completely agreed with the auditor: There is no way that any part of CIP-013 other than R1.2 can be strictly audited under the current CMEP.

But this time I followed up and asked Lew “Well, if you were an auditor, would you simply ignore all of the rest of CIP-013?” His answer was “No”. The audit team still can review the plan thoroughly, and if the team finds deficiencies it can issue an Area of Concern (as opposed to a Technical Non-Compliance finding, which can ultimately lead to a violation being assessed). The entity will be in no danger of ultimately being found to have violated CIP-013 R1.1 in this case, but they are well advised not to simply ignore the AoC. Instead, they should work diligently to rectify the deficiencies that the auditor found in the plan.

My takeaway – for NERC entities subject to CIP-013 - from these two conversations is “Don’t just focus your plan on the six items required by R1.2. Instead, make sure your plan addresses the three risk areas included in R1.1:

  1. Risks from procuring vendor equipment and software;
  2. Risks from installing vendor equipment and software; and
  3. Transitions from one vendor(s) to another vendor(s).
You may not actually be liable for a violation if you don’t do this, but the auditors aren’t going to be very happy with you, to be sure. Besides, it’s the right thing to do.”

And you know what? I sincerely doubt there’s any NERC entity with Medium or High impact BES Cyber Systems (i.e. an entity that is subject to CIP-013 compliance) that won’t do their best to develop a good supply chain cyber security risk management plan (which I hereby christen a SCCSRMP[i] – remember, you heard it here first!).

At this point, you may wonder why I’m writing this post at all. After all, it seems there really is no problem, right? It doesn’t matter that most of CIP-013 isn’t strictly enforceable, as long as NERC entities act as though it is.

Well, there’s more to it than that. You may have figured out that NERC’s CMEP is the villain in this post. This document sets out a compliance regime based on prescriptive requirements – and that regime is quite adequate for all of the NERC standards except the CIP family. The non-CIP standards (referred to as the 693 or O&P standards) are ultimately based on the laws of physics; for example, if two control centers don’t communicate properly and one misunderstands what the other ordered and does the wrong thing, there can easily be a serious outage. I have no issue with requirements being prescriptive in the case of the 693 standards.

However, cyber security isn’t physics. In cyber, it isn’t a matter of doing or not doing particular things; it’s a matter of statistics. If entity A doesn’t patch one server in its Control Center for one month, it’s highly unlikely there will be a major BES outage the next month, as a result of that omission. On the other hand, if all of the major electric utilities in one region (and their RTO, for good measure) don’t patch any of their servers for a year, the likelihood of some sort of major disturbance now becomes much higher.

This means that trying to base cyber security regulation on prescriptive requirements is IMHO a completely losing cause. The entities lose because their compliance expenditures escalate rapidly (and my poster child for this is CIP-007 R2 patch management. One entity told me that half of their compliance costs in their control centers – for all NERC standards – are due to this one requirement). And the public loses because a large portion of those expenditures (which the public pays for, of course!) doesn’t do anything to advance the security of the power grid. Moreover, because developing new NERC standards has become a hugely cumbersome process, major cyber threats like phishing and ransomware are nowhere addressed by CIP (for a discussion on these topics, see these posts: here, here, and here).

I used to think that rewriting CIP in a non-prescriptive (and risk-based and threat-based) format would fix these problems, but for a while now I’ve realized that something more is required. That something is a revision of CMEP, or even better a separate CMEP that applies just to CIP. If CMEP were rewritten as I would like, CIP-013 would suddenly become completely auditable. That is, it would actually be possible for an auditor to find an entity in violation if their SCCSRMP is hugely deficient, and even more importantly if they won’t take the steps needed to bring it up to snuff. Moreover, were the other CIP standards to be rewritten in a manner that emulates CIP-013, they would be completely auditable as well.

Would you like to know how I would rewrite CMEP (at a high level, of course)? I didn’t think so, but I’m going to tell you anyway. Here’s how I think a CIP-only CMEP should be written:

  1. First, the CIP standards need to be rewritten so they are all in the general form of CIP-013: develop a risk management plan, implement it, and review it annually.
  2. After the entity develops the plan, they need to submit it to their NERC region for review (this will require a large increase in staff or contractors for each region, I’ll admit. They will have to be true cyber security professionals, and will need to receive a lot of training in the ways of NERC and the electric power industry, assuming they don’t have that experience when they’re hired).
  3. The review of the plan isn’t an audit like what happens today. Rather it’s a collaborative effort between the reviewers (who may still have the title Auditor) and the entity, to see if the plan needs improvement - and if so how best to do that. After the review, the reviewers will give the entity a document outlining the steps it needs to take to improve the plan.
  4. The entity revises the plan and re-submits it. Assuming they have addressed the problems, the reviewers/auditors give them the green light to implement the plan.
  5. The entity implements the plan. Six months (or so) later, the auditors come in to judge how well the plan has been implemented. If they think the entity has done a good job, they issue them a passing grade. The entity then needs to maintain compliance with the plan they implemented.
  6. If the auditors think the entity has tried to implement the plan, but has fallen short in one way or another, they will describe what the entity did wrong and schedule another audit for say six months later to see if they’ve corrected the problems.
  7. However, if the auditors think the entity hasn’t really taken all of this seriously, and hasn’t really tried to implement the plan in an effective manner, then – and only then – will they consider assessing a violation.
Doing all of this will of course require a big increase in NERC’s budget, as well as a huge drafting effort. Is NERC likely to do this? I really don’t know. However, I do think that in the not-too-distant future NERC will face the question of whether they want to change or whether they want to give up regulation of grid security. I don’t think the ship will wait for them forever. The grid is too important.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.

[i] The auditor and I have had a good discussion of how to pronounce this acronym, or rather how to write the pronunciation. He says it should be written “Scuzzy-rump”. I harken back to the early days of the PC, when the SCSI interface was ubiquitous, so I prefer to write it “SCSI-rump”. In any case, the pronunciation is the same. And since the auditor has decreed it, he’s likely to issue an Area of Concern if he hears you pronouncing it differently

Monday, September 25, 2017

Risk in CIP-013

When FERC ordered NERC to develop a new standard to address supply chain security in Order 829 in July 2016, they specifically required “a new or modified Reliability Standard that addresses supply chain risk management for industrial control system hardware, software, and computing and networking services associated with bulk electric system operations.”

Note that FERC ordered that the new standard address “supply chain risk management”. Contrast this with the other CIP standards. For example, the purpose of standard CIP-007-6 is “To manage system security by specifying select technical, operational, and procedural requirements in support of protecting BES Cyber Systems against compromise that could lead to misoperation or instability in the Bulk Electric System (BES).”

Note that this says nothing about risk; the standard’s purpose is simply to “manage system security”. And how is this done? Through specifying “select technical, operation and procedural requirements”. In complying with and auditing these requirements, risk isn’t considered at all.

FERC could well have taken the same approach in ordering a supply chain standard. They might have simply provided a list of say 20 practices that contribute to supply chain security and told NERC to develop a requirement for each item on the list. In that case, CIP-013 would simply be a set of individual prescriptive requirements, like the CIP version 5 standards. This is why the CIP-013 standard just has three requirements. In essence, these are:

R1. Develop a supply chain cyber security risk management plan.
R2. Implement that plan.
R3. Review the plan annually.

Clearly, the risk management plan forms the heart of CIP-013 compliance. What is a risk management plan? It is a plan that

a)      Identifies all the current threats applicable to a particular subject area (in this case, supply chain cyber security);
b)      Classifies each threat according to the risk it poses; and
c)      Prioritizes remediation activities according to the degree of risk posed by each threat.

Why did FERC choose to take this approach? They state “In making this directive, the Commission does not require NERC to impose any specific controls, nor does the Commission require NERC to propose ‘one-size-fits-all’ requirements.” They also state “In particular, the flexibility inherent in our directive should account for, among other things, differences in the needs and characteristics of responsible entities and the diversity of BES Cyber System environments, technologies and risks. For example, the new or modified Reliability Standard may allow a responsible entity to meet the security objectives discussed below by having a plan to apply different controls based on the criticality of different assets.”

In other words, FERC is saying that, when it comes to supply chain security, all systems and vendors shouldn’t be treated the same in the new standard. Instead, the NERC entity’s supply chain cyber security risk management plan should take risk into account in determining what controls are appropriate for a particular vendor or system. By the same token, auditors shouldn’t require that an entity put in place the same controls for all of its vendors or systems; instead, the auditors should understand that the controls need to be appropriate to the level of risk in each case.

This should be understood as good news by NERC entities who have to comply with CIP-013. One of the common complaints about other CIP standards is that they don’t take risk into account; if a requirement mandates a particular control, it has to be applied in exactly the same way to every system in scope for that requirement, regardless of the risk posed by that system. This often results in a misallocation of resources to less-important activities vs. more-important ones (such as anti-phishing measures) that aren’t addressed in CIP at all.

For example, in CIP-007 R2 Patch Management, the entity is required to ascertain monthly, for every software application installed on a Medium or High impact BES Cyber System or Protected Cyber Asset, whether or not the vendor (or the designated patch source) has released a new security patch. There is no consideration of how important the software, or the system running it, is to the Bulk Electric System. There is also no consideration of whether the software vendor in question regularly releases security patches, or indeed whether they ever have released a security patch. Each vendor must be checked monthly.

Were CIP-007 R2 a risk-based requirement, the entity would be able to take a number of steps that would significantly reduce their compliance burden, without meaningfully impacting their security posture. For example, the entity could tier their software applications by the risk that the misoperation or failure of each application poses to the BES. For the lower-risk applications, the entity might not feel obliged to check for new patches every month. This step alone (and it is just one of a number that might be taken, were CIP-007 R2 a risk-based requirement) would noticeably reduce the compliance burden of this requirement.

Since many NERC entities are now beginning to plan for compliance with CIP-013, what does the fact that the standard is risk-based mean for their compliance plans? CIP-013 compliance will be very different from compliance with previous CIP standards, and a full discussion of how to comply is beyond the scope of this white paper. However, the risk-based nature of CIP-013 dictates the following:

1.      The entity needs to understand that the “risk” applicable to CIP-013 compliance is very different from the “risk” applicable to other cyber security standards, such as NIST 800-53. In the latter standard, systems are tiered by risk to the organization. For example, if the loss of a particular system could have a significant impact on the organization’s finances, it will be considered high risk. In CIP-013 (as well as other CIP standards or requirements that consider risk, including CIP-014 and CIP-007 R3), “risk” means “risk to the Bulk Electric System”. So even though the loss of a system would have a significant financial impact on the organization, that consideration is completely irrelevant in CIP-013. Conversely, the loss of a particular control system could have a large BES impact, but it might not even be noticed by personnel not part of Operations. Yet it would almost surely need to be classified as high risk for CIP-013 compliance purposes.

2.      CIP-013 only applies to Medium and High impact BES Cyber Systems. Should the entity simply assign a medium CIP-013 risk level to all Medium impact BCS, and a high risk level to all High impact BCS? While this is certainly permissible, it would probably be a mistake. This is because the Medium and High impact ratings are based solely on the asset (substation, Control Center, etc) at which the BCS is located, not on any inherent characteristics of the BCS itself. For example, most NERC entities would probably agree that the Energy Management System in a High impact Control Center would be high risk, since its loss would almost inevitably have a huge impact on the BES and on the area it manages. But how about the GPS clock? Maybe it should be classified as medium or even low risk, since its loss would cause much less impact on the BES than the loss of the EMS. Whatever the final classification, the entity now has the flexibility to assign different controls to the GPS clock than to the EMS. Especially for large entities with potentially thousands of BCS and hundreds of vendors in scope for CIP-013, this flexibility can mean a huge savings in compliance costs.

3.      There is nothing magic about having three risk levels. There will always be a tradeoff between the increased flexibility in controls afforded by having more levels and the increased risk of confusion and complication. The entity needs to decide early on how many levels are appropriate in its case; the more systems and vendors in scope at the entity, the more risk levels might be appropriate.

4.      Once “risk” is properly understood in the context of CIP-013 compliance, the entity needs to develop criteria for classifying both vendors and systems in scope by the risk they pose to the BES. This step isn’t explicitly required by CIP-013, but in a future audit the entity will most likely be asked to show their risk criteria to the auditor. If vendors and/or systems aren’t classified by any particular criteria but are simply assigned to their risk category according to somebody’s “gut feel”, the auditor is likely to be concerned; it will lead to the suspicion that the entity might be hiding the fact that they have chosen to give a lower risk rating than warranted to some vendors and/or systems – simply to make their compliance burden easier.

5.      When the risk criteria are identified, the entity should classify its vendors and systems (those in scope for CIP-013) by risk. At that point, the entity needs to determine the types of controls appropriate for each risk level.

Tuesday, September 19, 2017

Why is CIP-013 a Good Standard?

As I've hinted previously, I am now pivoting toward discussing CIP-013 (not that I'll completely ignore the rest of CIP, of course!). I'm doing this for two reasons. One is that CIP-013 compliance is going to require a huge effort, by utilities as well as vendors. The second is that CIP-013 is completely different from previous CIP standards, and compliance is completely different. Not only is it different, it's also very close to how I would like to see all of CIP be rewritten. In this post I discuss why I say that.

In July 2016, FERC issued Order 829, which ordered NERC to develop a standard for “supply chain risk management for industrial control system hardware, software, and computing and networking services associated with bulk electric system operations.” NERC developed and approved the standard and submitted it to FERC for their approval in September 2017. While FERC approval isn’t completely guaranteed, the new standard, CIP-013-1, will most likely come into effect in late 2019 or early 2020.

Experienced NERC CIP practitioners often repeat the mantra “Compliance doesn’t equal security.” This means that, while the current CIP standards CIP-002 through CIP-011 prescribe particular security practices to address particular threats, there are many threats (phishing, ransomware, cloud-based threats) that are simply not addressed at all. A comprehensive cyber security program needs to address all threats, not just those that happen to be included in NERC CIP.

But CIP-013 is different. CIP-013 essentially requires a NERC entity with High or Medium impact BES Cyber Systems to do two things: develop and implement a supply chain cyber security risk management plan (there are six particular items that must be addressed in that plan per CIP-013 R1.2, but the plan itself is in no way limited to those six items). Since a supply chain cyber risk management plan worthy of the name must by definition address all supply chain cyber risks, this in fact means that, as far as CIP-013 goes, there is no difference between compliance and security. Everything the NERC entity needs to do to secure their supply chain, they also need to do for CIP-013 compliance, and vice versa.

At first glance, NERC CIP practitioners may worry that CIP-013 will overwhelm them. It now seems that everything required for good supply chain cyber security is not only recommended but mandated by FERC and NERC – and therefore potentially subject to million-dollar-a-day penalties for non-compliance. How could this possibly be better than the existing CIP standards, which at least omit many important cyber threats like phishing and ransomware? How will NERC entities possibly comply with a standard that requires them to do “everything”?

Before answering this question, let’s consider why there need to be mandatory cyber security standards in the first place. Of course, there are few if any North American power market participants who don’t believe that cyber security is important. But they all face many competing demands for resources, and since security spending doesn’t usually produce immediately visible results, it often gets pushed to the bottom of the priority list, except when there are mandatory requirements to comply with.

Yet it is also true that mandatory standards like the current CIP-002 through CIP-011 distort cyber security spending priorities. This is because these standards (and similar standards in other industries) only address particular threats and security practices – for example, patch management – while completely omitting other threats and practices, like anti-phishing processes and technologies. But since patch management is required by CIP while anti-phishing is not, NERC entities will inevitably invest much more in patch management, even though phishing is one of the most important security threats today.

The ideal cyber regulation would be one that requires NERC entities to do what they would do anyway, given a hefty cyber security budget: They would identify the threats they face, determine the impact of each one, and remediate them based on their relative impact. Moreover, remediation of each particular threat is based on risk (so that, for example, a risky system gets much more attention than one that poses little risk). This ensures the most efficient allocation of security spending – i.e. the greatest “bang for the buck”. Yet this is almost exactly what CIP-013 does. Here’s why:

  • 1.      CIP-013 is objectives-based, meaning it states the objective and requires the entity to attain it. Too many of the other CIP requirements are prescriptive: They tell the entity how to achieve the objective, often in great detail. This creates a lot of compliance risk, and causes a big expenditure of resources on a lot of individual actions that in themselves do very little to enhance security. In CIP-013, the entity is required to develop and implement a good supply chain cyber security risk management plan, period. The content of the plan and the steps to implement it are completely up to the entity.
  • 2     CIP-013 is risk-based. While it is never officially stated in the requirements themselves, the excellent Implementation Guidance makes clear that risk is to be taken into account at every step of remediation. Vendors and systems should be categorized by risk, and remediation steps (including what is required of vendors in contract language and other commitments) should always be based on risk. This is what makes compliance with CIP-013 manageable: If it weren’t risk-based (and FERC specifically required this in Order 829), NERC entities would have to take exactly the same steps for their most strategic vendors as they do for their least strategic. And they would have to treat very important systems the same as fairly inconsequential ones.
  • 3.    The fact that CIP-013 requires a comprehensive plan means that the entity has to consider all supply chain security threats at one time, and prioritize their remediation based on impact. This is not possible under the other CIP standards, which simply require NERC entities to address particular threats without in any way allowing them to rank those threats against each other based on impact, and allocate remediation resources accordingly.

Friday, September 8, 2017

Here’s the Real Problem….

In two recent posts (here and here), I laid out the case for saying there will be no further development of the CIP standards in their present form, except for compliance with future FERC orders. But why is this a problem? Maybe NERC CIP is just fine as it is….No, I’m afraid that’s not the case.

In those two posts, I discussed one serious consequence of this situation: that NERC entities will never have definitive answers to any interpretation questions about CIP v5 and v6. And there are many of these!

This is of course a serious problem for NERC entities subject to the CIP standards, but it isn’t for the general public. The degree of protection against the consequences of a large-scale cyber attack that the CIP standards afford them (and I have said multiple times that CIP compliance has clearly made the industry much more secure than it would be otherwise; the problem is the huge and rising cost that goes along with the current prescriptive CIP compliance framework) is in no way diminished by the fact that there has been so much confusion over what the v5 and v6 standards mean. The lights will stay on, regardless of how many CIP compliance professionals spend long nights trying to figure out how to comply with ambiguously-worded requirements.

But there actually is another very serious problem that ultimately can and will affect the general public, due to the fact that CIP won’t ever be expanded: CIP will never be amended or expanded to address threats that the standards don’t currently address. CIP already doesn’t address some of the most important cyber threats today, and that problem will keep growing, since new threats are always appearing.

I’ll admit this is a problem that’s been around for a long time. Exhibit A is phishing – a threat that was probably recognized about 7 or 8 years ago (I don’t believe I’d ever heard this word until then, perhaps later than that). The current standards do nothing to address phishing (and don’t tell me that quarterly security awareness efforts are in any way adequate to mitigate any of the risk posed by phishing!); this isn’t surprising, given that the framework for CIP v5 and v6 was laid in 2008. That was when FERC issued Order 706, which approved CIP v1 but also ordered NERC to make a number of changes; CIP v5 was the culmination of the effort to address the directives in Order 706 (indeed, the drafting team that was put in place after 706, which drafted CIP versions 2 through 5, was called the CSO 706 drafting team, where CSO stands for “Cyber Security Order”).

A standard practice for NERC, in drafting a SAR for a new drafting team to address a FERC order, is to keep that SAR as narrow as possible – if possible, only including what is in the order itself. And with Order 706 – which weighed in at over 600 pages – NERC had all sorts of motivation not to go beyond the order. Even addressing what was in the order – or as much of it as NERC felt they could address – took five years! So, even though phishing quickly grew in importance as the years went on, NERC never even considered adding it to the threats addressed in CIP, since it wasn’t mentioned in Order 706. This means that now we have the situation in which at least one study found that 91% of cyberattacks start with a user clicking on a link or attachment in a phishing email (and both of the Ukraine attacks started that way!), yet CIP has absolutely nothing to say about phishing.

Of course, I’m not at all saying that utilities aren’t hyper-vigilant about phishing; in fact, the same study found that utilities had the third-lowest phishing success rate of the ten industries studied. I have heard of one large utility that has a “four strikes and you’re out” policy. Pursuant to that policy, regular test emails are sent to all employees. An employee who clicks for the first, second or third times is sent to increasingly serious training and given increasingly explicit warnings. On the fourth click, they’re fired, with no appeal possible (and in theory, the CEO isn’t exempt from this!). This may seem pretty harsh, but given the serious threat that phishing poses to any organization, in my opinion this is appropriate.

And phishing certainly isn’t the only threat not addressed by CIP. How about ransomware? How about purely destructive non-ransomware, exemplified by Not-Petya? How about the many possible threats that come with virtualization? How about the many threats attendant on storing BCSI in the cloud (which, as I discussed in a series of posts this year starting with this one, is allowed by CIP v5)? I’m sure there are a lot more that don’t come to mind at the moment.

None of these are new threats, yet not only are they not addressed by CIP now, none of them except virtualization have even been included in a NERC SAR (and as my recent post on virtualization asserted, the drafting team whose SAR this is included in will almost certainly never even submit the required new requirements and definitions to the NERC ballot body for approval). I think the last new threats that were addressed in CIP are the threats posed by Transient Electronic Devices and Removable Media; and these were only addressed because FERC ordered this (in Order 791 for Medium and High assets, and Order 822 for Lows).

And why aren’t these threats addressed in CIP v5 or v6, or at least set to be addressed by the CIP Modifications drafting team? Because addressing a new threat requires a tremendous multi-year effort: coming up with 3-4 major drafts of the new requirements and definitions, submitting these to the NERC ballot body, responding to the voluminous comments on each draft, submitting the final approved draft to the NERC Board, and finally to FERC for their approval. And should FERC decide they want some changes in the new requirements or definitions, a new drafting team (or perhaps the original one) will have to go through the same process again. It isn’t at all surprising that NERC practically has to have a gun at their head before they’ll propose that CIP be expanded to address a new threat.

Of course, NERC does have other ways of dealing with new threats, most notably the NERC Alerts, which have been used in the cases of the Aurora vulnerability and the Ukraine attacks, as well as in other cases. These aren’t mandatory standards, but they require NERC entities to report on what they are doing to address these threats – which is pretty close to mandatory, in my opinion. And as I said, probably all of the larger NERC entities are very attuned to new threats, and consider any new warning from the E-ISAC or other industry bodies as something that needs to be acted on.

But this just means that, in today’s utility environment, cyber threats are addressed using a hodge-podge of mandatory requirements (the CIP requirements), non-mandatory “requirements” (the NERC Alerts), and all sorts of different best practices. Except for CIP compliance, there is very little uniformity in how NERC entities are dealing with these threats, and even with CIP there is lots of variation among entities in how they comply.

Just as importantly, the fact that there isn’t any uniformity in how NERC entities deal with threats – both old and new ones – means that entities will almost always spend most of their time and resources on the threats included in the CIP standards (malware, remote access, etc), and less time and resources on threats not included in CIP (phishing, ransomware, etc). This is frankly a mis-allocation of resources, since in an ideal world each entity would look at all the threats it faces, rank them based on their potential impact, and prioritize their spending accordingly. Prescriptive CIP requirements inevitably result in too many resources being spent on certain activities, compared to others that might be of equal benefit but aren’t included in CIP at all. My poster children for this are CIP-007 R2 Patch Management and CIP-010 R1 Configuration Management. Both of these are important activities, but both consume disproportionate amounts of resources (especially in large organizations) vs. addressing threats like ransomware and phishing (if you have some time on your hands, you can find a rather voluminous discussion of this problem in this post).

You might now ask how I would fix this misallocation problem, given that there have to be mandatory standards (which I totally agree with). As some of you may know, this spring I started advocating a new critical infrastructure compliance regime that includes six parts:

1.                   The process being protected needs to be clearly defined (the Bulk Electric System, the interstate gas transmission system, a safe water supply for City X, etc).
2.                   The compliance regime must be threat-based, meaning there needs to be a list of threats that each entity should address (or else demonstrate why a particular threat doesn’t apply to it).
3.                   The list of threats needs to be regularly updated, as new threats emerge and perhaps some old ones become less important. Ideally, there will be an industry body that meets regularly to assess new threats, as well as document new mitigations. They will add important new threats to the list, and potentially remove threats that are now of less importance.
4.                   While no particular steps will be prescribed to mitigate any threat, the entity will need to be able to show that what they have done to mitigate each threat has been effective.
5.                   Mitigations need to apply to all assets and cyber assets in the entity’s control, although the degree of mitigation required will depend on the risk that misuse or loss of the particular asset or cyber asset poses to the process being protected.
6.                   It should be up to the entity to determine how it will prioritize its expenditures (expenditures of money and of time) on these threats, although it will need to document how it determined its prioritization.

Let’s focus on parts 2 and 3, since these are the ones that are important for this post. Part 2 essentially means that all of the CIP requirements should be replaced with just one, which reads something like “On a risk-adjusted basis, take steps to mitigate all threats on the current threat list, or document why a particular threat doesn’t apply to your organization.” And part 3 calls for an industry body to make regular updates to the threat list.

In my opinion, this is the right approach because it requires the entity to do what they would normally do in the absence of CIP: identify all current threats, rank them by impact, and allocate resources to mitigate them according to that ranking (with the stipulation that some threats may pose so little risk, or they may not be relevant to the entity for some reason or another, that they may be left out altogether – although the reasons for this will need to be documented!). But this process will be mandatory, not just one of those nice-to-haves that we all never quite get around to.

I hope you see how the subject of this post – the fact that it’s very unlikely any further cyber threats will be incorporated into the current NERC CIP compliance regime – is addressed by my third point. New threats will be fairly easily incorporated into my proposed new CIP compliance regime because there will no longer be any need to develop new requirements, or modify existing ones. The single requirement that I sketched out two paragraphs ago doesn’t ever need to be changed, regardless of the new threats that come along. The industry body I’m suggesting will be able to update the threat list simply through a majority vote (or perhaps a super-majority vote) of its members.

So could all of the problems with CIP be fixed by throwing out all of the current standards and replacing them with my single requirement, as well as the six points? The problem is that what I’m suggesting requires fundamental changes to NERC’s Compliance Monitoring and Enforcement Plan (CMEP) and to its Rules of Procedure (RoP), not just to the CIP standards.

Why do these documents need to be changed? For one example, just consider my point 3. The industry body I’m calling for would essentially have authority to modify what my single CIP requirement covers, by updating the threat list. There is certainly no provision in the two documents that would allow such a body to be constituted, or to have this authority.

Even more fundamentally, the way auditing would occur under my new compliance regime would be radically different from the way it is described in CMEP and the RoP now. These two documents describe auditing of requirements that mandate very specific procedures (e.g. almost all the requirements in the NERC 693 or “O&P” standards, as well as prescriptive CIP requirements like CIP-007 R2). These requirements can be audited very simply: Did the entity do what was required? If so, they are compliant; if not, they aren’t.

Think about how an auditor would have to determine whether the entity had complied with my single CIP requirement. Some of the components of that audit would include:

  1. Has the entity truly taken steps to address all of the threats on the current list, eliminating only those that aren’t applicable to it? And do I agree with the criteria the entity used to determine applicability of the threats?
  2. Since the entity is required to address threats based on the risk that each poses, have they developed a reasonable methodology for determining risk? And is the “risk” addressed in that methodology the risk to the Bulk Electric System, not to something else like the entity’s finances (of course, the whole point of CIP is to protect the BES, nothing more, nothing less)?
  3. Looking at each threat addressed by the entity, have they deployed effective means to address it? In other words, they may believe that a certain chant, repeated daily, will prevent malware infection of their BES Cyber Systems, but this isn’t an effective means of preventing malware.

I’m sure there’s more to auditing my single CIP requirement, but I think you can see that none of the three audit steps I’ve just listed would fit into the current NERC CMEP and RoP, although in fact I contend that CIP auditors are already partly moving in this direction. This is because some of the CIP requirements, like CIP-007 R3 Anti-Malware and CIP-010 R4 Transient Electronic Devices, are already non-prescriptive, objectives-based requirements. The only way to effectively audit these requirements is to take an approach something like this. But this situation – in which a few non-prescriptive CIP requirements are audited using a very prescriptive auditing framework – simply can’t last. And it would definitely not last if the current CIP regime were replaced with my single requirement and my six points.

Of course, we all know that the likelihood of CMEP and the RoP being so radically revised by NERC is pretty small. Does this mean I’m wasting my time by proposing a new compliance regime that will never be enacted? I don’t think so at all, and the reason I don’t is that I think there will be steadily growing pressure – mostly from Congress – on NERC and FERC to either a) make the necessary changes to the NERC compliance regime so that new cyber threats can be easily incorporated into CIP; or b) turn over cyber regulation of the electric power industry to another government body that isn’t constrained by CMEP and RoP. Consider this Congressional hearing sometime in the not-so-distant future.

Senate Committee Chairwoman: Thank you for coming before us, Mr. High NERC Official. We’ve asked you to come here today because we were disturbed to learn recently that the NERC CIP cyber security standards still haven’t been updated to include such important threats as phishing and ransomware. Can you explain to us why this is the case?

High NERC Official: I am pleased to explain that to you, Madame Chairwoman. This isn’t due to our not considering these threats to be of the highest importance. It is due to the fact that NERC’s Rules of Procedure require a deliberate, democratic process for drafting and approving proposed changes to the NERC CIP standards. We do intend to address all of these threats in the future, but we are currently working on some other changes to the CIP standards, such as new standards (ordered by FERC) addressing supply chain security and protecting communications between control centers. Once we have been able to draft and approve these and other new standards and requirements, we will be pleased to turn our attention to other threats like phishing and ransomware.

Senate Committee Chairwoman: Thank you for your honest answer. How long do you think it will be before we have new CIP requirements addressing these two threats, as well as other emerging threats that we don’t even know about today?

High NERC Official: We have already drafted the supply chain security standard and sent it to FERC for approval. We still have a number of other new standards and requirements – including some very important requirements regulating virtualization – that will take maybe three years to draft and approve. But at that point, I think we might very well be able to take up new threats like phishing.

Senate Committee Chairwoman: Excuse me, but I don’t consider phishing a “new” threat. But let’s go on. You say in three years you’ll be able to start addressing phishing. How long will it take, after that point, for a new NERC CIP anti-phishing requirement to be in effect?

High NERC Official: I would think we would have one in place in 2-3 years after we started drafting it.

Senate Committee Chairwoman: So it will be 5-6 years before there will be a requirement addressing phishing, is that correct? (HNO assents). And let’s suppose you could address ransomware on the same timetable, so an anti-ransomware requirement would also be in place 5-6 years from now? (HNO assents) Of course, I also have to assume that a new type of threat that appears say in 2018 will take much longer to address. Is that the case? (HNO assents) Mr. High NERC Official, I have to ask you: Do you agree that the North American Bulk Electric System is a critical national asset, whose substantial impairment would have a huge negative effect on the entire nation? (HNO assents) And do you agree that, while other methods are important, there needs to be a mandatory compliance regime, backed by substantial penalties, that will address all major threats to the security of the BES? And that the BES isn’t well protected until CIP addresses all major threats? (HNO assents, although he starts to look around nervously) Then how can you say that NERC is protecting the BES, when there are such gaping holes in the CIP standards – and there always will be such holes?

High NERC Official: (rising from his seat) I’m sorry, Madame Chairwoman. I need a short break to confer with some of my colleagues. Can we resume in half an hour?

Senate Committee Chairwoman: I’m afraid we don’t have time to do that. I want to thank you for your honest testimony today. I’m sure we will want to have further conversations with you in the future. This session is adjourned.

Senate Committee Chairwoman (aside to aide): Please contact the Secretaries of Homeland Security and Energy and ask them to come before this committee to let us know if they might possibly be able to address phishing and ransomware in something less than six years, if they were given responsibility for cyber regulation of the BES.

The views and opinions expressed here are my own and don’t necessarily represent the views or opinions of Deloitte.