The Implementation Date for CIP-003-7

In October of 2017, FERC issued a NOPR stating that they intend to approve CIP-003-7, which includes the revamped “LERC” requirement and also the requirement for Transient Cyber Assets and Removable Media used at Low impact assets. In November, I put out a post on the NOPR, which turned out to be the first of five parts.

In that post, I made two statements about the implementation date for CIP-003-7. First, I said that the implementation plan is for 18 months and the likely date for compliance was July 1, 2019. This was a mistake, because that date was just a little more than 18 months from the date I wrote the post. I was obviously assuming that FERC would approve the standard within a few weeks of when I wrote the post. This would have been a physical impossibility, since they were asking for comments on their proposed changes. The comment period would be a couple months, and wouldn’t start until at least January. Then FERC would have to analyze the comments and make their decisions on the various issues. There was no way FERC could approve CIP-003-7 for at least five or six months; I have no idea what I was smoking when I said otherwise. This means it is likely – and was always likely – that FERC won’t approve CIP-003-7 until the second quarter of this year. So the effective date for the standard will probably be January 1, 2020.

The second statement I made in the post was that NERC entities wouldn’t have to comply with the physical and electronic access control parts of CIP-003-6 R2 (specifically, sections 2 and 3 of Attachment 1) until CIP-003-7 comes into effect. In other words, implementation of these two sections of CIP-003-6 will be superseded by version 7.

I have continued to believe this was correct until today at the WECC CIP workshop in Boise, Idaho[i], when WECC auditors stated a couple times that compliance with both sections 2 and 3 of version 6 is still due on September 1 of this year – although they will allow the entity to describe what they have done using the language of v7[ii]. This will save entities from having to rewrite their documentation when v7 comes into effect.

I was of course surprised by this, but I wondered if I’d been wrong. So I went back to NERC’s Implementation Plan, where I found these words: “..this Implementation Plan clarifies that under Requirement R2 of CIP-003-7(i), the Responsible Entity shall not be required to include in its cyber security plan(s) any elements related to Sections 2, 3, and 5 of Attachment 1 until the effective date of CIP-003-7(i). Upon the effective date of CIP-003-7(i), the Responsible Entity’s cyber security plan(s) must include the elements required by Sections 2, 3, and 5 of Attachment 1 and the Responsible Entity must implement the controls included in its plan to meet the objectives of Sections 2, 3, and 5.”

This is very clear: NERC was saying that entities wouldn’t need to comply with the physical and electronic access control requirement parts of CIP-003-v6 until the effective date of CIP-003-7. FERC merely paraphrased this language. Specifically, in paragraph 45 of the NOPR, FERC says “NERC explains that the proposed implementation plan does not alter the previously-approved compliance dates for Reliability Standard CIP-003-6 other than the compliance date for Reliability Standard CIP-003-6, Requirement R2, Attachment 1, Sections 2 and 3, which would be replaced with the effective date for proposed Reliability Standard CIP-003-7.  NERC also proposes that the retirement of Reliability Standard CIP-003-6 and the associated definitions become effective on the effective date of proposed Reliability Standard CIP-003-7.” (my emphasis)

Does this mean I’m saying WECC entities should stop whatever they’re doing to implement physical and electronic access controls for Lows - and take a long vacation before they start the push again next year? I think you would be on solid compliance grounds if you did that, but in the unlikely event that FERC doesn't approve CIP-003-7, you'll be in a bad place. Whenever I have asked an entity whether they’re still going full bore to meet the September 1, 2018 date, they have inevitably said yes. 

What this points out is something that NERC entities have known for a long time: the whole business of effective dates of new or revised NERC standards needs to be revisited (in fact, I had a discussion of exactly that issue with one entity during the WECC workshop). This is certainly not the first time I’ve seen lots of confusion about the effective date for a standard.

Note: After posting this last night, I had email conversations this morning with auditors from two other NERC Regions, as well as WECC. The two other regions both said they were telling their entities that they expect the effective date for the physical and electronic access controls to be pushed back to the v7 effective date. However, a WECC auditor - that I'd also emailed this morning - replied to me that they are operating on the prudent assumption that FERC won't approve v7, meaning the 9/1/18 date will still be valid. As I said above, I regard this as very unlikely, but no entity should put aside their implementation of physical and electronic access controls at Lows now. The consequences would be very severe if FERC decides to surprise everybody and not approve CIP-003-7!

Also note that I did amend some of the language from the post last night, to clarify WECC's motivation in saying that the 9/1/18 date remains in effect. They are being very careful, and it is indubitable that, as of today, the effective date for the Low impact electronic access control requirement remains 9/1/18.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC.

If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post. To discuss this, you can email me at the same address or call me at 312-515-8996. 

---

[i] Which is a wonderful town, by the way!

[ii] Since CIP-003-7 does away with the LERC and LEAP terms but still allows the entity to comply in the same ways they could comply with Sections 2 and 3 in CIP-003-6 R2, this means that, if the entity complies with CIP-003-6 and later complies with CIP-003-7, they will need to rewrite their documentation but won’t need to make any actual changes to the physical or electronic access controls themselves – unless they want to, of course.

Are you going to RSA?

If you’re going to the RSA Conference the week of April 16 in San Francisco, I hope you’ll join me. I’ll be participating in two events:

First, on Thursday morning April 19 I’ll be participating in a panel entitled “How can we Regulate Critical Energy Infrastructure Security?” at 9:15 AM. You can reserve a seat here (I recommend signing up. When I attended last year, virtually all of the sessions having to do with security of control systems and critical infrastructure were filled, meaning they’d only let you in if a seat opened up - and then there was a waiting list. Since I hadn’t signed up, I missed almost all of these sessions).

My fellow panelists are Dr. Art Conklin of the University of Houston and Marcus Sachs of Coventry Computer, the former head of the NERC E-ISAC. The moderator of the panel is Mark Weatherford, former CISO of NERC.

Three hours later on Thursday at 12:30 PM, I will facilitate a “Birds of a Feather” conversation. The idea of these sessions is you can grab some lunch (or don’t grab it – your choice) and join me at Table H in the Golden Gate A room at the Marriott for a discussion of “How can we Regulate Cyber Security?” I deliberately chose this topic to be much broader than the topic of the panel discussion in the morning, since I’d like to hear people’s ideas and experiences with cyber regulation in general. I would be very interested in hearing what types of regulations and compliance regimes (i.e. auditing) work well for cyber and which don’t. There’s only room for eight people, so if this sounds interesting to you, you might want to try to get there early (since there’s no pre-signup for these sessions). Note that these sessions are only open to full conference participants.

And if you’d like to arrange a time to get together, I’m planning on being at the show and conference for its entire run, Tuesday through Friday. Please email me at the address below and we can set up a time and place.

Hope to see you then!

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC.

If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post. To discuss this, you can email me at the same address or call me at 312-515-8996.

I have Competition!

I am happy to announce a new blog focusing on developments in NERC CIP, written by my longtime friend Michael Johnson – a longtime SCADA and NERC CIP consultant, who is now an independent, as I am. To choose a name for his blog, Mike paid the same expensive branding consultants that I paid to find the name for mine – his is called Mike Johnson’s Blog. Those branding consultants – they’re worth their weight in gold!

Actually, Mike has been putting out posts for a while, but he’s been doing them on LinkedIn. He addresses various topics, but his favorites of late seem to be what’s going on with the current CIP drafting teams. Mike follows what they’re doing much more closely than I do, so I find it a great benefit to be able to read his summaries of what’s going on (as in his most recent post).

I persuaded Mike recently that there’s fabulous wealth to be had from blogging on Blogspot, so he moved over here (I don’t know whether the wealth came with him, but I’m sure it will). He’s moved a lot of his LinkedIn posts over (athough I know he’ll continue to post in both venues; I did that for a couple years). He’s also done something that I’d like to do if I run across a spare month or two: classify posts by content.

I’m looking forward to reading all of his posts!

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC.

If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post. To discuss this, you can email me at the same address or call me at 312-515-8996.

NERC’s Supply Chain Webinar: Good, but I still have Questions

I eagerly awaited NERC’s Supply Chain webinar yesterday. I found it was a good webinar on supply chain security, although the CIP-013 discussion raised more questions in my mind than it answered.

To start with the good news, I thought two of the presentations, by my friends Corey Sellers of Southern Companies and Joe Doetzl of ABB, were excellent – and I’m looking forward to seeing the slides for their presentations, as well as those of the other speakers, including Tobias Whitney of NERC. Corey, who is head of the team that drafted CIP-013 and guided it – with help from others, of course - through the shoals of four NERC ballots to deliver it to FERC in time to meet their one-year deadline, spoke mainly about the CIP-013 guidance document that the North American Transmission Forum (NATF) is putting together. This sounds like an excellent document, although Corey pointed out that it’s mainly focused on best practices for supply chain security, not CIP-013 compliance per se.

Joe Doetzl, who was with KCPL for many years and was an important part of the CSO706 drafting team as they drafted CIP versions 2, 3, 4 and the beginning of v5, gave a really excellent presentation on what you (i.e. a NERC entity) should expect from your software vendors (his slides had much more information than he could get through in his presentation, so I’m especially looking forward to seeing those).

Now to the not-so-good news (although I’ll say upfront that this isn’t bad news): I still have a few fundamental questions about how entities should comply with CIP-013, and these weren’t answered. Or if they were answered, the answers seem at odds with what is written in the requirements. Here’s what I mean:

1. My biggest question is what should be in the supply chain cyber security risk management plan, which is of course required by CIP-013 R1. In the presentations by Tobias and Corey, they focused exclusively (as far as I could see) on the six items that are required by R1.2. The implication is that these are the only things that must be in the plan. 
2. As I have pointed out in several posts (at least), CIP-013 R1.1 requires the entity to “identify and assess[i] cyber security risks to the Bulk Electric System” resulting from a) procuring vendor equipment and software, b) installing vendor equipment and software, and c) transitions from one vendor to another. The six items in R1.2 all have to do with a), not with b) or c). And even for a), there are clearly many more risks to be addressed than the risks underlying the six items in R1.2. For example, how about the risk that a vendor won’t follow good security practices for their own network, and will experience a breach where information on your software and how it is configured is stolen? This has happened with at least one SCADA vendor in North America in the past four or five years.
3. So is the entity required to identify and assess other risks than the risks behind the items in R1.2? I’m sure Tobias and Corey would answer this question in the affirmative, but then the question becomes “Given that there are a huge number of supply chain security risks – some with extremely small impact or probability – that could be identified, how many does the entity have to identify? Is it a certain number like 10 or 20? Or is it something like “all risks with high impact and probability”? And if that’s the case, who will determine which risks meet that description? The auditor? The entity? Someone else?
4. CIP-013 R1 is one of a category of what I call “plan-based requirements” which seem to have found favor with the CIP drafting teams in the last few years, as the only type of CIP requirement they should consider drafting. I am all for that, since I think – as do a lot of others – that the alternative - prescriptive requirements like CIP-007 R2 - just doesn’t work for cyber security, period. But, as I pointed out in this post (and a few subsequent ones), the only way I can see a plan-based requirement being successfully audited under NERC’s auditing procedures is to have criteria for what should be in the plan in the requirement itself. My best example of this is CIP-010 R4, where Attachment 1 (which is called out in R4, and is thus part of the requirement) provides a very detailed list of items that should be addressed in the plan for Transient Cyber Assets and Removable Media – e.g. software vulnerability mitigation, malicious code mitigation, etc. When reviewing an entity’s CIP-010 R4 plan, the auditor can fairly easily go through this list and determine whether the entity has addressed (and meaningfully addressed) each item on the list. Unfortunately, no such list is to be found in CIP-013-1 R1.
5. My second question is about the meaning of “mitigation” of risks (as I pointed out in the first end note below, this word was left out of R1, but clearly is meant to be in there). As far as I could see, both Tobias and Corey were only considering vendor contract language as a mitigation means for the six risks implied in R1.2. Yet, as I’ve pointed out in two recent posts (this one and this one), contract language is only one means of risk mitigation, and IMHO it isn’t anywhere near the best one.
6. But suppose you follow Tobias and Corey (and I’ll readily admit that both Order 829 and the NERC Implementation Guidance for CIP-013 take the same position as these two gentlemen) and first try to get your vendors to accept contract language for the six items in R1.2. And suppose a few of them absolutely refuse to insert the language you ask them to include. Furthermore, suppose these few vendors are absolutely critical to your organization, and it would be very disruptive and costly to try to replace their product with a competitor’s. What do you do then? Do you not have to do anything more to mitigate[ii] the risk?
7. To answer this question, look at the title of CIP-013: “Cyber Security – Supply Chain Risk Management” (my emphasis). You are supposed to manage each risk, not simply try one particular mitigation strategy and then give up if that fails. In this post, an auditor described an alternative strategy – and of course it’s one of many alternatives – that an entity could use if the vendor refuses to provide prompt notice of employees who either leave the company or no longer need access. In my opinion (and the auditor agrees with me), you need to be prepared to try multiple approaches to mitigating each risk that you identify.

However, I’m not waving a red flag yet. CIP-013 hasn’t even been approved by FERC, so there is still some time to address these questions[iii]. But – again, in my opinion – these two questions need to be addressed as soon as possible, since the entities will need to know the answers to both of them before they start developing their plans.

If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post. To discuss this, you can email me at the same address or call me at 312-515-8996.

---

[i] As I pointed out in this post, it seems an important word was left out here: mitigate. Your plan must describe how you will identify and assess risks, but the plan doesn’t have to say a word about what you will do about those risks! And since R2 just requires implementation of the plan, mitigation doesn’t automatically come in there, either. Of course, all of the other documents related to CIP-013, including FERC’s Order 829 and the NERC Implementation Guidance, make it abundantly clear that mitigation of these risks is the purpose of the standard – so I don’t recommend that you decide to stonewall it and only include identification and assessment of risks in your plan! But this does need to be fixed at some point. Assuming FERC orders some changes to CIP-013-1, as their NOPR strongly hinted they would, this should be included in the SAR for CIP-013-2.

[ii] Tobias seemed to strongly indicate this was the case, when he talked about what to do if a vendor won’t provide a means to authenticate identity and integrity of vendor-supplied software and patches. He didn’t talk about anything else you could do to mitigate the risk of a bad actor inserting malware into a patch, when there are clearly other things you could do (ask to download the patch from a secure FTP site, ask the vendor to burn the patch on a CD and overnight it to you, etc). BTW, I wasn’t able to ask a question about this because it was announced at the beginning of the webinar that there wouldn’t be time for questions from the online webinar audience.

[iii] Although the fact that FERC wants to shorten the implementation time for CIP-013 from 18 to 12 months, and that I believe they’ll approve the standard in Q2, means there isn’t much time remaining.

Complying with CIP-013 Part 6: A (near-fatal) Flaw in the Standard

I have come to realize that CIP-013 suffers from a very serious flaw. It would be a fatal one, were it not that the human spirit is infinitely resourceful, and I think the NERC Regions will rise to the occasion and develop a work-around for this flaw. While I have kinda sorta known of this flaw for a while, it’s only recently that I’ve been able to articulate it, and also realized what the solution is.

I want to emphasize that I still stand by what I said in a post last September: CIP-013 is the closest to my idea of the ideal NERC CIP standard of all the CIP standards, both those currently in effect and those that have “retired”. However, it nevertheless does suffer from the serious flaw, which I will describe in this post.

To understand the flaw, I need to go back to my series of posts late last year that discussed “plan-based” requirements (which includes the requirements in CIP-013, of course). In those posts, I came to the conclusion that a requirement to develop a plan can’t simply tell the NERC entity to develop a plan to mitigate a certain class of threats (in the case of CIP-013, these are supply chain threats), then leave it up to the entity to determine what threats they should address in their plan. The requirement to develop the plan needs to include a list of threats (although I called them “criteria” in one or two of the posts on plan-based requirements) that should be addressed in the plan. This should be a comprehensive list of all the threats that the drafting team felt should be included. Of course, the entity is always welcome to add to it, but the drafting team needs to assume that a threat that isn’t on their list usually won’t be addressed in the plans.

So does CIP-013 R1, which mandates that the entity develop a supply chain cyber security risk management plan, provide a list of threats that need to be addressed in the plan? As I pointed out in this post, R1.2 does list six types of mitigation (ordered by FERC) that need to be included in the plan – and these mitigations correspond to six particular supply chain threats. However, R1.1 says that the entity must “identify and assess” risks resulting from “(i) procuring and installing vendor equipment and software; and (ii) transitions from one vendor(s) to another vendor(s).” And I believe the word “all” needs to be assumed after “risks”, since otherwise it wouldn’t make sense (if you’re not going to address all risks, what are you going to address? Just those that begin with the letter A?)

I rephrase this list as (i) risks from procuring vendor equipment and software; (ii) risks from installing vendor equipment and software; and (iii) risks from transitions between vendors. The six mitigations in R1.2 all fall under the heading of procuring vendor software, and even then they hardly exhaust all the possible risks just in that one category; they don’t do anything to address risks in the other two categories.

So the serious flaw in CIP-013 R1 is that it requires development of a plan to mitigate supply chain threats (the requirement uses the word risks, but I prefer “threat” for several reasons) but doesn’t provide a list – beyond the six items in R1.2 – of threats that should be included in the plan. This means that someone auditing compliance with R1 only has two choices:

a)      Make up their own criteria for what should be in a plan and audit against that; or

b)      Restrict the audit to only the six items in R1.2. If these items are all sufficiently addressed in the plan, the entity doesn’t get a PNC. If they aren’t all sufficiently addressed, the entity is likely to get a PNC.

To be honest, neither of these is an acceptable choice. Clearly, for the auditor to make up their own criteria is completely unacceptable, meaning a) is off the table. But b) is also unacceptable, since R1.1 wants the plans to address a lot more threats than just the six threats that are implied by R1.2. Moreover, FERC said the same thing in Order 829, and NERC said it in the Implementation Guidance for CIP-013.

Option b) might be acceptable if it were likely that NERC entities would bend over backwards to identify supply chain risks that go well beyond the six items (threats) in R1.2. In that case, the auditor still couldn’t give the entity a PNC for not including a particular threat in the list, but they could certainly ding them if they listed a threat in their plan developed for R1 but didn’t take any steps to mitigate[i] that threat as they implemented the plan in R2.

However, I have two pieces of bad news for anyone who thinks this will happen:

1. There is no Easter Bunny; and
2. NERC entities aren’t going to bend over backwards to identify supply chain threats beyond the six threats referenced in R1.2. While they may identify particular threats that their Region told them they should identify (or that might be included in a future “NERC-approved” guidance document, say from the North American Transmission Forum), they simply aren’t going to search high and low for every threat they can think of and include it in their list. Even if they are already addressing a threat outside of the compliance process, just including it in the list will entail new paperwork, as well as compliance risk if their auditor decides their implementation of mitigations of that threat in R2 is inadequate.

So neither of the above options is acceptable. And it is very unlikely that the FERC commissioners will all read this post and immediately order that CIP-013 be rewritten to include a list of supply chain threats that must be addressed in the plan, since this would delay implementation for probably 2-3 years.  What is likely to happen? I (optimistically) think the Regions will develop a CIP-013 process roughly like what I outlined in this post at the end of last year – namely, that an entity will be allowed to have their Region review and suggest changes to their CIP-013 plan prior to starting implementation, and that the entity will also be able to request that their Region review and advise on their implementation of the plan, after they have started implementing it.

As part of the review of the entity’s plan, the Region will be able to suggest that there are threats (aka risks) the entity should address in their plan, which they haven’t addressed. It’s a good bet that, if an entity’s Region suggests they should add threats X, Y and Z to their plan, they will add them![ii] That’s why I think this is an acceptable solution to the problem posed by this serious flaw in CIP-013.

Let’s cut to the chase: Is what I’m suggesting completely “legal” by the NERC Rules of Procedure? I’ll bet it isn’t. But as far as I can see, the only other alternatives are the a) and b) options listed above. So there’s a choice between the somewhat illegal and the completely unacceptable. Which will it be?

If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post. To discuss this, you can email me at the same address or call me at 312-515-8996.

---

[i] There’s a slight problem here, which I only discovered while writing this post: R1.1 requires the entity to develop a plan to “identify and assess” supply chain risks, but it doesn’t require the plan to mitigate them! So in theory, the entity could develop a plan that simply listed a bunch of risks but didn’t propose to do anything about them. Of course, it’s clear that CIP-013 was ordered and developed in order to mitigate supply chain risks, so I can’t see this omission shutting down the implementation of CIP-013. But this omission does need to be fixed by amending the standard, assuming FERC orders other changes (in a second version, of course) when they approve CIP-013-1.

[ii] This arrangement will only work if the Region develops a standard set of threats that it wants included in CIP-013 plans, so that individual auditors can’t develop their own. If I were new to this business, I would also suggest that each region publish a list of threats that need to be included in the plans, but that would go way beyond what NERC would allow them to do. So the advice will need to be provided on a one-on-one verbal basis with individual entities. The compliance advice the Regions currently provide for other CIP standards is delivered in the same way, e.g. in the SGAS.

Complying with CIP-013, Part 5: An Auditor sets me Straight on Contract Language

My most recent post discussed the important question of the role of procurement contract language in CIP-013. My general point in the post was that I think it is a big mistake for a NERC entity to focus their CIP-013 compliance efforts solely, or even primarily, on getting the vendor to agree to incorporate particular language in their contract. After that post came out, an auditor wrote in to suggest some of my statements needed clarification. We exchanged a number of emails, and I’ve identified four important points that he raised.

I.                    Tales from Beyond the Scope

Early in the post, I quoted from a note that appears after CIP-013 R2: “the following issues are beyond the scope of Requirement R2: (1) the actual terms and conditions of a procurement contract; and (2) vendor performance and adherence to a contract.” Regarding (1), I had this to say: “The first part of the second sentence…says that the terms and conditions of a procurement contract are ‘beyond the scope’ of R2. What does this phrase mean? It specifically means that contract language is not the subject of R2 – and if an auditor asks to see an entity’s vendor contract, they can be turned down because it isn’t in scope.”

It turns out the truth is more nuanced. To quote from the auditor’s email: “For compliance finding purposes (i.e., determination of a PNC) the contract is out of scope.  However, if the entity states that the vendor will (or has promised to) do something, such as advise its customers within x days of identification of a vulnerability in its products, I want to see something from the vendor that demonstrates that, not just a verbal statement of assurance from the audited entity.  In many cases, the MSA, SLA, or contract T&C will address that expectation and it is appropriate for the entity to provide the written vendor assurances.”

What does this mean? It means that, if you assert that a vendor has promised to do X in a contract, the auditor can (and most likely will) ask for evidence of this. If you refuse to show him the contact and point to phrase (1) above, the auditor will probably say “OK, Mr. Smart Guy. Show me some other evidence they’ve made this promise.” At this point, you’ll wish you had a letter or email from the vendor making the same promise; presumably, you won’t feel you’re violating some legal principle by not showing him one of these.

But what if your only evidence is the contract – and you’ve been told by your lawyers never to show the contract to the auditors, even if they jump up and down and promise divine retribution for this refusal? In this case, the auditor technically can’t issue a PNC (potential non-compliance) finding, but they can certainly list in their report that you were unable to substantiate your assertion regarding the promise by vendor Y to do X.

So if you are going to be at all constrained in your ability to show a contract to the auditors, why even take the contract route in the first place? Get a letter from your vendor. This constitutes obtaining a vendor promise just as much as contract language does, and it will make your audit go much more smoothly. As the auditor pointed out above, other documents from the procurement process, such as an RFP stating terms the vendor agreed to, a Master Services Agreement or a Service Level Agreement, might also constitute evidence of the vendor’s promise.

II.                  Trying is Everything

What happens if you beg your vendor to promise to do something and they refuse – yet dropping them as a vendor is out of the question because of the cost of making the change, the unavailability of close substitutes, etc? Will you receive a PNC because of this? The auditor further states in the same email: “The key point in the contractual language exclusion is that the auditor cannot hold the entity at fault with a PNC if the entity was not successful in getting the vendor to agree to a CIP-013 requirement expectation.  And, failure of the vendor to comply with its contractual obligations or promises does not constitute a compliance failure on the part of the audited entity.  But the entity needs to demonstrate how it addressed the expectation during its procurement process.  The Plan should address how the entity will go about asking and the pre-contract negotiations and perhaps elements of the contract itself will demonstrate the Plan was followed.”

In other words, your supply chain cyber security risk management plan (which R1 requires you to develop, of course) must describe how you will go about requesting that your vendor promise to do something, as well as the different vehicles available for the vendor to document that promise. For each vendor, you should document how you tried to get them to make the promise. If they agreed to make it, then you need some document to verify that. If they didn’t agree, you should document that fact as well. To quote the auditor again: “The entity does need to keep records of any communication with the vendor; preferably date, time, with whom, and a summary of the discussion or agreement.  I like emails better than phone logs for the obvious reason; emails are better evidence than one-sided hand-written records.”

The auditor also points out that you don’t have to simply throw up your hands if the vendor tells you “no” the first time you ask for something, even though you know that finding another vendor for the same product isn’t an option, for whatever reason. How about negotiating, using tools that you (as a compliance or cyber security person, not a supply chain or legal person) have access to? The auditor also says “…if I cannot get, for example, prompt notifications of vendor staff terminations, then maybe I should reconsider my willingness to allow electronic or unescorted physical access to my BES Cyber Assets.  That is called a negotiation point.” Good idea!

Before I leave this topic, I want to point out something else: Your supply chain cyber security risk management plan must list risks and how you intend to mitigate them. Your first step to mitigate a particular vendor risk may well be getting the vendor to promise to take certain steps that will mitigate the risk. But if the vendor absolutely refuses to do this, are you now off the hook for that particular risk? Will your mitigation be “Well, we tried our best, but that vendor is as stubborn as a senior citizen mule!”?

Unfortunately, this isn’t mitigation – this is making an excuse for non-mitigation. There should usually be some step you can take to mitigate the risk in question, which doesn’t require the vendor to do anything at all. Let’s go back to the example the auditor just used, the risk that a vendor won’t provide prompt notification when an employee leaves who had access to your systems (this is the risk addressed by R1.2.3). The auditor suggested that, if the vendor refused to promise to give you this notification, you might then refuse to give their employees electronic access and/or unescorted physical access to their systems at your facilities. This isn’t just a negotiation tactic. It’s also a legitimate mitigation for this risk, since an employee who had just been fired and wanted to take out his anger on your systems would have a hard time doing so without electronic or unescorted physical access. He/she would have to come onsite and be escorted to the systems!

III.                Just Do It!

In my last post, I made this assertion: “I don’t think the entity needs to show any procurement documents at all, because I don’t think the entity needs to prove that they ‘initiated correspondence’ with the vendor to implement certain controls, if they can show other evidence of compliance. And what other evidence can the entity show? The best evidence will demonstrate that the vendor is actually doing what they were supposed to do!  After all, isn’t the goal of CIP-013 to protect the BES against supply chain risks, not simply document that you’ve asked your vendors to do certain things?”

In this statement, it seems I was flat out wrong. While it is important that you gather some evidence to show the vendor is actually keeping a particular promise[i], that evidence alone isn’t sufficient. In his email, the auditor further stated “I do have a problem with a “handshake” agreement.  I cannot evaluate that – it is essentially an attestation in the absence of supporting evidence.  Yes, the entity can show me that they received a notification of a vendor staff termination.  But, what prompted the notification?  Was there a contractual expectation or a vendor declaration that such (notification) would be sent by the vendor consistently and in a timely manner?  How do I know that the vendor will inform me of every applicable termination or reassignment?  Similarly, how do I know that the vendor will apprise me of every security vulnerability in their products?”

In other words, if a vendor makes a promise, it needs to be made in writing in some way (although contract language is just one of those ways). You also need to document either a) that the vendor is keeping that promise (using emails, etc); b) that you tried to get the vendor to promise something but you were turned down; or c) the vendor promised to do something but didn’t keep the promise. If either b) or c) is the case, you also need to document some other mitigation steps you took to address the risk in question – assuming there is a way you could mitigate the risk without the vendor’s cooperation (obviously, as required in R1.2.4, if the vendor absolutely refuses to tell customers about vulnerabilities it knows about but hasn’t yet made public, you will not be able to take steps to address those vulnerabilities - except perhaps by simply tightening up the normal security measures that would be applied to that system).

IV.                The Big Guys

Finally, the auditor also made a good point about how you mitigate vendor risks when it comes to big vendors like Microsoft, who obviously aren’t going to change their contract language for a single electric utility, no matter how large it might be. He says “…the plan can certainly address how procurement from the huge international players like Microsoft, Red Hat, Cisco, and Oracle will be handled.  I would prefer to see you get your networking gear from a Cisco channel partner, not eBay or some unknown used equipment reseller.  I would expect you to download software from a known, reputable source, not Sammy’s Software Shack.  Hopefully the vendor provides a digital signature (MD5 or SHA(2)-256 hash) of its software for verification purposes.  It is all about risk management.  You cannot eliminate all risk, but you can certainly reasonably mitigate it.”

If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com. Please keep in mind that Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post. To discuss this, you can email me at the same address or call me at 312-515-8996.

---

[i] Although you definitely don’t have to document every instance where the vendor kept the particular promise in question, as would be the case if we were talking about a prescriptive requirement like CIP-007 R2.