Regular readers of this blog (all three of you!) may have figured out that I’m writing two series of posts on CIP-013: one for NERC entities and one for vendors. The important thing to remember is that both series should be of interest to everybody. The two sets of concerns are almost completely the same, because a) vendors have all sorts of incentive to make sure their customers don’t get into trouble due to something they did or didn’t do; and b) most vendors have their own supply chain, which has to be secured in almost the same way as NERC entities will be securing theirs.
If you ask any NERC CIP compliance professional (or say a supply chain or legal professional at a NERC entity subject to CIP-013) what is the main focus of compliance with CIP-013, the upcoming supply chain security risk management standard, they will most likely tell you it’s contract language. This isn’t terribly surprising, since there was endless debate during the drafting/balloting periods on the role contract language would play in compliance. And I’ve heard that some utilities are already requesting security contract language from their vendors.
I must admit that I was very focused on contract language in the first few posts I wrote about CIP-013 last fall. But as I started my current posts on the standard this year, and focused more on the standard and the Implementation Guidance, I realized that it’s a mistake to think that contract language is in some way the ultimate goal of CIP-013. Far from it.
First, here’s a little quiz: Why was CIP-013 written? The possible answers are: a) to secure NERC entities’ supply chains; or b) to beef up language in vendor contracts. You’re right! The answer is a). Security is the object of CIP-013, not contract language. To see this, you first need to look at the requirements of CIP-013. Do the actual requirements ever refer to contract language? No, they don’t. The only time contract language is mentioned is in this note found after R2:
Implementation of the plan does not require the Responsible Entity to renegotiate or abrogate existing contracts (including amendments to master agreements and purchase orders). Additionally, the following issues are beyond the scope of Requirement R2: (1) the actual terms and conditions of a procurement contract; and (2) vendor performance and adherence to a contract.
Does either of these sentences say anything about contract language being an objective of CIP-013? The first sentence says that renegotiation of existing contracts isn’t required. I’ll admit this does somewhat imply that, when a new or existing contract is negotiated, there needs to be a discussion of contract language. But that isn’t the same thing as requiring that contract language be the primary means for compliance with the standard, even though so many people apparently believe that is the case.
The first part of the second sentence is even more interesting, since it says that the terms and conditions of a procurement contract are “beyond the scope” of R2. What does this phrase mean? It specifically means that contract language is not the subject of R2 – and if an auditor asks to see an entity’s vendor contract, they can be turned down because it isn’t in scope.
I’ve covered this ground before, but in a different context. In a series of three posts (starting with this one, following up with this one and concluding with this one), I addressed the idea that CIP-013 was unenforceable because of the above provision about contract language being out of scope. A staff member (and former auditor) from one region had made this assertion to me. He argued that NERC entities are required[i] to get their more-risky (i.e. important) vendors to agree to contract language for at least the six items listed in R1.2. However, since the NERC entity isn’t required to show the actual contract to the auditors, this means the auditors don’t have any way of verifying the entity’s assertion that they obtained the contract language.
An auditor (from another region) wrote in after that post to say that NERC entities aren’t required to get the vendor to agree to particular contract language in order to obtain the six items in R1.2 (I wrote about this in the second of the above posts). To quote him, “The requirement is essentially that the Registered Entity ask for the elements of the required process(es), not that the vendor agree to them. Therefore, the Standard is correct in that the language of the resulting contract is not in scope. (The procurement steps include) the Request for Quotation or Bid, Request for Contract Amendment, and/or any other Registered Entity-initiated correspondence that sets forth the expectations to be placed upon the vendor. As long as I see the expectations in the procurement documents issued by the Registered Entity, they have satisfied the requirement.”
I agree with the auditor that succeeding in getting language placed in a contract isn’t required, and that it should be sufficient that the Registered Entity “initiated correspondence that sets forth the expectations to be placed upon the vendor”. But I would go even further. I don’t think the entity needs to show any procurement documents at all, because I don’t think the entity needs to prove that they “initiated correspondence” with the vendor to implement certain controls, if they can show other evidence of compliance. And what other evidence can the entity show? The best evidence will demonstrate that the vendor is actually doing what they were supposed to do![ii] After all, isn’t the goal of CIP-013 to protect the BES against supply chain risks, not simply document that you’ve asked your vendors to do certain things?
Suppose you want to show your auditor that your vendor is providing “Notification…when remote or onsite access should no longer be granted to vendor representatives..” as required by R1.2.3. Let’s say the vendor has had several people leave their company in the last year who had electronic or physical access to the vendor’s systems at say one of your substations, and each time this has happened they’ve sent you an email asking you to remove the person’s access. All you should have to do is show those emails to the auditor. What possible further benefit would there be from also showing them procurement documents, let alone a contract? Of course, if you also have these, that’s fine. But if the vendor verbally agreed to do something, or just sent you a letter outside of the procurement process, but you can show they’re actually doing what they said they’d do, that should be sufficient evidence.
Let’s turn from the requirements of CIP-013 to the Implementation Guidance. This document has language that implies that contract language is a goal of the standard (although certainly not the only one), while at the same time it also says just the opposite. However, I can see how people who have read the document have come to believe that contract language is the primary control they should be focusing on.
First off, let me point out that there is nothing mandatory about the Implementation Guidance. The auditors aren’t bound to follow it in their audits, and the entities aren’t bound to follow it in their implementation of CIP-013. That being said, you can be sure most entities and all auditors will have read the document carefully and will usually try to follow it as often as they can. So it is important to at least understand what it says. On page 1, you’ll find this paragraph:
First, in developing their supply chain cyber security risk management plan(s), Responsible entities should consider how to leverage the various components and phases of their processes (e.g. defined requirements, request for proposal, bid evaluation, external vendor assessment tools and data, third party certifications and audit reports, etc.) to help them meet the objective of Requirement R1 and give them flexibility to negotiate contracts with vendors to efficiently mitigate risks. Focusing solely on the negotiation of specific contract terms could have unintended consequences, including significant and unexpected cost increases for the product or service or vendors refusing to enter into contracts.
Here, the Implementation Guidance is specifically warning the entity against relying solely on contract language. However, the message is very different when you get to pages 5-8, where the six specific items required under R1.2 are addressed. For each of these items, there are one or more processes listed. These are described as “Examples of ways that a Responsible Entity could, through process(es) for procuring BES Cyber Systems required by Part 1.2, comply with Parts 1.2.1 through 1.2.6.” (my emphasis) It seems that, at least for the six risks addressed in R1.2, the members of the Standards Drafting Team that wrote the Implementation Guidance were primarily considering controls that occur in the procurement phase of the vendor relationship – including contract language.[iii]
When you look at the 20 actual processes suggested under the parts of R1.2, you will notice that all but three of these begin with one of two phrases: “In an RFP or during contract negotiations, request that the vendor include in the contract provisions…” or “During procurement”. Of course, both of these phrases simply confirm the fact that whoever wrote this section of the Implementation Guidance was primarily looking at the process of obtaining contract language – or other processes focused entirely on the procurement phase of the vendor relationship – to achieve the mitigation of risk required in each of these six cases.
But in my opinion, the big emphasis on the procurement process in this section of the Implementation Guidance is a mistake. Essentially, what is being implied in this section is that the only possible way to coordinate security controls with vendors is during the procurement process, when obviously the customer has the most leverage over the vendor. This might be a good model for a purely commodity market, where you have a bunch of vendors competing viciously with each other and they all receive a very low margin (on presumably a high volume of sales). In this case, the customer clearly has to obtain everything they want from the vendor up front, and they need to negotiate simultaneously with multiple vendors and play them off against each other for concessions.
I don’t know about you, but I don’t think this is how it works in the market for control systems that run the grid. I am betting that almost all vendors in this market will do almost anything they can to make their customers happy, since they are counting on having a long-lasting relationship with them. And different vendors’ products are very different, so it’s impossible to play vendors off against each other, as in a commodity-type market.
Clearly, a customer’s leverage isn’t limited to the procurement process. I am sure that, in most cases, a field technician (backed up by their VP) with a serious complaint about a vendor, midway through the contract lifetime, will usually have as much leverage as a purchasing agent on the eve of contract renewal.[iv] Even though it is notoriously difficult to change vendors of control systems in the power industry, I really don’t believe that any vendor would last very long if they didn’t do everything they could to provide what their customers need.[v]
Again, I’m saying that I think the emphasis on the procurement process in this section of the CIP-013 Implementation Guidance is a mistake. I really doubt there are auditors who will insist that NERC entities need to wait for the procurement (or contract renewal) process in order to request any meaningful changes to vendor security controls. And it follows almost certainly that auditors won’t require that contract language be the primary vehicle for obtaining agreement on controls – especially since that’s the one type of evidence that the auditors won’t be able to see!
If I’m so sure that the emphasis of this whole section of the Implementation Guidance is wrong, how did it end up there in the first place? To answer that, you need to know that at least half of the members of the drafting team for this standard were supply chain people – and I’m speculating that this particular section was drafted by some of them.
For supply chain people, the procurement process is kind of the whole show, since that is when they have by far the most interaction with vendors, as well as when what they do and say will have the most impact. But they don’t get involved much with the day-to-day interaction with vendors that goes on in the field. So supply chain people are naturally going to consider the procurement process – and especially contract language – as the primary vehicle for influencing what vendors do. They don’t see how vendors often easily agree to requests that come through completely different channels.
The moral of this story? It would be a big mistake for NERC entities to focus on the procurement process – and especially on contract language – as the sole, or perhaps even the predominant vehicle for getting vendors to agree to implement security controls. There are lots of paths to achieve this goal, including simple phone calls from a field technician to their friend Joe at the vendor. However, what will need to change with CIP-013 compliance is that those calls or emails – when related to vendor controls the entity has decided need to be in place in order for it to comply with CIP-013 – will need to be logged and archived so they can be evidence of compliance.
If you would like to comment on what you have read here, I would love to hear from you. Please email me at firstname.lastname@example.org. Please keep in mind that Tom Alrich LLC can help you with NERC CIP issues or challenges like what is discussed in this post. To discuss this, you can email me at the same address or call me at 312-515-8996.
[i] The staff member based this on the fact that FERC seemed to have made this a requirement in their Order 829, in which they ordered NERC to draft this standard. The auditor replied that FERC’s statements have no bearing on how a standard should be interpreted, once they have approved it. You can read their back-and-forth in the second of the three posts I linked above.
[ii] And keep in mind that, if the vendor absolutely refuses to do something that the entity requested, the entity isn’t in violation for that reason. However – and this is important – they are still under obligation to if possible mitigate the risk in question, presumably using a different method that doesn’t require the vendor to do something for them. Of course, there are certainly some risks that it won’t be possible to mitigate without the cooperation of the vendor, such as the risk of someone who had access to your systems leaving the vendor’s employment, and your organization not being notified that this has happened (this is, of course, the risk addressed in R1.2.3). In these cases, if the vendor simply won’t cooperate, the entity isn’t obligated to change vendors; but the entity should document how they tried to obtain cooperation.
[iii] Of course, you need to keep in mind that the six items listed in R1.2 are there because FERC ordered these six items be specifically included in the standard. They aren’t in any way intended to be the only items that need to be included in the entity’s supply chain cyber security risk management plan. The plan needs to address each of the three risk areas listed in R1.1: a) risks from procuring vendor equipment and software, b) risks from installing vendor equipment and software, and c) risks from transitions from one vendor to another. The six items in R1.2 all correspond to risks that fall into category a). However, there are certainly many other risks that fall into that category, which aren’t included in these six. And of course, the six items don’t address any risks that fall into categories b) and c).
[iv] Of course, I’m not talking here about huge vendors like Microsoft and Adobe, for whom the electric power industry accounts for a very small part of their revenue. They will of course provide product support, but if XYZ Power calls up and demands that Microsoft insert a new feature into the current Windows version, I rate the likelihood they will succeed in this demand as very low.
[v] Of course, if anyone has a different view on this, I’d love to hear it. Send me an email or comment on this post.