Et tu, CISA?

Yesterday, I pointed out that the National Vulnerability Database has not only failed to perform one of its most important jobs – adding CPE names to all new CVE records, a process called “enrichment” – this year, but they put an exclamation point on that record during the last two weeks of the year (ending yesterday 12/30) when they only added 15 of the 1,181 CPEs they should have added during that period. This means that, during those two weeks, they were only operating at a rate of 1%, vs. 45% in the previous 50 weeks (even 45% is abysmal, but it’s 45 times better than 1%, if my math is correct).

Of course, when there’s a shortfall like that, you have to wonder whether the NVD (which is part of NIST, which is itself part of the Department of Commerce) has simply given up on creating CPEs. As I explained in the post, a CVE record – i.e., an announcement of a newly-identified vulnerability, with a textual description of the software product in which it is found – without a CPE name is invisible to automated searches using that product name. And since there have been almost 40,000 new CVEs reported in 2024, manually searching through the text of each of those records is probably not going to be a lot of fun.

I honestly don’t think the NVD is completely giving up on the idea of creating more CPEs, simply because it would be quite hard for anyone there to justify holding onto their job if they really do that. On the other hand, I find it hard to believe the NVD will ever make up for the ground they’ve lost over this year, since they would not only have to go back to creating CPEs at their old pace, but they would have to at least triple that rate, for three reasons:

1.      The volume of new CVEs this year is close to double what it was last year. If that rate continues next year, the NVD will need to more than double their effort just to keep up.

2.      Since they added less than half the CPEs they were supposed to add this year, in order to catch up to where they should be, they need to multiply their doubled rate by a factor of at least two. This means they really have to triple or quadruple their effort from last year. Do you think they can do that?

3.      In 2023, there were about 28,000 new CVEs, and I believe the NVD assigned at least one CPE to each one of those. Yet this year, they assigned fewer than 18,000 CVEs, which is 10,000 fewer than last year. If anything, the trend line of CPE assignments is sharply downward, not upward. And given the current growth rate of CVEs, there could quite easily be over 50,000 new CVEs next year. Even if the NVD were able to get back to the 2023 rate of 28,000 per year, that still leaves their backlog at 22,000 at this time next year – which happens to be about their current backlog. No matter what it is, this certainly isn’t progress.

Meanwhile, there’s been another development that only adds to this problem. Someone pointed out to me today that CISA put up this announcement on GitHub a month ago: “…as of December 10, 2024, CISA will no longer be adding CPE strings to the enriched dataset.” You may know that within a couple of months after the NVD started having its problems in February, CISA gallantly announced that it was stepping up to help and would be adding CPE names and a couple other items (CVSS scores and CWEs, I believe) to CVE records – this was called the “Vulnrichment” program. While the program didn’t produce a huge number of CPE names, the fact that CISA was doing this was a morale booster for people who still had some faith in the NVD (a group that no longer included me by that time).

However, it seems CISA has changed its mind about producing CPEs (although not the other items). Of course, they’re not providing any explanation for why they changed their mind. But I can certainly guess that, given the many broken promises from the NVD this year, CISA is getting a little fed up with them. So, just as CISA’s announcement of the Vulnrichment program last spring was a morale booster, their repudiation of that announcement in December has to be taken as a motion of no confidence, in both the NVD and in CPE itself. I’ll note that just a few days earlier, CVE.org employees had pointed out, in their annual CNA Workshop (which I wrote a post about), that they’re trying to learn more about purl. Indeed!

What’s the solution to this problem? Remember, the NVD was the leading vulnerability database in the world less than a year ago. What is it now? You might be inclined to say it’s just a fraction of its former self, but I want to point out that even that is a wild overestimation. Just because the NVD has managed to add CPEs to about 45% of its new CVEs this year, that doesn’t mean it’s providing 45% of the value that it did before that; in fact, it’s providing much less value.

Consider why you search the NVD, or any other vulnerability database, for vulnerabilities that apply to a software product your organization uses. Are you expecting to learn about only 45% of the vulnerabilities that affect that product? I doubt it. Don’t you really want to learn about every vulnerability that affects the product? Wouldn’t it be much better if someone told you that the 55% of vulnerabilities that you won’t find by searching the NVD will be easy to find if you also search XYZ database? However, nobody can do that, since there are no machine-readable identifiers to tell anyone what’s in the approximately 22,000 CVE records that don’t have CPE names (i.e., the backlog).

There is one thing you can do. If you’re concerned mostly about open source products, you shouldn’t be looking for those in the NVD; in fact, you should never have been doing that in the first place. The OSV and OSS Index open source databases both have a much larger collection of open source vulnerabilities than does the NVD. Moreover, searches in those two vulnerability databases are much more reliable than they are in the NVD.

Why are these two databases more reliable than the NVD? It’s because they use a much more reliable identifier than CPE: purl. You can read why purl is so much more reliable than CPE in this post, but one difference that should be very obvious by now is the fact that nobody needs to “create” a purl. In other words, no single government agency can gum up international vulnerability management activities by stopping work without any explanation (not that this would ever happen, of course 😊).

An open source product that’s made available in a package manager already has a purl, since it can be created by anybody who has the spec and also knows the name of the package manager, as well as the name and version string for the product in that package manager. Of course, all three of those items are readily verifiable.

You may be wondering why, given that purl is so much more reliable than CPE and that CPE is rapidly going down the tubes, we can’t use purl now to search for vulnerabilities in the NVD? There are two reasons:

1. The CVE Numbering Authorities (CNAs), who report new vulnerabilities to CVE.org (from where the records are passed on to the NVD and other databases that are based on CVE identifiers), have until this past May not been allowed to include any software identifier other than CPE in a CVE record.[i] In May, the new CVE v5.1 spec included a field for a purl. However, there has been probably zero use of that field, mainly because currently there’s no vulnerability database that can utilize CVE records that include purls. If the CNAs add purls to CVE records today, the records will be all dressed up with nowhere to go.

2. Purl currently does not provide a way to identify commercial software products, but only open source products distributed in package managers. The only currently available identifier for commercial software products is CPE. It’s not a good identifier for commercial products (or for open source products either, of course), but it’s the only one available today. Faute de mieux, as the French say.

I’m proud to announce that the OWASP SBOM Forum is starting to work on both of these issues now. You can read how we’re working on the second issue in this post that I linked earlier. However, we won’t get very far without more participation, as well as financial support (which can be in the form of donations of $1,000 or more to OWASP – a 501(c)(3) nonprofit organization – that are “restricted” to the SBOM Forum). If you can contribute time or funds, please email me.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.

My book "Introduction to SBOM and VEX" is now available in paperback and Kindle versions! For background on the book and the link to order it, see this post.

---

[i] If you aren’t intimately familiar with CVE records, CVE.org, the CNAs and similar exotica, you should go to this post.  

Has the NVD thrown in the towel?

I’ve been relying on three people to follow the ongoing saga with the National Vulnerability Database (NVD) for me: Andrey Lukashenkov of Vulners, Patrick Garrity of VulnCheck, and Bruce Lowenthal of Oracle. Early this morning (Chicago time) I emailed all three of them to ask for an update, since the last post I wrote on this subject (I’ve written many previous ones, which you can find by searching for NVD in the search bar at the top of this post) was on December 9.

Andrey replied first, but I must admit that he had an unfair advantage, since Patrick and Bruce both live in the US and Andrey lives in Spain. Regardless, I was shocked by what he told me: In the last two weeks of the year, the NVD “enriched” (i.e., added one or more CPEs to) only 15 CVE records out of a total 1,188. That’s just 1% (plus, all 15 were in the week beginning December 16; no CVEs at all were enriched during the last week in Andrey’s spreadsheet, which ended today, December 30).

To provide some perspective on this, in the first 50 weeks of this year, the NVD enriched 17,577 of 38,593 CVE records, or 45%. In previous years, the NVD enriched almost 100% of new CVE records, usually within a few days of receiving them from CVE.org. Of course, it was shocking that the NVD only enriched 45% of CVE records through mid-December, but – holiday or no holiday – it’s well beyond shocking that they seem to have literally thrown in the towel the last two weeks (after working during these weeks in previous years, with time off for the federal holidays).[i]

If you want to understand what this means, I’ll refer you back to the December 9 post. In brief, the problem is (and if you don’t understand any of the terms below, please read this post):

1.      New software vulnerabilities are reported to CVE.org by the CVE Numbering Authorities (CNAs). The CVE report includes a description of the vulnerability, a CVE number assigned to it by the CNA, and a textual description of the affected product or products.

2.      CVE.org includes the new record in their database and forwards it on to the NVD.

3.      Up until February 12, 2024, the NVD usually created the machine-readable CPE name, which is required for a user to look up vulnerabilities for a particular product in the NVD.

4.      If there is no CPE name attached to a CVE record in the NVD (as is the case with 55% of the new CVE records created this year), a user of the product that is described as affected in the text of the record will not see that CVE number when they search using the product’s CPE name. In other words, that CVE number will be invisible to searches for that product[ii].

5.      Of course, the user won’t see an error message when this happens. In fact, if that CVE is the only one listed for that product in the NVD, the user will just learn that there are no vulnerabilities that apply to that product. Of course, this isn’t true.

In summary, the fact that 55% of the 2024 CVE records in the NVD are invisible to automated product searches means that any search for vulnerabilities found in a product (including searches initiated by a scanning tool) is going to miss more than 55% of the vulnerabilities reported for the product this year (including the great majority of vulnerabilities reported after February 12).

It remains to be seen whether the NVD has given up on enriching CVE records altogether, but the fact that I even must mention that possibility shows how far the NVD has fallen. I don’t recommend that anybody bet the farm on the NVD coming back for perhaps a year or two, if even that. We need to move as quickly as possible toward introducing purl into the CVE ecosystem. More posts are coming on that subject.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.

My book "Introduction to SBOM and VEX" is now available in paperback and Kindle versions! For background on the book and the link to order it, see this post.

---

[i] In case you were wondering if the NVD has deigned to explain to us why this happened, I’ll point out that the last entry on their “news updates” page is from November 15.

[ii] The fourth paragraph of the December 9 post mentions, “..a few firms like Vulners (Andrey’s employer) and VulnCheck have taken it upon themselves to add their own CPEs to some of the unenriched CVE records..” This means you will find more vulnerabilities applicable to a particular product if you search either of those databases.

Why do we need to replace NERC CIP-007 R2?

I have written two recent posts about the need to replace (not just revise) the patch management requirement CIP-007 R2. Since both posts were written for other purposes as well, I’d like to summarize what they say and add some context that I previously left out.

The first post from October pointed out that the two most prescriptive CIP requirements, CIP-007 Requirement R2 and CIP-010 Requirement R1, are without doubt also the two most burdensome; this is because of the huge amount of documentation that the NERC entity must produce to prove compliance with each of them. The evidence must be provided for every physical (and soon, virtual) device in scope.

Moreover, the two requirements pose the biggest impediment to implementation of medium and high impact BES Cyber Systems in the cloud. This is because no CSP could ever produce the required device-level evidence, even if the CSP’s patch and configuration management programs in fact followed all the steps mandated by the two requirements. As every NERC compliance professional knows, “If you didn’t document it, you didn’t do it.”

I went on to point out that both problems – the overwhelming documentation demands for on-premises systems and the fact that it is virtually impossible for a CSP to provide the compliance evidence required by the strict wording of the two requirements – can be solved by replacing the current prescriptive wording of each requirement with a risk-based (or “objective-based”, to use NERC’s preferred term) requirement.

Doing this won’t be hard for CIP-010 R1, since the objective of the current prescriptive requirement won’t change in the new risk-based version: reduction of the risk that an inadvertent configuration change in a system will leave it vulnerable to a cyberattack. The difference is that the objective will be achieved in the revised requirement through risk management, not through providing careful documentation of each instance of compliance, as is the case currently.

However, in the October post I noted that the objective of the current CIP-007 R2 is to ensure that every security patch released for any software or firmware product installed on a system in scope is applied, or that the vulnerability is otherwise mitigated. There is no consideration of the risk posed by the vulnerability that is addressed by the patch; if the supplier (or other patch source) has released a security patch within the previous 35 days, either the patch needs to be applied or the vulnerability it addresses needs to be mitigated in some other way. The NERC entity doesn’t have the option of pointing out that applying the patch will address little or no risk, and therefore it would be better for them to spend their time applying patches hat do address risk, even if they aren’t required for compliance with CIP-007 R2.

In other words, the problem with CIP-007 R2 is that patching isn’t a security objective in itself; it’s a mitigation of the risk posed by one or more unpatched software vulnerabilities. If CIP-007 R2 is going to be rewritten as risk-based, it must be a vulnerability risk management requirement, not a patch application requirement.

2. In this post in December, I asked what a CIP vulnerability management requirement would look like. I pointed out how much more efficient that would be than the current patch management requirement.  This is especially true because at most 6-7 percent of all vulnerabilities (CVEs) are ever actively exploited. This means that a policy of applying every security patch that is released for a software product inevitably leads to a lot of wasted effort. However, this is the policy that is required by the current CIP-007 R2.

As I pointed out in the December post, the annual volume of newly identified software vulnerabilities (CVEs) when the original CIP patch management requirement was developed was a small fraction of what it is currently. This meant that organizations didn’t have a big backlog of unapplied patches; therefore, it was possible to apply every security patch that had been released for a software product.

Today, because of the huge volume of newly identified vulnerabilities – 35,000 this year alone vs. about 29,000 in 2023 - the biggest concern in vulnerability management is prioritizing patch applications, so they are as efficient as possible. In other words, risk needs to be reduced by the greatest amount possible, given the resources available for applying patches. This is why I stated in my December post that the new CIP-007 R2 should require that the vulnerability risk management plan prioritize patch applications “…according to criteria listed in the plan”.

Note that I’m not calling for the CIP-007 R2 vulnerability risk management plans to require use of specific criteria to prioritize patching vulnerabilities based on the degree of risk they pose. However, if a NERC entity just follows the FIFO method (first in, first out) for applying patches, they should be cited for that, since the patch’s time of arrival isn’t an indicator of the risk of the vulnerability being patched.

The most widely used vulnerability prioritization criterion today is probably CVSS score. However, many vulnerability management professionals will tell you that CVSS isn’t a measure of risk, but of severity. Risk is some combination of likelihood and impact. CVSS assumes that likelihood = 100%, so it only estimates the worst case impact. This doesn’t mean that CVSS should be ignored by an organization that is trying to prioritize patches according to the risk posed by the vulnerability or vulnerabilities they mitigate. But it does mean that CVSS should never be the only criterion you look at, or even the first criterion.

The best way to measure the risk posed by a particular vulnerability is by determining whether it is currently being exploited or is likely to be exploited, using these two measures:

1.      Whether the vulnerability (CVE) is in CISA’s Known Exploited Vulnerabilities catalog, and

2.      The vulnerability’s current EPSS score, which provides the likelihood (as a percentage between 0 and 1) that the vulnerability will be exploited within the next 30 days[i].

Using one or both of the above in your vulnerability prioritization methodology is much better than relying solely on CVSS, but you can use other criteria as well. These can include:

A.     CVSS, or even better, one or more components of the CVSS score. For example, one of the components estimates the degree of effort required to exploit the vulnerability. You can just pull out that component and include it as on of you criteria.

B.     A measure of the importance of an asset. Of course, since we’re talking specifically about NERC CIP compliance now, all assets will be important. However, some will be less so (e.g., a system handling I/O to a purely distribution substation vs a primary EMS server).

C.      A measure of the exposure of an asset, such as whether it is directly exposed to the internet, or whether it is in a DMZ vs. an ESP.

Of course, there are many other possible prioritization criteria. An organization should decide the right combination of criteria for their needs (and different areas of the organization might use different criteria).

Replacing CIP-007 R2 with a vulnerability risk management requirement will help accomplish three goals:

1.      It will reduce the amount of software vulnerability risk experienced by NERC entities with high or medium impact CIP environments, since entities will be able to prioritize their patch management efforts according to the amount of risk mitigated by each patch.

2.      It will greatly reduce the amount of compliance documentation required for NERC entities with high or medium impact on-premises CIP environments.

3.      It will go a long way toward eliminating the biggest obstacle to deployment of CIP-protected systems in the cloud today: the fact that CIP-007 R2 and CIP-010 R1 both require the CSP to provide evidence of prescriptive actions taken on a huge number of individual physical and virtual cyber assets over a three-year CIP audit period.

Of course, the Project 2023-09 Risk Management for Third-Party Cloud Services Standards Drafting Team (SDT) is just about to start drafting revisions and/or additions to the CIP standards that will finally make use of the cloud completely “legal” on the OT sides of all NERC entities. Since that team will almost certainly need to rewrite CIP-007 R2 and CIP-010 R1, why propose that these two standards be rewritten separately from that effort? There are two reasons why:

1.      Since CIP version 5 was implemented in 2016, I have continually been told that CIP-007 R2 and CIP-010 R1 are by far the most burdensome of all the requirements that apply in on-premises CIP environments. In other words, we need to fix these two requirements as soon as possible, regardless of any consideration of cloud use in the future.

2.      I recently estimated that it will be 2031 – i.e., seven years from now - before the new or revised CIP standards for the cloud are implemented. Since I think that rewriting CIP-007 R2 and CIP-010 R1 alone will require around 2 to 2 1/2 years, and since this needs to be accomplished as soon as possible, it makes sense to have a separate Standards Authorization Request (SAR) and SDT for this purpose. In fact, taking this item off the current SDT’s agenda will probably trim that agenda by two years, meaning the final CIP “solution” for the cloud could be in place by 2029 instead of 2031.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.

My book "Introduction to SBOM and VEX" is available! For context, see this post.

---

[i] For an excellent webinar recording that discusses EPSS, go here.

Was this the first risk-based CIP requirement?

On July 4, 2016, I wrote about a meeting I had just attended: the second meeting of the NERC “Project 2016-02  Modifications to CIP Standards” drafting team. Coincidentally, this team finally finished their work earlier this year, when the revised CIP standards to accommodate virtualization were approved by the NERC ballot body, approved by the NERC Board of Trustees and submitted to FERC for their approval.

This meeting occurred soon after an important turning point in the CIP standards:

1.        The CIP versions 5 and 6 standards had come into effect on July 1, 2016 – i.e., three days before I wrote the post.

2.        The “CIP Mods” team (as the new team came to be known) was created to address issues that had come to light as NERC entities were preparing for implementation of CIP v5 and v6.

In my post, I noted an issue that I had started writing about at the beginning of 2016:

…since the beginning of this year I have put out a few posts - such as this one and this one - that take a different tack. They don’t criticize the v5 wording, but do criticize the whole idea of what I call “prescriptive standards” – which is what all the CIP versions have been. It may be clear from these two posts that I think the prescriptive approach doesn’t work for cyber security standards, even though it might be fine for the other NERC standards. And since the requirements of CIP v5 and v6 are prescriptive ones (with two exceptions, which I’ll discuss below), it seemed certain that v7 will be prescriptive as well.

I cited two other posts in the above paragraph. Near the end of the first of those two posts (from February, 2016), I asked, “Is CIP v5/v6/v7 Sustainable?” I pointed to the new items that were on this new drafting team’s plate (or might be soon). I wondered whether they would result in even more prescriptive requirements (as if the CIP community didn’t have enough of those already). I added:

I have asked a small number of compliance people for NERC entities what percentage of every dollar they spent on CIP compliance actually went to cyber security and what percentage went to the paperwork required to prove compliance. Of course, there is no objective way to measure this, but the estimates I have received range from 35 percent to 70 percent. Maybe the average is 50 percent, but’s let’s assume my small sample is biased downward and the average is closer to 70 percent. Even this figure means that 30% of every dollar spent on CIP is only spent to prove compliance, not to improve cyber security. To phrase it differently, if we could figure out a way to get the percentage of spending that goes to security up to 90%, we would in principle have up to a 20% increase in security (budgets) without having to spend a single additional dollar.

I also quoted a respected utility cyber security manager who asked, when told that virtualization was next up in the new CIP standards lineup, “Why can’t we just be trusted to do the right thing, rather than have to spend a huge amount of time documenting our compliance for virtualized systems?”

That’s really the issue, isn’t it? Why should NERC entities have to spend so much time and money documenting compliance with prescriptive CIP requirements, when most other mandatory cybersecurity compliance regimes (PCI, HIPAA, etc.) have managed to develop requirements that let the entity focus on achieving an objective, not on taking a set of prescriptive steps to prove their compliance in every instance?

Of course, the answer to that question is that NERC entities should not have to spend so much time and money documenting compliance with prescriptive CIP requirements. How can the requirements be revised (or more importantly written differently in the first place) to eliminate this problem?

I didn’t answer that question in the February 2016 post, but I did in the July 4 post that referenced it. The second half of that post concentrated on the most important item on the drafting team’s agenda for that meeting: fixing the definition of “low impact external routable connectivity” (which was at the time affectionately known as LERC) found in the original version of CIP version 6[i].

LERC (a term that was invented for use with CIP v6) turned out to be a mistake and a big source of confusion when NERC entities prepared to comply with V6. Briefly, LERC was intended to be the low impact version of External Routable Connectivity (ERC), a term invented with CIP v5, which only applies in high and medium impact environments.

However, it turned out that the requirement where LERC applied (CIP-003-6 R2, Attachment 1 Section 3) implicitly allowed LERC to be “broken” by a number of factors; this led to long arguments between NERC entities and their auditors about whether LERC was completely broken by a particular measure like a data diode, or whether it was only partially broken by a measure like a firewall. In the first case, the entity would have been found compliant with the requirement; in the second case, it would not have.

It was quite interesting to watch the SDT, during the 2 1/2-day meeting, evolve from trying to make this a workable prescriptive requirement to realizing that couldn’t happen, since doing so would make both the requirement and the definition a hopeless mess. At that point, it seems “another observer” (although I can’t remember who that was) suggested that the SDT should “rewrite the requirement so that it simply stated that an entity with LERC needed to take steps to mitigate the risk posed by LERC and discuss different options for doing this in the Guidelines and Technical Basis section.”

In other words, the other observer realized it was a huge waste of time to torture the requirement until it yielded a fixed set of measures that the entity could take to “break” LERC (and thus comply with the prescriptive basis of the requirement). Instead, the person suggested it was better to “rewrite the requirement so it simply stated that an entity with LERC needed to take steps to mitigate the risk posed by LERC and discuss different options for doing this in the Guidelines and Technical Basis section.”[ii]

I found this to be quite interesting, because the other observer admitted it would never be possible to treat this as a purely prescriptive requirement, for which the Responsible Entity would be given an up-or-down judgment of compliance. Instead, the entity needed to decide what was the best way for them to mitigate the risk in question; it was up to the auditor to decide whether the measures the entity took sufficiently mitigated the risk.

Folks, this was heady stuff! Here was a revised NERC CIP requirement for which the goal of compliance was simply to mitigate the risk addressed by the requirement, without trying to a) prescribe the steps required to mitigate that risk or b) attempt to measure the total risk that was mitigated by the steps the entity took (which would have been a fool’s exercise, in any case).

In other words, this may have been the first NERC CIP requirement (and perhaps the first NERC requirement, period) whose objective is clearly risk mitigation[iii] and nothing more (there already were at least two requirements in CIP version 5 that were in fact risk-based, including CIP-007 R3 anti-malware and CIP-011 R1 information protection. However, neither one of them explicitly mentions risk or risk mitigation).

Since 2016, other risk-based (which NERC calls “objectives-based”) requirements, and even whole standards (CIP-012, CIP-013 and CIP-014) have been developed and put into effect. Moreover, I am reasonably sure that after prescriptive requirements like CIP-007 R2 and CIP-010 R3 were developed as part of CIP version 5 (in 2011 or 2012, probably), not a single new prescriptive CIP requirement has been developed. Moreover, I’m certain that no new prescriptive CIP requirements ever will be developed; there clearly is no more appetite for them in the NERC community.

This leaves open the big questions, How are risk-based CIP requirements being audited today? And how should they be audited? These questions are especially important because any new or revised CIP requirements that will apply to assets in the cloud will have to be risk-based. Will developing procedures for auditing risk-based CIP requirements require changes to the NERC Rules of Procedure? Given that I haven’t found anyone who can tell me exactly what is required to change the RoP, that might not be a lot of fun…

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.

My book "Introduction to SBOM and VEX" is available! For context, see this post.

---

[i] That definition was “Direct user-initiated interactive access or a direct device-to-device connection to a low impact BES Cyber System(s) from a Cyber Asset outside the asset containing those low impact BES Cyber System(s) via a bi-directional routable protocol connection.”

[ii] The “Guidelines and Technical Basis” section of CIP-003 has been renamed “Supplemental Materials” and is now found at the end of the CIP-003-8 standard (the current version) starting on page 33. It provides diagrams of ten possible methods which the entity might take to mitigate the risk posed by LERC, without saying which of those is the “best” or the “most compliant”.

[iii] CIP-003-8 R2 Attachment 1 Section 3 doesn’t explicitly mention risk. However, it requires the entity to “Permit only necessary inbound and outbound electronic access as determined by the Responsible Entity.” Since the entity is allowed to determine for itself what constitutes “necessary” access and since any judgment about whether something is necessary must always consider risk, this makes it a risk-based requirement in my book.

What should a NERC CIP vulnerability management requirement look like?

In October, I wrote this post titled “NERC CIP needs vulnerability management, not patch management”. That post made the argument – which I’ve made before – that compliance with Requirement CIP-007 R2 is hugely expensive relative to the security benefit it provides. My most important evidence for this was the fact that CIP-007 R2 requires the NERC entity, every 35 days, to identify every security patch that has been issued for each version of each software or firmware product installed within their ESP, that has been issued in the previous 35 days. It doesn’t matter whether the vulnerability or vulnerabilities mitigated by the patch pose high, medium or zero risk; the patch needs to be evaluated and applied within 35 days. If it can’t be applied in that time, the Responsible Entity must develop and implement a mitigation plan for the vulnerabilities addressed in the plan.

When the original patch management requirement CIP-007-1 R3 appeared with NERC CIP version 1 in 2008, the requirement wasn’t spelled out in such exquisite detail, but it was very similar: The NERC entity needed to “establish and document a security patch management program for tracking, evaluating, testing, and installing applicable cyber security software patches for all Cyber Assets within the Electronic Security Perimeter(s).” As is the case with the current version, there wasn’t a word about the degree of risk posed by the vulnerability addressed in the patch.

However, in 2008 this wasn’t as big a problem as it is today, since far fewer vulnerabilities were identified at that time; therefore, far fewer patches were issued. The first vulnerabilities identified with a CVE were reported in 1999; a little more than 300 were identified in that year. Even 15 years later in 2014, only 7,928 CVEs had been reported in total. How many CVEs have been reported as of last week? 274,095, of which 38,000 have been reported so far this year (a huge jump from last year, by the way). In other words, almost five times as many new CVEs have been reported so far in 2024 alone as were reported in total during the period 1999 – 2014.

In fact, any company that tries today to apply every patch they receive for every one of the software products they use is sure to fail; yet that is exactly what CIP-007 R2 requires. How do organizations not subject to CIP compliance keep from being overwhelmed by patches? They must prioritize them. Of course, it’s best to prioritize them by the degree of risk posed by the vulnerability or vulnerabilities that are mitigated by each patch. What’s the best way to do that?

A lot of organizations prioritize vulnerabilities based on CVSS score; in fact, that used to be considered the best way to do it. However, CVSS doesn’t measure risk; it measures severity of impact of the vulnerability – and even that is very hard to measure in a single score, since the severity of impact will vary greatly by the importance of the affected system, how well the network is protected, etc.

There is now a growing consensus in the software security community that the best measure of the risk posed by a vulnerability is whether it is currently being exploited by attackers. There’s a big reason for using that measure: Only about six percent of CVEs are ever exploited in the wild.[i]

Of course, this means that, in retrospect, an organization that tries to apply every patch will be wasting at least 80-90% of the time and effort they spend in patching. While it’s not possible to reduce this wasted time to zero, it’s certainly possible to reduce it substantially by prioritizing patch application for vulnerabilities in CISA’s Known Exploited Vulnerabilities catalog or values close to 1.0 in FIRST’s EPSS score[ii] - as well as using other measures like the impact if the system is compromised.

However, there’s one type of organization that’s unable to prioritize patch applications: that’s a NERC entity with high or medium impact systems.[iii] This is the most important reason why the CIP-007 R2 patch management requirement needs to be replaced with a risk-based vulnerability management requirement. What would the new CIP-007 R2 look like? Very roughly, I think it should require the Responsible Entity to develop and implement a vulnerability management plan to:

1.      Identify new vulnerabilities that apply to software and firmware products installed on medium or high impact BES Cyber Systems, EACMS, PACS or PCAs that they operate.

2.      Assign a risk score to each vulnerability, based on criteria identified in the plan.

3.      Identify a score that is the “do not fix” threshold. The NERC entity will normally not apply a patch for a vulnerability whose score is below that threshold, although there will always be special cases in which the vulnerability needs to be patched regardless of its score.

4.      Regularly investigate security patches released by the vendors of those products.

5.      Prioritize application of those patches according to criteria listed in the plan; apply patches in order of priority, if possible.[iv]

6.      If a patch cannot be applied when it normally would be, determine what alternate mitigation(s) for the vulnerability addressed in the patch might be applied.

How will compliance with this new requirement be audited? The entity will need to provide two types of evidence:

A.     The plan itself, including a narrative explaining how the vulnerability management plan makes much more effective use of resources and mitigates more risk than would a plan to apply every applicable patch, regardless of the degree of risk mitigated.

B.     Evidence that the plan was implemented and that it significantly lowered the entity’s level of software cyber risk. This could include information such as a comparison of a group of vulnerabilities that were patched vs. a group that were not patched, along with the average risk scores of the two groups; of course, the comparison should show that the scores of the patched vulnerabilities were much higher than those of the unpatched vulnerabilities.

C.      Along with the quantitative evidence, qualitative evidence that the plan was implemented and was effective in lowering the entity’s level of software risk. For example, there could be a narrative like, “An example of the success of our plan is CVE-2024-12345. This vulnerability was on CISA’s KEV list when the patch was released. We assigned the vulnerability our highest risk score of 5 and applied the patch the day it was released. After we applied the patch, three major cyber incidents were reported in which CVE-2024-12345 was the primary attack vector.”

Note that, unlike the current CIP-007 R2, compliance with this vulnerability management requirement will not mandate that the NERC entity be able to provide evidence of every instance of compliance – e.g., evidence that the entity (or their tool) checked with every software or firmware vendor in scope every 35 days to determine whether any new security patches were available.

Even more importantly, the NERC entity will not need to provide evidence on an individual device basis, as is usually required for CIP-007 R2 compliance. Instead of having to identify individual BES Cyber Assets that were patched, the entity will identify – for example – vulnerabilities on the CISA KEV list that were patched, without having to point to individual devices.

This is especially important for one reason: Cloud service providers will never be able to provide compliance evidence on a device basis, since they’re not equipped to do that. This means that the current CIP-007 R2 (and CIP-010 R1, which also requires evidence on a device basis) will continue to be an obstacle to cloud use by NERC entities with high and medium impact BES Cyber Systems until CIP-007 R2 is changed to a risk-based (or “objectives-based”, to use NERC’s preferred term) requirement. The risk that patch management mitigates is the risk posed by unpatched vulnerabilities, so the only way that CIP-007 R2 can be made “cloud friendly” is to replace it with a vulnerability management requirement.

Of course, we can assume that all of CIP will ultimately be rewritten (or replaced) as part of the huge effort required to make use of the cloud fully “legal” for NERC entities. However, as I pointed out recently, it will likely be 6-7 years before that is accomplished. Does the power industry have no choice but to wait that long?

What I just wrote about CIP-007 R2 points to a possible interim solution, which may “only” require rewriting two requirements: CIP-007 R2 and CIP-010 R1 (configuration management). I’ve just described what needs to be done with the former requirement. As for the latter, I think CIP-010 R1 can remain a configuration management requirement, but it can be reworded so that it is no longer device-based and so that it focuses on the risk of inadvertent misconfiguration.

I think both of these requirements could be rewritten and gain approval from NERC and FERC in 3 to 3 ½ years, which is half the time required for the full “CIP in the cloud” revisions that the Project 2023-09 Risk Management for Third-Party Cloud Services Standards Drafting Team is now working on. The full set of revisions will ultimately be needed, but I believe most cloud use by NERC entities could be made “legal” just by rewriting these two requirements.

However, doing this will require a different SDT – one that is just focused on these two requirements. Besides accelerating the development of the replacements for CIP-007 R2 and CIP-010 R1, constituting a second SDT will allow the current SDT to finish their work at least a year or two earlier than 2031. 

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.

My book "Introduction to SBOM and VEX" is available! For context, see this post.

---

[i] I owe this observation to Chris Hughes, who writes the excellent blog on software security called Resilient Cyber. I recommend you subscribe to it, if software security is a concern of yours. The post where I found this figure is here.

[ii] EPSS scores can change daily, so it is important to update them daily, if you’re basing your patch prioritization in part on EPSS.

[iii] Of course, NERC auditors are human beings; they’re unlikely to issue a Notice of Potential Violation (NPV) if an entity convinces them they don’t have the resources required to apply or mitigate every patch that’s been released for every piece of software found in their ESP. But it’s a big waste of time for a NERC entity to have to prove it. The fact is that today, an organization that does have those resources is the rare exception that proves the rule.

[iv] There may be reasons why some patches should not be applied strictly according to their assigned priority. For example, if the target systems are found in different physical locations, and it will save a lot of time if all patches due to be applied in one location are applied at once.

News from the folks at CVE.org Part II

My previous post described the set of videos that CVE.org posted recently from their annual CNA (CVE Numbering Authority) Workshop earlier this month. I said I found a number of statements very interesting, but first I wanted to introduce readers to how CVEs are identified and used and who the players are, since most readers probably only have vague ideas on these topics.

Since I wrote that introduction on Friday, I’m now ready to tell you about at least some of the nuggets of wisdom that I found (you may find it helpful to refer to the previous post if any of the terms or acronyms below aren’t familiar to you):

CVE.org and the NVD

Since the National Vulnerability Database (NVD) is where most of us learn about CVEs, we often think it is the source of CVE information. That isn’t true. CVE.org, which used to be called just “MITRE”, is the source of information on CVEs. This information is found in “CVE Records” that are available in the CVE.org database.

The CVE records are passed on from CVE.org to the NVD and are incorporated into their database, but the NVD’s CVE records don’t include all the information that is found in the same records in the CVE.org database. One of the speakers in the workshop emphasized that just looking for information about a CVE in the NVD will cause you to miss the additional information that’s available in CVE.org. If you are trying to research a CVE in depth, you need to look in both databases.

The word from Microsoft  

Lisa Olsen of Microsoft, a longtime participant in the CVE ecosystem, made a number of interesting points:

·        Microsoft usually reports 80-100 new CVEs on every Patch Tuesday.

·        For some of their products, they create CPE names and include them in the new CVE records.

·        When they create a CPE for a version of Windows, they include the build number. This is important, since builds are really the “versions” for Windows. Windows 10, 11, etc. are more like separate products, since they are constantly updated during the multiple years they’re usually available. To identify the specific codebase in which a Windows vulnerability is found, you have to know the build number.

·        Lisa pointed out that in some CVE reports, the “affected” version of the product refers to the fixed version of the product (i.e., the version to which the patch has been applied), while in other reports (usually from different CNAs), the “affected” version is the unpatched version. This is a huge difference, of course, since it means some organizations may be applying patches to versions of a product to which the patch has already been applied. Lisa said the new CVE schema will allow the CNA (which is in many cases the developer of the affected product) to indicate which case applies. However, it seems to me there should be a rule: The “affected” product is always the one in which the vulnerability has not been patched.

·        Microsoft also is going to start publishing CVE records that have a new category of CVE: one that doesn’t require customer action to resolve. I believe she meant these are vulnerabilities that have already been resolved by a means other than patching – e.g., a configuration change in the software itself; yet Microsoft believes it is still important for the user to know that the vulnerability is present in their software, even though it is not currently exploitable.

Art Manion

One of the most respected figures in the CVE “universe” is Art Manion, formerly vulnerability analysis technical manager at the CERT Coordination Center at Carnegie-Mellon and someone who continues to be very involved with CVEs as a CISA contractor. He made a number of interesting points, including:

·        The key to CVE.org’s mission is the CNAs, since they identify and report all CVEs. CVE.org greatly prefers the “federated” approach, in which the CNAs are given a lot of freedom in how they report CVEs (in fact, he noted that only three of the many fields in the CVE specification are required; the others are all optional).

·        Another way of saying the above is that using the word “must” with the CNAs won’t usually get you anywhere. The CNAs are volunteers. CVE.org is very careful about not alienating them by threatening to reject CVE reports if certain fields aren’t present, etc. There are a lot of complaints about CVE reports not including this or that field, but these concerns should always be approached with the attitude of, “Do you want a half-full glass or no glass at all?”

·        A number of industries are reporting very low numbers of vulnerabilities, including healthcare and autos. This is why I usually say the vulnerabilities you need to worry about most are the ones that have never been reported to CVE.org (as well as to other vulnerability reporting entities like ICS-CERT and GitHub Security Advisories). Of course, when these are suddenly exploited, they’re known as “zero day vulnerabilities”. However, these are often vulnerabilities that have been known for a while – but just to the developer, which never reported them.

·        It would be nice to have a VEX capability in CVE. That is, instead of saying, “Product X is affected by CVE-2024-12345”, the CVE report would effectively say, “Even though Product X contains component ABC and ABC is affected by CVE-2024-12345, Product X is not affected by that vulnerability.”

·        The new version of the PCI-DSS standards for protection of payment card data by the retail industry requires that vendors to the industry report more vulnerabilities. In other words, “We don’t believe the fact that you haven’t reported any vulnerabilities for your product really means you don’t have any. It just means you have decided to endanger your customers by not reporting them. This has to stop.”

·        It would be good to include in the CVE record whether CISA has placed that vulnerability on its Known Exploited Vulnerabilities (KEV) list. Since almost every organization has many more vulnerabilities to patch than it has hours in the day to patch them, prioritization of vulnerabilities is a key issue. Prioritizing vulnerabilities that are on the KEV list is a great strategy. I agree with Art that this information should be included in the CVE record. 

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.

My book, Introduction to SBOM and VEX, is available here!