Late at
night last week, a rock was thrown through my window, wrapped in a note from a
NERC entity that obviously didn’t want to be identified. The note asked, “What is your take on
Windows XP and CIP Version 5? If you’re
running XP, are you non-compliant?” As
you’re probably aware, support for XP ended last year, so there are no more
patches available for vulnerabilities.
On the other hand, many entities still have systems (mainly in
substations and generating stations) that run custom code on top of XP. It’s a multi-year process to rewrite and
re-validate that code for a new OS.
I
immediately emailed this question to an unnamed compliance auditor who
responded promptly that, from a strictly compliance point of view, running XP
doesn’t make you non-compliant.
Regarding patching, he said “Patching is not an issue. Because XP
is past end-of-life, no patches are available, therefore there are none to
assess for applicability. In the extremely unlikely event an emergency
patch is released for XP, the entity will have to deal with it.” He went on to say “The rest of the
requirements are manageable.” To repeat,
from a strictly compliance point of view, running XP isn’t an issue.[i]
I then
raised a further issue with the auditor: “The big problem I can see with XP is that new
vulnerabilities will appear and there won’t be patches for them. Does the entity have any responsibility to
mitigate these new vulnerabilities?” His
answer was, “Mitigation of new vulnerabilities is a good security practice but
in the absence of patches, there is nothing in the Standard to compel the
entity to mitigate identified vulnerabilities. They only have to mitigate when
they cannot install an applicable patch.”
Of course, there are no longer any patches for XP; therefore, there is no obligation to mitigate new XP
vulnerabilities as they’re identified.
However, the
auditor did go on to say, “Obviously, they need to get on with replacing the XP
environments - again, for reasons of good security and utility practice and not
because the CIP standards compel them. They are at increasing risk because
often vulnerabilities found in currently supported versions of Windows will
also be present in Windows XP. So, they need to lock down their XP
environments as much as they possibly can to keep them from being the cause of
the first blackout due to a cyber-security incident.”
So there you
have it. Running XP isn’t going to
expose you to compliance risk, but it certainly does expose you to security risk. It’s certainly a good idea to try to take
other steps to mitigate new vulnerabilities, since patches are no longer
available. And it’s an even better idea
to look for ways to move from XP to a supported OS as soon as this is feasible
without impacting reliability.
Coming up
next:
What if you
still have OS/2?
What if you
still have Windows NT?
What if you
still have OpenVMS?
The views and opinions expressed here are my
own and don’t necessarily represent the views or opinions of Honeywell.
[i]
The auditor also said, “We have seen a couple of entities still running XP in
the control center. But they were well on the way of migrating to Windows
7. We still see Windows XP Embedded systems in the field. XP
Embedded is supported through the end of next year, so hopefully the entities
are working with their vendors to replace/upgrade the systems before XP
Embedded reaches end of support.”
In reading the first post, I came away with an even greater amount of 'head shaking' upon reading the following:
ReplyDeleteRunning XP isn’t going to expose you to compliance risk, but it certainly does expose you to SECURITY risk. It’s certainly a good idea to try to take other steps to mitigate new vulnerabilities, since patches are no longer available. And it’s an even better idea to look for ways to move from XP to a SUPPORTED OS as soon as this is feasible without impacting reliability.
My first question is how can an Entity be CIP v5 "compliant" running Windows XP or any software which cannot be patched or supported? My second question, and perhaps a cynical one, how does the Regional Entity expect the level of security to improve by not citing EOL software or hardware as a security risk?
The point behind my comment to your blog post was to ask if CIP v5 does not require retirement/replacement of EOL software, especially for critical infrastructure software systems, then how will its implementation improve the level of resilience and reduce intentional interruption of power grids by malware or software intrusions?
Software security should not be a 'point in time' consideration, but a lifecycle perspective. As a system/software lifecycle issue, security is an attribute that is enhanced and maintained over the period of operation by patches, but software patching was never intended to forestall the inevitable and clearly evident 'requirement' to migrate systems as the software's operational end-of-life drew near. Migration planning, which begins the very day a software put into 'production', is no less an ongoing system security responsibility than patching.
The use of COTS software as system components and managing its commercial lifecycle from 'release' to 'obsolescence' is not a new topic. But if 'running XP (outdated and unsupported software) isn't going to expose an Entity to compliance risk under CIP v5, then why does running unpatched software? With all due respect to the 'auditor' whose opinion you cited in your post, his/her position not to cite an Entity for running unsupported or obsolete (EOL) software like Windows XP is illogical, both from a compliance and audit perspective since in either case, the same result occurs: failure to mitigate evident risks.
Why would an Entity buy and then install COTS software or hardware that the vendor does not support? Why would an Entity buy COTS software or hardware and not purchase maintenance or support? Again, the rationale for auditing patch management isn't to simply 'check the box' but to cite and penalize behavior that results in a less secure or resilient system.
In response to the previous comment: "Why would an Entity buy and then install COTS software or hardware that the vendor does not support? … how does the Regional Entity expect the level of security to improve by not citing EOL software or hardware as a security risk?"
ReplyDeleteI fear the comment confuses cyber security and compliance. The two serve a singular goal but with entirely different vectors. It also highlights a here-to-fore "control-center-centric" view of the CIP Compliance that no longer jives with the reality of V5 changes. The paradigm is completely understandable (mea culpa, btw) just based upon the systems most affected by Versions 1 - 3. The lion's share of systems coming into compliance, from my perspective at least, are large transmission resources. While the mostly IT-related EMS systems of control centers upgrade on the common IT basis of 3 to 5 years, the designs of highly tailored systems integrated into a large HVDC expect lifespans of 15 to 30 years.
An argument may exist here about cyber security, but not under the language of CIP-007-5. On the opposite side, I would point to ATM machines. The vast majority of reported security exploits at the ATM revolve around skimmers, not software vulnerabilities. Yet, as far as I know, the majority of ATMs still run OS/2 2.1 as a core operating system.
Love it or hate it, compliance must be to the Standards, not to best cyber security practices or even wishful thinking. Case in point, as a regional auditor (not anyone Tom recently mentioned), I once had to write a possible violation to an entity with an excellent whitelisting infrastructure. It was wonderfully secure but still sadly non-compliant, as they had neither antivirus nor technical feasibility exceptions. Before decrying my team's actions as ridiculous, we must first face facts. The industry provided resources to form the Standards Drafting Teams, those teams made the--let's face it--poor choice of prescribing a particular technology in a requirement rather than identifying a security control. Since then, we've all learned and Version 5 says "malware prevention," which establishes a control not a technology.
If my state established a law that required all automobiles painted black, there's no doubt I would work hard to change it; but if I persisted in driving my white vehicle, a traffic ticket should come as no great surprise.
I believe the term "Patch Management" also misleads. It confuses people into believing that compliance must do something technical with software updates. It really doesn't, or at best, only marginally and after-the-fact. A better term would be something like "Security Update Tracking" or "Patch Information Evidence" or anything else that helps the user understand that any "Management" aspects of patching should actually (and only) happen under CIP-010.