It was already becoming apparent
to me that the scale of the damage that could be caused by the SolarWinds
attacks is of an order of magnitude larger than the damage caused by any
previous cyberattack, including NotPetya and Wannacry. Today I got more
evidence that this is indeed the case.
Damage to SolarWinds itself
Today I had an email discussion
with someone I often discuss cybersecurity issues with. I mentioned that all Orion
users probably need to simply disconnect the system, as CISA has already
ordered government users to do. I pointed out that it may take many months for
SolarWinds to fix the problems that led to the attacks (i.e. the attacks on
their customers, courtesy of the backdoor that SolarWinds unwittingly pushed
out in a patch).
I also speculated that many customers
won’t want to wait that long to be able to manage their networks, meaning they
might abandon SW before they’re able to fix the problem. The company may
suddenly find themselves in a severe cash flow problem and might have to seek
bankruptcy protection – not even considering the lawsuits that will soon start
to flow (and the revelation
that an update server was accessible with a password of “solarwinds123” won’t
help their position in lawsuits, even though that probably didn’t have an
impact on the breach itself. However, it certainly speaks to their mindset
regarding security).
My friend clearly thought I was
exaggerating. He acknowledged that the attackers had probably taken advantage
of the 6 months or so that they’ve lain in networks undiscovered to move
laterally within the organizations affected (including the State Department,
DoE, FERC and DHS) and plant all sorts of malware in other networks within the organization,
as well as exfiltrated mountains of data.
But as for Orion itself, he
thought the problem was like other software compromises: The supplier just
needs to get all of their customers back running a known clean version, and
make sure they keep their nose clean going forward. Of course, this alone would
be a massive project, but at least it would be a bounded one. If SW needed to
borrow money to fund this, they could probably do it, because the end would be
clearly in sight, no matter how distant.
However, I pointed out to him that
there’s no way SW could get their customers to install anything that
they send out now: a “clean” version of the software, a patch, a chocolate chip
cookie recipe, etc. Their software build process has been compromised,
and evidently for a long time. Moreover, they clearly have no idea how it
happened, since they haven’t been able to provide any good explanation (of
course, this is the worst thing a company can do when they’re hacked).
In fact, I’m not sure I know of
any official statement from them in the last week, other than their SEC filing
that hoped to smooth things over by pointing out that “fewer than” 18,000 of
their customers had applied the infected patch. Of course, it’s very reassuring
that at most there are no more than 18,000 direct victims of this attack, rather
than say 18 million.
So SolarWinds has to rip their
entire build process apart and reconstruct it from scratch, make sure it’s
working properly and is secure, and finally start providing code to their
remaining customers (both of them) again. But even that won’t give their
customers confidence, unless there are wholesale changes in management, and
especially their internal security team. First, someone from outside will need
to come in, figure out which of their management are part of the problem and
which are part of the solution (and it might be hard to identify the latter). Then
they will need to bring in their new team. And only then can the company be
trusted to put in place a new development environment that is unlikely to be
compromised.
The bottom line on this is it will
be a loooong time before SolarWinds will be able to get back to business as
usual. Whether they’ll still have any customers at all at that time is
questionable.
Damage to everybody else
So SolarWinds has got a long road to
travel before they can be at anywhere close to a “normal” state again. What
about everyone else?
Slate.com put out an excellent article
on Friday titled “The SolarWinds Hack Is Unlike Anything We Have Ever Seen
Before”. The first paragraph contains these two sentences: “In the coming year,
we won’t just be fighting about who was responsible or figuring out how this
happened or assessing the fallout or repairing affected systems. That whole
time, government and private sector systems will continue to actively be
breached because of the malware that was surreptitiously included in updates to
the SolarWinds Orion products.”
The article goes on to point out
that in previous big attacks, like Equifax, Sony Pictures, and the Office of
Personnel Management, there were clear targets and clear sources of the
attacks. If you know the source of the attacks, you can trace, in any
organization attacked, what that source did, including what data they accessed,
what user accounts they compromised, what suspicious software they installed,
etc. You can “be relatively confident the incident was confined to a particular
department or target system and that wiping and restoring those systems would
be sufficient to remove the intruders’ presence.” This would of course take a
lot of work, but again it would be a bounded problem.
But since Orion servers have either
direct or indirect access to everything on the network (direct in the case of
switches, routers, firewalls, etc. but indirect in the case of servers and
workstations – see this
post for an explanation of the difference between these two cases), literally
everything on the network could have been attacked – or could still be
attacked.
The article turns the screw once
again: “Even more worrisome is the fact that the attackers apparently made use
of their initial access to targeted organizations, such as FireEye and
Microsoft, to steal tools and code that would then enable them to compromise
even more targets.” In other words, there’s no reason to believe the Russians haven’t
penetrated – or still could penetrate - a lot more organizations than the “less
than” 18,000 SolarWinds customers they have probably already penetrated.
And there’s another big group of organizations
that may already be compromised, or could still be compromised: that’s the
customers and business partners of all those other organizations: i.e. the Early
18,000, plus all of the companies compromised with the stolen FireEye and
Microsoft tools. And perhaps the customers and business partners of those
customers and business partners, etc. etc. The author states “So when I say the
SolarWinds cyberespionage campaign will last years, I don’t just mean, as I
usually do, that figuring out liability and settling costs and carrying out
investigations will take years (though that is certainly true here). The
actual, active theft of information from protected networks due to this breach
will last years.”
Yet the author turns the screw
again, revealing another whole dimension to the problem: “Another element
adding to the challenge of trying to clean up this mess will be the
thoroughness of the compromise of each individual system. Many cyberespionage
activities begin with phishing campaigns or stolen credentials, which are then
used to deliver malware to targeted systems. Those credentials, depending on
whom they belong to and how much access that individual has, can be very
effective ways to gain a toehold in a protected computer system, but they’re
also very easy to change or reset when the compromise is discovered.”
However, “none of that matters
with a breach like SolarWinds that granted intruders broad access to the entire
network of every system it was installed on. Additionally, SolarWinds had
apparently persuaded many of its customers that its Orion products needed to be
exempt from existing antivirus and security restrictions on their computers
because otherwise it might look like a threat or be unable to function properly[i].”
The author concludes by saying “So
the access that the intruders had using the SolarWinds updates goes far beyond
the access granted by many initial cyberespionage compromises, and the number
of potential targets is enormous—and only growing every time we learn about the
ways that each of those targets may have been leveraged to access new victims.
As we continue to unravel all the different strands of this compromise, the
federal government would do well to assume that its computer systems are still
being actively infiltrated and not imagine that, simply having discovered this
breach, they are anywhere close to reaching the end of it.”
Other than that, I’d say we’re in
pretty good shape…
Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at tom@tomalrich.com.
[i]
The author points out that SolarWinds wasn’t unique among network management
and security vendors in requesting this be done.
No comments:
Post a Comment