Posts Tagged: SolarWinds breach


12
Jan 21

SolarWinds: What Hit Us Could Hit Others

New research into the malware that set the stage for the megabreach at IT vendor SolarWinds shows the perpetrators spent months inside the company’s software development labs honing their attack before inserting malicious code into updates that SolarWinds then shipped to thousands of customers. More worrisome, the research suggests the insidious methods used by the intruders to subvert the company’s software development pipeline could be repurposed against many other major software providers.

In a blog post published Jan. 11, SolarWinds said the attackers first compromised its development environment on Sept. 4, 2019. Soon after, the attackers began testing code designed to surreptitiously inject backdoors into Orion, a suite of tools used by many Fortune 500 firms and a broad swath of the federal government to manage their internal networks.

Image: SolarWinds.

According to SolarWinds and a technical analysis from CrowdStrike, the intruders were trying to work out whether their “Sunspot” malware — designed specifically for use in undermining SolarWinds’ software development process — could successfully insert their malicious “Sunburst” backdoor into Orion products without tripping any alarms or alerting Orion developers.

In October 2019, SolarWinds pushed an update to their Orion customers that contained the modified test code. By February 2020, the intruders had used Sunspot to inject the Sunburst backdoor into the Orion source code, which was then digitally signed by the company and propagated to customers via SolarWinds’ software update process.

Crowdstrike said Sunspot was written to be able to detect when it was installed on a SolarWinds developer system, and to lie in wait until specific Orion source code files were accessed by developers. This allowed the intruders to “replace source code files during the build process, before compilation,” Crowdstrike wrote.

The attackers also included safeguards to prevent the backdoor code lines from appearing in Orion software build logs, and checks to ensure that such tampering wouldn’t cause build errors.

“The design of SUNSPOT suggests [the malware] developers invested a lot of effort to ensure the code was properly inserted and remained undetected, and prioritized operational security to avoid revealing their presence in the build environment to SolarWinds developers,” CrowdStrike wrote.

A third malware strain — dubbed “Teardrop” by FireEye, the company that first disclosed the SolarWinds attack in December — was installed via the backdoored Orion updates on networks that the SolarWinds attackers wanted to plunder more deeply.

So far, the Teardrop malware has been found on several government networks, including the Commerce, Energy and Treasury departments, the Department of Justice and the Administrative Office of the U.S. Courts.

SolarWinds emphasized that while the Sunspot code was specifically designed to compromise the integrity of its software development process, that same process is likely common across the software industry.

“Our concern is that right now similar processes may exist in software development environments at other companies throughout the world,” said SolarWinds CEO Sudhakar Ramakrishna. “The severity and complexity of this attack has taught us that more effectively combatting similar attacks in the future will require an industry-wide approach as well as public-private partnerships that leverage the skills, insight, knowledge, and resources of all constituents.”


7
Jan 21

Sealed U.S. Court Records Exposed in SolarWinds Breach

The ongoing breach affecting thousands of organizations that relied on backdoored products by network software firm SolarWinds may have jeopardized the privacy of countless sealed court documents on file with the U.S. federal court system, according to a memo released Wednesday by the Administrative Office (AO) of the U.S. Courts.

The judicial branch agency said it will be deploying more stringent controls for receiving and storing sensitive documents filed with the federal courts, following a discovery that its own systems were compromised as part of the SolarWinds supply chain attack. That intrusion involved malicious code being surreptitiously inserted into updates shipped by SolarWinds for some 18,000 users of its Orion network management software as far back as March 2020.

“The AO is working with the Department of Homeland Security on a security audit relating to vulnerabilities in the Judiciary’s Case Management/Electronic Case Files system (CM/ECF) that greatly risk compromising highly sensitive non-public documents stored on CM/ECF, particularly sealed filings,” the agency said in a statement published Jan. 6.

“An apparent compromise of the confidentiality of the CM/ECF system due to these discovered vulnerabilities currently is under investigation,” the statement continues. “Due to the nature of the attacks, the review of this matter and its impact is ongoing.”

The AO declined to comment on specific questions about their breach disclosure. But a source close to the investigation told KrebsOnSecurity that the federal court document system was “hit hard,” by the SolarWinds attackers, which multiple U.S. intelligence and law enforcement agencies have attributed as “likely Russian in origin.”

The source said the intruders behind the SolarWinds compromise seeded the AO’s network with a second stage “Teardrop” malware that went beyond the “Sunburst” malicious software update that was opportunistically pushed out to all 18,000 customers using the compromised Orion software. This suggests the attackers were targeting the agency for deeper access to its networks and communications.

The AO’s court document system powers a publicly searchable database called PACER, and the vast majority of the files in PACER are not restricted and are available to anyone willing to pay for the records.

But experts say many other documents stored in the AO’s system are sealed — either temporarily or indefinitely by the courts or parties to a legal matter — and may contain highly sensitive information, including intellectual property and trade secrets, or even the identities of confidential informants.

Nicholas Weaver, a lecturer at the computer science department at University of California, Berkeley, said the court document system doesn’t hold documents that are classified for national security reasons. But he said the system is full of sensitive sealed filings — such as subpoenas for email records and so-called “trap and trace” requests that law enforcement officials use to determine with whom a suspect is communicating via phone, when and for how long.

“This would be a treasure trove for the Russians knowing about a lot of ongoing criminal investigations,” Weaver said. “If the FBI has indicted someone but hasn’t arrested them yet, that’s all under seal. A lot of the investigative tools that get protected under seal are filed very early on in the process, often with gag orders that prevent [the subpoenaed party] from disclosing the request.”

The acknowledgement from the AO comes hours after the U.S. Justice Department said it also was a victim of the SolarWinds intruders, who took control over the department’s Office 365 system and accessed email sent or received from about three percent of DOJ accounts (the department has more than 100,000 employees). Continue reading →


16
Dec 20

Malicious Domain in SolarWinds Hack Turned into ‘Killswitch’

A key malicious domain name used to control potentially thousands of computer systems compromised via the months-long breach at network monitoring software vendor SolarWinds was commandeered by security experts and used as a “killswitch” designed to turn the sprawling cybercrime operation against itself, KrebsOnSecurity has learned.

Austin, Texas-based SolarWinds disclosed this week that a compromise of its software update servers earlier this year may have resulted in malicious code being pushed to nearly 18,000 customers of its Orion platform. Many U.S. federal agencies and Fortune 500 firms use(d) Orion to monitor the health of their IT networks.

On Dec. 13, cyber incident response firm FireEye published a detailed writeup on the malware infrastructure used in the SolarWinds compromise, presenting evidence that the Orion software was first compromised back in March 2020. FireEye said hacked networks were seen communicating with a malicious domain name — avsvmcloud[.]com — one of several domains the attackers had set up to control affected systems.

As first reported here on Tuesday, there were signs over the past few days that control over the domain had been transferred to Microsoft. Asked about the changeover, Microsoft referred questions to FireEye and to GoDaddy, the current domain name registrar for the malicious site.

Today, FireEye responded that the domain seizure was part of a collaborative effort to prevent networks that may have been affected by the compromised SolarWinds software update from communicating with the attackers. What’s more, the company said the domain was reconfigured to act as a “killswitch” that would prevent the malware from continuing to operate in some circumstances.

“SUNBURST is the malware that was distributed through SolarWinds software,” FireEye said in a statement shared with KrebsOnSecurity. “As part of FireEye’s analysis of SUNBURST, we identified a killswitch that would prevent SUNBURST from continuing to operate.”

The statement continues:

“Depending on the IP address returned when the malware resolves avsvmcloud[.]com, under certain conditions, the malware would terminate itself and prevent further execution. FireEye collaborated with GoDaddy and Microsoft to deactivate SUNBURST infections.”

“This killswitch will affect new and previous SUNBURST infections by disabling SUNBURST deployments that are still beaconing to avsvmcloud[.]com. However, in the intrusions FireEye has seen, this actor moved quickly to establish additional persistent mechanisms to access to victim networks beyond the SUNBURST backdoor.

This killswitch will not remove the actor from victim networks where they have established other backdoors. However, it will make it more difficult to for the actor to leverage the previously distributed versions of SUNBURST.”

It is likely that given their visibility into and control over the malicious domain, Microsoft, FireEye, GoDaddy and others now have a decent idea which companies may still be struggling with SUNBURST infections.

The killswitch revelations came as security researchers said they’d made progress in decoding SUNBURST’s obfuscated communications methods. Chinese cybersecurity firm RedDrip Team published their findings on Github, saying its decoder tool had identified nearly a hundred suspected victims of the SolarWinds/Orion breach, including universities, governments and high tech companies.

Meanwhile, the potential legal fallout for SolarWinds in the wake of this breach continues to worsen. The Washington Post reported Tuesday that top investors in SolarWinds sold millions of dollars in stock in the days before the intrusion was revealed. SolarWinds’s stock price has fallen more than 20 percent in the past few days. The Post cited former enforcement officials at the U.S. Securities and Exchange Commission (SEC) saying the sales were likely to prompt an insider trading investigation.


15
Dec 20

SolarWinds Hack Could Affect 18K Customers

The still-unfolding breach at network management software firm SolarWinds may have resulted in malicious code being pushed to nearly 18,000 customers, the company said in a legal filing on Monday. Meanwhile, Microsoft should soon have some idea which and how many SolarWinds customers were affected, as it recently took possession of a key domain name used by the intruders to control infected systems.

On Dec. 13, SolarWinds acknowledged that hackers had inserted malware into a service that provided software updates for its Orion platform, a suite of products broadly used across the U.S. federal government and Fortune 500 firms to monitor the health of their IT networks.

In a Dec. 14 filing with the U.S. Securities and Exchange Commission (SEC), SolarWinds said roughly 33,000 of its more than 300,000 customers were Orion customers, and that fewer than 18,000 customers may have had an installation of the Orion product that contained the malicious code. SolarWinds said the intrusion also compromised its Microsoft Office 365 accounts.

The initial breach disclosure from SolarWinds came five days after cybersecurity incident response firm FireEye announced it had suffered an intrusion that resulted in the theft of some 300 proprietary software tools the company provides to clients to help secure their IT operations.

On Dec. 13, FireEye published a detailed writeup on the malware infrastructure used in the SolarWinds compromise, presenting evidence that the Orion software was first compromised back in March 2020. FireEye didn’t explicitly say its own intrusion was the result of the SolarWinds hack, but the company confirmed as much to KrebsOnSecurity earlier today.

Also on Dec. 13, news broke that the SolarWinds hack resulted in attackers reading the email communications at the U.S. Treasury and Commerce departments.

On Dec. 14, Reuters reported the SolarWinds intrusion also had been used to infiltrate computer networks at the U.S. Department of Homeland Security (DHS). That disclosure came less than 24 hours after DHS’s Cybersecurity and Infrastructure Security Agency (CISA) took the unusual step of issuing an emergency directive ordering all federal agencies to immediately disconnect the affected Orion products from their networks.

ANALYSIS

Security experts have been speculating as to the extent of the damage from the SolarWinds hack, combing through details in the FireEye analysis and elsewhere for clues about how many other organizations may have been hit.

And it seems that Microsoft may now be in perhaps the best position to take stock of the carnage. That’s because sometime on Dec. 14, the software giant took control over a key domain name — avsvmcloud[.]com — that was used by the SolarWinds hackers to communicate with systems compromised by the backdoored Orion product updates.

Armed with that access, Microsoft should be able to tell which organizations have IT systems that are still trying to ping the malicious domain. However, because many Internet service providers and affected companies are already blocking systems from accessing that malicious control domain or have disconnected the vulnerable Orion services, Microsoft’s visibility may be somewhat limited.

Microsoft has a long history of working with federal investigators and the U.S. courts to seize control over domains involved in global malware menaces, particularly when those sites are being used primarily to attack Microsoft Windows customers.

Microsoft dodged direct questions about its visibility into the malware control domain, suggesting those queries would be better put to FireEye or GoDaddy (the current domain registrar for the malware control server). But in a response on Twitter, Microsoft spokesperson Jeff Jones seemed to confirm that control of the malicious domain had changed hands. Continue reading →


14
Dec 20

U.S. Treasury, Commerce Depts. Hacked Through SolarWinds Compromise

Communications at the U.S. Treasury and Commerce Departments were reportedly compromised by a supply chain attack on SolarWinds, a security vendor that helps the federal government and a range of Fortune 500 companies monitor the health of their IT networks. Given the breadth of the company’s customer base, experts say the incident may be just the first of many such disclosures.

Some of SolarWinds’ customers. Source: solarwinds.com

According to a Reuters story, hackers believed to be working for Russia have been monitoring internal email traffic at the U.S. Treasury and Commerce departments. Reuters reports the attackers were able to surreptitiously tamper with updates released by SolarWinds for its Orion platform, a suite of network management tools.

In a security advisory, Austin, Texas based SolarWinds acknowledged its systems “experienced a highly sophisticated, manual supply chain attack on SolarWinds Orion Platform software builds for versions 2019.4 HF 5 through 2020.2.1, released between March 2020 and June 2020.”

In response to the intrusions at Treasury and Commerce, the Department of Homeland Security’s Cybersecurity and Infrastructure Security Agency (CISA) took the unusual step of issuing an emergency directive ordering all federal agencies to immediately disconnect the affected Orion products from their networks.

“Treat all hosts monitored by the SolarWinds Orion monitoring software as compromised by threat actors and assume that further persistence mechanisms have been deployed,” CISA advised.

A blog post by Microsoft says the attackers were able to add malicious code to software updates provided by SolarWinds for Orion users. “This results in the attacker gaining a foothold in the network, which the attacker can use to gain elevated credentials,” Microsoft wrote.

From there, the attackers would be able to forge single sign-on tokens that impersonate any of the organization’s existing users and accounts, including highly privileged accounts on the network.

“Using highly privileged accounts acquired through the technique above or other means, attackers may add their own credentials to existing application service principals, enabling them to call APIs with the permission assigned to that application,” Microsoft explained.

Malicious code added to an Orion software update may have gone undetected by antivirus software and other security tools on host systems thanks in part to guidance from SolarWinds itself. In this support advisory, SolarWinds says its products may not work properly unless their file directories are exempted from antivirus scans and group policy object restrictions. Continue reading →