The ongoing breach affecting thousands of organizations that relied on backdoored products by network software firm SolarWinds may have jeopardized the privacy of countless sealed court documents on file with the U.S. federal court system, according to a memo released Wednesday by the Administrative Office (AO) of the U.S. Courts.
The judicial branch agency said it will be deploying more stringent controls for receiving and storing sensitive documents filed with the federal courts, following a discovery that its own systems were compromised as part of the SolarWinds supply chain attack. That intrusion involved malicious code being surreptitiously inserted into updates shipped by SolarWinds for some 18,000 users of its Orion network management software as far back as March 2020.
“The AO is working with the Department of Homeland Security on a security audit relating to vulnerabilities in the Judiciary’s Case Management/Electronic Case Files system (CM/ECF) that greatly risk compromising highly sensitive non-public documents stored on CM/ECF, particularly sealed filings,” the agency said in a statement published Jan. 6.
“An apparent compromise of the confidentiality of the CM/ECF system due to these discovered vulnerabilities currently is under investigation,” the statement continues. “Due to the nature of the attacks, the review of this matter and its impact is ongoing.”
The AO declined to comment on specific questions about their breach disclosure. But a source close to the investigation told KrebsOnSecurity that the federal court document system was “hit hard,” by the SolarWinds attackers, which multiple U.S. intelligence and law enforcement agencies have attributed as “likely Russian in origin.”
The source said the intruders behind the SolarWinds compromise seeded the AO’s network with a second stage “Teardrop” malware that went beyond the “Sunburst” malicious software update that was opportunistically pushed out to all 18,000 customers using the compromised Orion software. This suggests the attackers were targeting the agency for deeper access to its networks and communications.
The AO’s court document system powers a publicly searchable database called PACER, and the vast majority of the files in PACER are not restricted and are available to anyone willing to pay for the records.
But experts say many other documents stored in the AO’s system are sealed — either temporarily or indefinitely by the courts or parties to a legal matter — and may contain highly sensitive information, including intellectual property and trade secrets, or even the identities of confidential informants.
Nicholas Weaver, a lecturer at the computer science department at University of California, Berkeley, said the court document system doesn’t hold documents that are classified for national security reasons. But he said the system is full of sensitive sealed filings — such as subpoenas for email records and so-called “trap and trace” requests that law enforcement officials use to determine with whom a suspect is communicating via phone, when and for how long.
“This would be a treasure trove for the Russians knowing about a lot of ongoing criminal investigations,” Weaver said. “If the FBI has indicted someone but hasn’t arrested them yet, that’s all under seal. A lot of the investigative tools that get protected under seal are filed very early on in the process, often with gag orders that prevent [the subpoenaed party] from disclosing the request.”
The acknowledgement from the AO comes hours after the U.S. Justice Department said it also was a victim of the SolarWinds intruders, who took control over the department’s Office 365 system and accessed email sent or received from about three percent of DOJ accounts (the department has more than 100,000 employees).
The SolarWinds hack also reportedly jeopardized email systems used by top Treasury Department officials, and granted the attackers access to networks inside the Energy, Commerce and Homeland Security departments.
The New York Times on Wednesday reported that investigators are examining whether a breach at another software provider — JetBrains — may have precipitated the attack on SolarWinds. The company, which was founded by three Russian engineers in the Czech Republic, makes a tool called TeamCity that helps developers test and manage software code. TeamCity is used by developers at 300,000 organizations, including SolarWinds and 79 of the Fortune 100 companies.
“Officials are investigating whether the company, founded by three Russian engineers in the Czech Republic with research labs in Russia, was breached and used as a pathway for hackers to insert back doors into the software of an untold number of technology companies,” The Times said. “Security experts warn that the monthslong intrusion could be the biggest breach of United States networks in history.”
Under the AO’s new procedures, highly sensitive court documents filed with federal courts will be accepted for filing in paper form or via a secure electronic device, such as a thumb drive, and stored in a secure stand-alone computer system. These sealed documents will not be uploaded to CM/ECF.
“This new practice will not change current policies regarding public access to court records, since sealed records are confidential and currently are not available to the public,” the AO said.
James Lewis, senior vice president at the Center for Strategic and International Studies, said it’s too soon to tell the true impact of the breach at the court system, but the fact that they were apparently targeted is a “a very big deal.”
“We don’t know what the Russians took, but the fact that they had access to this system means they had access to a lot of great stuff, because federal cases tend to involve fairly high profile targets,” he said.
Sadly a lot of that data can be monetized by really evil folks. Extortion or simply offering someone a look at what the feds are doing.
Any chance they got Trump’s tax returns?
Didn’t the IRS leak those? There was a whole lot of nothing wrong there…
The Government is suppose to be opening up that PACER database to the public as a free service with no charges. I don’t know how much truth is in from what I read on Bleeping Computers.
You did notice the article said “sealed records”, right?
You do know what “sealed” means in this context, right?
No? Think (or google) before you speak.
It also said:
“the vast majority of the files in PACER are not restricted and are available to anyone willing to pay for the records.”
Seems his comment was valid
Valid perhaps, but unrelated to the topic of sealed records.
But still related to the article. Stop being dense.
Thanks for these recent posts, Brian.
We all need to be concerned about the provenance of all of our software and hardware tools – network switches, monitoring, build systems, backups, development environments.
I’m a frequent user of Jetbrains products but also use many from other countries without knowing who controls the software or firmware.
My most recent concern is with a fairly large number of backup companies that have ties to the Soviet Union orbit. Some of these companies also put themselves in charge malware detection. Should I be concerned?
“have ties to the Soviet Union orbit”. Our brains are tainted in that Natasha is after Squirrel, even now. Just as likely China. Attribution is hard and the 3 letter guys haven’t used the ‘highly likely’ signal yet..
No doubt about it. I gotta get me another hat.
This breach grows deeper and deeper…the breadth is really stunning.
As in all military Ops, C3I is executed upon the victim with diversion tactics, Stay Woke:
Command
Communication
Control
& Intelligence
Do you have a mobile app or plan to create one for easy access to your news and investigations?
Use the RSS feed
This implies that any informants working with the FBI against organized crime have been exposed.
I hope that I am incorrect, but I seriously doubt it.
…no, aouscourts does not have access to fbi records, informants are not stored there and never have been…
Maybe not informants (I don’t know), but certainly witness statements and depositions are under seal. Can you imagine if maybe a witness were to have an “accident” prior to trial?
Hopefully they printed these off, so that when Hillary, baracky boy obama, nancy PIGLOSI, chuck chuck schumer, and others pay chinese hackers to mess with these documents, it won’t matter.
And so the sealed indictments against Donald Trump don’t get lost.
The breach was on Trump’s watch.
He didn’t even say anything about it, so butt-hurt about twitter.
Thanks for playing politics, but you lost. No need for tears,
grow up someday.
Some of these records could give advance notice to the perpetrators that one or more of their people could be under investigation. That would allow them to get their agents out of the country before they are arrested.
It still stuns me that there is no Federal agency that requires software on government computers have source code open to them and analyzed. Updates should only be allowed from government controlled proxies.
…you have no idea what you’re talking about…
…aouscourts does not have a record until a case goes to court, i. e., an indictment if it’s a crime, or a lawsuit if it’s a civil action…
…investigation data before that is never at aouscourts.,..
I guess I took
“full of sensitive sealed filings — such as subpoenas for email records and so-called “trap and trace” requests that law enforcement officials use to determine with whom a suspect is communicating via phone, when and for how long.
“This would be a treasure trove for the Russians knowing about a lot of ongoing criminal investigations,” Weaver said. “If the FBI has indicted someone but hasn’t arrested them yet, that’s all under seal. A lot of the investigative tools that get protected under seal are filed very early on in the process, often with gag orders that prevent [the subpoenaed party] from disclosing the request.”
too literally. My bad.
…he’s not the expert he thinks he is obviously…
All updates should be done locally, off the network.
The government should not do updates like microsoft does.
Some of the information may have allowed the perpetrators to see if any of their agents were under investigation. This would have helped them get them out of the country before arrest.
It still stuns me that there is no Federal agency that requires software providers also provide source code and that analyzes it for threats. Updates should be only to government proxy servers and then allowed internally after analysis. Or get off the cloud!
I’d like to hear a lot more about attribution, because there’s more than one group of people interested in access to that info. US intel is the most potent hacker out there and they’re masters at obfuscating their presence. And why would the Russians be so stupid as to leave RATS lying around when they’re perfectly capable of implementing in-memory malware with little or no detectable hard drive persistence? They had a Golden Ticket hack and exposed it by sloppy exploit activity? And, wow, why was this information even accessible via internet? The value is so obviously high to so many that handling the information cries out for more secure management. Who can you believe anymore?
Attribution is indeed hard. But not impossible.
It takes work and a lot of people putting the pieces together.
TTPs have been a valuable component for deriving attribution. Analysts in both public and private sectors have gotten quite good at attribution down the the APT group level.
So we could either have some trust in the experts who do this for a living… or wait for it to be released/leaked like from the Reality Winner dump, which confirmed the attribution and depth of Russian intrusion previously.
Sorry, there were several agencies concerned with the safety of software and operating systems prior to our current cost cutting moves. Those were cost cuttingly combined into one agencie. And a political person placed in charge, rather then someone with best practices in mind. That’s how you sink a ship of state. But that is where we are now. We have to find the best and brightest, and get them redesigning our infrastructure. And as in all redesigns, it will take longer then needed, it’s not a one or two year, but a multi generational program to safety our systems. And, we only have heard of the public side, did they get into our secret squirrel stuff? We would never hear.?
I’m no computer expert, but, I always say: If it is important data, keep it off the network.
When the IT guy wanted to put my computer onto his network, I lost a hard drive.
The second time it was requested, I said no, as it was my computer.
Never lost any of my data, but the server drive crashed…I warned that it was going to happen, and it did: lost 2 weeks of data, computer guy had to work many hours to fix it.
The boss learned a big lesson: Listen to your employees!
PACER = Public Access to Court Electronic Records
…with an emphasis on Public Access…
…pure speculation by non-experts about access to so-called sealed records…
Do they not understand what a sealed record is? Seriously?
Sounds like your IT guy is not up to snuff. No offense to you while employee input should be considered it should be IT and Management should be the final decision makers on such matters. If you IT guy was doing his job and or being listened to a backup and restoration plans should be in place for all network based data. In a situation where there are more then a handful of computers individual backups become impractical.
Any open source software is involved in all the breach?
Brian Krebs is one of the few in media I still trust. More likely than not, the information the article in this post is accurate.
One responder who said that a database list of FBI informants is not on the AO’s systems is correct. But matters related to getting search warrants approved by Federal judges, including the names of informants related to those warrants, as well as various people involved in a sealed indictment may well be in the system.
This all happened under your watch, no?
Who are you referring to?
Wrong Krebs, the other fellow which you are referring to, worked for CISA. CISA puts out alerts and is not the entity, or rather the multiple entities, who are responsible for the security of the departments penetrated.
There are multiple agencies responsible for protecting the compromised systems. The espionage campaign was designed to not trigger early warning systems placed into foreign networks that alert intelligence agencies to an attack. Instead it targeted a software vendor (SolarWinds) and other 3rd parities who supplied the government with their products.
The hack started in SolarWinds, and possibly their development platform (still being investigated), also bypassing their internal detection and warning systems.
Once it penetrated those systems, the exploit was delivered through a software update into government systems and other SolarWinds customer’s networks.
This then allowed a backdoor to be opened that the nation state operatives bypassed security with and could remain undetected while they moved laterally through the network to gather and exfiltrate intelligence.
The other Krebs (of CISA) was a convenient target for blame and undeserving of it.
The hack went undetected by the entire US intelligence, signals and security community, aided by a divisive and ill prepared White House administration, who continued to ignore the warnings of the skilled adversary they faced from the GRU, and especially the SVR.
So no, it did not happen under this Kreb’s watch (especially not this Chis Krebs), or even the other Kreb’s watch.
And if you are wondering how they managed to get away with it, once they have penetrated the network…
“The actor sets the hostnames on their command and control infrastructure to match a legitimate hostname found within the victim’s environment. This allows the adversary to blend into the environment, avoid suspicion, and evade detection.”
They also used Golden SAML, a type of technique often referred to as a ‘Golden Ticket’, and other techniques, including a previously unseen memory dropper (now coined TEARDROP) to also evade detection.
The other Chris Krebs, formerly of CISA, was fired by Trump for correcting voter fraud disinformation.
…Not at all why he was fired, but way to make it political with misinformation.
Yes, it was.
Chris Krebs was fired hours after contradicting the president regarding false claims of fraud. A full month before the Solarwinds compromise was known. If Krebs was fired because of Solarwinds… then the president knew of a major hack and didn’t tell anyone?
Yes, actually he was fired directly after that disclosure.
There was no other reason given. You didn’t even try.
Spreading pseudo-disinformation is your accusation, lol?
That’s like Trump calling you fat. Sure guy.
see the following article regarding why the other Chris Krebs was fired, (who is not the same person as Chris Krebs of Krebs On Security).
https://www.npr.org/2020/11/17/936003057/cisa-director-chris-krebs-fired-after-trying-to-correct-voter-fraud-disinformati
Krebs On Security is Brian Krebs.
Brian “Chris” Krebs. You know, like Ruth’s Chris Steakhouse.
Haha!
Security Vet – this appears to be a documented Supply Chain Attack could you please help me understand how we are using code and or test capabilities developed outside the US. Thought there were rules/guidance/laws that prevented this? What am I missing?
…the US gov buys things made outside the US all the time, lowest price, speed of delivery, etc…
…even if the product was made in the USA, the perps got into it over the internet which is global…
…and update servers are typically AWS or Akami and are also global…
Main problem with this seems to be what happen when the code is closed and not public. No one can know what the code inside does except for the programmers themselves, so you literally transfer your security measures to them.
Seems to be a fair choice to do in the private sector, but it seems that doing it as a public agency has a whole different significance, and I repeat, there is no way to control what the software developers put in there, or miss to check, when the code isn’t open.
…well, yes and no…
…yes, we are turning much over to the s/w developers, open or closed, which is both good and bad depending on how much and what kind of testing goes on…
…open s/w can be made very insecure and closed proprietary can be very secure, depends on how skilled the user is and how much and what kind of testing goes on…
…in the case of solarwinds et al too much trust was given to them and it burned us…
I can;t speak to Solarwinds specifically but I’ve worked with and for a business software vendor and it very much concerned me how little concern the coders seem to have with regards to the security aspect of the product. That’s not to say they didn’t care about security only that it was too low on the priority list for them IMHO especially when your software deals with accounting/finance.
This was a very interesting piece to read. Glad I found the site (thanks to a co-worker)
But the main difference is that with the open source code, any organization can make their own evaluation, where with closed only the company that make the software can make that assessment. Both can be secure or insecure, and the users still a key part of it, but only one gives an organization the necessary info to assess the real risk.
…sure, if the org has the skill to evaluate the software they have, make a risk judgement in their environment, etc., which many don’t…
Brian, I still remember your prior observation on
“Immutable Truths About Hacking and Information on the Internet”
“-If you connect it to the Internet, someone will try to hack it.
-If what you put on the Internet has value, someone will invest time and effort to steal it. . . .”
So true, so true …
Gotta love the email message from Ubiquiti. It has many of the characteristics of a phishing scam email:
– Generic “Dear Customer” salutation TICK
– Conveys a sense of urgency TICK
– Links go to a site that is not “ui.com” TICK
– Requests a password reset TICK
Spot on! Why do (probably) legit companies still include links in their emails at all and why do mail programs still render them as clickable and even allow the URL to differ from the label text? I know it’s a minor pain to have to navigate to the site yourself but these days it is entirely necessary.
It seems that the Solorwinds hack has penetrated PACER, which is used by every lawyer. The attack occurred to the end user (the lawyers) when they initiated an account change. the infection was sent and then, boom. Everything not backed up, 🙁
Since this ransomware is “Crysis” variant with a .GITF extension, it is new.
BE CAREFUL, ANY message from PACER could be compromised.
There’s a reason DOD’s (and others’) security requirements include “no network connections.”
You can do at least that at home for financial and other personal records (tax) by running a disconnected system that, once running stably, is NEVER UPDATED. (You are not going to be able to do the certification required before updating so let it be.)
Have a connected system for browsing and assume it it as leaky as a sieve. Assume updates are malicious. Never put anything on there you wouldn’t want on the front page of the Washington Post.
Never updating is a stupid idea.
Just because something is stable, doesn’t mean it’s not vulnerable.
DOD requirements don’t say, “no connections”.
Yes, air gaps are useful, but there are other ways to compromise. Instead, we use something called Defense in Depth… Rather than naive and simplistic rules like you suggest.
Even the most secure, top secret systems, have updates and patch management programs.
Anyone not patching just because they fear supply chain compromise of updates… Will be fired for good cause.
Although major supply chain hacks like this are sensational news stories…. They are extremely rare and require the most sophisticated attackers.
Far more likely, in the real world, are the vulnerabilities and exploits that would be mitigated by timely updates.