Last week, Microsoft Corp. made headlines when it scored an unconventional if not unprecedented legal victory: Convincing a U.S. court to let it seize control of a Chinese Internet service provider’s network as part of a crackdown on piracy.
I caught up with Microsoft’s chief legal strategist shortly after that order was executed, in a bid to better understand what they were seeing after seizing control over more than 70,000 domains that were closely associated with distributing hundreds of strains of malware. Microsoft said that within hours of the takeover order being granted, it saw more than 35 million unique Internet addresses phoning home to those 70,000 malicious domains.
First, the short version of how we got here: Microsoft investigators found that computer stores in China were selling PCs equipped with Windows operating system versions that were pre-loaded with the “Nitol” malware, and that these systems were phoning home to subdomains at 3322.org. The software giant subsequently identified thousands of sites at 3322.org that were serving Nitol and hundreds of other malware strains, and convinced a federal court in Virginia to grant it temporary control over portions of the dynamic DNS provider.
Microsoft was able to do that because – while 3322.org is owned by a firm in China — the dot-org registry is run by a company based in Virginia. Yet, as we can see from the graphic above provided by Microsoft, Nitol infections were actually the least of the problems hosted at 3322.org (more on this later).
To learn more about the outcome of the seizure, I spoke with Richard Boscovich, a senior attorney with the company’s digital crimes unit (DCU) who helped to coordinate this action and previous legal sneak attacks against malware havens. Our interview came just hours after Microsoft had been cleared to seize control over the 70,000+ subdomains at 3322.org. I asked Boscovich to describe what the company was seeing.
“The numbers are quite large,” he said. “Just a quick view of what we’ve been seeing so far is upwards of 35 million unique IP [addresses] trying to connect with the 70,000 subdomains.”
Certainly IP addresses can be very dynamic — a single computer can have multiple IP addresses over a period of a few days, for example. But even if there were half as many infected PCs than unique IPs that Microsoft observed reporting to those 70,000 domains, we’d still be talking about an amalgamation of compromised PCs that is far larger than any known botnet on the planet today. So how certain was Microsoft that these 35 million unique IPs were in fact infected computers?
“We started identifying what our AV company blocks,” Boscovich explained. “We saw a lot of different types of malware, from keyloggers to DDoS tools and botnets going back there. Our position would be if you’re reaching out to these 70,000 subdomains, that the purpose would be you’re directed there to be infected or you are already infected with something. And that something was up to 560 or so malware strains we identified [tracing back] to 3322.org.”
COLLATERAL DAMAGE?
Microsoft’s past unilateral actions against malware purveyors and botnets have engendered their share of harsh reactions from members of the security community, and I fully expected this one also would be controversial. I wasn’t disappointed: Writing for Internet policy news site CircleID, longtime antispam activist Suresh Ramasubramanian warned that Microsoft’s action would cause “extremely high collateral damage,” both to innocent sites and to ongoing investigations.
“So, in the medium to long term run …all that Microsoft DCU and Mr. Boscovich have achieved are laudatory quotes in various newspapers and a public image as fearless and indefatigable fighters waging a lone battle against cybercrime,” Ramasubramanian wrote. “That manifestly is not the case. There are several other organizations (corporations, independent security researchers, law enforcement across several countries) that are involved in studying and mitigating botnets, and a lot of their work just gets abruptly disrupted (jeopardizing ongoing investigations, destroying evidence and carefully planted monitoring).”
Boscovich said Microsoft worked hard to focus its legal request on 3322.org subdomains that appeared to be doing little else than serving as controllers, updaters or data repositories for malware operations. He noted that the 70,000 domains the court granted it control over were only a small subset (less than 3 percent) of the 2.75 million subdomains currently host at 3322.org.
“There’s always a balancing act,” the Microsoft lawyer told me. “You want to make sure you do it in such a way to minimize collateral damage on legitimate sites. The unique aspect of this action was the great lengths that we went to make sure that we surgically took out and sinkholed 70,000 subdomains out of a domain hosting 2.75 million subdomains total. We developed technology along with Nominum where we were able to — once a domain was pointed to us — to only take out those 70,000, allowing all of the other subdomains which are beyond the scope of our order to simply resolve and not be impacted.”
Boscovich added that Microsoft and Nominum will be working with Internet service providers to help clean machines seen reporting to the hostile 3322.org sites.
“A lot of people in the security community like to do a lot of research, they like sit on these things and see what’s happening, but sometimes the right thing to do is get to the victims, tell them that they’ve been victimized, tell them that they’re victimizing others, and help clean them up,” he said.
Other luminaries in the security research space expressed surprise at the breadth of Microsoft’s latest legal action, but said it was too soon to say how much of an impact it would have on the malware ecosystem. Dan Hubbard, chief technology officer at OpenDNS, said his firm has been blocking all 2.75 million subdomains at 3322.org for almost two years.
“We very rarely get complaints, and even today we see 1.1 million requests [attempting to go to] 3322.org with zero complaints,” Hubbard said. “The vast majority of it is not good.”
But he said he wonders what Microsoft is going to do with all of the sensitive information flowing through the sinkholed 3322.org domains. As I noted in my previous piece, subdomains at 3322.org have long been associated with targeted malware used in espionage attacks against U.S. and other Western corporations.
“There is going to be quite a bit of sensitive information that’s coming across the flow, from credit card details to proprietary company records,” Hubbard said. “It will be interesting to know what are the bounds around that, what they do with that data, and are they going to inform companies that are impacted.”
Joe Stewart, a senior security analyst with Dell SecureWorks, agreed, calling the scope of the interception order “unprecedented.”
“It’s a little bit surprising that Microsoft went to the lengths they did,” Stewart said. “That they are intercepting or trying to intercept millions of malicious requests while still allowing service to operate is unprecedented, sort of like they’re acting as ‘the great firewall of Microsoft.’ It’s not the sinkholing of these subdomains that’s novel, it’s that they’ve injected themselves legally between this service in China and its users. Handled responsibly, it could be a good thing.”
WILL THE REAL JOHN DOE PLEASE STEP FORWARD?
Like others before it, this latest legal salvo by Microsoft seeks to unmask individuals behind the alleged criminal activity at 3322.org. It does this using so-called “John Doe,” requests, which are legal proceedings that can enable a plaintiff or prosecutor to gather information on a number of individuals, in a bid to learn their identities and/or to prove they were parties to a conspiracy.
I asked Boscovich if Microsoft’s John Doe requests in previous targeted botnet takedowns had produced any leads. Specifically, I wanted to know if there were any updates to the John Does named in connection with its targeting of the Kelihos spam botnet. In that case, Microsoft identified 31-year-old Andrey N. Sabelnikov of St. Petersburg, Russia, a former system developer and project manager for Agnitum, a Russian antivirus firm.
“In the Kelihos case, we named the Russian AV…the individual that we alleged was the developer of the code for Kelihos,” Boscovich said. “We’re resolving that case now, and very shortly you’ll hear a statement that will be coming out.”
Shortly after that story broke, Sabelnikov vehemently refuted Microsoft’s allegations, saying he had never participated in the management of botnets or any other similar programs. But according to Boscovich, Microsoft will soon be publishing a statement that says otherwise.
“I think that once you see the statement that he agreed to that we’re going to publish in the next couple of days on the Kelihos case, I think that will put that to rest. I think we’ve been pretty accurate that when we name someone we know who they are. And there have been a lot of cases referred to law enforcement, and a lot of the evidence based upon which they’re much further along now based upon the stuff that we have done. So anybody who thinks that these things are not effective, from purely an identification of individuals behind it is concerned, they’re wrong.”
Please Microsoft, don’t start calling people to tell them they have a virus. Especially don’t outsource it.
You are being called by an Indian support company called iYogi. Its not Microsoft outsourcing Microsoft support but a boiler room operation. Brian has written about this and their connection with companies like Avast.
I know that, but it’s hard enough to get people educated to not fall for this crap without the good guys doing it too.
Has MS published the sinkhole IP they are using for the malicious domains? Or are they just monitoring the DNS queries themselves?
They haven’t published it for general consumption, but they have shared it with trusted members of the security community. ThreatSTOP and others are blocking/alerting on connections to the IPs.
In our case, you will see a hit on “SinkHole”. Any system connecting to a sinkhole should be re-imaged.
We provide both free tools and a commercial service to protect from, and assist in the remediation of, botnets.
I very much agree upon Joe Stewart’s statement. Intercepting so much of sensitive data, solely upon Microsoft sounds bleek. Having a collaborative panel to justify the same would be a great idea!
I think the question of whether to take down a domain like this vs leaving it up while under investigation has to be decided on a case by case basis.
If the domain and its impact is relatively small, and it is located in a legal jurisdiction where there is at least a chance of local law enforcement providing an effective response and cooperating with international law enforcement once the investigation is completed, then it should be left up in order to catch those responsible.
In this case, we have a VERY large malware system which is actively compromising large numbers of people and the odds of catching the originators are probably less due to the fact that it is in China, the originators status within China is unknown, and China is known for not extraditing its citizens, so I’d say taking the system down was the better option.
Zero complaints for blocking dns queries is likely. 3322 is, as far as DNS goes, not going to be used very much for running any service that’ll attract a large number of dns queries for any individual hostname.
Family members connecting from their schools or offices to access their webcam, a small business that uses it to provide a hostname for dsl + wifi / vpn routers at a bunch of remote offices so their tech doesn’t need to go to each office to fix stuff.
Most of those are just not going to be using 3rd party services like opendns. Especially *not* in China. So – there’s absolutely zero surprise if opendns sees a large number of malware queries and nothing else. There’s a lot of malware hostnames on 3322, agreed 100%
My argument has not been with the sinkholing as much as with the way that Microsoft went about it. The one argument against siezing and sinkholing a legitimate but heavily abused domain located in China of all places is the extremely unfortunate timing of this action, just before the WCIT starts up with several calls to “free the internet from the US government and make it international” .. led by China, among others. That call seems to include a lot of wording about cybersecurity, so – again, unfortunate.
The question still remains, did Microsoft engage with CN-CERT and other organizations in China (with whom they are familiar enough) and what was the result?
China wants a ‘free’ internet? The irony …
If Antivirus outfits incorporated freely available technology into their products, it would have a massive upfront impact on BotNets. I cannot understand why they have ignored this technology for so many years, and have even gone as far as actively discouraging their use in support forums.
Thousands of IT support folks including myself have been using a free tool called Combofix for years with enormous success. AV software even identifies and blocks combofix as Malware while trudging along fine ignoring hundreds of peices of spyware on the same machine. Its beyond comprehension! We could not have waited for Microsofts enormously belated actions to get a handle on it.
From the official website for Combofix:
Known issues
Some antivirus software may detect ComboFix as malicious; for example it uses NirCmd, which is considered as a backdoor by many antivirus software.
ComboFix may disrupt internet connectivity.The majority of times only a simple fix is required.
ComboFix may attempt deletion all files from the system drive on systems infected with a rootkit.
Haven’t used it or know much about it myself but it does seem on the face of it to be an application that could cause considerable harm to user systems if they aren’t competent in it’s use.
Want to know what else can can significant harm to users? Malicious software. I’m sure, being blunt, most users would rather be infected with malicious software than lose all their data entirely.
I am wary of free security tools. I’m not sure if I should trust Combofix. I use SpyBot, but again, there are worldwide download sites for the dat file and that makes me nervous since I do not know whom to trust in the freeware world. I think it would be great if Kaspersky and the like would incorporate those tools. Maybe they do and do not advertise that. I guess I am a sucker when I think if I buy the anti-virus at Best Buy that it is the real deal, no malware included.
“I am wary of free security tools.
Depends on the source. There are some useful security features built-in Windows: create/use standard user accounts, disable the UAC authentication prompt for standard user accounts, software restriction policy whitelisting via gpedit.msc or parental controls, Internet Explorer security zones, just to highlight a few. Also, Microsoft provides free security software for their customers to download and install: EMET, Microsoft Security Essentials, DropMyRights (for Windows XP admin accounts). If one has Windows, all of these features and tools are free.
Avira, Avast and Avg are well-known companies that provide free versions of their anti-virus software. And there’s lots more free security software out there (see http://www.techsupportalert.com/content/probably-best-free-security-list-world.htm, for example ) as well as responsible reviews.
It also depends on the knowledge and experience of the user as some free security software is clearly targeted towards advanced users (e.g., ComboFix).
I say harden Windows with Microsoft’s offerings first, then worry about adding 3rd party security software if you still think that you need it or want it. However, all the security software in the world won’t help much if one doesn’t follow Brian’s 3 rules here:
http://krebsonsecurity.com/2011/05/krebss-3-basic-rules-for-online-safety/
P.S. For Linux and BSD, the vast majority of security tools are both free and open-source.
I know what you mean, Craig. However, Combofix has been looked at six ways from sunday by some of the best computer fix-it pros in the biz, and has been recommended by them for years. If that’s not an endorsement, I don’t know what is.
I’m guessing I have cleaned over 500 computers since they released combofix and have very few problems outside of getting it into the system to run it but once it starts its rock n roll. Most definitely have not lost data because of running the tool. The sad part is effective alternatives outside of a complete reformat are zero, zip, nada. There are some tools that will show partial successes but CF is the king of all rootkit removers.
We too have used ComboFix for years on truckloads of systems with very good results and no data loss. It is radical surgery and not for the meek. But data loss is not a concern beyond the fact that data loss can occur just by turning the system on.
Why can’t mainstream & Microsoft learn from the likes of ComboFix and Malwarebytes? An age old question, I guess we will never know….
If it worked and was free, they would get it into their software asap, because it would give them an edge against competitors.
See http://xkcd.com/808/
While I mostly agree with The Economic Argument of your link, there is a nuance missed. Example: Certain alternative health care can reduce health care costs, but is not done due to profit motive (can’t patent => can’t profit as much => don’t invest + take steps to discourage anyone else using it so that they have to pay you for your product (even if it is less capable)). Also the case for ComboFix?
Actually, there is precedence for such a thing. Trend Micro acquired the free HijackThis tool in 2007:
http://news.cnet.com/Trend-Micro-acquires-HijackThis-antispyware-tool/2100-7355_3-6167308.html
Has the acquisition of HijackThis helped Trend Micro’s bottom-line? I don’t know, but I strongly suspect that they believed it would. Btw, the tool is still free.
Our experience @ ThreatSTOP is the inverse. Tying things too tightly to a clear ROI (like with adwords) has been ineffective, but doing what we think is right in the hope of it ha worked.
We have always provided a free “community” version of our service, for those who share log data back (log data is used to feed the detection algorithm). That wound up being a good source of leads and goodwill, so we expanded it to provide check IP and check log functionality, for free, without registration (just a valid e-mail to verify you aren’t a bot and to give us at least a shot at selling something) on our website.
That was so successful that we have developed an IP reputation equivalent of HijackThis, our ThreatCHECK app, which will enumerate all TCP and UDP connections to/from the machine it’s run on during the reporting period, and give you the option to correlate with our IP rep database, or just put it in your clipboard where you can do whatever you want with it.
None of these tools were cheap to develop, but they have been the most effective marketing we have done, and it makes us feel good to help even if it doesn’t’ result in a sale.
At least in IT, and in security in particular, it’s very possible to do well while doing good. It helps us to help the community, and the data we get from logs helps improve our detection.
Boscovich says: “A lot of people in the security community like to do a lot of research, they like sit on these things and see what’s happening, but sometimes the right thing to do is get to the victims, tell them that they’ve been victimized, tell them that they’re victimizing others, and help clean them up,”
Well said!
And ThreatSTOP is providing free tools to let users find out PRE-NAT (since all MS and their ISP see is the post-NAT) which internal hosts are compromised.
More in our blog:
http://blog.threatstop.com/2012/09/21/nitol-takedown/
FYI, this looks like this is a link to the complaint, motion, and evidence filed in the case: http://www.noticeofpleadings.com