Late last month I wrote about Citadel, an “open source” version of the ZeuS Trojan whose defining feature is a social networking platform where users can report and fix programming bugs, suggest and vote on new features, and generally guide future development of the botnet malware. Since then, I’ve been given a peek inside that community, and the view so far suggests that Citadel’s collaborative approach is fueling rapid growth of this new malware strain.

The CRM page shows democracy in action among Citadel botnet users.

A customer who bought a license to the Citadel Trojan extended an invitation to drop in on that community of hackers. Those who have purchased the software can interact with the developers and other buyers via comments submitted to the Citadel Store, a front-end interface that is made available after users successfully navigate through a two-step authentication process.

Upon logging into the Citadel Store, users see the main “customer resource management” page, which shows the latest breakdown of votes cast by all users regarding the desirability of proposed new features in the botnet code.

In the screen shot to the right, we can see democracy in action among miscreants: The image shows the outcome of voting on several newly proposed modules for Citadel, including a plugin that searches for specific files on the victim’s PC, and a “mini-antivirus” program that can clean up a variety of malware, adware and other parasites already on the victim’s computer that may prevent Citadel from operating cleanly or stealthily. Currently, there are nine separate modules that can be voted and commented on by the Citadel community.

Drilling down into the details page for each suggested botnet plugin reveals comments from various users about the suggested feature (screenshot below). Overall, users seem enthusiastic about most suggested new features, although several customers used the comments section to warn about potential pitfalls in implementing the proposed changes.

Citadel users discuss the merits of including a module to remove other parasites from host PCs.

The customer resource management page also reveals that although the principal authors of the Citadel Trojan treat this as their day job, they try their best to have a life on the weekends. A notice prominetly posted to the Citadel CRM homepage reads:

Please note regarding the Help Desk in the Jabber chat & CRM page:

Daily from 10.00 to 00.30
Sat, Sun – closed, you can write us offline.
All requests and questions will be processed on Monday.

The collegial atmosphere being cultivated by the Citadel authors appears to have hastened the malware’s maturity, according to researchers at Seculert. In a blog post published Wednesday, researchers there said that they’d observed at least five new versions of Citadel since first spotting the malware on Dec. 17, 20011.

Seculert’s Aviv Raff said that means the miscreants behind Citadel are pushing out a new version of the Trojan about once a week.

“The only similar Trojan who got close to this pace was the so called ‘SpyZeus’ Trojan,” Raff said. “Others, including ZeuS itself, took between a month to several months to release a new version.”

Anyone who’s run a Web site is probably familiar with the term “malvertising,” which occurs when crooks hide exploits and malware inside of legitimate-looking ads that are submitted to major online advertising networks. But there’s a relatively new form of malware-based advertising that’s gaining ground — otherwise harmless ads for illicit services that are embedded inside the malware itself.

At its most basic, this form of advertising — which I’m calling “crimevertising” for want of a better term — has been around for many years. Most often it takes the form of banner ads on underground forums that hawk everything from cybercriminal employment opportunities to banking Trojans and crooked cashout services. More recently, malware authors have started offering the ability to place paid ads in the Web-based administrative panels that customers use to control their botnets. Such placements afford advertisers an unprecedented opportunity to keep their brand name in front of the eyeballs of their target audience for hours on end.

The author of the Blackhole exploit pack is selling ad space on his kit's administration page, as seen in this screenshot.

A perfect example of crimevertising 2.0 is the interface for the Blackhole Exploit Kit, crimeware that makes it simple for just about anyone to build a botnet. The business end of this kit is stitched into hacked or malicious Web sites, and visitors with outdated browser plugins get redirected to sites that serve malware of the miscreant’s choosing. Blackhole users can monitor new victims and the success rates of the compromised sites using a browser-based administrative panel.

In the screen shot above, the administration panel of a working Blackhole exploit kit shows two different ads; both promote the purchase and sale of Internet traffic. And here is a prime example of just how targeted this advertising can be: The most common reason miscreants purchase Internet traffic is to redirect it to sites they’ve retrofitted with exploit kits like Blackhole.

I wanted to find out how much it would cost to place such targeted ads, so I chatted up the author of this kit — a hacker who uses the nickname “Paunch.” He said an ad that would run on administration panels across the entire Blackhole user base would cost me $700 per month. He declined to say just how many “impressions” that money would buy, or exactly how many Blackhole users there are today.

But it’s probably quite an audience: According to security firm Sophos, Blackhole is now by far the most popular method of delivering drive-by attacks. In its 2012 Security Threat Report, the company found that “in the second half of 2011, 67% of [malware] detections were redirections on compromised legitimate sites. Of these, approximately half are believed to be redirections to Blackhole exploit sites.”

Interestingly, when Paunch doesn’t have ads to run from paying customers, he runs ads for his own ancillary services. In the screen shot below (taken from a different working Blackhole exploit kit) Paunch can be seen pitching his subscription-based malware obfuscation service.

I suppose it’s possible that miscreants could try to place malware-laced crimevertisements in a bid to hijack the browsers of other hackers, but that’s probably unlikely to happen as long as malware authors like Paunch are manually reviewing purchased ads and disallowing anything other than plain text. In the end, crimeware kit buyers may have more to fear from a kit’s author himself: The author of the infamous SpyEye botnet creation kit once acknowledged adding a hidden backdoor to his software that let him remotely access all customer installations.

More than two months after authorities shut down a massive Internet traffic hijacking scheme, the malicious software that powered the  criminal network is still running on computers at half of the Fortune 500 companies, and on PCs at nearly 50 percent of all federal government agencies, new research shows.

Source: FBI

The malware, known as the “DNSChanger Trojan,” quietly alters the host computer’s Internet settings to hijack search results and to block victims from visiting security sites that might help scrub the infections. DNSChanger frequently was bundled with other types of malware, meaning that systems infected with the Trojan often also host other, more nefarious digital parasites.

In early November, authorities in Estonia arrested six men suspected of using the Trojan to control more than four million computers in over 100 countries — including an estimated 500,000 in the United States. Investigators timed the arrests with a coordinated attack on the malware’s infrastructure. The two-pronged attack was intended to prevent miscreants from continuing to control the network of hacked PCs, and to give Internet service providers an opportunity to alert customers with infected machines.

But that cleanup process has been slow-going, according to at least one security firm. Internet Identity, a Tacoma, Wash. company that sells security services, found evidence of at least one DNSChanger infection in computers at half of all Fortune 500 firms, and 27 out of 55 major government entities.

“Yes, there are challenges with removing this malware, but you would think people would want to get this cleaned up,” said Rod Rasmussen, president and chief technology officer at Internet Identity. “This malware was sometimes bundled with other stuff, but it also turns off antivirus software on the infected machines and blocks them from getting security updates from Microsoft.”

Computers still infected with DNSChanger are up against a countdown clock. As part of the DNSChanger botnet takedown, the feds secured a court order to replace the Trojan’s DNS infrastructure with surrogate, legitimate DNS servers. But those servers are only allowed to operate until March 8, 2012. Unless the court extends that order, any computers still infected with DNSChanger may no longer be able to browse the Web.

Rasmussen said there are still millions of PCs infected with DNSChanger. “At this rate, a lot of users are going to see their Internet break on March 8.”

Tom Grasso Jr., an FBI supervisory agent at the National Cyber Forensics & Training Alliance in Pittsburgh, Pa., said the DNSChanger Working Group — the industry and law enforcement coalition that’s handling the remediation — has been discussing what to do about the upcoming deadline, but he declined to offer specifics.

“We’re certainly exploring all different options to minimize whatever impact there’s going to be on a lot of people,” Grasso said.

Even if the DNS Changer working group manages to get the deadline extended, the cleanup process will likely take many years.  At least, that’s been the experience of the the Conficker Working Group, a similar industry consortium that was created to help contain and clean up infections from the infamous Conficker Worm. That working group was formed in 2009, yet according to the group’s latest statistics, nearly 3 million systems remain infected with Conficker.

Given the Conficker Working Group’s experience, shutting down the surrogate DNS network on March 8 may actually be a faster — albeit more painful — way to clean up the problem.

“I’m guessing a lot more people would care at that point,” Rasmussen said. “It certainly would be an interesting social experiment if these systems just got cut off.”

Individuals in charge of a large network can learn if any systems are infected with DNSChanger by sending a request to one of the members of the DNS Changer Working Group. Home users can avail themselves of step-by-step instructions at this link to learn of possible DNSChanger infections.

Where do you come down on the decision to extend the Mar. 8 deadline? Register your vote in the poll below. Feel free to sound off in the comments.

Underground hacker forums are full of complaints from users angry that a developer of some popular banking Trojan or bot program has stopped supporting his product, stranding buyers with buggy botnets. Now, the proprietors of a new ZeuS Trojan variant are marketing their malware as a social network that lets customers file bug reports, suggest and vote on new features in upcoming versions, and track trouble tickets that can be worked on by the developers and fellow users alike.

A screenshot of the Citadel botnet panel.

The ZeuS offshoot, dubbed Citadel and advertised on several members-only hacker forums, is another software-as-a-service malware development. Its target audience? Those frustrated with virus writers who decide that coding their next creation is more lucrative and interesting than supporting current clients.

“Its no secret that the products in our field — without support from the developers — result in a piece of junk on your hard drive. Therefore, the product should be improved according to the wishes of our customers,” Citadel’s developers claim in an online posting. “One problem is that you have probably experienced developers who ignore your instant messages, because there are many customers but there is only one developer.”

In the following excerpt, taken from a full description of Citadel’s innovations, the developers of this malware strain describe its defining feature as a social networking platform for malware users that is made available through a Web-based portal created by the malware itself.

“We have created for you a special system — call it the social network for our customers. Citadel CRM Store allows you to take part in product development in the following ways:

- Report bugs and other errors in software. All tickets are looked at by technical support you will receive a timely response to your questions. No more trying to reach the author via ICQ or Jabber.

-Each client has the right to create an unlimited number of applications within the system. Requests can contain suggestions on a new module or improvements of existing module. Such requests can be public or private.

-Each client has a right to vote on new ideas suggested by other members and offer his/her price for development of the enhancement/module. The decision is made by the developers on whether to go forward with certain enhancement or new module depending on the voting results.

-Each client has the right to comment on any application and talk to any member. Now it is going to be interesting for you to find partners and like-minded people and also to take active parts in discussions with the developers.

- You can see all stages of module development, if it is approved other members. We update the status and time to completion.

- You may pay a deposit, if module is approved (50%). After the deposit is paid by the members, the project starts moving forward, so that the money is paid directly to coders and there will be no laziness or inaction. Everything is clear: every stage of development is thoroughly shown.

-Easy jabber [instant message] notification of new member or developer comments, or the availability of new custom applications.

The Citadel store lets users file and track bug reports, and request and vote on new features.

Citadel may be the first notable progeny of ZeuS since the ZeuS source code was leaked online last year. The authors claim that it includes a number of bug fixes for the most recent ZeuS version, including full support for grabbing credentials from victims using Google Chrome. Also bundled with this update is a component that can record and transmit videos of the victim’s screen activity.

The basic Citadel package — a bot builder and botnet administration panel — retails for $2,399 + a $125 monthly “rent,” but some of its most innovative features are sold as a la carte add-ons. Among those is a $395 software module that allows botmasters to sign up for a service which automatically updates the bot malware to evade the last antivirus signatures. The updates are deployed via a separate Jabber instant message bot, and each update costs an extra $15.

Citadel also boasts a feature that hints at its creator’s location(s). According to the authors, if the malware detects that the victim’s machine is using a Russian or Ukrainian keyboard, it will shut itself down. This feature is almost certainly a hedge to keep the developers out of trouble: Authorities in those regions are far less likely to pursue the Trojan’s creators if there are no local victims.

The Citadel bot builder.

It will be interesting to see if these malware developers hold true to their word. The growth of a more real-time, user-driven and crowdsourced malicious software market would be a truly disturbing innovation. For now, the miscreants behind Citadel appear upbeat about their chances of ushering in such a reality.

“It’s very interesting for us to work with our clients,” they wrote in an online forum posting. “A lot of authors write in forums that they ‘support the product,’ but at the end the updates only come out once every three months or the author disappears forever. Problem is in author’s motivation. You support us, we support you. It is easy.”

Amnesty International‘s homepage in the United Kingdom is currently serving malware that exploits a recently-patched vulnerability in Java. Security experts say the attack appears to be part of a nefarious scheme to target human rights workers.

The site’s home page has been booby trapped with code that pulls a malicious script from an apparently hacked automobile site in Brazil.  The car site serves a malicious Java applet that uses a public exploit to attack a dangerous Java flaw that I’ve warned about several times this past month. The applet in turn retrieves an executable file detected by Sophos antivirus as Trojan Spy-XR, a malware variant first spotted in June 2011.

A woman who answered the phone this morning at Amnesty International’s research and policy branch in the U.K. declined to give her name, but said she would pass on the information about the break-in. The site remains compromised.

This is hardly the first time Amnesty International’s sites have been hacked to serve up malware. The organization’s site was hacked in April 2011 with a drive-by attack.  In November 2010, security firm Websense warned Amnesty International’s Hong Kong Web site was hacked and seeded with an exploit that dropped malware using a previously unknown Internet Explorer vulnerability. 

The UK site is not particularly popular – its global rank is 90,203 according to Alexa.com – but the chances are good that the attackers behind this are not after financial data. It appears more likely that the exploit maybe part of an ongoing campaign by Chinese hacking groups to extract information from dissident and human rights organizations.

The attack against the Amnesty International’s Hong Kong site last year loaded malware that belongs to a notorious family of backdoor Trojans from China. According to a ThreatExpert analysis of the malicious Java file currently being served by Amnesty’s UK site, the malware downloaded appears to be associated with China.

Paul Royal, a research consultant with Barracuda Networks, said the attack fits the profile of previous campaigns against human rights non-governmental organizations.

“Certain countries use zero day exploits and other techniques to gain electronic information about the activities of human rights activists,” Royal wrote in an email to KrebsOnSecurity, noting that the site appears to have been compromised since at least Dec. 16.  “Of course, a subset of these activists are too smart to click on links in even well-worded spearphishing emails. But what if you compromised a website frequented by these activists (e.g., Amnesty International)? Then your targets come to you. The context-specific damage potential is significant.”

These attacks highlight the importance of staying up to date on security patches. In the case of Java, removing oft-targeted software that you don’t really need may be a safer option. Either way, tools like Secunia’s Personal Software Inspector or FileHippo’s Update Checker can help you stay on top of the latest security updates for popular software titles.

Update, 12:59 p.m. ET: Barracuda Labs just published a blog post about this.

Update, Dec. 24, 9:40 a.m. ET: Emerson Povey, digital communications editor for Amnesty International UK, wrote in to say that the exploit has been removed from the site.

Thousands of Twitter accounts apparently created in advance to blast automated messages are being used to drown out Tweets sent by bloggers and activists this week who are protesting the disputed parliamentary elections in Russia, security experts said.

Image: Twitterbot.info

Amid widespread reports of ballot stuffing and voting irregularities in the election, thousands of Russians have turned out in the streets to protest. Russian police arrested hundreds of protesters who had gathered in Moscow’s Triumfalnaya Square, including notable anti-corruption blogger Alexei Navalny. In response, protesters began tweeting their disgust in a Twitter hashtag #триумфальная (Triumfalnaya), which quickly became one of the most-tweeted hashtags on Twitter.

But according to several experts, it wasn’t long before messages sent to that hashtag were drowned out by pro-Kremlin tweets that appear to have been sent by countless Twitter bots. Maxim Goncharov, a senior threat researcher at Trend Micro, observed that “if you currently check this hash tag on twitter you’ll see a flood of 5-7 identical tweets from accounts that have been inactive for month and that only had 10-20 tweets before this day. To this point those hacked accounts have already posted 10-20 more tweets in just one hour.”

“Whether the attack was supported officially or not is not relevant, but we can now see how social media has become the battlefield of a new war for freedom of speech,” Goncharov wrote.

I’ve been working with a few security researchers inside of Russia who asked not to be named for fear of retribution by patriotic Russian hackers or the government. Since Trend’s posting, they’ve identified thousands of additional accounts (e.g., @ALanskoy, @APoluyan, @AUstickiy, @AbbotRama, @AbrahamCaldwell…a much longer list is available here) that are rapidly posting anti-protester or pro-Kremlin sentiments to more than a dozen hashtags and keywords that protesters are using to share news, including #Navalny.

A review of the 2,000 Twitter accounts linked above indicates that most of them were created at the beginning of July 2011, and have very few tweets other than those meant to counter the protesters, or to simply fill the hashtag feeds with meaningless garbage. Some of the bot messages include completely unrelated hashtags or keywords, seemingly to pollute the news stream for the protester hashtags.

In addition, almost all of the bot accounts are mostly following each other, with a handful of exceptions:  It appears that most of the auto-created accounts that are flooding the protester hashtags are following the Twitter account @master_boot, which looks like it belongs to an actual user. In fact, one of Master_boot’s 17,000+ followers recently tweeted to inquire about Twitter bots. The person behind the @master_boot account did not immediately respond to requests.

Interestingly, the Kremlin leadership appears to be using their Twitter accounts to bash those calling the recent elections a fraud. Reuters is reporting that Russian President Dmitry Medvedev caused shock and jeers on Wednesday after an obscene insult directed at political opponents appeared on his official Twitter feed. According to cached copies of the feed and a notification of the post received by a Reuters reporter, Medvedev’s tweet read:

“It has become clear that if a person writes the expression ‘party of swindlers and thieves’ in their blog then they are a stupid sheep getting f****d in the mouth .”

It wasn’t long ago that I felt comfortable recommending CNET‘s download.com as a reputable and trustworthy place to download software. I’d like to take back that advice: CNET increasingly is bundling invasive and annoying browser toolbars with software on its site, even some open-source titles whose distribution licenses prohibit such activity.

Although this change started this summer, I only first became aware of it after reading a mailing list posting on Monday by Gordon “Fyodor” Lyon, the software developer behind the ever useful and free Nmap network security scanner. Lyon is upset because download.com, which has long hosted his free software for download without any “extras,” recently began distributing Nmap and many other titles with a “download installer” that bundles in browser toolbars like the Babylon toolbar.

CNET’s own installer is detected by many antivirus products as a Trojan horse, even though the company prefaces each download with the assurance that “CNET hosts this file and has scanned it to ensure it is virus and spyware free.” CNET also has long touted download.com’s zero tolerance policy toward all bundled adware.

Lyon said he found his software was bundled with the StartNow Toolbar, which is apparently powered by Microsoft‘s “Bing decision engine.” When I grabbed a copy of the Nmap installer from download.com and ran it on a test Windows XP machine, CNET’s installer offered the Babylon Toolbar, which is a translation toolbar that many Internet users have found challenging to remove.

The CNET download installer that I got for Nmap from download.com was made by CBS Interactive (CNET Networks was acquired by CBS in 2008), and it is detected as malicious by three antivirus products at Virustotal.com. When I unpacked the installer from the Nmap program and scanned just the installer, 10 out of the 39 antivirus products detected the file as either a Trojan horse or adware.

Lyon said CNET is violating Nmap’s distribution license, which bars any distribution that “integrates/includes/aggregates Nmap into a proprietary executable installer, such as those produced by InstallShield.”

“Of course the problem is that users often just click through installer screens, trusting that download.com gave them the real installer and knowing that the Nmap project wouldn’t put malicious code in our installer,” Lyon wrote. “Then the next time the user opens their browser, they find that their computer is hosed with crappy toolbars, Bing searches, Microsoft as their home page, and whatever other shenanigans the software performs! The worst thing is that users will think we (Nmap Project) did this to them!”

Nmap isn’t alone. Wireshark, another free and widely-used network analysis and security tool, also was being bundled with toolbars through download.com. That is, until Wireshark open source director Gerald Combs sent CBS a cease and desist letter.

Combs said download.com had been distributing Wireshark since the early 2000s, back when it was still known under its former name, “Ethereal.”

“It is a little ironic, that you’re downloading these security tools but [download.com] makes you run through this gauntlet of crapware to get them,” Combs said.

Interestingly, CNET does not offer the download installer for “registered users;” those who are registered are offered a direct download link. Also, it appears that software makers who pay CNET to have a “premium listing” can avoid the installer being bundled with their offering.

The CNET download installer will still let users decline the toolbar installations, but the default is of course to install the software. I have asked CBS for comment on the apparent discrepancy between download.com’s no-adware policy and its practices, and will update this blog post when I hear from them.

As I was researching this, I found that I’m a little late to the party on this one. Thanks to that ExtremeTech post, I found this link, in which CNET explained part of the rationale for rolling out this download installer, in a blog posting on July 25, 2011:

2. Why is Download.com making this change?

Our testing has shown that as many as half of all people who initiate a download fail to complete the download and install their software. The Download.com Installer improves the process by stepping the user through their download and enabling them to more easily find and execute your software’s installer. Other download sites employ similar solutions, but we believe that ours provides more security and utility as well as better consumer protections.

3. How does the Download.com Installer improve the download experience?

By downloading with the Download.com Installer the user is guaranteed that the file they install on their system came directly from Download.com. Only software that is tested spyware-free and hosted on Download.com’s secure servers may be delivered via the Installer.

In addition, thanks to the clear steps provided by the Installer, the percentage of users who are able to complete the download process increases significantly when using the Installer for their downloads.

Finally, Download.com is supported primarily by advertising, and we include offers for additional downloads from advertisers as part of our Installer process. Unlike other download sites that employ similar ad-supported technologies, however, our Installer is limited to a single offer that is carefully screened to ensure compliance with the Download.com Software Policies.

4. Is all software on Download.com delivered via the Installer?

No. The Download.com Installer was rolled out in July 2011 to a limited number of Windows software downloads. At this time we are still evaluating its performance and incorporating feedback from the user and developer communities.

There you have it, readers. If you’re unhappy about this development, let CNET/CBS know how you feel. These toolbar deals no doubt have the potential to earn CNET a lot of money: Download.com is a very heavily visited site, and according to Alexa it is the 174th most-visited site on the Internet. But CNET should be more consistent and up-front about its adware policies, or risk losing that ranking in a hurry.

In the meantime, it’s always a good idea to download software directly from the source whenever possible, and to pay close attention to the prompts during the installation process.

The Wall Street Journal this week ran an excellent series on government surveillance tools in the digital age. One story looked at FinFisher, a remote spying Trojan that was marketed to the governments of Egypt, Germany and other nations to permit surreptitious PC and mobile phone surveillance by law enforcement officials. The piece noted that FinFisher’s creators advertised the ability to deploy the Trojan disguised as an update for Apple’s iTunes media player, and that Apple last month fixed the vulnerability that the Trojan leveraged.

Image: spiegel.de

But the WSJ series and other media coverage of the story have overlooked one small but crucial detail: A prominent security researcher warned Apple about this dangerous vulnerability in mid-2008, yet the company waited more than 1,200 days to fix the flaw.

The disclosure raises questions about whether and when Apple knew about the Trojan offering, and its timing in choosing to sew up the security hole in this ubiquitous software title: According to Apple, as of June 2011, there were approximately a quarter billion installations of iTunes worldwide.

Apple did not respond to requests for comment. An email sent Wednesday morning to its press team produced an auto-response stating that employees were already on leave for the Thanksgiving holiday in the United States.

I first wrote about this vulnerability for The Washington Post in July 2008, after interviewing Argentinian security researcher Francisco Amato about “Evilgrade,” a devious new penetration testing tool he had developed. The toolkit was designed to let anyone send out bogus automatic update alerts to users of software titles that don’t sign their updates. I described the threat from this toolkit in greater detail:

Why is this a big deal? Imagine that you’re at an airport lounge, waiting to board your flight, and you pop open your laptop to see if you can hop on an open wireless network. Bear in mind that there are plenty of tools available that let miscreants create fake wireless access points for the purposes of routing your connection through their computer. You connect to that fake network, thinking you can check your favorite team’s sports scores. A few seconds later, some application on your system says there’s a software update available. You approve the update.

You’re hosed.

Or maybe you don’t approve the update. But that may not matter, because in some cases, auto-update features embedded in certain software titles will go ahead and download the update at that point, and keep nagging you until you agree to install it at a later date.

Evilgrade leveraged a flaw in the updater mechanism for iTunes that could be exploited on Windows systems. Amato described the vulnerability:

“The iTunes program checks that the binary is signed by Apple but we can inject content into the description as it opens a browser, with a malicious binary so that the user thinks its from Apple,” Amato said of his attack tool.

Emails shared with KrebsOnSecurity show that Amato contacted Apple’s security team on July 11, 2008, to warn them that the iTunes update functionality could be abused to push out malicious software. According to Amato, Apple acknowledged receipt of the report shortly thereafter, but it did not contact him about his findings until Oct. 28, 2011, when it sent an email to confirm his name and title for the purposes of crediting him with reporting the flaw in its iTunes 10.5.1 patch release details. Interestingly, Apple chose to continue to ignore the vulnerability even after Amato shipped a significant feature upgrade to Evilgrade in Oct. 2010.

The length of time Apple took to patch this significant security flaw is notable. In May 2006, I undertook a longitudinal study of how long it took Apple to ship security updates for its products. In that analysis, I looked at two years’ worth of patches issued to fix serious security bugs in Apple’s Mac OS X operating system, as well as other Apple software applications like iTunes. I found that on average, 91 days elapsed between the date that a security researcher alerted Apple to an unpatched flaw and the date Apple shipped a patch to fix the problem. In that study, I examined patch times for four dozen flaws, and the lengthiest patch time in that period was 245 days.

Amato said he’s not sure why Apple took so long to fix his bug, which he said should have been trivial to correct.

“Maybe they forgot about it, or it was just on the bottom of their to-do list,” he said.

Public attention to digital surveillance tools being marketed to law enforcement agencies worldwide is spurring discussion about whether antivirus companies are doing all they can to unmask these intruders. Mikko Hypponen, chief research officer for Finnish security firm F-Secure, first blogged about FinFisher in March 2011, when protesters in Egypt took over the headquarters of the Egyptian State Security and gained access to loads of confidential state documents, including those that appear to show the government purchased licenses for the program.

Hypponen said F-Secure has stated unequivocally that it will detect any malware that it knows about, regardless of whether the malware is being actively used by government authorities for surveillance. But he said not all antivirus companies have made similar public commitments.

“There is no real discussion or industry-wide agreement on it,” Hypponen said. “The way it goes down is that [antivirus] companies have no idea which Trojans they get are governmental Trojans or just the usual stuff. There must be many more governmental Trojans that we and others detect but don’t know are being used for government surveillance.”

As for the years that Apple took to patch the iTunes update flaw, Hypponen said he’s stumped, but inclined to give the company the benefit of the doubt.

“It is an unusually long time to patch anything, so it doesn’t make much sense,” he said.

For more details on FinFisher, see Der Spiegel’s fascinating coverage of how this surveillance Trojan was marketed.

One note of caution about upgrading your software that I hope is clear from this post: Staying up-to-date with the latest security patches is one of the surest ways to keep your system secure from malware and intruders. But whenever possible, try to do your updating from a network that you trust and control. Otherwise, you may be placing far too much trust in the security of the update mechanisms built into the software you use.

Update, 3:11 p.m. ET: An earlier version of this story incorrectly stated that Amato was able to exploit the iTunes update flaw on OS X systems. While Apple’s advisory states that this flaw is present on OS X systems that lack the iTunes 10.5.1 patch, Amato said he was unable to replicate the problem on OS X systems during his research.

The U.S. Department of Homeland Security today took aim at widespread media reports about a hacking incident that led to an equipment failure at a water system in Illinois, noting there was scant evidence to support any of the key details in those stories — including involvement by Russian hackers or that the outage at the facility was the result of a cyber incident.

Last week, portions of a report titled “Public Water District Cyber Intrusion” assembled by an Illinois terrorism early warning center were published online. Media outlets quickly picked up on the described incident, calling it the “first successful target of a cyber attack on a computer of a public utility.” But in an email dispatch sent to state, local and industry officials late today, DHS’s Industrial Control Systems Cyber Emergency Response Team (ICS-CERT) said that after detailed analysis, DHS and the FBI “have found no evidence of a cyber intrusion into the SCADA system of the Curran-Gardner Public Water District in Springfield, Illinois.” The ICS-CERT continued:

“There is no evidence to support claims made in the initial Fusion Center report – which was based on raw, unconfirmed data and subsequently leaked to the media – that any credentials were stolen, or that the vendor was involved in any malicious activity that led to a pump failure at the water plant,” the ICS-CERT alert states. “In addition, DHS and FBI have concluded that there was no malicious or unauthorized traffic from Russia or any foreign entities, as previously reported.  Analysis of the incident is ongoing and additional relevant information will be released as it becomes available.”

The statement is the most strongly worded yet from DHS refuting the alleged cyber incident in Illinois. The story broke on Nov. 17, when Joe Weiss, managing partner of Applied Control Solutions, a security consultant for the control systems industry, published a blog post about a disclosure he reported reading from a state terrorism intelligence center about a cyber intrusion into a local water plant that resulted in the burnout of a water pump. The break-in reportedly allowed intruders to manipulate the supervisory control and data acquisition system, or “SCADA” networks that let plant operators manage portions of the facility remotely over the Internet. Within hours of that post, media outlets covering the story had zeroed in on the Curran-Gardner Water District as the source of the report.

Weiss has repeatedly declined to share or publish the report, but he cited large portions of it in my story from last week. The language and details reported in it stand in stark contrast to the DHS’s version of events. According to Weiss, the report, marked sensitive but unclassified, stated:

“Sometime during the day of Nov. 8, 2011, a water district employee noticed problems with a SCADA system. An information technology service and repair company checked the computer logs of the SCADA system and determined the system had been remotely hacked into from an Internet provider address located in Russia. The SCADA system that was used by the water district was produced by a software company based in the US. It is believed the hackers had acquired unauthorized access to the software company’s database and retrieved the usernames and passwords of various SCADA systems, including the water district systems.”

“Over a period of 2-3 months, minor glitches have been observed in remote access to the water district’s SCADA system. Recently, the SCADA system would power on and off, resulting in the burnout of a water pump.”

“This network intrusion is the same method of attack recently used against the MIT Server,” the water district alert stated. “The water district’s attack and the MIT attack both had references to PHPMyAdmin in the log files of the computer systems. It is unknown at this time the number of SCADA usernames and passwords acquired from the software company’s database, and if any additional systems have been attacked as a result of this theft.”

Weiss blogged about the ICS-CERT statement, and said he can’t figure out how the two accounts could be so different. He notes that the day after his blog post, Don Craven, chairman of the Curran-Gardner Water District, was quoted on a local ABC News affiliate television interview saying that there was “some indication that there was a breach of some sort into a software program, a SCADA system, that allows remote access to the wells and the pumps and those sorts of things” (see video below).

“The real thing that bothers me is how could there be such substantial amount of information provided where a lot of it is really a simple yes or no situation,” Weiss said. “Was there a Russian [Internet] address involved or wasn’t there? The Illinois facility also said their technician had observed these abnormalities for 2-3 months. Well, either he did or he didn’t.”

The ICS-CERT communique also mentioned another alleged hacking incident of a water facility in Texas that was widely reported last week. In that incident, a hacker using the nickname “pr0f” claimed to have gained access to a water control systems plant, and posted a series of screen shots to prove his accomplishment.

Regarding the alleged hack in Texas, the ICS-CERT would only say it is still investigating:

“In a separate incident, a hacker recently claimed to have accessed an industrial control system responsible for water supply at another U.S. utility,” the ICS-CERT alert continued. “The hacker posted a series of images allegedly obtained from the system. ICS-CERT is assisting the FBI to gather more information about this incident. ICS-CERT has not received any additional reports of impacted manufacturers of ICS or other ICS related stakeholders related to these events. If DHS ICS-CERT identifies any information about possible impacts to additional entities, it will disseminate timely mitigation information as it becomes available. ICS-CERT encourages those in the industrial control systems community who suspect or detect any malicious activity against/involving control systems to contact ICS-CERT.”

My story from last week quoted Michael Assante, president and CEO of the National Board of Information Security Examiners and a former chief security officer for the North American Electric Reliability Corporation (NERC), expressing concern that initial reporting on cyber-related SCADA incidents often turns out to be inaccurate.

But Weiss said the complete reversal makes no sense, and that “something doesn’t smell right.” By way of example, he points to the fact that while media reports on the claimed hack of the Texas facility made today’s DHS Daily Infrastructure Report, the Illinois incident is noticeably absent from any of the recent editions of that report.

“What this is essentially saying is the state intelligence centers shouldn’t put anything out unless DHS approves it,” Weiss told KrebsOnSecurity. “It says either Illinois is incompetent or DHS is covering something up.”

State fusion centers, most of which were formed under a joint project between DHS and the Justice Department between 2003 and 2007, collect data from government and private sector sources. Some of the centers have produced warnings that have been a tad controversial. For example, a report in 2009 from the Virginia Fusion Center warned that certain historically black colleges were potential hubs for terror related activity, and identified hacktivism as a form of terrorism.

The proprietors of shadowy online businesses that have become synonymous with cybercrime in recent years were arrested in their native Estonia on Tuesday and charged with running a sophisticated click fraud scheme that infected with malware more than four million computers in over 100 countries — including an estimated 500,000 PCs in the United States. The law enforcement action, dubbed “Operation Ghost Click,” was the result  of a multi-year investigation, and is being called the “biggest cybercriminal takedown in history.”

Vladimir Tsastsin, in undated photo.

Estonian authorities arrested six men, including Vladimir Tsastsin, 31, the owner of several Internet companies that have been closely associated with the malware community for many years. Tsastsin previously headed EstDomains Inc. a domain name registrar that handled the registrations for tens of thousands of domains associated with the far-flung Russian Business Network.

Reporting for The Washington Post in September 2008, I detailed how Tsastsin’s prior convictions in Estonia for credit card fraud, money laundering and forgery violated the registrar agreement set forth by the Internet Corporation for Assigned Names and Numbers (ICANN), which bars convicted felons from serving as officers of a registrar. ICANN later agreed, and revoked EstDomains’ ability to act as a domain registrar, citing Tsastsin’s criminal history.

Also arrested were Timur Gerassimenko, 31; Dmitri Jegorov, 33; Valeri Aleksejev, 31; Konstantin Poltev, 28 (quoted in the above-linked stories as the spokesperson for EstDomains); and Anton Ivanvov, 26. All six men were arrested and taken into custody this week by the Estonian Police and Border Guard. A seventh defendant, a 31-year-old Russian national named Andrey Taame, is still at large.

Source: FBI

Indictments returned against the defendants in the U.S. District Court for the South District of New York detail how the defendants allegedly used a strain of malware generically known as DNS Changer to hijack victim computers for the purposes of redirecting Web browsers to ads that generated pay-per-click revenue for the defendants and their clients. U.S. authorities allege that the men made more than $14 million through click hijacking and advertisement replacement fraud.

DNS Changer most often comes disguised as a video “codec” supposedly needed to view adult movies. It infects systems at the boot sector level, hooking into the host computer at a very low level and making it often very challenging to remove. This malware family didn’t just infect Microsoft Windows systems: Several versions of DNS changer would just as happily infect Mac systems as well. Other variants of the malware even hijacked DNS settings on wireless home routers. The FBI has posted several useful links to help users learn whether their systems are infected with DNS Changer.

Feike Hacquebord, senior threat researcher for security vendor Trend Micro, called the arrest the “biggest cybercriminal takedown in history.” In a blog post published today, Hacquebord and Trend detail the multi-year takedown, which involved a number of front companies, but principally an entity that Tsastsin founded named Rove Digital:

In 2009 we obtained a copy of the hard drives of two C&C servers that replaced advertisements on websites when loaded by DNS Changer victims. On the hard drives we found public SSH keys of several Rove Digital employees. These keys allowed the Rove Digital employees to log in on the C&C servers without password, but with their private key. From log files on the servers we were able to conclude that the C&C servers were controlled from Rove Digital’s office in Tartu.

Rove Digital had also been running a fake AV / rogue DNS affiliate program called Nelicash. We were able to download a schema of the infrastructure for the fake AV part. From a Nelicash C&C server we discovered data on victims who bought fake AV software. Among the purchases of victims, there were several test orders placed by employees of Rove Digital from IP addresses controlled by Rove Digital in Estonia and the US. This shows that Rove Digital was directly involved in the sales of the fake AV.

From the same Nelicash C&C server we were also able to download a detailed planning of the deployment of new rogue DNS servers in 2010 and 2011. Every day, Rove Digital spread a new malware sample that changed systems’ DNS settings to a unique pair of foreign servers. We checked DNS Changer Trojans for a couple of days and we learned that these Trojans changed DNS settings of victims exactly according to their plan.

We collected much more evidence but we are unable to include them all here. All of our findings indicate that Rove Digital is committing cybercrimes on a large scale indeed and is directly responsible for the large DNS Changer botnet.

As its name suggests, DNS Changer works by hijacking the domain name system (DNS) server settings on a computer; these settings point to Internet servers that are responsible for translating human-friendly domain names like example.com into numeric Internet addresses that are easier for computers to understand. DNS Changer swapped out victims’ legitimate DNS server settings with the addresses of DNS Servers controlled by Rove Digital. Armed with that control, the defendants could redirect any part of the Web browsing session on an infected user’s computer.

This presented a unique challenge for the law enforcement officials and private security experts who sought to dismantle the fraud network. Experts had identified a large number of rogue DNS servers that were owned by front companies tied to Rove Digital, and indeed secured a court order to seize control over those servers. But experts warned the FBI that seizing the rogue DNS servers without first putting in place a backup system would effectively kill Internet access for the four million computers worldwide that were infected with DNS Changer.

In response, the court appointed the job of swapping out the rogue DNS servers for clean ones to Internet Systems Consortium (ISC), a California nonprofit that maintains BIND, a DNS software package that is widely used throughout the Internet.

“The big concerns came when all the evidence had built up on the law enforcement side, and people said, ‘Hey, there are millions of infected systems whose DNS is wrong,’” said Barry Greene, president and CEO of ISC. “We really wanted to keep people from having their DNS shut down, and everyone calling the help desk at their ISP or security provider to complain that their Internet wasn’t working.”

In a press call with reporters, FBI officials said they would be working with industry to help notify ISPs about customers infected with DNS Changer.

“It’s a complicated cleanup because the malware they put on there is boot-sector stuff,” Greene said. “So we’re not finished. We just finished phase 1, which is law enforcement putting handcuffs on people and making sure we don’t black out people on the ‘Net. The press release and outreach is phase two, and cleanup is phase three. We’ll be doing that for some time, I think.”

Officials from the FBI and the U.S. Attorney for the Southern District of New York said they would seek to extradite the defendants to the United States. An FBI official told reporters that four of the arrested have been charged in Estonia and will probably face trial and any judgment over in that country before being extradited. The FBI said it would concentrate on extraditing two of the men arrested — Anton Ivanov and Valeri Aleksejev — neither of whom were charged in Estonia but were arrested provisionally.

The U.S. government has had some success in extraditing Estonian cybercriminals. Sergei Tsurikov, an Estonian man convicted of participating in the coordinated $9 million ATM heist against RBS Worldpay in late 2008, was extradited to the U.S. last year after serving part of his time in an Estonian prison. Tsurikov is currently being processed through an federal jail in Atlanta.

A copy of the indictments returned against the seven men is available here (PDF). This link from Estonian news outlet Delfi includes several pictures of the arrest and seizure of equipment from Rove Digital properties.