Peter Bennett first suspected his own Web site might have been turned into a spam-spewing zombie on the night of Nov. 11, when he discovered that a tiny program secretly uploaded to his site was forcing it to belch out ads for rogue Internet pharmacies.
Bennett’s site had been silently “infected” via an unknown (at the time) vulnerability in a popular e-commerce software package. While most site owners probably would have just cleaned up the mess and moved on, Bennett — a longtime anti-spam vigilante — took the attack as a personal challenge.
“Spammers always know it is me attacking their resources in whatever form that takes,” Bennett said. “In other words, I make myself a target because I have a clue or two about server security and defense and just love taunting them to crank them up.”
And taunt them he has. For years, the New Zealand resident was part of a ragtag band of anti-spam activists, or “antis,” that helped to bring down infamous pill spammer Shane Atkinson and other junk e-mail purveyors. After taking a break from anti activity in 2007 to pursue other professional goals, Bennett – now 50 – seems eager to jump back into the fray.
In the interim, however, spammers have been refining their techniques. Like reluctant conscripts in a global guerilla army, hundreds — sometimes thousands — of legitimate Web sites are now enslaved each month and sold to criminals who use them to blast out spam and host spam sites. The attackers Bennett is tracking mainly pick on orphaned Web sites running Linux with insecure, unpatched software packages (Bennett says his site was hacked thanks to a zero-day bug in OScommerce, a popular e-commerce software program).
Bennett found that his Web site was part of a larger botnet of at least 1,200 compromised sites that was being used to send roughly 25 million junk e-mail messages each day, although he said it appears the botnet is used for spam runs only intermittently.
“They only run the botnet once a week or so at a time, and then shut it off,” Bennett said.
An ad soliciting EvaPharmacy affiliates.
The hacked sites in the botnet Bennett identified mainly advertise one of three types of rogue pill sites: MyCanadianPharmacy, Canadian Family Pharmacy, and Canadian Health&Care Mall. The latter has been tied to a pharmacy affiliate program called EvaPharmacy, one of the few remaining pharmacy affiliate programs that pays members to promote fly-by-night pill sites via spam.
I’ve separated the hacked sites identified by Bennett into different lists, broken down by the type of pharmacy programs they are currently promoting. A fourth category, labeled “Other,” includes those hacked pages that appear to be mainly erected to beef up the search engine ranking of rogue pharmacy sites. Click the links below for a text file listing the compromised sites in each group (please take care with those links; some of the sites may also host malicious code):
Most people understand the need to keep their computers up-to-date with the latest security patches, and that failure to do so could let bad guys turn those systems in spam zombies. Web sites, as it turns out, also can be zombified if deprived of proper care and feeding. I suspect many Web site owners either aren’t aware of the threat or do not want to apply updates out of laziness or because they have their Web site set up just right and are afraid that patches may screw things up.
“The big problem is many of the web sites get themed, which means upgrading the site is a lot of work and often does not get done,” Bennett said.
In any case, it is clear that a more comprehensive approach to helping people secure their Web sites is long overdue. Unfortunately, the very organizations that should be leading by example — federal and state government Web sites — often set the worst example, and are frequently compromised and hijacked by spammers — sometimes for months or years on end. The U.S. Agency for International Development, for instance, is doing a bang up job helping to develop international spam cartels by hosting dozens of pages promoting rogue online pharmacies.
On Dec. 14, the Obama administration announced that it was teaming up with Google, Microsoft and a host of other tech giants to establish a nonprofit organization targeting illegal Internet pharmacies. Victoria Espinel, the White House intellectual property enforcement coordinator, described the nonprofit group as “comprised of companies that serve as Internet choke points and was in response to a call from the administration for private efforts to police illegal pharmacies.”
It will be interesting to see what — if any — impact this group may have on the overall problem. For now, the job of policing the Internet for illegal pharmacies will continue to fall to individuals and volunteers, like Bennett.
“I’ll keep at it, because I enjoy it,” he said. “But the war on rogue pharmacies has to be fought on many levels. The fact that hackers are creating Web site botnet armies needs to be dealt with, as lack of action just emboldens the hackers.”
Why aren’t all these USG problems being reported to US-CERT by the people that find them?? https://forms.us-cert.gov/report/ US CERT is the official arm that’s supposed to alert federal agencies about exploits.
If they don’t then you have another column to write about how ineffective and unresponsive US CERT is in performing its job and the Administration, therefore, deserves the D it got in Information Security!
Good call L Lou; I imagine Brian has a lot of stories to fall back on about government ineptitude in IT security. He’s just waiting for a slow news day! 🙂
“The big problem is many of the web sites get themed, which means upgrading the site is a lot of work and often does not get done,” Bennett said.
What does “get themed” actually mean?
“theming” here means adding site specific content by manually changing portions of the site code or css.
The problem is that these changes typically rely on specific behavior of the version of the code / css at the time of the modifications.
If you “update” the site, you can easily break either the behavior or the specialized css.
My wife and I use a commercial webhost for two sites. Hers is an image gallery with about 100 GB of hi-res photos and such using coppermine. Mine is for a Linux User Group which I run and uses Joomla and SMF.
I recently found out that both sites had injected PHP exploit code in virtually all the php files. I only discovered it in the first place because one of the changes must’ve broken our forum and so I went looking for the cause of the problem.
It took me a few days of painstaking work to remove the exploit code although in a few cases I had to reinstall the server software.
I was able to analyze some of the code using a base64 decoder and saw references to coppermine in the code so I think it was probably what got originally hacked and allowed the attacker in. The coppermine was about 6 months behind because my wife uses a lot of themes she’s customized and I was scared to break things AND because coppermine has no way (that I can find anyway) of notifying its users of updates other than the users having to go to their forum and find the announcement. I think Joomla might have been 1 point release behind.
All I want it an email list I can sign up for. Joomla has one. SMF doesn’t but at least the updater is built into the administration itself so you can check there quite easily.
I still don’t know what the cost was intended to do as I didn’t keep all the files related to it. Was more interested in just scrubbing the websites.
SANS and Secunia have mailing lists, though they are broad in subject. You could setup a free email account and pump them all in there. Use an email client to setup rules to to forward only the software that you want to see to your real email address. Might be able to use RSS, never looked into that myself.
http://www.sans.org/newsletters/risk/
http://secunia.com/advisories/secunia_security_advisories/
Or a gmail account will allow you to set “filters” to do the same thing, without client software.
Thanks for the suggestions. That certainly helps but yeah, I’d rather not see updates for every vulnerability out there and I’d like to know about bug fixes, too.
This exemplifies how as the good guys batten down the hatches, the bad guys continue to probe for other weaknesses and exploit them. It’s also an indication of how important it is to patch software. I understand the risk that patching may break something. But, any good admin has a solid process in place to test and then have some kind of mitigation plan should things go awry. The consequences to not patching are too great compared to the risks of patches breaking something. 🙁
@xAdmin “But, any good admin has a solid process in place to test and then have some kind of mitigation plan should things go awry.”
For the .gov sites, this should be a no-brainer. But for the small business that downloads “SuperCart2010” to their $7.95-per-month hoster and installs it via the easy-to-use-wizard, standing up a Store in less than 30 minutes, that’s as far as it goes. Even if there is a “software update” button in the Admin control panel (admin/admin, BTW), it’s not likely to be used.
I’ve pointed out hijacked sites to a number of SMB’s, and while all were grateful for the info, none had a clue (or a plan) on how to fix the problem.
Thanks for the reply. That’s what I wanted to say in response but wasn’t sure if anybody would think I was just lazy or dodging my responsibility. I’m not a programmer and wouldn’t know bad code from good code. It took several queries to my webhost (who also doesn’t really consider it their problem either to fix) before they’d explain to me what to look for in the php files to remove.
The first time this happened (because Google classified my site as having malware) they just basically said you have exploit code in your files. I asked how to find it. They said, oh just look at the code inside your php files and remove anything that shouldn’t be there. These are bundled applications. I didn’t even know that the php code is supposed to look like. I ended up removing some very old static webpages thinking they were probably the ones with the exploit code. Months later, the forum was broken so I asked why and they told me there was exploit code. This time when I asked, they told me how to find it by giving me an example of what to look for:
<?php /**/eval(base64_decode('
which was followed by what looked like gibberish but of course, was in fact obsfucated executable code. And they told me it'd be in the top of the files.
Call me uninformed or uneducated if you like, but I've been using computers for 30 some years and didn't know how to look for malware until I was given these instructions.
It's my fault for not keeping my server software up-to-date but it wasn't out of laziness or stupidity. It still takes more time than it should and sometimes breaks things. I'm willing to spend the time (especially after having been burned by it), but a lot of people won't, at least not until it happens. And in many cases, they won't even know it's happened until months later.
Fantastico-type scripts helps but don't always support the software you want, and so far I've only noticed that SMF makes updates actually easy, point and click.
The hacking doesn’t end with websites like Bennet’s, which in this case was used (among other things) as a spam deployment server and a redirection creator.
I note that most of the sites in those lists of urls you link to are still infected.
Here’s an example of the pattern one of these takes:
URL from the text file:
hxxp://www.jordancolecciones.com/images/guest.html
That file contains very simply obfuscated JavaScript which redirects the user to:
hxxp://www.rxsleepmeds.net
Watch your browser’s status bar while that site loads. You’ll notice that all of the images, css files and other assets live on a variety of individual IP addresses:
hxxp://163.22.64.9:8080/images/mcp/logo.jpg
hxxp://163.22.64.9:8080/images/mcp/pp_valentine.jpg
Here’s the complete list of IP’s serving out images for this particular site:
123.100.251.62
163.22.64.9
188.132.221.245
188.132.221.246
95.154.241.38
Here’s where, respectively, each of these is hosted:
Webvisions.net, Singapore
Asia Pacific Network Centre (APNIC) in (I think) China
marsglobaldatacenter.com in Prague, Czech Republic (same for the next one)
Ideal Hosting in Turkey
Each of those IP addresses has also been hacked by someone suppporting the EvaPharmacy group. They’ve been doing this since at least 2006. Each of them is a Unix, or Linux or FreeBSD server running one or two exploits which were custom written for this group.
It doesn’t stop there. The IP address for the target URL is 219.143.16.157.
That’s hosted by Beijing Telecom Corporation. That one may be the only IP actually operated by the EvaPharmacy spammer.
DNS is provided by ns1.rxsleepmeds.net and ns2.rxsleepmeds.net, hosted on the following server IP’s:
60.190.201.133 [Shengzhou Teacher Learning School]
202.100.91.132 [Feitian Internet Company, also in China.]
Any attempt to visit these raw IP addresses shows that none of these was set up to host this particular website, or its images. Seriously: would a Teacher Learning School be hosting a rogue pharmacy site to pay the bills? I guess it’s possible but come on. Here’s one example:
hxxp://163.22.64.9/
Many of them show no website at all, indicating that they weren’t even intended to host a website at all. This explains why these hackers use port 8080. Whatever the server was originally intended to do, it now also hosts images for these websites in a distributed fashion.
Each of these websites has a very poor, very easy to guess root password. This hacking has been extremely widespread and nobody has done anything about it. Reporting these IP addresses results in absolutely zero action from anyone. The criminals behind EvaPharmacy and their infrastructure know this, and they’ve been provided what is essentially “free” (aka: stolen) web hosting, DNS and asset distribution for many years now. Everything from the server that sends the spam to the server that places the redirections to each of the servers that support the output of the target website has been hacked by this group. For this one site alone I showed eight distinct servers, only one of which may not have been hacked by this group, and that’s on top of Bennet’s server which was also hacked by this group.
ISP’s are completely ineffective at stopping this or preventing it. As long as that’s still the case, these widespread hacks are going to continue unabated.
SiL
marsglobaldatacenter.com may be bulletproof hosting too. Their website doesn’t show much interest in attracting business, and some sites that come up on reverse IP of 219.143.16.157 are currently hosted at 188.132.221.244. There also seems to be difference of opinion whether they are located in Prague or Turkey:
http://www.magic-net.info/black-list-checker.dnslookup?black=188.132.129.13&Check_RBL=Check
By my reckoning there are 887 compromised websites in those lists. They range from
720sticks.com
8d8d.co.cc
909saloon.com
abendmusik.com.mx
adsel.net.pl
to
ywisdom.com
z7chin.com
zapercomercial.com.br
zaraf11.home.ro
Each web site’s domain name will have a registrar, a hosting IP with an ISP, possibly a registrant email address in the WHOIS, and probably an email address or feedback form in the web site’s Contact Us page.
It would be great to have an automated process to alert the relevant contacts that these 887 web sites are compromised, and some helpful advice on how to upgrade the server to make it less vulnerable.
Getting web server admins to act on compromised servers is not an easy task, especially at this volume.
“Getting web server admins to act on compromised servers is not an easy task, especially at this volume.”
In many cases, the preferred method of contact consists of a webform on the site, so that is under the control of the spammers, too. And a lot of the sites were set up by people with short attention spans who haven’t looked at them in years.
Sometimes a host or registrar will relay a message to the website owner (which is helpful, since email from them is less likely to be mistaken for spam/scams, and because they may have a better contact email address than the public whois). But others simply refuse to get involved because they say their customers’ compromised websites are not their problem.
One problem for long running sites is that junior or entry sys-admins cutting their teeth on “stable sites” are neither trained nor taught some diagnostic fundamentals and are instilled with keeping status quo by management.
Long gone is the experienced team that setup the site or consulted. The contract is in “maintenance” and likely any upgrade is a billable cost to the customer.
The answer is to bake in the effort to regularly reference http://www.dnsbl.info/ and other sites like it. The results properly interpreted are important.
Bake in running portscanners/webscanners on a variable schedule against ones own site. Most often its is also a major newbie blunder to not know how to to this.
Its not like anyone hasn’t figured out the HIDS or link validity checkers; let alone entire website CMS that can aid in detecting tampering.
Once the basics these are done the admin can move up to weblog and fw log analysis.
If people shift paradigm; to monitor by the “bit” not flat rate traffic they would really look into why they have traffic spikes at 2am on an odd Thursday. Website checking and fw logs would also reveal all the re-directed traffic.
The tools we have are already here. Its time to use them often, well and insist they get baked into the hosting/ISP system as de facto.
Its amazing how organizations chase SEO but don’t do the basics of IT housekeeping which yield a treasure of customer contacts anyhow.
Look I really can’t see what all the fuss is about here. Perhaps someone would like to explain it to me?? Now please don’t misunderstand me, I do NOT like spam emails.
However I run a number successful e-commerce sites and I pay Hostgator a set amount of money each month for unlimited space and bandwidth. If somebody cracks my sites and they start sending mass emails I’m no better off if I stop it speaking frankly. I still pay the same amount of money and if I “muck about” with my websites there is the possibility of them going offline and losing me money! How am I a winner in this scenario exactly??
That’s why I’d just let it ride and send the emails. Do I like it? No! Would I do anything about it? NO!
Good point Julie;
And thus is the problem. This is where I feel, if the government really wants to help reduce the problem, they need to provide some kind of assistance, both technical and tax wise(as in tax break).
This way, maybe the problem could be mitigated without a break in service, and repair costs could be managed.
Just a side comment: You must have pretty good spam filtering for your mail box, for you to be publishing it like that!
Would “Julie” really be so totally clueless as to say something like that and then post her real email address? More likely she has an enemy who wants to bring the wrath of the spam-burdened masses down on her head by posting a troll message like that and putting her name on it.
Obviously, having spam sent from your own server’s IP address will get your site blacklisted everywhere within hours. Even sending an automated vacation message to the spoofed “from” address on a spam email may get you on a blacklist that will make it impossible to send your own legitimate emails through your mailserver. Anyone who runs “several successful ecommerce sites” ought to know that having a spam spewing trojan sharing her IP address would be internet suicide. And if her domain got red ratings from MyWot and SiteAdvisor as a result, she’d have a lot of difficulty repairing her reputation even if she moved to a different host.
Yes AlphaCentauri:
However, I meet very smart business people every day, who are otherwise clueless to impending disaster on their web side of the business.
Well, I’m trying to be charitable in case Julie is a victim here. If she’s really that clueless, her self-centered attitude is going to end up giving her a major dose of bad karma.
They might do more than just use your sites to send spam. They might harvest your customers emails and information and sell it to the highest bidder. They might use your websites to host phishing websites so customers expose their login information which would let hackers access their credit card information and could lead to identity theft. Your sites may stop functioning altogether. That was how I discovered my site had been exploited, when one of them broke.
“On Dec. 14, the Obama administration announced that it was teaming up with Google, Microsoft and a host of other tech giants to establish a nonprofit organization targeting illegal Internet pharmacies.”
So my question is this: With many of the best and brightest tech giants located here in the US, why doesn’t the government team with Google and Microsoft on a grander scale to battle this type of attack? You would think that all three parties would have a vested interest in protecting our network from external attackers.
Online security is no longer a game of “I’ll protect my stuff, you protect yours, and everyone will be happy.” Now we need to work to protect each other, since everyone’s security is dependent on everyone else’s.
I second that notion!
“[W]hy doesn’t the government team with Google and Microsoft on a grander scale to battle this type of attack?”
Two reasons:
(1) There might be some First Amendment problems here.
(2) Private enterprise doesn’t want to give the government any more access to snoop around without a subpoena than it already has.
Bennett is doing great work! Thanks for bringing one forward. Legitimate Canadian pharmacies carrying the CIPA Seal do not engage in any spam. One of our top safety messages for online pharmacy is that patients should never order medications from a spam message. I regularly speak to callers and answer emails regarding the Bad sites you’ve documented in your article and people are relieved they contacted us before falling prey to the spam offer. There are guidelines for safely ordering prescription drugs online at http://www.cipa.com.
Bennett is a known net abuser and spammer – don’t buy this rubbish. He was kicked from a major NZ isp for spamming and abuse, took them to Court to ‘clear his name’ and lost. On the day he lost he fired off hundreds of spam emails via a semi-anonymous remailer purporting to be from a usenet nobody, ‘Jamie Baillie’.
This man is a menace and an idiot, he is not some kind of hero.