Buying Battles in the War on Twitter Spam

The success of social networking community Twitter has given rise to an entire shadow economy that peddles dummy Twitter accounts by the thousands, primarily to spammers, scammers and malware purveyors. But new research on identifying bogus accounts has helped Twitter to drastically deplete the stockpile of existing accounts for sale, and holds the promise of driving up costs for both vendors of these shady services and their customers.

Twitter prohibits the sale and auto-creation of accounts, and the company routinely suspends accounts created in violation of that policy. But according to researchers from George Mason University, the International Computer Science Institute and the University of California, Berkeley, Twitter traditionally has done so only after these fraudulent accounts have been used to spam and attack legitimate Twitter users.

Seeking more reliable methods of detecting auto-created accounts before they can be used for abuse, the researchers approached Twitter last year for the company’s blessing to purchase credentials from a variety of Twitter account merchants. Permission granted, the researchers spent more than $5,000 over ten months buying accounts from at least 27 different underground sellers.

In a report to be presented at the USENIX security conference in Washington, D.C. today, the research team details its experience in purchasing more than 121,000 fraudulent Twitter accounts of varying age and quality, at prices ranging from $10 to $200 per one thousand accounts.

The research team quickly discovered that nearly all fraudulent Twitter account merchants employ a range of countermeasures to evade the technical hurdles that Twitter erects to stymie the automated creation of new accounts.

“Our findings show that merchants thoroughly understand Twitter’s existing defenses against automated registration, and as a result can generate thousands of accounts with little disruption in availability or instability in pricing,” the paper reads. “We determine that merchants can provide thousands of accounts within 24 hours at a price of $0.02 – $0.10 per account.”

SPENDING MONEY TO MAKE MONEY

For example, to fulfill orders for fraudulent Twitter accounts, merchants typically pay third-party services to help solve those squiggly-letter CAPTCHA challenges. I’ve written here and here about these virtual sweatshops, which rely on low-paid workers in China, India and Eastern Europe who earn pennies per hour deciphering the puzzles.

The Twitter account sellers also must verify new accounts with unique email addresses, and they tend to rely on services that sell cheap, auto-created inboxes at Hotmail, Yahoo and Mail.ru, the researchers found. “The failure of email confirmation as a barrier directly stems from pervasive account abuse tied to web mail providers,” the team wrote. “60 percent of the accounts were created with Hotmail, followed by yahoo.com and mail.ru.”

Bulk-created accounts at these Webmail providers are among the cheapest of the free email providers, probably because they lack additional account creation verification mechanisms required by competitors like Google, which relies on phone verification. Compare the prices at this bulk email merchant: 1,000 Yahoo accounts can be had for $10 (1 cent per account), and the same number Hotmail accounts go for $12. In contrast, it costs $200 to buy 1,000 Gmail accounts.

Finally, the researchers discovered that Twitter account merchants very often spread their new account registrations across thousands of Internet addresses to avoid Twitter’s IP address blacklisting and throttling. They concluded that some of the larger account sellers have access to large botnets of hacked PCs that can be used as proxies during the registration process.

“Our analysis leads us to believe that account merchants either own or rent access to thousands of compromised hosts to evade IP defenses,” the researchers wrote.

Damon McCoy, an assistant professor of computer science at GMU and one of the authors of the study, said the top sources of the proxy IP addresses were computers in developing countries like India, Ukraine, Thailand, Mexico and Vietnam.  “These are countries where the price to buy installs [installations of malware that turns PCs into bots] is relatively low,” McCoy said.

PAYPAL, DOUBLE-DIPPING AND STOCKPILING

The researchers paid for most of the accounts using PayPal, which means that most Twitter account sellers accept credit cards. They also found that freelance merchants selling accounts via Fiverr.com and other sellers not associated with a static Web site were the most likely to resell credentials to other buyers — essentially trying to sell the same accounts to multiple buyers. This was possible because the researchers made the decision not to change the passwords of the accounts they purchased.

They found that bulk-created Twitter accounts sold via Fiverr merchants were also among the shortest lived: 57 percent of the accounts purchased from Fiverr sellers were cancelled during the time of their analysis. In contrast, Web storefronts like buyaccs[dot]com (pictured at left) had only five percent of their purchased accounts eventually detected as fraudulent.

Turns out, most of the Twitter account merchants stockpile huge quantities of accounts in advance of their sale; the researchers determined that the average age of accounts for sale was about 30 days, while some sellers routinely marketed accounts that were more than a year old. For these latter merchants, “pre-aged” accounts appeared to be a proud selling point, although the researchers said they found little correlation between the age of an account and its ability to outlive others after purchase.

THE TAKEDOWN

Twitter did not respond to multiple requests for comment. One of the researchers named in the paper — Berkeley grad student Kurt Thomas — was a Twitter employee at the time of the study; he also deferred comment to Twitter. But the other researchers say they had full cooperation from Twitter to test the efficacy of their merchant profiling techniques. They focused on building unique signatures that could be used to identify accounts registered by each of the 27 merchants they studied, based on qualities such as browser user agent strings, submission timing, signup flow and similarly-named accounts.

Vern Paxson, a professor of computer sciences at UC Berkeley and a key researcher at the International Computer Science Institute, said that in cooperation with Twitter the group analyzed the total fraction of all suspended accounts that appeared to originate from the 27 merchants they tracked. They found that at its peak, the underground marketplace was responsible for registering 60% of all accounts that would go on to be suspended for spamming. During more typical periods of activity, the merchants they tracked contributed 10–20% of all spam accounts.

Paxson said that when Twitter went back and applied the group’s merchant signatures to all of the Twitter accounts registered during the ten months of the study, they were able to disable 95 percent of all fraudulent accounts registered by those 27 merchants, including those previously sold but not yet suspended for spamming. Only .08 percent of those accounts that were cancelled asked to be unsuspended, and the researchers believe that 93 percent of those requests were performed by fraudulent accounts abusing the unsuspend process.

Immediately after Twitter suspended the accounts, the researchers placed 16 new orders for accounts from the 10 sellers with the largest stockpiles; of the 14,067 accounts they purchased, 90 percent were dead on arrival due to Twitter’s previous intervention.

“There was a fair amount of confusion on the [black hat hacker] forums about what Twitter was doing,” Paxson said. When the researchers requested working replacements, one of the merchants responded: “All of the stock got suspended….Not just mine…..It happened with all of the sellers….Don’t know what twitter has done….”

Within a few weeks, however, the bigger merchants were back in business, and the templates the researchers built to detect accounts registered by the various merchants began to show their age: Of the 6,879 accounts they purchased two weeks after Twitter’s intervention, only 54 percent were suspended on arrival.

Nevertheless, Paxson said Twitter is actively working to integrate their techniques into its real-time detection framework to help prevent abuse at signups. The trick, he says, is finding a sustainable way to fine-tune their merchant signatures going forward and continue to stay ahead in the arms race.

“We would love to keep doing this, but the hard part is you kind of have to keep doing the buys, and that’s a lot of work,” Paxson said. “The signatures we have created so far are definitely useful to them and they’ve gotten a lot of traction out of it already in actively suspending accounts that match those signatures. But as soon as the account merchants get wise, they change things slightly and our signatures no longer match.”

As such, the paper concludes that a long term disruption of the fraudulent Twitter account marketplace requires both increasing the cost of account registration — perhaps through additional hurdles such as phone verification — and integrating at-signup time abuse classification into the account registration process.

For more on this research, see: Trafficking Fraudulent Accounts: The Role of the Underground Market in Twitter Spam and Abuse (PDF).