A ridiculous number of companies are exposing some or all of their proprietary and customer data by putting it in the cloud without any kind of authentication needed to read, alter or destroy it. When cybercriminals are the first to discover these missteps, usually the outcome is a demand for money in return for the stolen data. But when these screw-ups are unearthed by security professionals seeking to make a name for themselves, the resulting publicity often can leave the breached organization wishing they’d instead been quietly extorted by anonymous crooks.
Last week, I was on a train from New York to Washington, D.C. when I received a phone call from Vinny Troia, a security researcher who runs a startup in Missouri called NightLion Security. Troia had discovered that All American Entertainment, a speaker bureau which represents a number of celebrities who also can be hired to do public speaking, had exposed thousands of speaking contracts via an unsecured Amazon cloud instance.
The contracts laid out how much each speaker makes per event, details about their travel arrangements, and any requirements or obligations stated in advance by both parties to the contract. No secret access or password was needed to view the documents.
It was a juicy find to be sure: I can now tell you how much Oprah makes per event (it’s a lot). Ditto for Gwyneth Paltrow, Olivia Newton John, Michael J. Fox and a host of others. But I’m not going to do that.
Firstly, it’s nobody’s business what they make. More to the point, All American also is my speaker bureau, and included in the cache of documents the company exposed in the cloud were some of my speaking contracts. In fact, when Troia called about his find, I was on my way home from one such engagement.
I quickly informed my contact at All American and asked them to let me know the moment they confirmed the data was removed from the Internet. While awaiting that confirmation, my pent-up frustration seeped into a tweet that seemed to touch a raw nerve among others in the security industry.
The same day I alerted them, All American took down its bucket of unsecured speaker contract data, and apologized profusely for the oversight (although I have yet to hear a good explanation as to why this data needed to be stored in the cloud to begin with).
This was hardly the first time Troia had alerted me about a huge cache of important or sensitive data that companies have left exposed online. On Monday, TechCrunch broke the story about a “breach” at Apollo, a sales engagement startup boasting a database of more than 200 million contact records. Calling it a breach seems a bit of a stretch; it probably would be more accurate to describe the incident as a data leak.
Just like my speaker bureau, Apollo had simply put all this data up on an Amazon server that anyone on the Internet could access without providing a password. And Troia was again the one who figured out that the data had been leaked by Apollo — the result of an intensive, months-long process that took some extremely interesting twists and turns.
That journey — which I will endeavor to describe here — offered some uncomfortable insights into how organizations frequently learn about data leaks these days, and indeed whether they derive any lasting security lessons from the experience at all. It also gave me a new appreciation for how difficult it can be for organizations that screw up this way to tell the difference between a security researcher and a bad guy.
THE DARK OVERLORD
I began hearing from Troia almost daily beginning in mid-2017. At the time, he was on something of a personal mission to discover the real-life identity behind The Dark Overlord (TDO), the pseudonym used by an individual or group of criminals who have been extorting dozens of companies — particularly healthcare providers — after hacking into their systems and stealing sensitive data.
The Dark Overlord’s method was roughly the same in each attack. Gain access to sensitive data (often by purchasing access through crimeware-as-a-service offerings), and send a long, rambling ransom note to the victim organization demanding tens of thousands of dollars in Bitcoin for the safe return of said data.
Victims were typically told that if they refused to pay, the stolen data would be sold to cybercriminals lurking on Dark Web forums. Worse yet, TDO also promised to make sure the news media knew that victim organizations were more interested in keeping the breach private than in securing the privacy of their customers or patients.
In fact, the apparent ringleader of TDO reached out to KrebsOnSecurity in May 2016 with a remarkable offer. Using the nickname “Arnie,” the public voice of TDO said he was offering exclusive access to news about their latest extortion targets.
Snippets from a long email conversation in May 2016 with a hacker who introduced himself as Adam but would later share his nickname as “Arnie” and disclose that he was a member of The Dark Overlord. In this conversation, he is offering to sell access to scoops about data breaches that he caused.
Arnie claimed he was an administrator or key member on several top Dark Web forums, and provided a handful of convincing clues to back up his claim. He told me he had real-time access to dozens of healthcare organizations they’d hacked into, and that each one which refused to give in to TDO’s extortion demands could turn into a juicy scoop for KrebsOnSecurity.
Arnie said he was coming to me first with the offer, but that he was planning to approach other journalists and news outlets if I declined. I balked after discovering that Arnie wasn’t offering this access for free: He wanted 10 bitcoin in exchange for exclusivity (at the time, his asking price was roughly equivalent to USD $5,000).
Perhaps other news outlets are accustomed to paying for scoops, but that is not something I would ever consider. And in any case the whole thing was starting to smell like a shakedown or scam. I declined the offer. It’s possible other news outlets or journalists did not; I will not speculate on this matter further, other than to say readers can draw their own conclusions based on the timeline and the public record.
WHO IS SOUNDCARD?
Fast-forward to September 2017, and Troia was contacting me almost daily to share tidbits of research into email addresses, phone numbers and other bits of data apparently tied to TDO’s communications with victims and their various identities on Dark Web forums.
His research was exhaustive and occasionally impressive, and for a while I caught the TDO bug and became engaged in a concurrent effort to learn the identities of the TDO members. For better or worse, the results of that research will have to wait for another story and another time.
At one point, Troia told me he’d gained acceptance on the Dark Web forum Kickass, using the hacker nickname “Soundcard“. He said he believed a presence on all of the forums TDO was active on was necessary for figuring out once and for all who was behind this brazen and very busy extortion group.
Here is a screen shot Troia shared with me of Soundcard’s posting there, which concerned a July 2018 forum discussion thread about a data leak of 340 million records from Florida-based marketing firm Exactis. As detailed by Wired.com in June 2018, Troia had discovered this huge cache of data unprotected and sitting wide open on a cloud server, and ultimately traced it back to Exactis.
Vinny Troia, a.k.a. “Soundcard” on the Dark Web forum Kickass.
After several weeks of comparing notes about TDO with Troia, I learned that he was telling random people that we were “working together,” and that he was throwing my name around to various security industry sources and friends as a way of gaining access to new sources of data.
I respectfully told Troia that this was not okay — that I never told people about our private conversations (or indeed that we spoke at all) — and I asked him to stop doing that. He apologized, said he didn’t understand he’d overstepped certain boundaries, and that it would never happen again.
But it would. Multiple times. Here’s one time that really stood out for me. Earlier this summer, Troia sent me a link to a database of truly staggering size — nearly 10 terabytes of data — that someone had left open to anyone via a cloud instance. Again, no authentication or password was needed to access the information.
At first glance, it appeared to be LinkedIn profile data. Working off that assumption, I began a hard target search of the database for specific LinkedIn profiles of important people. I first used the Web to locate the public LinkedIn profile pages for nearly all of the CEOs of the world’s top 20 largest companies, and then searched those profile names in the database that Troia had discovered.
Suddenly, I had the cell phone numbers, addresses, email addresses and other contact data for some of the most powerful people in the world. Immediately, I reached out to contacts at LinkedIn and Microsoft (which bought LinkedIn in 2016) and arranged a call to discuss the findings.
LinkedIn’s security team told me the data I was looking at was in fact an amalgamation of information scraped from LinkedIn and dozens of public sources, and being sold by the same firm that was doing the scraping and profile collating. LinkedIn declined to name that company, and it has not yet responded to follow-up questions about whether the company it was referring to was Apollo.
Sure enough, a closer inspection of the database revealed the presence of other public data sources, including startup web site AngelList, Facebook, Salesforce, Twitter, and Yelp, among others.
Several other trusted sources I approached with samples of data spliced from the nearly 10 TB trove of data Troia found in the cloud said they believed LinkedIn’s explanation, and that the data appeared to have been scraped off the public Internet from a variety of sources and combined into a single database.
I told Troia it didn’t look like the data came exclusively from LinkedIn, or at least wasn’t stolen from them, and that all indications suggested it was a collection of data scraped from public profiles. He seemed unconvinced.
Several days after my second call with LinkedIn’s security team — around Aug. 15 — I was made aware of a sales posting on the Kickass crime forum by someone selling what they claimed was “all of the LinkedIN user-base.” The ad, a blurry, partial screenshot of which can be seen below, was posted by the Kickass user Soundcard. The text of the sales thread was as follows:
Soundcard offering to sell what he claimed was all of LinkedIn’s user data, on the Dark Web forum Kickass.
“KA users –
I present you with exclusive opportunity to purchase all (yes ALL) of the LinkedIN user-base for the low low price of 2 BTC.
I found a database server with all LinkedIN users. All of user’s personal information is included in this database (including private email and phone number NOT listed on public profile). No passwords, sorry.
Size: 2.1TB.
user count: 212 million
Why so large for 212 million users? See the sample data per record. There is lot of marketing and CRM data as well. I sell original data only. no editz.
Here is index of server. The LinkedIN users spread across people and contacts indexes. Sale includes both of those indexes.
Questions, comments, purchase? DM me, or message me – soundcard@exploit[.]im
The “sample data” included in the sales thread was from my records in this huge database, although Soundcard said he had sanitized certain data elements from this snippet. He explained his reasoning for that in a short Q&A from his sales thread:
Question 1: Why you sanitize Brian Krebs’ information in sample?
Answer 1: Because nothing in life free. This only to show i have data.
I soon confronted Troia not only for offering to sell leaked data on the Dark Web, but also for once again throwing my name around in his various activities — despite past assurances that he would not. Also, his actions had boxed me into a corner: Any plans I had to credit him in a story for eventually helping to determine the source of the leaked data (which we now know to be Apollo) became more complicated without also explaining his Dark Web alter ego as Soundcard, and I am not in the habit of omitting such important details from stories.
Troia assured me that he never had any intention of selling the data, and that the whole thing had been a ruse to help smoke out some of the suspected TDO members.
For its part, LinkedIn’s security team was not amused, and published a short post to its media page denying that the company had suffered a security breach.
“We want our members to know that a recent claim of a LinkedIn data breach is not accurate,” the company wrote. “Our investigation into this claim found that a third-party sales intelligence company that is not associated with LinkedIn was compromised and exposed a large set of data aggregated from a number of social networks, websites, and the company’s own customers. It also included a limited set of publicly available data about LinkedIn members, such as profile URL, industry and number of connections. This was not a breach of LinkedIn.”
It is quite a fine line to walk when self-styled security researchers mimic cyber criminals in the name of making things more secure. On the one hand, reaching out to companies that are inadvertently exposing sensitive data and getting them to secure it or pull it offline altogether is a worthwhile and often thankless effort, and clearly many organizations still need a lot of help in this regard.
On the other hand, most organizations that fit this description simply lack the security maturity to tell the difference between someone trying to make the Internet a safer place and someone trying to sell them a product or service.
As a result, victim organizations tend to react with deep suspicion or even hostility to legitimate researchers and security journalists who alert them about a data breach or leak. And stunts like the ones described above tend to have the effect of deepening that suspicion, and sowing fear, uncertainty and doubt about the security industry as a whole.
Another real good article, keep them coming !
Desentupimentos 24 horas em Lisboa
Really interesting account….!
However, having been the victim of multiple company breaches and security intrusions, stolen Medical office computers, Anthem, OPM, IRS, and more, I don’t know….I can’t really feel sorry for the victim companies when they have slack a** security and then are politely, or rudely, shown that they should have kept a lock on the barn door.
I concur with Sarah. I’ve had my data hacked at least four times now that I’m aware of. How many other times has it been left out for all to see and exploit by companies or government agencies that don’t even employ minimally effective security measures? I have zero (yep, ZERO) empathy for anyone who collects personal data who fails to safeguard that information, and who then is exposed to public anger. Wouldn’t it be wonderful if hackers would be monetarily rewarded by the companies or agencies for exposing lousy security, instead of being ostracized or at the very least criticized? Save “victim shaming” for the entity who lost my data. Those entities have earned every bit of negative publicity they receive. I think the hackers (or “security consultants” if you prefer) are doing the real ‘victims’ (me, you, other persons who had very personal information carelessly “lost) a very real, “good Samaritan” like service. Many thanks to ’em.
I think part of the problem with some smaller and mid sized companies is the cost vs benefit. You can spend 100,000 or more on security, hiring analysts etc, but if a determined hacker wants to get in, they get in. It doesnt seem to matter if youre military grade security or not, everybody is getting hacked. Banks, governments, even security companies themselves. I understand we want to make things harder for the criminals, but how much are you willing to spend to do so, especially if you do, then get hacked anyway?
Well, don’t forget that part of the benefit is also damage mitigation. The idea isn’t just to keep the bad guys out, it’s also to provide ways to investigate if they do get in, limit what they get or what they damage if they get in in, trace as best you can what they took or compromised, and fix what they did.
Too much of the image of infosec is about keeping the bad guys out. That’s a huge part of it, but another huge part is handling things if they do get in. That’s actually a very large part of the profession.
Its called going undercover Brian. Why make it hard for good guys to do their jobs? Yeesh.
I don’t think undercover LEOs offer to sell the private info of non-undercover LEOs (even as a feint) without permission/coordination etc. Krebs indicates this wasn’t the first breach of trust.
No one is claiming the authority of LE in the article – and yes undercover LE pretend to sell out other LE sll the time it’s called a setup. Read the book by FBI undercover legend Jack Falcone called “Making Jack Falcone “
Alex is right. There is a right way and a wrong way to engage threat actors. The wrong way involves making a fake sales post and prompting the affected company to issue a statement about how your sales post is not related to an actual breach that happened against them.
There are actual, real threat intelligence companies that speak to threat actors, and none of them do crap like this.
If he had asked LinkedIn for money, this might count as racketeering. This is why people don’t trust security researchers.
@secret actually, people don’t trust security researchers because they are also a random stranger who can access their data/life/info/private things just as easily or if they are a good security researcher then easier than a blackhat/criminal… Just because you call yourself a whitehat doesn’t mean people are okay with you also having access to their personal lives. Take this for example: a deranged shadow lurking rapist creep hacks your Webcam and enjoys watching you jerk off with a butt plug or, a security researcher finds a hole (no pun intended) and access to your camera, and subsequently sees the same act as the weirdo, are you still not as equally embarrassed /pissed off/feel exploited and paranoid is either of the 2 happened? It’s a catch 22—it’s not supposed to be, nor will it ever be, a glorious image pasted onto security researchers—if that’s what your looking for, I suggest going to the dark side where you can brag and look cool to your fellow blackhat peers, because it ain’t gunna happen on the other side. If you’re a whitehat, your in it for the cause and because you actually care, and that should be more than enough validation and satisfaction then what the average Joe thinks of you.
Undercover LEOs DO sell stuff to entrap people. They have IA to cover their a**, a brotherhood of constituents. There are also lots of dirty cops and have been since who-knows-when. They set up sting ops to sell drugs and to entice people to ’employ’ sex workers.
Literally, they will confiscate drugs, turn around and sell it to bust distributors and buyers. Same for prostitution but they have smokin hot cop ladies impersonating prostitutes to bust “johns”. The difference being: taxpayers pay for and legitimize their efforts; security researchers are in some cases freelance…….maybe mercenaries of a sort. If a hacker (white, grey, or black hat) is engaged in ‘research’ without permission, they might claim white hat, but essentially they’re taking without permission they’re thieves and should be equally prosecuted as any outright black hat hacker would be subject to.
In this new era of the EU’s GDPR, it will be interesting to see whether they go after the idiot companies (many of whom do business globally and are subject to these regulations), who fail to take the most basic steps to secure their data.
Sadly, we have no such regulations here in the States — perhaps one reason that so many companies just don’t seem to care about data security. They don’t have any (dis)incentive.
Very well written. Enjoyed.
Oh how the mighty have fallen, Brian. It was only a matter of time before you run out of relevant and interesting material and now write about the researchers that feed you stories.
That’s kind of what he’s been doing for a while now by mostly targeting the under-18 script kiddies that feed him stories, you don’t really see Krebs writing about the more serious actors.
How does Brian dare to give us this free not so relevant and interesting material.
Arnie has quite the sense of humor
Researchers have an obligation to walk a fine line… if you go off that line you should be called out.
What I find interesting is Brian sits on the board of another small cybersecurity startup company that does exactly the same thing he is describing here. The difference is he writes about Hold all the time and in those cases, the information discovered is acceptable and evidently does not walk the same grey line.
Please.
Brain’s panties are obviously in a bunch over something, and i bet it aint listed in this article.
@Hold
someone seems to be angry… and/or jealous.
Relax and breath.
Says right there what he’s upset about. That someone is using his name without asking him first.
Pfft… nice try. Might be time to get back under your bridge…
Good Story. Though, I find it plain unnerving when companies like LinkinIn deny network breaches, but their user data has been compromised thru some other source, and many times used by marketing firms, who then were breached. Your tweet said it all.
The exposed LinkedIn data is the public profile data. It’s SUPPOSED TO BE EXPOSED. That’s why this “massive breach” and the researcher that peddled it is so fucking lame.
Interesting story. Hum? Folks, ready to critique, but forgot to aim? The interesting part, the cloud is not encrypted. Oh, who would have guessed the host is ready to read. Guess they will have to change their policy’s. And update their services. Maybe charge more for encryption.
And the other part, remember not all the cowboys wore white hats, and therefore “you” should trust all white hatted cowboys? Or researchers?
Geez, a little fish in a big pond suddenly comes up with a monster index out of nowhere and admires BK sooo much that he just can’t resist name dropping. And to use something so easily found to be fake? Horse before cart thing…
No intention of selling? Yeah, right. That would go over well with someone who just handed over ~$12000.
You write: “had exposed thousands of speaking contracts via an unsecured Amazon cloud instance.” — so I’m curious what this means exactly. Is it an S3 bucket? An EC2 instance? An open MongoDB port?
I definitely get the impression that the “linkedin” TB database is via some kind of open port on a noSQL like MongoDB (which regularly is used without authentication).
Anyway, please keep up the work. The longer articles take a lot of time to write/edit. If only one of the many distributed systems of trust were used more to authenticate reputations.
I’m beginning to wonder when we, your faithful readers, are going to find out that “Brian Krebs” (alt name “Bribe Snark) is really a 20-something Ukrainian created to fake us into believing there really are good people in the world.
OK, just kidding…..?
lol, I met Brian once during a conference. He’s much older than the photo, but eh, my photo’s online are 10 to 20 years old too.
Is that you, Pasha? How many anagrams did it take you to come up with that? I’m totally stealing it.
This sounds like the government: “they will eventually sell it, leak it, lose it, or get hacked and be relieved of it.”
That probably inspired Tom Clancy to say: “The information is all out there, if you go looking for it, and the classified stuff just comes from analyzing the unclassified stuff and connecting the dots… “
I’d like to see a discussion of the recent Newegg breach. I had to change my credit card as a result, because I had shopped with them during the exposure period. This breach was revealed in quite a few tech journo sites but not with the Krebs treatment.
“Researchers have an obligation to walk a fine line… if you go off that line you should be called out.”
Agreed. Obviously this individual had no problem reaching out to Brian numerous other times. Suddenly it’s too much trouble to tell him he is using his sanitized data in a sting? Inconsiderate at best, unprofessional in my eyes. White Hats aren’t assumed, they have to be maintained by specific behavior.
Nice post
http://www.fb.com Nice post
>When Security Researchers Pose as Cybercrooks, Who Can Tell the Difference?
What about when it’s the other way around?
I like that this sloppy PhD student (on Twitter) also claims to be a full PhD (on LinkedIn).
The story makes it sounds like he’s a weasel, but also a good, clever source.
So I wonder why the author decided to out him.
What I find interesting is Brian sits on the board of another small cybersecurity startup company that does exactly the same thing he is describing here. The difference is he writes about Hold all the time and in those cases, the information discovered is acceptable and evidently does not walk the same grey line.
Please.
Brain’s panties are obviously in a bunch over something, and i bet it aint listed in this article.
Only need to submit your comments once here, Vinny. Congrats on the PhD.
Congratulations on completing the PhD.
I reread this article after reading your comments, because I was concerned that BK outed you for pissing him off. Then it occurred to me that you left him no choice.
First off, BK’s board title is honorary, not a voting position. You’d know that if you bothered to read their disclosures. He’s got no reason to out you for that.
If Holden screwed up by becoming the subject of an investigative report, BK would have an ethical obligation to resign the title and tell the full story.
Next, the classes for a PhD should have taught you that it’s necessary for researchers to keep your hands clean. Getting the bad guy isn’t your job. Focus instead on things you are supposed to do with your smarts: mitigation of risk, protect systems and property, increase knowledge, explore flaws.
Cops are given leeway to limit economic and physical harms through the Dark Arts of deception, undercover surveillance, and traps. Leave oppressive government tactics to them.
Something else you should’ve learned in PhD classes: don’t show dirty laundry to a journalist who’s ethically required to reveal it. That’s how you go from source to subject.
I’m only getting involved to say thank you to Readership1 for pointing out that my twitter needed to be updated. I am a full PhD now. It’s changed.
So then “bingo/Vinny” since you know so much about brians panties, why are they in a bunch?
1. Many people trying to use the cloud / AWS are untrained (aka idiots).
2. Why does AWS think they are not?
3. Setting data to public read should be a very difficult thing to do.
4. AWS is mostly to blame here.
Came across an interesting article called “How Russian Spies Infiltrated Hotel Wi-Fi to Hack Victims Up Close” on Wired.com. It looks like Russians got caught outside a couple of buildings with equipment they should not have had outside the buildings of their targets. This might be something Krebs will want to look a little deeper into since he has more connections than the Wired authors.
https://www.wired.com/story/russian-spies-indictment-hotel-wi-fi-hacking/
“This is not spy versus spy.” That author should’ve pointed out that the Mad magazine comic strip always had the spies trying to hurt each other.
One of the techniques used by the “Russian spies,” Wifi spoofing, was mentioned in this (1) article. In it, Brian describes a convenience/flaw of mobile phones: they readily connect to wifi networks that are using the same name as one already trusted.
The same flaw/convenience exists in laptops and tablets. A spy would only need to set up a fake Wifi hotspot with the same SSID as the hotel’s wifi network to manipulate what victim devices transmit online.
A similar exploit technique is used by free and democratic governments to spy on their citizenry with “stingray” devices (2) that mimic cell phone towers, by the way. These things can track phone locations and intercept data in real time. Lovely not.
(1) https://krebsonsecurity.com/2016/04/new-threat-can-auto-brick-apple-devices/
(2) https://nakedsecurity.sophos.com/?s=stingray
Hi ! Yesterday, bank has denied my request for a checking account. For that moment I didn’t know that banks have my credit history. So I went to the internet and found information on https://checkgain.com/ about ChexSystems database, report and score. Now I know why they denied my account.
+×+×+×+×+×+×+×+×+×+×+×
I suspect your comment is spam, based on grammar and that the website you describe is not ranked well by Alexa.com or SimilarWeb.com, and the domain was registered anonymously.
Nor is it related to a real financial site.
Definitely comment spam, Readership. I noticed he/she left the same comments a couple times in another one of Kreb’s recent posts, but was addressing someone named “Aldus”—totally out of context and unrelated to the post.
I don’t recommend visiting that link.
Good read.
Now Bloomberg is reporting that a major telecom has had similar chips implanted and actually has the Co-CEO of Sepio sited as the source.
See https://www.bloomberg.com/news/articles/2018-10-09/new-evidence-of-hacked-supermicro-hardware-found-in-u-s-telecom