Verizon Enterprise Solutions, a B2B unit of the telecommunications giant that gets called in to help Fortune 500’s respond to some of the world’s largest data breaches, is reeling from its own data breach involving the theft and resale of customer data, KrebsOnSecurity has learned.
Earlier this week, a prominent member of a closely guarded underground cybercrime forum posted a new thread advertising the sale of a database containing the contact information on some 1.5 million customers of Verizon Enterprise.
The seller priced the entire package at $100,000, but also offered to sell it off in chunks of 100,000 records for $10,000 apiece. Buyers also were offered the option to purchase information about security vulnerabilities in Verizon’s Web site.
Contacted about the posting, Verizon Enterprise told KrebsOnSecurity that the company recently identified a security flaw in its site that permitted hackers to steal customer contact information, and that it is in the process of alerting affected customers.
“Verizon recently discovered and remediated a security vulnerability on our enterprise client portal,” the company said in an emailed statement. “Our investigation to date found an attacker obtained basic contact information on a number of our enterprise customers. No customer proprietary network information (CPNI) or other data was accessed or accessible.”
The seller of the Verizon Enterprise data offers the database in multiple formats, including the database platform MongoDB, so it seems likely that the attackers somehow forced the MongoDB system to dump its contents. Verizon has not yet responded to questions about how the breach occurred, or exactly how many customers were being notified.
The irony in this breach is that Verizon Enterprise is typically the one telling the rest of the world how these sorts of breaches take place. I frequently recommend Verizon’s annual Data Breach Investigations Report (DBIR) because each year’s is chock full of interesting case studies from actual breaches, case studies that include hard lessons which mostly age very well (i.e., even a DBIR report from four years ago has a great deal of relevance to today’s security challenges).
According to the 2015 report, for example, Verizon Enterprise found that organized crime groups were the most frequently seen threat actor for Web application attacks of the sort likely exploited in this instance. “Virtually every attack in this data set (98 percent) was opportunistic in nature, all aimed at easy marks,” the company explained.
It’s a fair bet that if cyber thieves buy all or some of the Verizon Enterprise customer database, some of those customers may be easy marks for phishing and other targeted attacks. Even if it is limited to the contact data for technical managers at companies that use Verizon Enterprise Solutions, this is bound to be target-rich list: According to Verizon’s page at Wikipedia, some 99 percent of Fortune 500 companies are using Verizon Enterprise Solutions.
What’s the “closely guarded underground cybercrime forum”?
He’d tell you… but then he’d have to kill you.
Right before the SWAT team mows him down….
Closely guarded, prolly at the behest of anti-malware multinationals – already guilty of extortion – who would stand to lose the most were “cybercrime” to suddenly end. Kinda like cops dealing crack or psychiatrists prescribing methamphetamine – if you CAN guarantee an income, why wouldn’t you? Corporations assuring their future is natural progression – it ain’t like VW didn’t lie, Standard Oil didn’t kill mass transit or Johns-Manville didn’t conceal the dangers of asbestos…
If you don’t own stock in producers of aluminum, you should. Then you’d stand to profit from spreading complete nonsense on the internet.
Actually truer than you fear… https://en.wikipedia.org/wiki/General_Motors_streetcar_conspiracy
That would be the TOR anonymous forum.
Whoops. It does not inspire one with confidence when the big boy on the block craps the bed.
Nothing will top the OPM breach–ever.
Don’t say that. There’s still that huge data center complex in Utah.
And walmart hasn’t had their personnel/pos systems breached yet either that we know of although I’m certain that bad actors are trying.
I think it’s more likely even criminals have a line they won’t cross, you know?
What line? setting foot in Walmart?
I’m hoping that they never see a POS breach
Oh Walmart has been hacked not only myself but friends and family members have had their bank accounts drained beginning approximately 2 weeks before Christmas and it was all at walmart
I encountered the walmart issue as well. But there wasn’t any money in my bank account because I’m broke so about $2,000 of various attempts ranging from $20-$500 were bounced back for insufficient funds. All attempted purchases were at Walmart and the only place I’d used my card in over 5 months was at Walmart or Walmart online. I operate in cash because I’m a waitress.
Huge difference between criminals using a cloned card at walmart vs criminals breaching their hris/pos systems.
Just because you haven’t seen news of a Walmart breach doesn’t mean it’s safe… Or hasn’t been infiltrated. AFAIK; it has.
this is what my enterprise rep responded to this article with
Hi xxx,
I’m pretty sure it doesn’t. This is talking about a different portal and server than what XXX account is on. I am going to reach out to double check but I will need a few days to get an answer for it. I will let you know as soon as I know for sure.
Thank you,
XXX
Verizon Wireless
VZW is an entirely different business unit from VZB. (Disclosure: I used to work for VZ, ironically on the DBIR itself, but no longer have any affiliation with them).
Thanks for the reply, my rep just replied back. But i still don’t feel “safe” but do we ever feel “safe” =-)
Actually that was quicker than I thought. They notified all of the account managers that had customer’s affected. I wasn’t contacted but an account manager in my office got the letter with a contact. I called them and they verified that your account is not on the Verizon Enterprise Server, the XXX account is on the MyBiz portal (it’s a litte different). XXX information is safe!
Actually VZW and Verizon both use the same enterprise portal. This is not to be confused with consumer portals.
And this brings up the point that this breach apparently involves Verizon’s 1000’s of “enterprise customers” only and none of Verizon’s 100M+ “consumer customers.”
I wonder if they’re using their own QSAs to test the efficacy of their security controls?
if they are I’m sure the qsa can be bought off easy enough to pass you. this is why 3rd party auditors are smart to offer other services you can buy to launder the passing fee. besides that pci standards are open for compensating controls and wild interpretations to ensure you have legal due diligence of security without having to be secure (security consts money). :-p
Anytime you are going around bragging about how secure you are, it is like putting a big “kick me” sign on your posterior. Remember the founder of Life-Lock? 😀
Here is my social 457-55-5462! have a free trial!
HA! Yup! 🙂
Hello Todd Davis!
The company I work for provides a mobile phone and the service is through Verizon. Does this breach affect end-users like me? Or only the people within my company that actually interact with Verizon? or am I totally off base here?
No. Maybe. Yes.
It depends on how your company setup the account. Yes if they used your company provided email. Not as much if they used a generic IT support email. Of course they have your cell number and can try any number of that style of scam.
Aye
Who day gonna call? Ghostbusters?
Hey Veirzon. How’s that 19k employee layoff from last October working out for you?
Wouldn’t a more appropriate title be “Verizon Business accidentally gives away its enterprise customers data?”
When companies cannot adequately protect their data, they are really just giving it away, rather than someone stealing it.
You serious Clark?
Why the F has veirzon had not purchased this info from the hacker the moment it came out for sale , geez like they could buy for 50-75K get with it . Sh%$ its not like they don’t have any money.
Same reason the entity in possession of the data can’t really be trusted to decrypt the data they stole from you in a ransomeware attack. These types generally don’t abide by rules and are by nature unpredictable. What’s to stop them from reselling it a hundred times?
Nothing wrong with making the attempt, Viper. Especially when it comes to pocket change like that.
I’m with you VAUD and SAM. I would think large companies would act similar to many shipping companies when pirates take over ships (which happens monthly, btw), or like mexican kidnapping negotiators do (which sadly is a thriving industry in Mexico) – if you’ve got the money, definitely pay it. They’re not going to ‘kill the hostage’ so to speak, BECAUSE you paid, etc.
And some great additional perspective by MEDICALQUACK. Got me thinking…
I say its a no brainer corp playbook in this type of event to pay the money quick even if it is significantly higher than the paltry price of $100k mentioned in the article (the legal retainer alone in the event of even a small breach for a Fortune 1000 Corp is prob $300-500k at least) Pay the money, and/or maybe even offer a kicker or bonus for some kind of proof of Exclusivity or similar. Sure, sure, you can never be sure it wasn’t copied etc. but there would be some deal point you could ask for that could possibly further limit the chances for additional exposure…but this is not necessarily the case with these large breaches…the core value in the data pool you are buying is realised in the early, pre-aware days of the breach. Time is the enenemy of cyber thieves. The more times that data pool gets sold, the more time there is for the company to notify those affected (unlikely but possible, and the more time there is for a white hat master like Krebs or security researcher(s) to discover it and alert the world. So in a real world scenario where one hacker is paying another hacker a real $100k dollars, there are inherent and mutual considerations understood by the criminal parties which make the prospect of arranging an exclusive sale an acceptable economic premise between the nefarious parties.
We must remember that the computer scientist (the hacker) behind the original breach/hack, may not (gasp!) be motivated by the desire for widespread financial destruction and consumer financial ruin, or by a desire to be a terrorist super-villain bent on world domination…In the stories I’ve read and interviews watched with accomplished hackers and cyber-kingpins who reflect on their crimes, often times the motivating factors behind their digital transgressions discussed are not closely correlated with the desire for wealth or the seduction by old- lady-Greed. I would argue that at the top of the talent pool in digital security, I’m not talking script-kiddies here, I’m talking the savants and Equation-Group caliber minds) White-Hat and Black-Hat mastery are similarly rooted in an intellectual calling that goes beyond the drive for wealth-as-an-end-in-itself. Often the real motivator behind such considerable feats (in my opinion) lies somewhere closer to the challenge of the hack itself, the David v Goliath, the high stakes 3-D chess match, the psychological identity found in being a genius-punk intellectual..and sure, basic human vanity or the competitive gene that pushes individual to be the very best in wherever their passion lies…
And after the real achievement is reached (the breach + exploit) they often surely say to themselves,”…Any yeah, I think I should be paid for being such a badass…get myself a car with working windows. ” The money is often as much a trophy as it is a means to a new Ferrari. They are not anti-capitalist, not for the vast majority, but they are not primary capitalists either. Greed is not what blinds them (in contrast to the pure-play ponzi schemer, bribable politician, or drug trafficker… )
While you can’t negotiate with terrorists, you can (often) negotiate with hackers.
Ironically, $100k is probably a lot of money to the person or group behind this intrusion. $100k is DEfinitely NOT a lot of money for any Fortune 1000 company when faced with such a potentially ruinous scenario (and especially not for a company like Verizon that does $130BILLION in revenue yearly!!!!) . Even mid-level IT staff should have access to discretionary emergency funds specifically to deal with events like this. To me it would be a no brainer. And even still, as I type this Saturday morning, I bet Verizon has still yet to purchase this data…I would have found a way to purchase it within the hour of reading of its existence. Cost-of-doing billion dollar business in the new digital world. Companies can deal with it on the front end without question, or they can pay through the noses of its shareholders if they pay for it on the backend.
Data breaches like this are not unlike nuclear meltdowns. With every hour, the radiation from the fallout exponentially increases and spreads. If you can, the first thing you do is send somebody, send the janitor if you need to, but send somebody in there to turn off the reactor.” IT security pros should be practicing and planning for this event as if it absolutely will happen and when it does every minute counts. The bank will be robbed, and when it is, give them the money, . Before you go spending $500k per year on data-breach insurance premiums, put together a bitcoin wallet with $300k in liquid BC, on hand, ready to transfer if (when) a code red event happens. Contingent Digitally Liquid Discretionary Fund – is a standard operating procedure in my consulting playbook.
I think if I were Brian, ‘talking to the kidnappers first’, before anyone outside of the highest levels of the global criminal tech underworld even knows a whisper, as he has been so many times, I would try brokering the sale direct to the companies. (The master hacker simply wants $100k…which would cost $500k through me, etc. I don’t think I would think twice about it and I would still feel like a white-hat boss) No doubt this opportunity presents itself on a daily basis for Brian…but there is an altruism to great journalism, a special kind of integrity that I so respect in his case, especially when it coincides so directly with financial opportunity.
There were FBI agents indicted in the Silk Road investigation for taking advantage of the one and only opportunity they had to make an easy $400k…It was so easy and opaque, even the integrity flawed FBI agent couldn’t pass up the temptation to use their position to make insane amounts of easy money.
And with humility and composure, Krebs continues, day in and day out, reporting with the integrity of Frontline, on things that a $20billion in Corp IT Security spend a year doesn’t know, and makes it look easy.
Aside from being first, with like ..every significant financial (non political) hack in the past few years, the journalistic integrity he demonstrates with his consistency is so cool. He could have jumped ship so to speak way way long ago. If he wanted, I bet Krebs could leverage his Donnie Brasco like status in the cyber underworld and raise a $20mil backed opportunity Fund with one phone call. But he’s not in it for the money. He can’t be. And it is in this way that he is not unlike the criminal adversaries he exposes…
(sorry to rant, but my initial comment spawned into an opportunity to express my admiration for your work Mr. Krebs, and as a loyal reader of yours for the past 4 years, it has been long overdue. As a verizon customer, thanks for your work on this story in particular)
Krebs for Cyber Czar 2017
It’s electronic data. He can sell it a million times over, to anyone who wants it. It’s not like he stole a piece of hardware and Verizon needs it back, because they still have the data, too. It’s not even like credit card numbers that will get canceled and no longer be worth buying in a few months. He can be selling the same data next year, and it will be nearly as valuable.
Verizon doesn’t need to be customer 1,000,001 paying for information they already have.
Thanks Brian for touching on “repackaging” and selling of consumer data. I’ve been beating that one for quite a while trying to get folks to wake up.
How do I know about that? I used to do data transfers from one data base to another (legally of course for folks changing medical software programs) and it’s not hard to do once you have your tools. It’s been a few years since I got out of the business as more went to the web versus storing on local servers but if you know how to clean, de-normalize and map, you can fix up any old data base and do what you want with it.
It’s just like any tool, you either do good things and are a trusting individual or you are like the folks in your story as it’s the same batch of basic tools that do the work and there’s folks that do a lot better job today than what I did as well. MongoDB didn’t even exist when I was providing services as such Company A has you in a data base as does Company B, but both have other data that each doesn’t have. Company C comes along and buys up Company A’s data and Company B’s data and created a brand new data base and can sell even stolen data as new with no reference to A or B for the source. How about that—scary indeed! Look at the Data Exchange site and it’s wide open and it’s not alone.
http://ducknetweb.blogspot.com/2015/02/the-data-exchange-buy-and-sell-data-on.html
I’ve had a somewhat quiet campaign and I was supposed to present it to my Congressman last week but they had to reschedule, but we do need to license and index ALL data sellers. Who are they? I’ve been promoting this idea for a long time as you should have a “license” to sell data. We license people for much less, i.e. a hair dresser..lot less risk there than having your data all over the place.
https://www.youcaring.com/medical-fundraiser/help-preserve-our-privacy-think-anthem/258776
Of course you won’t see companies like Verizon high on this idea as they would have to get licensed Companies that sell data should be licensed, period and that number should accompany their sales. The World Privacy folks thought it was a good start as well, as how do you begin to regulate anything without knowing who the players are. Data Selling is what creates breaches as its easy money. Heck I even had one of the Anonymous folks, who actually do like this idea as well, tell me it’s a piece of cake to scrub and fix up data bases to look like new for sale. Also remember, crooks are selling data and if they can dress it up to where it looks legal, they don’t have to resort to the dark web and thus their chances of getting caught or even recognized is lot less and of course we all lose as consumers. The more the data is queried and remixed, the greater the change of flaws and heck some of it is flawed to begin with. Sadly, bad and flawed data today gets the same price as good data as nobody looks at the data, only the query results.
http://ducknetweb.blogspot.com/2015/02/anthem-data-breachcrooks-want-to-sell.html
Great links. Very interesting. I’ve had frequent talks with a close friend of mine lately who works specifically in programattic advertising…data repackaging, obfuscating, reselling etc. is rampant among many very large, very expensive vendors. its a problem and your articles really added layers to my understanding of the business of data. Thanks for sharing
I for one am not surprised. Having worked at verizon for 10+ years as a Sys Admin it was bound to happen, and will happen more. With VZ constantly cutting head count and Admin to server ratio reaching over 600 servers per admin they cant keep up. Things get overlooked and security gets lax. They only care about maximizing shareholder value and will cut corners to achieve it.
I think you can guess that virtualization is the most likely reason for that 600-to-1 ratio. And the cost savings of virtualization does show up on the bottom line.
“”The seller of the Verizon Enterprise data offers the database in multiple formats, including the database platform MongoDB, so it seems likely that the attackers somehow forced the MongoDB system to dump its contents.”
A bit of an analytic leap there. The blog post is almost 2 years old. And references a Verizon Wireless database vs the b2b side of things.
Huh? What’s more than 2 years old? The company itself says this is Verizon Enterprise, not Wireless, data.
Seriously?
In the article, when you mention MongoDB you have it linked to this article. The hyperlink exists in the quoted part I already posted.
https://www.mongodb.com/blog/post/how-and-why-verizon-wireless-chose-mongodb
…which again, is almost 2 years old and reference MongoDB usage for Verizon Wireless which has nothing to do with Verizon Enterprise.
So saying it was probably a MongoDB dump from within Verizon Enterprise might not be totally accurate.
Have to agree with John. Certainly possible that it was in a mongodb at verizon, but it is also possible that the attackers just put the data into mongodb.
Mongo is a a very convenient JSON-style key:value type store. So, if they got data from any verizon system that happened to use a key:value format, it could certainly be the case that they are just packaging it in mongo because it’s easy.
We are force to give these companies our personal information and they can’t keep it safe.
Wrong. If they (VZW) have your data, its because you gave it to them, along with a large sum of money.
Yes I did willingly give them my data. If I had not wanted to share it, I could get rid of my cell phone and my laptop and my desktop. And just stick with my land line.
Oh wait, I have to ” volunter” to share my personal info if I want a land line….
They still don’t get it do they?
There is a bigger story getting ready to unfold here. My wife and I both received phishing texts connected to our bank information on Monday. Upon contacting the bank we were told that our bank and numerous others were getting hammered because any customer with a Verizon account was being probed by these phishing texts. It appears to me that this Verizon hack is a lot larger than their Enterprise customers. I know that banks in Colorado were involved. Once again these companies refuse to be truthful and are about as irritating as the people doing the hacking.
Unfortunately it is illegal for a financial institution in the US to share with you what company was breached. Often we know . Which is BS because people instantly think their Bank/CU was breached and want to move their finances elsewhere.
It’s a BS law that hurts more businesses by breaking trust, and all the while people will find out eventually, usually.
It doesn’t surprise me at all that Verizon was breached, given the kind of security practices showcased in the customer portal for their consumer FIOS business (you can log into that portal and see your WiFi router’s WPA2 passphrase displayed in plain text, for example. Also, when I added a new service to my existing account, they sent me a “Welcome to Verizon” email containing the login credentials for my existing account – username and password in plain text).
I know the enterprise customers are under a separate business unit, but still.. it doesn’t give me the warm fuzzies when I see that kind of stuff going on in the consumer side.
I have verizon, should I be worried? ???
No
She may be a Fortune 500 company with Verizon circuits.
They just lost my business!!!
Good bye 1 user out of 1.5million
Of course the right wing Republican multinational corporation commits this against our country constantly, encryption of their data is sooooo Communist, ya know?
Hmmm
Once again; where the hell is the NSA, the FBI and if there perps are foreign the CIA on solving this crime? The short answer I am sure is no where. Not because they can’t, but because they won’t.
where the Anonymous when you need them
I just noticed that in the address bar of my browser, Verizon Enterprise Solutions is the verifier of the ‘green lock’ (site identity information) for virgin mobile usa. Should there be concerns?
Organized crime versus hackers… no difference anymore.
It still bugs me that none of these breached companies come clean with what went wrong and what they’re doing to get better. Maybe Verizon Enterprise will break that mold. But probably not.
That’s why I wrote “Bullseye Breach,” an educational book disguised as an international thriller about how an ad-hoc group of good guys deals with a breach of 40 million credit card numbers at a fictional retailer named Bullseye Stores. A security blogger named Henry Lincoln also plays a role in the story. The book website is http://www.bullseyebreach.com. Enjoy.
– Greg Scott
You forgot the baked beans.
Ok, Verizon found a flaw (I’de like to get something a little more specific)
But were their computers/servers fully updated and patched (I’m sure they were….but I’m curious)?
Someone soon needs to figure out a way for our information to stop being compromised in this fashion. I have had mine stolen at least twice in the last 2 years and it is getting very frustrating. When will someone figure out a better way to identify ourselves so that we cannot be screwed in the future?
Do we know what the security vulnerability was that caused the breach? Was it a plugin? Was is a SSL/TLS vulnerability?
Mr. Krebs, Target was hacked again about a week ago. I have been “fighting” with them for days trying to get them to come out and tell their customers to change their passwords at least. Several customers who had gift cards stored in their Target.com and Target app lost them. Funny thing is, most of them seem to have been used on the same day, at the same location. Target is blaming anyone else but themselves. Coincidentally, a few days after the hacking they decide to perform maintenance on their website and everyone had their gift cards blocked for at least 24 hours. This confirms the fact that there is something going on. Why do they keep getting hacked again and again?? And why won’t they tell the truth? I don’t expect to get my money back, but this is really messed up.
Nice irony! Even the watchers are vulnerable.