At Experian, Security Attrition Amid Acquisitions

T-Mobile disclosed last week that some 15 million customers had their Social Security numbers and other personal data stolen thanks to a breach at Experian, the largest of the big American consumer credit bureaus. But this actually wasn’t the first time that a hacking incident at Experian exposed sensitive T-Mobile customer data, and that previous breach may hold important clues about what went wrong more recently.

On Dec. 30, 2013, T-Mobile said it notified a “relatively small” number of customers that unauthorized access to a file stored on servers owned by Experian had exposed Social Security numbers and driver’s license numbers. The mobile provider identified the breached vendor as Decisioning Solutions, an identity-proofing and authentication company that was acquired by Experian in April 2013. We’ll revisit this acquisition in a few moments.

Over the past week, KrebsOnSecurity has interviewed a half-dozen security experts who said they recently left Experian to find more rewarding and less frustrating work at other corporations. Nearly all described Experian as a company fixated on acquiring companies in the data broker and analytics technology space, even as it has stymied efforts to improve security and accountability at the Costa Mesa, Calif. based firm.

Jasun Tate worked for a year until April 2014 as a chief information security officer delegate and risk consultant at Experian’s government services and e-marketing business units. Tate said he and several of his colleagues left last year after repeatedly running into problems getting buy-in or follow-up support for major projects to beef up security around Experian’s growing stable of companies handling sensitive consumer and government data.

“What the board of directors at Experian wanted security-wise and the security capabilities on the ground were two completely different things,” Tate said. “Senior leadership there said they were pursuing a very aggressive growth-by-acquisition campaign. The acquisition team would have a very strict protocol on how they assess whether a business may be viable to buy, but the subsequent integration of the business into our core security architecture was just a black box of magic in terms of how it was to be implemented. And I’m not saying successful magic at all.”

Another recent former security employee at Experian who agreed to talk on condition of anonymity said it was clear that the company’s board was not well-informed about the true state of security within the company’s various business units.

“When I was there, the board was very big on security and wanting to invest in it and make sure we were doing what we needed to do in order to avoid situations just like this,” the source said. “In my opinion, there’s no way the board was told the whole story, because if they had been then things wouldn’t be where they are are now. We wouldn’t be talking about this. Some things had to have been hidden or spun in a way to look positive somehow.”

BLACK BOX MAGIC

Not long after it acquired the above-mentioned Decisioning Solutions in April 2013, Experian folded the company into its Decision Analytics platform — a unit which provides credit and noncredit data, customer analytics and fraud detection to lenders, cable and satellite companies, telecommunications firms, third-party debt collectors, utilities and to state and federal government entities.

Within hours of the latest T-Mobile breach news hitting the wires, KrebsOnSecurity was contacted by an anonymous source who sent this author a Web link that, when clicked, opened up a support ticket within that Decision Analytics platform in the United Kingdom — with absolutely no authentication needed. That support ticket I viewed appears to have been filed by someone in an office cube at Experian’s data center in Costa Rica who was requesting hardware support for a component of the company’s Global Technology Services division.

That particular support ticket was relatively uninteresting, but according to my source anyone could view countless other support tickets filed via the support portal for Experian’s Decision Analytics platform.

The same source demonstrated how modifying just one or two numbers at the tail end of that link revealed requests for access to networked file shares from across a range of Experian’s business units. The requests included specific names of network shares, usernames, userIDs, and LanIDs, as well as email addresses, phone numbers of Experian personnel requesting and approving the changes.

The support site also apparently allowed anyone to file support tickets, potentially making it easy for clever attackers who’d studied the exposed support tickets to fabricate a request for access to Experian resources or accounts on the system.

In addition, experts I spoke with who examined the portal said the support site allowed anyone to upload arbitrary file attachments of virtually any file type. Those experts said such file upload capabilities are notoriously easy for attackers to use to inject malicious files into databases and other computing environments, and that having such capability out in the open without at least first requiring users to supply valid username and password credentials is asking for trouble.

KrebsOnSecurity sought comment from Experian to find out if it knew that its Decision Analytics support portal allowed anyone to view the tickets within. The company said in a statement that it had disabled the portal in response to what appeared to be unauthorized access to it and had notified law enforcement.

“We take any unauthorized access to our systems very seriously, and when we detected the unauthorized activities, we shut down the website and notified law enforcement,” the company said in a statement. “Our credit database and core infrastructure were not impacted – nor could they be accessed through this website.  This site was a legacy version of a service to enable clients and internal users to create and log tickets for issues they may have and we had already deployed its replacement solution.”

ANOTHER ACQUISITION DISASTER

The two separate breaches of T-Mobile customer data that were caused by break-ins at Experian are dwarfed by the security disaster that ensued in the months following March 2012. That was the date Experian acquired Court Ventures, Inc., a company that aggregated, repackaged and sold public record data from more than 1,400 state and county sources.

“This acquisition strengthens Experian’s consumer data assets in North America and is a further step in Experian’s strategy to extend its global lead in credit information and analytics,” the company says on its Web site.

The only problem, experts say, was that Experian’s acquisition team neglected to do “due diligence” on Court Ventures’ existing customers to ensure that Experian was not inheriting legal or security problems that could haunt it down the road.

Unfortunately for Experian and countless Americans, hidden among Court Ventures’ customers was a human Trojan horse: A 22-year-old Vietnamese man named Hieu Minh Ngo was running an online service called Superget.info that catered to identity thieves, and he’d secretly gained access to Court Ventures and other data brokers by posing as a private investigator based in the United States.

Ngo’s business had attracted more than 1,000 paying customers, mainly identity thieves seeking to open new lines of credit in victims’ names, and/or to file phony tax refund requests in their names with the states and the Internal Revenue Service.

With the Experian acquisition of Court Ventures, Ngo’s customers had access to records on more than 200 million Americans, according to documents filed by U.S. prosecutors who successfully obtained a guilty plea and 13-year prison sentence for Ngo earlier this year.

Experian told Congress the oversight had not provably led to any harm for consumers. “There’s been no allegation that any harm has come, thankfully, in this scam,” said Tony Hadley, Experian’s senior vice president of government affairs and public policy, testifying in front of the Senate Commerce Committee in Dec. 2013. Meanwhile, investigators with the U.S. Secret Service and U.S. Justice Department have continued to arrest and prosecute many of Ngo’s customers for a variety of identity theft crimes, including new account fraud and tax refund fraud.

Experian also is now battling a class action lawsuit over Ngo’s identity theft service. In addition, several state attorneys general are reportedly investigating Experian over the Superget.info identity theft service that Ngo operated.

Now, in the days following revelations of a new breach at Experian involving the personal data on 15 million T-Mobile customers, lawmakers in the Senate are again demanding answers. The senators said the incident demonstrates the need for new legislation that addresses “both consumer notification and sets minimum security requirements for companies that collect and store such sensitive consumer data.”

In letters to T-Mobile CEO John Legere and Experian CEO Brian Cassin, the senators stated that the breach is “is extremely troubling to us given the sensitive nature of the compromised personal data, and its particular value to identity thieves,” especially with the exposure of social security numbers. The letter noted that according to the Department of Justice, 64 percent of the 17.6 million victims of identity theft in 2014 experienced a direct financial loss resulting from personal information fraud.

“This is particularly distressing based on your companies’ reported breach, because victims of personal information fraud lost an average of $7,761 compared to victims of bank or credit card fraud who lost an average of $780,” the lawmakers wrote. The remainder of their letter is here.

The credit bureau giant says consumers needn’t worry: The company is offering free credit monitoring for affected consumers through its ProtectMyID service, an highly profitable business unit of Experian (readers considering credit monitoring should read this article first).

GOOD LEADERS MATTER

Multiple former Experian security employees said that for years Experian seemed to be making great progress on establishing security as a core priority across the mammoth company. All interviewed directly attributed that progress to the leadership of then-chief information officer (CIO) John Finch, who helped hire and build up a staff of nearly 30 talented professionals to monitor Experian’s security “brain” — the “security operations center” or SOC for short. The SOC is designed to pull together real-time alerts about cyber attacks, as well as provide assessments about vulnerabilities across the organization’s far-flung computers and networks.

After Finch was lured away to take the CIO job at the Bank of England, many of the major in-progress projects designed to bake security into all aspects of Experian’s business ground to a halt, the former employees said on condition of anonymity. Core members of the Experian security team soon began seeking employment elsewhere. A year after Finch’s departure, morale suffered and the staff of the company’s SOC had dwindled from nearly 30 to about a dozen.

“I don’t have any ill will toward the company but what happened there was just a culmination of wrong decisions made outside of the security team’s control,” said one of Experian’s former security employees. “One of the challenges in security is being looked at as more than just an audit check box or the jerk that just wants to take the system down. But John saw security as an asset rather than a cost center, and a significant investment was made to grow security and mature the process. We got new tools, we got the architecture operating the way it should, and we began to get more collaboration from other teams which wasn’t there before.”

Finch could not be reached for comment. But the employee said after Finch’s departure, everything changed.

“We had a period of time there where security was viewed in a positive light, and things weren’t being swept under the rug for the sake of uptime.” the employee said. “He left and it kind of went the opposite direction.  Once the leadership changed, the focus changed to controlling costs and not taking systems down for maintenance, and investments started disappearing from a lot of areas. We were in the middle of putting into operation certain tools to do next-generation detection of [cyber] threats, but we weren’t able to get many of them out into production. And that’s how Experian wound up where they are now.”

In a written statement provided to KrebsOnSecurity, Experian maintained that security remains a top priority for the company.

“Experian is committed to continuous investments in upgrading talent, processes, and technologies needed to protect our systems,” the statement reads. “Specifically, we made incremental investments of tens of millions of dollars in the last three years alone to strengthen our security positions. We employ over 200 highly skilled and experienced info sec professionals, and we supplement our own resources with leading third parties, as needed.  As part of our continuous investment, we have increased the size of our security team over the last three years and upgraded our core competencies in information security.”