November 11, 2013

Facebook is mining data leaked from the recent breach at Adobe in an effort to help its users better secure their accounts. Facebook users who used the same email and password combinations at both Facebook and Adobe’s site are being asked to change their password and to answer some additional security questions.

Message that Facebook has been sending to certain users whose information was found in the stolen Adobe user data.

Message that Facebook has been sending to certain users whose information was found in the stolen Adobe data.

Facebook spokesman Jay Nancarrow said Facebook is constantly on the lookout for data leaked from other breach incidents that may endanger accounts of its own users. Nancarrow said that the social networking service has similarly acted in the wake of other high profile breaches to determine if any of its own users’ credentials may have been affected.

“We actively look for situations where the accounts of people who use Facebook could be at risk—even if the threat is external to our service,” said Nancarrow, who declined to say exactly how many Facebook users were seeing the above message. “When we find these situations, we present messages like the one in the screenshot to help affected people secure their accounts.”

In a breach first announced on this blog Oct. 3, 2013, Adobe said hackers had stolen nearly 3 million encrypted customer credit card records, as well as login data for an undetermined number of Adobe user accounts. Earlier this month, Adobe said it had actually notified more than 38 million users that their encrypted account data may have been compromised. But as first reported here on Oct. 29, the breach may have impacted closer to 150 million Adobe users.

What’s more, experts say Adobe appears to have used a single encryption key to scramble all of the leaked user credentials, meaning that anyone who computes, guesses or acquires the decryption key immediately gets access to all the passwords in the database. In a detailed analysis of the enormity of Adobe’s blunder, Paul Ducklin of Sophos describes how researchers managed to work out a decent chunk of the encrypted user passwords just by comparing the leaked data to other large password breaches and to password hint information included in the Adobe account cache.

Update, 2:07 p.m. ET: Looks like Diapers.com and Soap.com sent similar notices to their customers on Sunday. A hit tip to readers Arthur and Dave for sharing copies of the emails from those two sites.

Update, Nov. 12, 4:07 p.m. ET: The initial story seems to have confused a number of readers, perhaps because I left out an explanation of what exactly Facebook did. As a result, many readers seem to have hastily and erroneously concluded that Facebook doesn’t properly secure its users passwords if it can simply compare them in plain text to the Adobe passwords that have already been worked out.

As I proffered in a follow-up comment on this story, Facebook and any other company can take any of the Adobe passwords that have already been guessed or figured out and simply hash those passwords with whatever one-way hashing mechanism(s) they use internally. After that, it’s just a matter of finding any overlapping email addresses that use the same password. Facebook’s Chris Long confirmed that this is more less what the company did.


60 thoughts on “Facebook Warns Users After Adobe Breach

  1. DefendOurFree

    http://www.forbes.com/sites/oreillymedia/2013/11/09/how-secure-is-your-old-and-inactive-user-data/

    Excerpt:

    First, the “illegal access” to source code, then the news of lost customer data to the tune of 2.9 million, then upping that to 38 million, but really actually (maybe?) 150 million. The larger number is expired accounts—or something.

    Adobe spokesperson Heather Edell said that the 38 million accounts are *active* accounts, and that weasel word seems to explain the rest. They have reset the passwords of the inactive accounts, but personally I’d prefer that they delete them. Adobe has had the usual response of buying free credit monitoring for the hacked *active* accounts.

    (snip)

    So, as per Adobe, the breach involved about 150 million users; but are only taking responsibility for what they consider “active” users, which is the 38 million.

  2. Matt Leidholm

    How could Facebook know that you’re sharing a password between them and Adobe, unless they’re (a) using some sort of reversible encryption instead of a good, expensive one-way hash, or (b) using the same unsalted hash Adobe was using?

    Even if an encrypted password database is leaked, a company shouldn’t be able to match it unless it is reversible or if the two companies were using identical encryption techniques, right? What am I misunderstanding?

    1. lucascp

      This caught my attention too. I think Adobe didn’t use a one-way hash function, they were using a block cipher, so even if Facebook used the same method, the key would’ve been different.

      So, they either discovered the key, which seems unlikely, or they just sent this warning to everyone who used the same email, regardless of password.

      1. Matt Leidholm

        I think if they were just checking emails, Facebook would have used fuzzier language than “you were using the same password in both places.” More likely is that before discarding the plaintext password, Facebook is sending it to Adobe to check against the hacked database. Adobe might be hosting a private web service to do this check, or may have simply shared with Facebook the key to reverse the encryption themselves.

        1. BrianKrebs Post author

          I don’t have special knowledge one way or the other, but one thing FB could do is look at the corpus of email and password pairs that have already been worked out by researchers. Take a look at the Sophos post linked to in this blog post or have a gander at any of several other blog posts that sought to tease out the most commonly used passwords from the Adobe set, and that’s a pretty good chunk.

          FB could easily take those worked out passwords (123456, password, password1, e.g.) and then hash them with whatever hashing method they use internally, and the look for any overlapping email addresses for FB users that also have that corresponding hash. That would be one way to do it, AFAICT.

          1. Chris Long

            I work at Facebook on the security team that helped protect the accounts affected by the Adobe breach. Brian’s comment above is essentially spot on. We used the plaintext passwords that had already been worked out by researchers. We took those recovered plaintext passwords and ran them through the same code that we use to check your password at login time.

            Like Brian’s story indicates, we’re proactive about finding sources of compromised passwords on the Internet. Through practice, we’ve become more efficient and effective at protecting accounts with credentials that have been leaked, and we use an automated process for securing those accounts.

            1. Mara Alexander

              As a large number of us have experienced, Facebook’s security “Automated process” leaves a lot to be desired, and with no way of contacting anyone to discuss an issue (unless you’re a world renown blogger on the level of Brian Krebs).

              I do Web Development. As part of a website redesign package I offer to either setup a Facebook page or customize/manage one they already have. Like legions of users out there, many of my clients have trouble navigating anything on Facebook, and wouldn’t know HOW to set up an admin account, so I often have to use their login in order add myself as a page admin in order to set the custom username, adjust the page settings, etc.

              All typical “Facebook page admin” activities….and yet a clients Facebook page got shut down because I:

              1. Logged in using valid login credentials;
              2. Set the username (to the business name);
              3. Added 2 admins (the actual admin, who had been using the page owner’s login, and then myself).

              THAT tripped the “zOMG this page iz haxOred” Facebook alarm.

              The two options I was presented with to reclaim the page were to either login with a previously used browser, which I had the admin do and that didn’t work; or to upload a photo ID from the page owner, which isn’t possible because he’s not only is he on the road but he doesn’t even own a scanner.

              I created graphics briefly listing what had happened, giving my real name (I’ve been on FB since 2007) and what I had done (which should all be listed in the logs), and uploaded those, hoping I might reach a human at Facebook.

              I received an automated reply that my credentials had been received…. and then nothing. NOTHING. I have submitted a plea through every contact form I can find on Facebook that might apply (you have over 150), and NOTHING.

              Meanwhile the clients Facebook page remains in limbo. Maybe I’ll recommend G+ to him instead of Facebook.

              So, yes, while these measures sound wonderful, I have no doubt that Facebook’s implementation of them will be a cluster. Many, many users don’t have a scanner, and won’t be willing to trust Facebook with a copy of their driver’s license or passport even if they did! Requesting some other type of info, like a challenge question, etc… maybe. But trusting Facebook with a photo ID? I don’t think so.

              1. DefendOurFree

                I had a very similar experience with a client and his Facebook page. I was never, ever able to reclaim it. The client was never able to log in. I’m not sure Facebook gives allows for webmasters. I don’t offer doing Facebooks for clients anymore.

                1. Mara Alexander

                  Thank you, it’s good to know it wasn’t just me. I’ve read many horror stories about really well-known companies having this happen, and the manpower and public pleading it took for *them* to get control back of their own Facebook pages… so I don’t hold out much hope.

                  And yet when I’ve been on the reporting end of issues, it’s a whole ‘nother world.

                  I’ll probably follow your lead, and refuse to help clients with their Facebook pages. At least if they get themselves locked out (which will happen), then I won’t fee; guilty, even though I did nothing wrong.

                  1. JustSayin

                    “their own facebook pages” …. If they want their own they can build it. Otherwise it is not theirs. Bitch and complain, feel entitled even. Until you build it and support it STFU.

                2. Mara Alexander

                  Oh, and….Facebook does give allowances for webmasters, because they have the function for the page owner to add additional admins. I always recommend clients have several trusted people as admins, because you never know when Facebook is going to have a bad hair day and just shut down a page for no reason (not even the “non-reasons” I listed here).

                  And Facebook wonders why there’s so many abandoned pages…

                  1. DefendOurFree

                    Thanks for that tip on the ‘add admins’ feature. My incident was back in 2009. I don’t think they had that feature then. I’ll look at that and see about adding back social media profile options.

              2. sublimnl

                The problem seems to be with the follow up customer support – not the security system.

                A system that can detect an abnormality when a new IP address logs in, and automatically follows the steps to set a username and add in two admins – when those are not standard procedures that user does – seems to be a good flagging system.

                1. R

                  Customer? You seem to believe you are Facebook’s client. Don’t ever forget you’re their product. And products don’t have any rights to customer service.

                  1. Product

                    Indeed, an important reminder/valuable info for those who aren’t aware of it. Facebook’s customers are advertisers. Facebook users are the product.

                    Still, kudos are in order to Chris Long for doing something; the shortcoming is not surprising, given God said: We have the words “Done is better than perfect” painted on our walls to remind ourselves to always keep shipping.

              3. Allan Miller

                Before you switch over to Google+, pretend this had happened there, and try to get hold of a person at Google. Oh, and good luck with that, by the way.

                1. Mara Alexander

                  Oh, I totally agree with you, Allan. And just to make my position clear, I *hate* G+. With the passion of a thousand suns kind of hate.

      2. Jason

        My email address was included in the Adobe leak, and I also use it for Facebook. As such, I can confirm that Facebook is not sending that message out indiscriminately, because I used different passwords on the two sites and Facebook didn’t send me the notification.

        1. Jason

          My vote is that Facebook is sending its emails+password hashes to Adobe. Adobe then (so conveniently has the ability to) decrypts the coorisponding password in its database, uses identical hashing method, and then compares.

          1. DefendOurFree

            Any chance they share a database due to the partnership they entered in 2009?

            http://blogs.adobe.com/sarthak/2009/04/facebook_and_adobe_partnership.html

            Adobe and Facebook® announced the availability of the new ActionScript 3 Client Library for Facebook Platform, a free and open source programming language library for developers creating applications using Facebook.
            Officially supported by Facebook and Adobe, the ActionScript 3 Client Library for Facebook Platform is expected to be a complete resource that supports all Facebook application programming interfaces (APIs) including Facebook Connect.

            Right now the license on the code is an MIT license, as found on the Google Code page.

            Adobe has also launched a new developer center for Facebook at http://www.adobe.com/devnet/facebook.

            1. Chris Adams

              Those are completely unrelated things: one is a library which software developers download to integrate Facebook support into their own applications while the other is a database used by Adobe’s store & support services. Those probably aren’t even in the same division.

    2. Joe C

      Since they are able to decrypt the Adobe passwords, they can check to see if the adobe password matches your facebook password using the same method they do to determine if you typed the right password when logging into facebook. I would imagine they take Adobe’s decrypted passwords, run their hash function on it, and compare it to the hashed version they have stored in their user database.

      1. Phil K.

        Yes, which is the same way they check your password when you submit it.

        They have the plaintext password by decrypting adobe’s database. So, they compare that to your stored password on facebook via the exact same mechanism as when you log in.

        Of course they can check if a plaintext word is your password. It happens every single time you sign in.

    3. Andrew Barratt

      I think you might be missing when Facebook does this.
      1)
      First of all – assume Adobe shared with facebook either the plaintext passwords or the encryption key. Or Facebook as done some crypto analysis for really common passwords. Given that a couple of other sites have done similar adobe might have shared this with them.

      2) during the facebook login process – when facebook receives your login credentials over SSL – it then has the plain text by which it can A- hash in order to compare with its hash for authentication purposes and B – use to look up in the table from 1) above. (then of course free).

      if you can login and fb find a match then I suspect you get this alert.

      1. Nate

        The hashing should be done client side, so only the hashed versions is ever transmitted.

        They could do it the way you suggest, but that’s bad news. As long as the hashing is done client side, SSL shouldn’t matter for simple password storage.

        1. Chris Adams

          Client-side hashing doesn’t actually improve security in any meaningful way because you have to use JavaScript from the server to perform the hash operation. If you don’t trust the server to perform the hash correctly, you also can’t trust the code it gives you to perform that hash.

          The only thing which will change that dynamic will be browsers implementing the WebCrypto API so the authentication-related code will be a standard, well-tested library provided by your browser rather than a blob of JavaScript which can silently change without notice. Even that, however, really doesn’t help with passwords as much as adding two-factor authentication because the browser itself can still be compromised.

        2. Joe C

          If hashing is done client-side, and passwords are leaked, i dont need to know the actual passwords to use them. I can just authenticate with the hashes…

    4. A S

      Hi Matt,

      Agree with you completely…How could Facebook do a hash compare unless their own database was using unsalted hashes (same as Adobe)…

      Scary crap…now hackers know they can do the same with FB’s user account DB.

  3. Dr. Neal Krawetz

    Hi Brian,

    Perhaps I misunderstood… but I thought that the Adobe passwords were part of a larger package that also contained Adobe source code. (Please correct me if I’m wrong.)

    Assuming that is the case, then when Facebook acquired the package containing the Adobe passwords, they also acquired the stolen Adobe source code. This makes me wonder how soon before Facebook incorporates Adobe proprietary functionality into Facebook’s own graphics systems, such as Instagram. This would lead to major lawsuits about copyright and patent infringement, as well as possession of stolen technology.

    1. BrianKrebs Post author

      I have no idea whether FB acquired any source code of Adobe’s, but doing what you suggested seems like it would a bone-headed move, and I doubt they would do that.

  4. Scott Herbert

    @Matt,

    They could be just looking at the password you send them when you log in.
    i.e. something like

    $email= $_Get[’email’];
    $password = $_Get[‘password’];
    if(IsValidUser($email,$password)){
    if(InHackedPasswordList($email,$password)){
    displayWarning();
    }
    echo “your loged in, welcome”;
    } else {
    echo “nar get lost!”;

    1. Matt Leidholm

      But unless Adobe gave Facebook the key to the block cypher (which isn’t out of the question, and probably more likely than finding some reference to it in the stolen source code), how would they implement the InHackedPasswordList method?

      1. Andrew Philips

        They could use Adobe’s website to check the username/password combo. This could be done through (A) web scraping (public login on a web page), (B) a published (by Adobe) web API, or (C) a private web API created by agreement between the two companies (presumably for this purpose). Facebook could check Adobe’s public web login without permission from Adobe (suggestion A), although Adobe might then block Facebook because FB looks to them like someone trying to crack account passwords. That is, Facebook need do nothing more complicated than what a cracker would be doing, except that Facebook gets a head start by using a password given to them by the user themselves instead of the cracker having to make educated or brute force guesses.

        Nothing nefarious and no keys or algorithms exchanged.

      2. Jason

        I replied at the wrong level above. Whoops.

        My vote is that Facebook found all of its user’s email addresses that were in the database, and sent those to Adobe along with the salts and hashes. Then, Adobe just has to decrypt the passwords (how conventient) and hash them with the provided salts to compare.

        1. aaaa

          That’s not how hashes are suppposed to work. Hashed are normally a one way encryption. Hashes for common passwords are known (given salts)
          , but complexes one are not and very expensive to calculate.

          1. Daniel15

            The key part there being “meant to”. Adobe were encrypting their passwords (not hashing them) using Triple DES, so they do have the ability to decrypt them.

          2. Daniel15

            The key part there being “supposed to”. Adobe were encrypting their passwords (not hashing them) using Triple DES, so they do have the ability to decrypt them.

  5. dave

    I actually got a similar message from soap.com. it only stated I had used the same email address as found in the breached list however. (Ironically it looks like someone had used my email address as a dummy account or miss typed their actual email address for adobe, b/c I personally never signed up with adobe to begin with… I use GIMP… so I was surprised to get a message from adobe saying to reset my password)

  6. The Oregano Router

    Like people should trust Facebook with privacy issues, maybe their is some irony here

  7. charisse

    It’s getting so a data breach, even if your password is not compromised, is almost as time consuming to remedy as having a credit card misused.

  8. IA Eng

    I don’t have have need for the social media sewer and potential outbreak of information gathering. It really boggles thy mind when people will complain about the NSA and their ways, when the common folk simply spill thier information on the social media sites. For me, I have better things to do than to sit and put up with all that nonsense. Not worth my time in any facet of life.

    So, Facebook takes a password list and tries these passwords on their site? Though thats proactive, its also a knee jerk reaction. They have to assume that a hacker / cracker or otherwise MIGHT use their account. I don’t know if thats an invasion of privacy or not. BUT it ADDS to complexity of the situation. It gives the crooks another piece of the puzzle. If a person is using the same password on a major corporation website, and now, on a social media site, what are the odds of them using the same password on a banking site? Hummmmmm.

    If a Crook has PII, the facebook questions are merely a small hoop to jump through. They may ask questions about their facebook activity, but thier are ways to look at cached information and potentially dig out an answer.

    In my opinion, if a social media site can take the time to go through millions of user names and passwords – they are focusing on an issue that they have no control over. So the person changes the password. Woopie. Old habits die hard and they probably used, or reused the same password on both sites once again. These social media sites need to get a grip on what’s happening internally and severely limit the use code on the sites to limit the amount of people who are subject to becoming infected each day. The social media sites want to make it way too convenient to do things, but security suffers for it.

    They can come on in and stand on the podium and spew their comments about security (by obscurity) to many, and do a proactive brain washing to most, and people believe them. In the end its all the marekting and corporate left over hogwash that remains. I’d like to see a report on the average number of people on a social media site, and the percentage of infections that happen. I would have to guess that the rate is higher than most people think.

    1. Leave It To Beaver

      Definitely no superiority complex here folks. Not in the slightest.

      1. IA Eng

        Everyone has opinions, some matter some don’t. Hide behind that 1970’s title and I expect comment like this from a simpleminded, stereotypical person who has more time logged on the social media sites than they actually work.

        Pick your poison, I am not into social media sites. Your time on them probably makes up for this, tenfold.

  9. Brian NotKrebs

    The Facebook dialog box says “you were using the same password”.

    How do they know that? Were the passwords not hashed with a random salt? Which, if true, how can Facebook compare those hashes with their own unless Facebook is also storing unsalted passwords.

    1. BrianKrebs Post author

      Given the volume of responses from people who had this same conclusion, I can only assume I should have been more explicit in the article. The fact that Facebook or any other company can do this says nothing about their security. They can do this because a significant percentage of the Adobe account passwords already have been cracked.

      All that a company needs to do is take the list of known Adobe account emails, and then see how many of those have passwords that researchers have already worked out (there are many, many blog posts and web pages where this is being done publicly). A company like Facebook then just takes its own internal one-way hashing mechanism and then runs it against those figured-out Adobe passwords, and sees how many of them match hashes for existing Facebook users. It’s really very simple and straightforward.

  10. Canuck

    Adobe finally got around to sending me a notice email today about the breach. How many others got notices two weeks later?

  11. not me

    Wow! Just read the Sophos article in the link. What a incredibly foul blunder on Adobe’s part. From the comments it looks like not many folks bothered to read the details. If you have not spent the time reading the article you really should, it details out exactly how they examined the encryption and how they worked through the password data.

    Fascinating read, Thank you Brian for pointing me to it.

  12. Greg

    Why could facebook just not give their username+salt and hash code function in private to adobe and politely ask that they generate the hashed result for all of the exposed password.

    There should be no privacy nor security issue here:
    – the email addresses are already public knowledge
    – adobe doesn’t get to see the facebook passwords
    – adobe doesn’t need to reveal their DES3 key to facebook
    – facebook doesn’t get to know the user’s passwords

  13. Winski

    What truly awful software these Klowns produce…. It’s just laughable that individuals and companies still buy this garbage.

  14. Alex

    I have set up my Facebook, Google and Outlook accounts, so that if somebody tries to access my account from another computer, I am sent a code via SMS. Same if I log out and want to log in again. Nobody can get in without the code, so I feel pretty safe. I’ve also locked my password for internet banking.

  15. QHoster

    Seems Adobe made a real mess again. Really scary Abobe, Facebook and other huge systems to be hacked – affecting millions of computers worldwide.

  16. DefendOurFree

    http://arstechnica.com/security/2013/11/fbi-warns-hacking-spree-on-government-agencies-is-a-widespread-problem/
    FBI warns hacking spree on government agencies is a “widespread problem”

    Excerpt:

    The FBI is warning that members of the Anonymous hacking collective have breached computers belonging to multiple government agencies and made off with sensitive information in a series of attacks that started almost a year ago, according to a published report.

    The warning is linked to the case of a British resident indicted two weeks ago on charges that he hacked databases belonging to the Department of Energy, the Department of Health and Human Services, the US Sentencing Commission, and other US agencies, Reuters reported Friday, citing an FBI memo. Lauri Love, prosecutors have alleged, exploited a flaw in Adobe’s ColdFusion Web application development software and used his access to install backdoors that allowed him to return on subsequent occasions.

    “The majority of the intrusions have not yet been made publicly known,” Thursday’s FBI memo stated. “It is unknown exactly how many systems have been compromised, but it is a widespread problem that should be addressed.”

    (snip)

  17. Alon

    I’m quite appreciated of reading such a good article you have posted Brian.
    Infact, I would say that this should be concerned among the public as a good imaginary module for safety mode of technology. Keep sharing ahead.

  18. DefendOurFree

    http://defendourfreedoms.net/2011/02/05/pdf-files-vulnerable-to-security-attack-cautions-researcher.aspx

    PDF Files Vulnerable to Security Attack, Cautions Researcher

    According to security researcher Julia Wolf with California based ‘FireEye,’ there is a new security flaw within the Adobe PDF file arrangement that reportedly exposes its documents to security assault.

    Said the researcher that while the majority of organizations used PDF files as a standard way for documenting their data on computers as well as anywhere on the Web, the particular file format provided low security to data that could be easily altered on the basis of which browser or operating system was running.

    She further stated, while at the Germany held 27th Chaos Computer Club seminar, that attackers could effortlessly manipulate certain PDF functions for waging assaults on targeted networks. ‘In Audit’ published this in news on January 4, 2011.

    Citing one example, Ms. Wolf said that software within Acrobat Reader could be run via data networks having low security that could begin scanning a database within a specific network subsequent to the printing of PDF files utilizing the printer of that network.

    Moreover, given that PDF files can combine Flash, video, and audio files along with JavaScript and other similar formats that are potentially less secured, it implies that these files can conceal programs and data rendering them as probable environments for assault

  19. Patricia Boyle

    Well, I was affected by the Adobe security breach — somebody used my credit card to make a large fraudulent purchase. Thankfully I check my accounts daily and was able to notify my financial institution immediately and have the card cancelled.

Comments are closed.