Almost weekly now I hear from an indignant reader who suspects a data breach at a Web site they frequent that has just asked the reader to reset their password. Further investigation almost invariably reveals that the password reset demand was not the result of a breach but rather the site’s efforts to identify customers who are reusing passwords from other sites that have already been hacked.
But ironically, many companies taking these proactive steps soon discover that their explanation as to why they’re doing it can get misinterpreted as more evidence of lax security. This post attempts to unravel what’s going on here.
Over the weekend, a follower on Twitter included me in a tweet sent to California-based job search site Glassdoor, which had just sent him the following notice:
The Twitter follower expressed concern about this message, because it suggested to him that in order for Glassdoor to have done what it described, the company would have had to be storing its users’ passwords in plain text. I replied that this was in fact not an indication of storing passwords in plain text, and that many companies are now testing their users’ credentials against lists of hacked credentials that have been leaked and made available online.
The reality is Facebook, Netflix and a number of big-name companies are regularly combing through huge data leak troves for credentials that match those of their customers, and then forcing a password reset for those users. Some are even checking for password re-use on all new account signups.
The idea here is to stymie a massively pervasive problem facing all companies that do business online today: Namely, “credential-stuffing attacks,” in which attackers take millions or even billions of email addresses and corresponding cracked passwords from compromised databases and see how many of them work at other online properties.
So how does the defense against this daily deluge of credential stuffing work? A company employing this strategy will first extract from these leaked credential lists any email addresses that correspond to their current user base.
From there, the corresponding cracked (plain text) passwords are fed into the same process that the company relies upon when users log in: That is, the company feeds those plain text passwords through its own password “hashing” or scrambling routine.
Password hashing is designed to be a one-way function which scrambles a plain text password so that it produces a long string of numbers and letters. Not all hashing methods are created equal, and some of the most commonly used methods — MD5 and SHA-1, for example — can be far less secure than others, depending on how they’re implemented (more on that in a moment). Whatever the hashing method used, it’s the hashed output that gets stored, not the password itself.
Back to the process: If a user’s plain text password from a hacked database matches the output of what a company would expect to see after running it through their own internal hashing process, that user is then prompted to change their password to something truly unique.
Now, password hashing methods can be made more secure by amending the password with what’s known as a “salt” — or random data added to the input of a hash function to guarantee a unique output. And many readers of the Twitter thread on Glassdoor’s approach reasoned that the company couldn’t have been doing what it described without also forgoing this additional layer of security.
My tweeted explanatory reply as to why Glassdoor was doing this was (in hindsight) incomplete and in any case not as clear as it should have been. Fortunately, Glassdoor’s chief information officer Anthony Moisant chimed in to the Twitter thread to explain that the salt is in fact added as part of the password testing procedure.
“In our [user] database, we’ve got three columns — username, salt value and scrypt hash,” Moisant explained in an interview with KrebsOnSecurity. “We apply the salt that’s stored in the database and the hash [function] to the plain text password, and that resulting value is then checked against the hash in the database we store. For whatever reason, some people have gotten it into their heads that there’s no possible way to do these checks if you salt, but that’s not true.”
CHECK YOUR ASSUMPTIONS
You — the user — can’t be expected to know or control what password hashing methods a given site uses, if indeed they use them at all. But you can control the quality of the passwords you pick.
I can’t stress this enough: Do not re-use passwords. And don’t recycle them either. Recycling involves rather lame attempts to make a reused password unique by simply adding a digit or changing the capitalization of certain characters. Crooks who specialize in password attacks are wise to this approach as well.
If you have trouble remembering complex passwords (and this describes most people), consider relying instead on password length, which is a far more important determiner of whether a given password can be cracked by available tools in any timeframe that might be reasonably useful to an attacker.
In that vein, it’s safer and wiser to focus on picking passphrases instead of passwords. Passphrases are collections of multiple (ideally unrelated) words mushed together. Passphrases are not only generally more secure, they also have the added benefit of being easier to remember.
According to a recent blog entry by Microsoft group program manager Alex Weinert, none of the above advice about password complexity amounts to a hill of beans from the attacker’s standpoint.
Weinert’s post makes a compelling argument that as long as we’re stuck with passwords, taking full advantage of the most robust form of multi-factor authentication (MFA) offered by a site you frequent is the best way to deter attackers. 2fa.directory has a handy list of your options here, broken down by industry.
“Your password doesn’t matter, but MFA does,” Weinert wrote. “Based on our studies, your account is more than 99.9% less likely to be compromised if you use MFA.”
Glassdoor’s Moisant said the company doesn’t currently offer MFA for its users, but that it is planning to roll that out later this year to both consumer and business users.
Password managers also can be useful for those who feel encumbered by having to come up with passphrases or complex passwords. If you’re uncomfortable with entrusting a third-party service or application to handle this process for you, there’s absolutely nothing wrong with writing down your passwords, provided a) you do not store them in a file on your computer or taped to your laptop or screen or whatever, and b) that your password notebook is stored somewhere relatively secure, i.e. not in your purse or car, but something like a locked drawer or safe.
Although many readers will no doubt take me to task on that last bit of advice, as in all things security related it’s important not to let the perfect become the enemy of the good. Many people (think moms/dads/grandparents) can’t be bothered to use password managers — even when you go through the trouble of setting them up on their behalf. Instead, without an easier, non-technical method they will simply revert to reusing or recycling passwords.
I haven’t noticed others mentioning this but I use following scheme on the sites I seldomly visit but which require an account. It is that I create a throwaway password and each time I access that site I will reset the password.
Most people have no concept of basic information processing or security. They are literally incapable of thinking through the processes involved.
Thanks Bob! You must be the village idiot who likes to be dramatic at the same time of doing consdescending reach-outs. Maybe you should strike a balance, and therefore, be more realistic in your views of others. I know, your arrogance gets in the way….too bad..
Except that in this case, Bob is 100% correct. What sort of arrogance does it take to lie in wait for the first person who comments and then pounce on them like that?
In my 25+ years of enterprise IT I can attest to the fact that the vast majority of end users have NO clue what passwords or any other form of security are for and see them as a corporate bureaucratic nuisance to be sidestepped at all costs.
look in the mirror for the idiot.
gracious. do you understand how the global monetary system works or do you not use cash and cards ?
So my question is, why don’t web services hash userid’s as well as passwords? Especially sites that insist on using e-mail addresses as userid’s?
The problem with that is usually websites will need (or in some cases want) your email address for announcements, notifications, marketing etc, which is a valuable asset. If your email address is hashed, they can’t access it, so their own user base essentially becomes uncontactable outside of the site.
Users like not having to remember account numbers.
It’s true that having two unrelated secrets would be better than one piece of public information and one reused piece of information.
But, also, eventually, your login name is going to leak, at which point it’s no longer a secret and the attack is repeated as in the current state.
It’s better to add MFA (twofactorauth.org as Brian mentions) and encourage pass phrases (as Brian mentions) than it is to try to use random logins and deal w/ very angry users.
It’s painful enough for users who have 2-10 email addresses and have to guess which account to use for rarely used sites, imagine having to write down an account table for every site?
Sure, if you use a password manager, then worrying about remembering login names isn’t a big deal, but at that point, neither is using incredibly long randomly generated pass phrases. And again, there’s more security there.
Great article, thank you!
One thing I take issue with is you suggesting that writing down passwords could me ok. Writing down passwords is not a good practice. Sooner or later, a password that is written down may end up being left laying around somewhere.
You suggested that users can write down the password and store it in a locked cabinet or safe etc. However it does not sound practical for a user to visit a locked safe at home each time they forget their password, as very often they are trying to use the password away from home or not anywhere close to the safe or cabinet.
Passphrases are a better option, and I am glad you mentioned those also. So for anyone having issues remembering passwords, I would recommend passphrases.
For the home user, writing down passwords is absolutely fine. They don’t even need to lock them in a cabinet, unless their family members are a threat. If one’s home is broken into, they’ve got bigger problems.
Anything is fine if you are okay with that level of security. But from my experience, physical security concerns tend to be even more lacking than electronic security. I have seen MANY more instances of compromises coming from storage of passwords in unsafe locations and social engineering than anything else. Depending on where you work, writing down a password and storing it in an insecure location can even result in negligence charges… But that is FAR from everywhere. Just my $0.02…
I only have to remember a couple passwords and those are for my email addresses. Those passwords are long and crazy looking, but I can remember them. All other sites I just create a new password every time I log in.
For the comment crew who were insistent that Glassdoor couldn’t be salting their hashes if credential stuffing attacks worked in testing- How do they think it normally works when you enter your password for a site? Anything Glassdoor does or doesn’t do on the backend is completely irrelevant to the data a user actually supplies when they authenticate- it’s just a password.
And in the case of these tests, that’s exactly what’s being entered at the beginning of the meat grinder- a password used by accounts linked to the same email address or username. If the resulting output/ground meat (however many grinders are turning along the way) exactly matches the stored value, then the user either used the same password, or, in a vanishingly unlikely coincidence, there was a hash collision. Either way, if a bad guy entered that password manually in an attempt to authenticate as that user, they’d succeed, so the user should change it.
I recently watched a google talk by Frank Abagnale. I look forward to the day when passwords are no longer required. He said 5 years.
A few years ago, I changed my user ID on Netflix which was an email account. I sat amazed & horrified as I watched someone/thing enter a password 3X. Needless to say, I changed it back.
Microsoft is now trying to push people into not using a password by using Windows Hello that contains both face ID and biometric (this one is a USB based system) and PIN.
You’re right that it will come 5 years.
My dad aged 74 adopted password manager two years ago and he absolutely loves it. He used 7 character piss-poor password before, so he could type it fast with sausage fingers… Now he has autofill from lastpass.
While initially I tried to get him on Bitwarden, UI didn’t work for him. Lastpass isn’t my personal fav, nor I’d go ahead with autofill, but his overall security skyrocketed – random +20 digit auto-filled password beats the hell out of 7 character password.
Now, he doesn’t understand security too well, but he trusts me and he agreed that it’s faster to use password manager.
Password managers would have a lot better uptake if they marketed themselves as “convenience tools to fill out password forms faster” rather than
“security is important, don’t reuse your passwords because leaks, poor crypto practices and hunter2 is really not cool, md5 is not good for passwords, salt is not a condiment”.
When I tried to convince my friends to pick up any password manager, it flew other their head. They even struggled with concept of haveibeenpwned, saying that their ‘spooky1’ password hasn’t been leaked because it’s their password (uhhh, aargh!). Then, they saw me use it, saw the speed – even via extension and clicking login manually, they were sold.
Just stop using email address’ as username, it makes no sense security and privacy wise. It like having no pin code on your credit card, and not using the chip.
What if you use a different email address for every site, and a service like ManyMe?
Modern cracking methods try word combinations. If you want to know how secure a password is, check it with the John The Ripper password cracker. Increasing the length of a non-random password or using passphrases isn’t effective unless the number of cracker database elements is at least four; perhaps more. I use a password manager and two methods for the passwords that I have to type: (1) a serial number that I can go look at or hide in plain sight or (2) a randomly generated string followed by a mnemonic for that use. (If you type any of the latter passwords often, you’ll quickly remember the random part.) One method I’ve also considered using is a string like “dywtwfmb” composed of the first letter of each word of a passphrase that you can remember.
Just use simple, long passwords, people.
Do not make it complicated. Randomness in a password’s characters only has a marginal effect on difficulty to crack it, because it frustrates a dictionary-based attack. But that’s it.
And it’s harder to remember a complex password.
“Dorothy wears ruby slippers in Dropbox” or “Senator.Warren.Smells.Like.Hotmail” better than “£#£#9#rudud43wegd.”
Easier to type, too.
Using a central cloud storage or password manager (or a sticky note attached to your cublice wall) for all your stupidly complex passwords just adds another attack point.
Or point of failure.
The best password is long and easy to remember, so you’ll be able to use it without jotting it down.
In case someone cares, here’s why a long password beats a complex one:
https://math.stackexchange.com/a/1934499
That last paragraph is why we have to change the entire password concept. People simply arent going to do all that. They will reuse passwords.
A lot of people dont hardly have to go online and we expect them to memorize haxor language words and have them remember them 6 months later. So you can tell grandma to use &hHyyud%6 and 7hhuuuYsa& for passwords but in 6 months when she cant get into those two sites they are going to get changed back to Lassie12. And every IT guy knows the worst time is when you force everyone to change passwords.. for a good month, people forget the new one.
phone authentification which is just starting to be a thing.. not two factor but just phone authentification is better than passwords. Its something they have on them and never forget and yeah 2factor is better than just phone but the entire password idea just sucks. I get we used the idea we had at the time. but its way way past time to replace it.
Passphrases are more fun to come up with an easier to remember too. Make them super weird, like “volcano koala sprinting doorknob” or random junk like that =)
Confession – I reuse passwords across sites. I also use my email address as username wherever it’s allowed (it’s easier to remember). I also encourage my relatives to write down passwords on a notepad with pencil and paper.
The problem with many of these articles is not classifying where strong passwords are critical. Strong, non-reused passwords are critical on any website’s relating to your financial data. Other websites perhaps where the currency in question is one’s reputation (facebook, instagram, etc) probably calls for strong passwords too.
But these critically important authentications are VASTLY outnumbered by “junk” sites – sites which require usernames/passwords for their own dumb reasons – perhaps just to track me, without any actual benefit to me. For these sites, I don’t really care if the credentialing is compromised – the cost is to low to care about. For all these sites, I reuse passwords with abandon.
Regarding writing down passwords – I encourage less literate computer users to write down there strong, unique passwords on a notepad, which they keep in a desk drawer. It’s a akin to a key to a safe for most – if a bad entity has access to your desk drawer to access your keys, then you’ve got more to worry about than a compromised password.
I reuse a password template for junk sites. I use a password manager for real sites. Password templates are fun. I’m not saying Grandpa is going to have fun with them – but all sorts of rules can be used to derive a password for a particular FQDN that can be discoverable in a few tries by someone who knows the template. I’m not going to share mine, but I’ve used it for over 20 years. The cool part is that you can determine which sites were compromised when you get those emails stating “I have all your secrets {password}”. Great, I vaguely remember creating an account on that site to read an article.
I re-use passwords 🙂 I re-use them on all the sites that don’t contain any sensitive information. If someone wants to hack my account and make a nefarious comment on the New York Times comment section, by all means feel free. They don’t have my real name anyway, or any other identifying info.
As for the ones I care about, they’ll never ‘guess’ them before we have quantum computing 🙂
In any event, the idea of password protecting accounts is failing little by little. And if they don’t figure out a different way to do it, we will eventually be left with no *practical* way to secure them. That is the elephant in the room.
I would love an article on the best way to handle passwords for the elderly or stubborn. It’s so hard getting others to use unique passwords for every site I just wish there was a easy way. I get so much flack trying to get them to use a password manager.
To make generating, and remembering, a personal password fun, I suggest to seniors an approach like this: Remember a zip code (5 digits) or the phone number (10 or more digits) of a favorite public place. If you forget the numbers you can look ’em up.
The phone number of the Metropolitan Museum in NY is: (212) 570-3951
The phone number of the Louvre in Paris is: +33 (0)1 40 20 53 17
The zip code of Jones Beach, NY, is: 11793
Their phone number is: (516)785-1600
Second hint: Take the name of the web site and spend some time inventing some wild and wooly associations with that name.
Amazon is a river in Brazil. Also: Amazons were mythical race of warrior women from ancient Greece. Kind of amazin’ ain’t it?
Gmail is like G-man. Think of Eliot Ness. Also: Gmail has “mail” in it. Think of US Post Office, letters, stamps, envelopes.
Yahoos are not very civilized persons. Yahoo! is a cry of joy or victory. Huzzah! Horray! O Happy Day!
Microsoft is tough. DOS anybody remember the last version of DOS? Microsoft is headquartered in Redmond. Remember the Red man of Barsoom?
Everybody has some quirky memory to associate with these things. Use those quirky memories to build a series of 4 or 5 “words.”
If I have iOS devices, I clump similar characters together. I avoid WhEnEv3r and Wh0t1sThis? Each shift requires changing iOS’s onscreen keyboard. I rely on the length of the passphrase (22+ characters is common).
I memorize just a few essential passwords. I just changed on of them. Hm. Better go and re-enter it a couple times to to reinforce my memory.
BTW, I’m over 65. I like setting myself memory challenges.
mobile lost
If you don’t trust pw managers and autofill, long and random passwords, stored in doc that is encrypted by gpg, pgp, or one of many other tools. Cut and paste instead of autofill. One pw to unlock others; no third-party pw manager; advantage of long and also no dictionary words besides.
Two factor is great until the site starts texting you or letting someone steal/’rent’ your number. Sites are using tfa as an excuse to lure us into giving out our phone numbers. Damn them for spoiling a good thing.
Advice like “just use a long password like ‘Have.a.nice.but.strange.day’ instead of a complicated one” ignores the reality that password rules vary widely, but commonly contain constraints like “Must contain at least uppercase, lowercase, number, and special character (but not that special character), the name of a protected minority, and a fast food condiment” or “use any length you want, from eight all the way up to sixteen characters.” Dain bramage is the most common denominator.
I’m amazed nobody mentioned XKCD’s comic about Password Strength:
https://www.xkcd.com/936/
Also, the first part of “Password Reuse” seems relevant:
https://www.xkcd.com/792/
The problem with this is that, it’s so much easier to crack.
Just use the same brute force attack, but instead of letters and numbers, use whole words.
So instead of 8 letters to test repeatedly, you have much less words to work with
We can change the length, but since the algorithm is so simple, you know where I’m going with this.
It’s a method to make it even easier, ironically.
I always wonder about computer time spent by actions as those of Glassdoor. You write in your article of millions or even billions of hacked accounts to be checked against. A single hash may cost 10 millisecond (most hashing algorithms have this time as input variable). When one wants to check 10^9 (a billion) userid’s, with one hash costing 10^(-2) seconds, you need 10^7 seconds (is 1157 days) of computer time.
Where do I go wrong in this calculation, or does Glassdoor use massive parallel systems with for instance 10^7 processors so that the time goes down to 1 second?
I find it a pity that nobody so far answered my question.
In the meantime I found that a general advice for the time of a single hash is somewhere between 250 and 500 msec, that is 25 to 50 times longer than my assumption of 10 msec. The computer time needed for Glassdoor’s check then goes up to somewhere between 20,000 and 50,000 days.
The reason for these long hashing times is obvious, it is to make cracking of password hashes prohibitively expensive. It then occurred to me that Glassdoor’s procedure is essentially a hack of their own database. In that sense the people that worried about Glassdoor’s safety measures are right: if Glassdoor can crack the entries in its database, others can too.
There is no cracking of the password occurring.
Glassdoor downloads the breached credentials from the source. They take paulw@gmail.com‘s password and feed it in to the login via API as if the user were submitting and check if it authenticated.
If it authenticates then it is a match and off goes the reset email.
This is the same thing attackers are doing daily against sites, minus the reset emails.
@Nick: The way I understood Glassdoor’s procedure was the following. Upon registration of a new user, a salted hash of his/hers password is checked by Glassdoor against a billion plain text passwords found on the dark net.
In order to do that, the billion passwords undergo, one by one, the very same procedure as the password of the new user. That is, the plain text password from the dark net is salted and hashed with the salt of the new user and compared to the salted and hashed password of the new user. This takes 250 to 500 msec per password and because it is done for a billion plain text passwords, the procedure takes in total 20,000 to 50,000 days. When a match is found, one could say that the password of the new user is “cracked” (in the sense that its plain text version is known).
Now that I’m writing this, it occurs to me that Glassdoor probably takes userids into account as well and only performs a check if the userid of the new user is found on the dark net. Say the userid is found 5 times, then only 5 passwords have to be checked (a matter of 1.2 to 2.5 seconds) instead of a billion passwords. That makes the Glassdoor procedure completely feasible.
A breached password tool is likely, >90%, an offline tool. So running it against the database over a couple of days is pretty easy. Another thing to note here is that tuning a hash to take a lot of iterations is much less effective nowadays than it used to be. You can thank the proliferation of crypto currencies for libraries and tools that can run almost any hash you might use for a password on a GPU with a speed up factor usually between 10 and 100 times. That means if you want a hash to take 10ms per attack you will be spending .1 to 1 second per customer login attempt to protect it.
actually if a few of these idiots who have nothing better to do with their time and resources than to destroy the lives and personal property of people they have never met, were publicly executed, this would stop real quick….We need some tough anti hacking and cyber behavior laws in place…..these morons have had carte blanche on the internet for way too long…
Though I can’t support your concept of public execution… I believe that the penalties and threats used to discourage electronic commission of crime HORRENDOUSLY lacking… Much of this can be explained by how difficult it is to prove beyond a reasonable doubt who was the actual guilty party, however, I DO agree that the penalty of such should be effectively life-ending, and anyone found to willfully contribute to commission of such crimes share a similar fate.
seems like an overly complicated process to compare hacked pwords with customers current pwords.
Wouldnt it be easier to just take the list of emails address that match the hacked addresses, take that password and feed it into an internal app to see if it permits login.
Then notify customer.
This is exactly what the story says Glassdoor is doing.
Great article, as usual, Brian!
Since I don’t want my passwords stored on the web, in whatever form, I use an encrypted spreadsheet, specifically a LibreOffice one that is platform independent and works on Windows, macOS and Linux operating systems equally. I believe it’s encryption is AES256. Microsoft Excel spreadsheets can also be encrypted, but is not as platform independent.
For passphrases that are both memorable and have words that aren’t related, I recommend DiceWare, https://www.rempe.us/diceware/
There is a spreadsheet implementation of this in both LibreOffice and Microsoft Excel, however as I no longer host websites, a current version is no longer available online. An old copy for example purposes is available on the Wayback Machine: http://web.archive.org/web/20161020235344/http://happycattech.com/security-apps/spreadsheet-diceware-passphrase-generation
Wait a minute though, doesnt this also mean these sites (GlassDoor etc.) are not using a random salt in the hash, at least at a site level? Maybe I’m a security noob but shouldn’t my hash at Site A be different at Site B even if I used the same creds?
I use a password manager on my iphone, Strongbox, which uses the PasswordSafes (psafe3 file type). I don’t trust online password managers like 1password etc as if they get hacked, my pw’s might get compromised. So I use the psafe3 format and store it on my dropbox (I have more faith in dropbox than 1password etc 😉 ). I then share this file via dropbox across all my devices so it’s always up to date.
Anyway Strongbox on the iphone lets me create multiple word passphrases so I thought great, I’ll change all my logins to use passphrases instead of %6GfrF43… I soon came to find out that not all websites allow more than a certain amount of characters, so you’re sometimes limited to 8/12/16 characters only… SMH! So much for passphrases.