In the wake of widespread media coverage of the Internet security debacle known as the Heartbleed bug, many readers are understandably anxious to know what they can do to protect themselves. Here’s a short primer.
The Heartbleed bug concerns a security vulnerability in a component of recent versions of OpenSSL, a technology that a huge chunk of the Internet’s Web sites rely upon to secure the traffic, passwords and other sensitive information transmitted to and from users and visitors.
Around the same time that this severe flaw became public knowledge, a tool was released online that allowed anyone on the Internet to force Web site servers that were running vulnerable versions of OpenSSL to dump the most recent chunk of data processed by those servers.
That chunk of data might include usernames and passwords, re-usable browser cookies, or even the site administrator’s credentials. While the exploit only allows for small chunks of data to be dumped each time it is run, there is nothing to prevent attackers from replaying the attack over and over, all the while recording fresh data flowing through vulnerable servers. Indeed, I have seen firsthand data showing that some attackers have done just that; for example, compiling huge lists of credentials stolen from users logging in at various sites that remained vulnerable to this bug.
For this reason, I believe it is a good idea for Internet users to consider changing passwords at least at sites that they visited since this bug became public (Monday morning). But it’s important that readers first make an effort to determine that the site in question is not vulnerable to this bug before changing their passwords. Here are some resources that can tell you if a site is vulnerable:
https://www.ssllabs.com/ssltest/
http://heartbleed.criticalwatch.com/
https://lastpass.com/heartbleed/
As I told The New York Times yesterday, it is likely that many online companies will be prompting or forcing users to change their passwords in the days and weeks ahead, but then again they may not (e.g., I’m not aware of messaging from Yahoo to its customer base about their extended exposure to this throughout most of the day on Monday). But if you’re concerned about your exposure to this bug, checking the site and then changing your password is something you can do now (keeping in mind that you may be asked to change it again soon).
It is entirely possible that we may see a second wave of attacks against this bug, as it appears also to be present in a great deal of Internet hardware and third-party security products, such as specific commercial firewall and virtual private network (VPN) tools. The vast majority of non-Web server stuff affected by this bug will be business-oriented devices (and not consumer-grade products such as routers, e.g.). The SANS Internet Storm Center is maintaining a list of commercial software and hardware devices that either have patches available for this bug or that will need them.
For those in search of more technical writeups/analyses of the Hearbleed bug, see this Vimeo video and this blog post (hat tip once again to Sandro Süffert).
Finally, given the growing public awareness of this bug, it’s probable that phishers and other scam artists will take full advantage of the situation. Avoid responding to emailed invitations to reset your password; rather, visit the site manually, either using a trusted bookmark or searching for the site in question.
NMAP can also be used to scan for Heartbleed if you include a script:
http://rollingwebsphere.blogspot.com/2014/04/scanning-for-heartbleed-with-nmap.html
Thank you Brian for giving information on this and Lenny for pointing out a solution to scan offline for the vulnerability. I’m managing some storage appliances which obviously are not accessible from the Internet. While waiting for an official stayement from the vendor I was looking for a solution to see if they are wulnerable without an online scan.
What’s distressing about this is finding out EXACTLY who is now safe and who isn’t. For example, it seems that everyone agrees that Yahoo was vulnerable, but has now fixed things, so a Yahoo user could update their password, presumably safely at this point.
Apple seems to be a huge unknown. Your first site says http://www.apple.com is OK. But Qualys gives http://www.apple.com an “F” WHAT THE??
Mashable says outlook.com was never affected, but Qualys gives the site MULTIPLE Fs, and your first site implies there MAY be a problem. WHAT THE???
The SSL Labs/Qualsys test is checking more than Heartbleed, thus an F can be due to some other weakness or deviation from best practice.
In the two examples you cited, run the test again and on the F sites, drill down by clicking the IPA hyperlink to see the detailed report.
Sargent Schultz reporting. ( http://www.youtube.com/watch?v=UmzsWxPLIOo )
I went to the three largest banks in the U. S. according to http://www.relbanks.com/top-us-banks/assets .
1. https://www.ssllabs.com/ssltest/analyze.html?d=www.chase.com
2. https://www.ssllabs.com/ssltest/analyze.html?d=www.bankofamerica.com
3. https://www.ssllabs.com/ssltest
/analyze.html?d=online.citibank.com
The server rating guide is available at
https://www.ssllabs.com/downloads/SSL_Server_Rating_Guide_2009e.pdf
HA!HA! Gotta love old Sarge! 😀
Thanks for the clarification!!
The only site on my list was Yahoo that was ready to change password, the other sites weren’t, in any way, critical, even if they were compromised. YMMV
Were you actually able to change your Yahoo password? I’ve attempted to do this countless times, but after inputing my old password, new password and clicking “continue, ” nothing happens. LastPass wants me to confirm the change, which I don’t do, but nothing changes on Yahoo’s side. I’ve used numerous different browsers and even different operating systems, but I’ve had no luck.
Yes – but I had to switch to Firefox because none of the other browsers would work. I’ve had to run CCleaner once and a while with various browsers to gain functionality also. Go figure!?
I was using the latest Firefox with ABP and NoScript, on Vista Ultimate x64. I’ll list my various blended defenses if you wish.
JCitizen, I applaud your goodwill and help. However listing one’s defenses to try to help can itself be turned into an infosec phishing scheme and make you less secure. Should someone ask for that information, I would be extremely suspect.
Ideally, the blackhats should gain nothing from an event.
regards,
Lee
I must admit that my blended defense scheme is really not a good enterprise practice; so I hesitate boring anyone with my personal details. Anymore, I prefer tools that work in, and despite of, an infected environment, in the kernel space, to resist malware manipulation. As long as one looks at it that way, and keeps everything updated, and takes full advantage of the operating system’s inherent security features, I think most people will be golden at their home office.
Well, I never had any success with the normal “change password” process, but I did finally achieve it by going through Yahoo’s password “reset” process.
Apple OS X Server is secure because OS X uses OpenSSL v 0.9.8y, a version that precedes the heartbeat “upgrade.” You can check this yourself with the command:
$ /usr/bin/openssl version
OpenSSL 0.9.8y 5 Feb 2013
However, anyone who downloaded and used Macports probably has a heartbleed-vulnerable version of openssl installed, like this:
$ /opt/local/bin/openssl version
OpenSSL 1.0.1e 11 Feb 2013
Macports just released a patch, do this:
$ sudo port selfupdata; sudo upgrade openssl
Furthermore, the Tunnelblick VPN project used a openssl version that is susceptible to heartbleed. Tunnelblick is also patched now, but according to According to OpenVPN’s James Yonan, “the tls-auth option should protect against this vulnerability”. I don’t know how much confidence I should put in his word “should”.
I’d like more specifics on the vulnerability of certificate-based OpenVPN servers.
Bottom line: If you were running a web server that used /opt/local/bin/openssl or an OpenVPN server without the tis_auth directive, to the best of my knowledge, you must assume that ALL credentials on your box are compromised — anything that was ever in computer memory like everything in your keychain, all account passwords, all private keys, literally everything.
I believe that’s “selfupdate” not “selfupdata”
OpenVPN’s heartbleed FAQ says that if the server specifies the tis-auth directive, then there is no heartbleed attack without the TLS certificate. Then it becomes a question of how widely available your server’s TLS certificates are: low risk for a few users with certificates stored in secure keychains, or high risk for a large number of users with know confidence how the certificates are stored.
i thought the PayPal reaction was interesting:
https://www.paypal-community.com/t5/PayPal-Forward/OpenSSL-Heartbleed-Bug-PayPal-Account-Holders-are-Secure/ba-p/797568#
[excerpted]
We would like to assure you that with regards to the Heartbleed bug:
1) Your PayPal account is secure
2) Your PayPal account details were not exposed in the past and remain secure
3) You do not need to take any additional action to safeguard your information
4) There is no need to change your password
According to a doc on Gethub of a scan of a large number of sites on the 8th, before a lot of wholesale patching had occurred, PayPal is not shown as vulnerable because it had no SSL (I don’t know if this equates to non vulnerable because smth else is used instead it that the test couldn’t be run because SSL was missing.)
https://github.com/musalbas/heartbleed-masstest/blob/master/top1000.txt
It is also interesting to note that running the SSL Labs test on paypal returns an F on one of their servers because it is vulnerable to a MiM attack!
For http://www.paypal.com – Qualys SSL Labs reports server IP 23.73.82.234 as an F with a bad protocol setting – This server is vulnerable to MITM attacks because it supports insecure renegotiation. Grade set to F
The other two servers rate an A-
By the way, how did you come up with the paypal statement? I didn’t receive any email from paypal (or any other site), was it posted somewhere on the paypal site?
((Out of an abundance if caution and because since I started using iCloud Keychain across all my apple devices (which makes best practice pw creation and pw changes really easy) I changed my paypal pw anyhow.))
As always, great information.
Thank you Brian.
While the testing sites are appreciated, I am getting inconsistent results on whether sites are vulnerable (sites that LastPass says are vulnerable are showing up as clean on some of the other links you provided). As a helpful gesture, LastPass has generated a list for me of vulnerable sites in my own vault. However, when I cross checked a few random sites against my vault of sites (I use it for password management) sites that show up as bad in its own heartbleed checker are not flagged when you do a security check on your own vault; as an example, the website for E*Trade, which shows up as vulnerable on some lists. While it shows up as bad in the LastPass individual website heartbleed checker, when I run the security check against the entry for it in my own vault, it was not flagged.
LastPass is recommending that you NOT change passwords on sites that have the vulnerability, but to wait (which will require re-validation), which seems consistent with your own recommendation that you may have to change your password more than once on sites that are vulnerable.
Brian. There remains a significant problem with these testing sites of false negatives. This is true of sites like LastPass which show a date. The reason is because many CAs do not consider reissuing a certificate cause to update the date that a certificate was issued. So for example LastPass shows a site like Fark.com as vulnerable even though it has been publicly stated by the administrators that it has been patched.
A site like LastPass is apparently doing nothing but checking to see if the site uses OpenSSL and then looking at the certificate date. If the date is old, it assumes it has not been patched. A horrible methodology.
Because of this CA update problem there is no effective third-party way to tell WHEN a websites vulnerability to this bug ended.
I don’t know – Qualys always seemed golden. Whenever I’ve contacted the server administrators about their reports, they have thanked me and related a successful patch or reconfiguration. I’ve never been told it is not accurate. Subsequent tests, after corrections always showed success, or pointed out other mistakes.
I used to use another service that has since left my memory, but it was a little less detailed than Qualys, but always turned out to be correct in the end, as well.
I wonder, wouldn’t companies that know they have been vulnerable at a certain point in time and have patched their systems but have never informed their users about the exposure fail on practicing due care and could they be sued for that?
http://mashable.com/2014/04/09/heartbleed-bug-websites-affected/
Is a great list of websites that are common, their status and even their quote on what their status is.
It seems to me it would be a good move for all financial and e-commerce sites to at least acknowledge they are working on the problem. If a site knows they are secure it would boost customer confidence to know that. Each site would benefit from posting a status message on their log in page to keep their users and customers up to date.
Contacted PayPal immediately upon reading some of the posts on this article, and testing their site – They say they are escalating the server issue to a higher level. Qualys reports they indeed are patched for the Heartbleed bug, but need to get it together on one other issue.
Does anyone know if tomcat on a windows server is likely to be vulnerable? I believe it’s based on OpenSSL, but OpenSSL isn’t actually installed as far as I can tell unlike a Linux host.
Looks like it should be affected:
https:// stackexchange.com/questions/55139/does-the-heartbleed-vulnerability-affect-apache-tomcat-servers-using-tomcat-nati
Roughly: Tomcat -> APR -> OpenSSL
The question is what new holes have the NSA buried in the latest version?
Open SSL? It wouldn’t surprise me if that “thousand eyes” didn’t miss something in the code.
Or 1000 hands out for a bribe to include it in the distro.
Well, a thousand eyes seem to have overlooked this one.
What would be interesting to know if whether Google and Facebook (which were not affected AFAIK) knew about this in advance and fixed it themselves, have their own implementations of SSL, or are running a patched up version of 0.9.8 or 1.0.0.
Hmmm? Interesting!
I don’t it’s correct to say that they weren’t affected.
I know that Akamai acknowledged being given advance notice. I’d assume that same applies to Google and Facebook.
Finally the elephant in the room steps up!
This vulnerability has been around for almost two years, only those who could afford to pay for this info could use it before last Monday.
Consider how many sites you may have visited in the last two years and whom has had the information available to them. Maybe not criminals or bad guys? Maybe they just did not use it yet?
Call me paranoid but it seems like changing passwords would be a start. I guess anything over the last few years could have been exposed to “someone”.
If PayPal itself was not vulnerable, what if you shopped on a vulnerable merchant web site (if there was one) and used PayPal as your payment method? Could your PayPal account and password be visible through the merchant’s web site? Sorry if this is a dumb question.
I’d assume yes, because you don’t know which server Paypal will select to handle the exchange. I let Paypal know about their protocol vulnerability. They are patched for Heartbleed, but there is more ways an MITM attack can progress. Only one of their 3 servers flunked the test for protocol configuration.
Brian Krebs is on the New York Times website today again talking about this very issue. Way to go Krebs !
http://www.nytimes.com/2014/04/10/technology/flaw-calls-for-altering-passwords-experts-say.html
There’s our man again! Taking the lead, while others are repeating what he’s already said. Way to go, BK!
Thanks for pointing out the article.
Poor photography, unfortunately.
The bright reflection in the lower left quadrant inexcusable. And a NYT photographer produced this picture?
George,
It’s not a reflection. That light source is a digital projector that was on when the photographer was taking pictures. He was trying different angles to get an artsy photo that incorporated the light from the projector. The inclusion of the light in the picture wasn’t an accident.
Checked krebsonsecurity.com with all four of the URL’s provided in the article.
Results :
First :
test never completes, the progress indicator stops just before completion, the “circle” keeps going around and around
Second:
Gives a grade of B
Third :
“Good news ! You are not vulnerable”
(Says the same for Yahoo. too)
Fourth (quoting):
Site: krebsonsecurity.com
Server software: Apache
Vulnerable: Likely (known use OpenSSL)
SSL Certificate: Unsafe (created 5 months ago at Nov 19 00:00:00 2013 GMT)
Assessment: Wait for the site to update before changing your password
My take : whom do I believe (not for this site, but in general) ??
I had similar results. Three of five validation sites said my bank was OK. Two were iffy, one of those being identical to your second negative result.
This is about integrating information.
If a site is using Apache and isn’t currently vulnerable, it’s quite likely that it was vulnerable. Thus, if the certificate has an issue date from before Tuesday, you should probably not trust it and bug that site to have a new certificate issued (with an issue date / not-before from “now”! — apparently some CAs list stale issue dates for new certificates?!) and deployed.
It is important to note that hackers have been reported as actively exploiting this problem since at least November 2013. Additional analysis of logs from honeypots may show that it has occurred even earlier.
Ref: http://arstechnica.com/security/2014/04/heartbleed-vulnerability-may-have-been-exploited-months-before-patch/?comments=1
Thus, it is important to know not just if a site you visit is now safe, but was it ever unsafe. Your credentials could have been stolen months before (and queued up for future bad usage).
I expect the fallout from this to be long and slow as hackers craft exploits using stolen server keys.
I also worry long term about hardware appliances embedded with OpenSSL that will never get patched and always be suspectible.
OK, in that case, change passwords on all sites that are now not vulnerable. Oh boy, this is the tip of the iceberg, isn’t it. The solution relies on everyone with the power to make the corrections to do the right thing. No wonder the security community is taking this so seriously.
All sites that have ever used OpenSSL, that is.
True – but bear in mind that other vulnerabilities can exist at any site if the server is not set up properly or they have flaky certificates, etc. So just because they have OpenSSL, and the patch, does not mean they may not have other problems.
Also sites that do not use OpenSSL may have other issues if the server administrator is not practicing due diligence.
The most likely scenario is stealing usernames, passwords, account nos, etc, that show in plain text in the 64k of random server memory scarfed down with the exploit.
Certificates are less useful:
1) Everyone will be scrambling to get new certs in the next few weeks.
2) A stolen cert can be used for a Man in the Middle attack, but a MITM attacks is “noisy”, kind of like a burglar using a saw or a hatchet instead of coming in through an unlocked door.
3) A stolen cert can be used to playback a previously recorded stream of encrypted data, even if the site upgrades their cert. So, somebody would have had to record a stream of data, expecting to decrypt it with the quantum computer in their basement. Now it’s a lot easier *if* they managed to get their hands on a cert for their recorded stream.
So far no one has sprung up selling any stolen certs – BK is probably on the lookout – it would be a big story to break. Meanwhile, change your passwords.
Harry S hit upon something that has been bugging me for the last couple of days. A lot of these sites that are quoted as being ‘safe’, i.e. running a newer version of OpenSSL that isn’t vulnerable, doesn’t state whether or not those same sites–or any sites marked as safe–EVER used a vulnerable version of OpenSSL in the last 2 years that Heartbleed has been around.
So at this point, I’m operating under the assumption that all of my passwords are likely compromised. However, at the speed sites are patching themselves, it will take a long, long time before I will be able to change them all safely [without having to worry about them being compromised again via this vulnerability]. Is there anything that can be done in the meantime? Simply not log into the vulnerable sites? Scrub those accounts of all personally identifiable information in case hackers -do- log in? Create a new linked email address that is only for Heartbleed accounts and nothing more?
Here in Canada in the midst of tax season the CRA – Canada Revenue Agency – Canada’s version of the IRS minus the kicking in of doors, shuttered part if their website yesterday and it is still closed due to Heartbleed.
I’m proud of the CRA for taking this action. It gives them time to:
1. Update all server software
2. Investigate to see if there is anything wrong with their servers (files that don’t belong – inserted by hackers who recovered administrative credentials, configuration which may have been changed by such a hacker)
3. Issue and deploy new certificates to all servers
4. Expire and replace all administration credentials
And since everyone gets an extension for the days CRA is closed, they can focus on fixing their servers / resetting their personal credentials everywhere else.
Has there been ANY confirmed report of a exploit using Heartbleed that lead to data loss by ANY web site?
As noted by several others above, the lastpass checker is terribly misleading. Please remove the link! It gives false positives on many sites that were never vulnerable to begin with and are not vulnerable now.
Otherwise, great post and list of resources.
One very important thing that a lot of media sites are not mentioning when it comes to password changes is that even IF a site fixes itself, and then you change your password… unless the site has revoked its old SSL cert, issued a new one and created a new public/private key pair, changing your password very well may be for naught.
Every site affected by Heartbleed should be operating under the assumption that their keys are now in the wild and exploitable.
Hi Brian, et al… kindly check (with a https website of your own liking & see if you think there is a tempest in a teapot brewing – or not? I just tested> https://www.discover.com/ [on these sites…]
LastPass shows:
Site: http://www.discover.com
Server software: Apache
Vulnerable: Likely (known use OpenSSL)
SSL Certificate: Unsafe (created 3 weeks ago at Mar 24 00:00:00 2014 GMT)
Assessment: Wait for the site to update before changing your password
http://filippo.io/Heartbleed/#www.discover.com
Heartbleed test
If there are problems, head to the FAQ
Enter a URL or a hostname to test the server for CVE-2014-0160.
All good, http://www.discover.com seems fixed or unaffected!
http://heartbleed.criticalwatch.com/
Heartbleed Tester
Heartbleed explained
Test for CVE-2014-0160
OpenSSL Buffer Over-read
Learn more about Heartbleed on learning.criticalwatch.com/heartbleed
Enter Website to be tested
https://www.discover.com/
[was entered in proper field]
Result from post:
Great News! your are not vulnerable to heartbleed
Moreover:
https://www.ssllabs.com/ssltest
gives https://www.discover.com/
a ‘Grade’ of A-
Overall I’m seeing that quite a few folks are complaining that LastPass is spewing false positives. Just Sayin’…
Best Regards,
Mica
Well, after all, Lastpass is not an authority on complete SSL testing; and they use more than one factor to check that you are at the right site. So I’m not sure I really care that they setup a custom checker just for the heartbleed issue, and not really much of anything else.
I think they were just trying to cover that one subject for their concerned customers. IT geeks will go elsewhere for a more expert over view.
While LastPass isn’t authority on SSL, in principle, I agree with their view here:
If you were vulnerable in the past, then it’s possible that the certificate you were using before you fixed heartbleed had its private key information compromised.
Also, to make it easier for all lay people, everybody who could have been impacted should replace their certificates (using new private keys). You could look at this as “an abundance of caution”, but it’s really the only easy and clear line anyone can draw.
Download and run Nmap along with the Heartbleed script if you run any devices with an IP address. Any admin should be able to do it.
http://rollingwebsphere.blogspot.com/2014/04/scanning-for-heartbleed-with-nmap.html
I have verified that it works. i.e. the script spotted one of my boxes running openssh version 1.0.1c. I’ve patched it with 1.0.1g and rerun the scan.
Boy, I can’t wait for the “Internet of things”. And were we not assured that “the cloud” is secure? Guess not.
I’ve been neck-deep in patching my company’s servers against Heartbleed over the past two days, and discovered a few things:
1) Updating the OpenSSL 1.0.1 libraries to 1.0.1g on a Linux server is realtively easy.
2) When an application statically links to vulnerable OpenSSL libraries, the application must be updated.
3) When an application dynamically links to vulnerable OpenSSL libriaries which are included within the application’s folder structure, you may have to update the aplication, because it is difficult to tell the exact compiler options (assembler routines, external engines, etc) which were used to build the libraries.
4) For systems which are not vulnerable to Heartbleed because they use older versions of OpenSSL — these systems may not be well-protected against the BEAST vulnerability. The SSL Server tester at Qualys (SSL LABS) can test for this.
5) After updating software, servers often need restarting.
6) Self-signed SSL certificates should then be regenrated, while CA-issued SSL certificates should be re-issued (and the old certs revoked).
Forgive my naivety but is the crux of this problem that we need to change some of our passwords? We should be doing that on a regular basis. Of course these breaches are going to continue so the importance of good Internet practice and being well informed is what we need to be focused on. Thanks Brian for helping us all be aware of this latest breach.
Executive summary:
1. Sites need to deploy the fix and replace/ expire possibly compromised authentication data.
2. You need to replace your passwords – but not before 1.
Every year there is some serious security exploit.. Seems for 2014 this is the one … A lot of fun for days to patch servers …
Did I try to enter something I should not have ?
My comment has not appeared.
Now I see : my comment is awaiting moderation.
What help is there for those of us who are not as computer savvy as you folks who have posted are?
What can we as seniors do, in order to protect our mutual fund or IRA? Insuring that our account is not going to have a zero balance, the next time we log on to view it?
Can anyone help me sort through the “open SSL’ business, one step at a time, please?
Where do we start? What do we do?
Many thanks for taking the time to respond!
Joaquin
Joaquin,
1. Choose a new password for each of your critical accounts. Make the PW strong and unique for each site.
2. Go to each of your critical accounts and change your password.
3. Repeat the above steps every two days for the next six days.
4 Check your accounts via phone every day and set up alerts on your accounts that will notify you if your balance changes outside of what is normal for you.
5. Watch for fake, and real, requests to change your PW. Confirm all such requests by calling the published 800 number, or your broker, or better yet stop into branch.
6. Some financial organizations allow account holders to set up auto text messages if the balance changes. Set that if if your organization offers it.
It is always a good idea to change your PWs from time to time. What you want to do here is change your PW after the server had been fixed. Until the server is fixed you will not be safe. I believe this is the worst security breech in the history of the Internet. This has ramifications far beyond the obvious.
You do not need to switch out your router, modem, computer etc… At home.
Good luck!
Mike
I, for one, am sad to see this exploit exposed and attacked. For about three years now I have been making about $100 a week fixing hacked Yahoo email accounts.
I am glad you thought about VPN. OpenVPN is vulnerable for instance.
But there is a missing bit in this article: POP, IMAP and SMTP over SSL. Many mail servers will need an update.
Jabber/XMPP also offers a vulnerable service over SSL.