November 24, 2015

All new Dell laptops and desktops shipped since August 2015 contain a serious security vulnerability that exposes users to online eavesdropping and malware attacks. Dell says it is prepping a fix for the issue, but experts say the threat may ultimately need to be stomped out by the major Web browser makers.

d3llAt issue is a root certificate installed on newer Dell computers that also includes the private cryptographic key for that certificate. Clever attackers can use this key from Dell to sign phony browser security certificates for any HTTPS-protected site.

Translation: A malicious hacker could exploit this flaw on open, public networks (think WiFi hotspots, coffee shops, airports) to impersonate any Web site to a Dell user, and to quietly intercept, read and modify all of a vulnerable Dell system’s Web traffic.

According to Joe Nord, the computer security researcher credited with discovering the problem, the trouble stems from a certificate Dell installed named “eDellRoot.”

Dell says the eDellRoot certificate was installed on all new desktop and laptops shipped from August 2015 to the present day. According to the company, the certificate was intended to make it easier for Dell customer support to assist customers in troubleshooting technical issues with their computers.

“We began loading the current version on our consumer and commercial devices in August to make servicing PC issues faster and easier for customers,” Dell spokesperson David Frink said. “When a PC engages with Dell online support, the certificate provides the system service tag allowing Dell online support to immediately identify the PC model, drivers, OS, hard drive, etc. making it easier and faster to service.”

“Unfortunately, the certificate introduced an unintended security vulnerability,” the company said in a written statement. “To address this, we are providing our customers with instructions to permanently remove the certificate from their systems via direct email, on our support site and Technical Support.”

In the meantime, Dell says it is removing the certificate from all Dell systems going forward.

“Note, commercial customers who image their own systems will not be affected by this issue,” the company’s statement concluded. “Dell does not pre-install any adware or malware. The certificate will not reinstall itself once it is properly removed using the recommended Dell process.”

The vulnerable certificate from Dell. Image: Joe Nord

The vulnerable certificate from Dell. Image: Joe Nord

It’s unclear why nobody at Dell saw this as a potential problem, especially since Dell’s competitor Lenovo suffered a very similar security nightmare earlier this year when it shipped an online ad tracking component called Superfish with all new computers.

Researchers later discovered that Superfish exposed users to having their Web traffic intercepted by anyone else who happened to be on that user’s local network. Lenovo later issued a fix and said it would no longer ship computers with the vulnerable component.

Dell’s Frink said the company would not divulge how many computers it has shipped in the vulnerable state. But according to industry watcher IDC, the third-largest computer maker will ship a little more than 10 million computers worldwide in the third quarter of 2015.

Zakir Durumeric, a Ph.D. student and research fellow in computer science and engineering at the University of Michigan, helped build a tool on his site — https://zmap.io/dell — which should tell Dell users if they’re running a vulnerable system.

Durumeric said the major browser makers will most likely address this flaw in future updates soon.

“My guess is this has to be addressed by the browser makers, and that we’ll seem them blocking” the eDellRoot certificate. “My advice to end users is to make sure their browsers are up-to-date.”

Further reading:

An in-depth discussion of this issue on Reddit.

Dan Goodin‘s coverage over at Ars Technica.

Dell’s blog advisory.

Update, 1:15 a.m. ET: Added link to Dell’s instructions for removing the problem.


44 thoughts on “Security Bug in Dell PCs Shipped Since 8/15

  1. Dennis

    Man, when will they learn, hah?

    I personally immediately rip out whatever version of OS image they pre-installed on my desktop and put a fresh copy of pure Windows that I got directly from Microsoft. This is the only way to ensure that you have a clean PC w/o any crapware, spyware, listen-ware, or whatever Dell and Lenovo like to install there.

    1. Trevor Bavoom

      > This is the only way to ensure that you have a clean PC…

      Unfortunately, that alone is not enough.

      Lenovo was recently caught using Microsoft Windows Platform Binary Table feature to inject their “Lenovo Service Engine” into the Windows operating system from the motherboard firmware after a full wipe and re-install of Windows, even using alternative media.

      See the article “CAUGHT: Lenovo crams unremovable crapware into Windows laptops – by hiding it in the BIOS
      How Microsoft made it possible, and how to truly purge it” (http://www.theregister.co.uk/2015/08/12/lenovo_firmware_nasty/) for details.

    2. Jonathan E. Jaffe

      The problem with a rip/strip/plant is you lose any custom device drivers shipped already installed. Did that once and disabled a specialty hard drive.

      That having been said, given that there an increasing number of diversions from the supply chain where the (body cams, USB memory sticks etc) get infected then repackaged, nothing, even brand new in the box machines can be considered safe.

      Better to boot before connecting to network and run every virus scanner you trust. Even that wouldn’t have stopped this one. “It isn’t a virus, its a feature! Ooops

      Jonathan

  2. femtobeam

    Certs again!

    Remember the fake Microsoft certs that redirected users to the fake Microsoft update sites? That went on for over a year during the transition to Azure (Dell Servers). I do not believe that there is a “pure” copy of Windows. Bloat ware is bad, but there are also bad certs for hardware/software combos as I learned with HP and Cyberlink. Google has started handling certs in the Chrome browser. Even new Apple iMacs have a long list of bad certs in the root. Why are they there in the first place? How to delete? Which ones to delete? How to prevent them from even being on the endpoints? Why aren’t they all being dealt with by the browser?

    Brian, we need a cert tutorial.

  3. Eric

    Brian, the link with Dell’s instructions sends me to “http://chrome-extension//gbkeegbaiigmenfmjfclcdgdpimamgkj/views/app.html”
    Which makes no sense to me…

    1. timeless

      That’s https://chrome.google.com/webstore/detail/office-editing-for-docs-s/gbkeegbaiigmenfmjfclcdgdpimamgkj?hl=en

      a.k.a. “Office Editing for Docs, Sheets & Slides”
      i.e., the link is to a .docx, and you have an add-on which can handle it.

      You should be thankful. My browser doesn’t have such an Add-on installed, so it downloaded it and offered to open the file in Pages.app, which couldn’t (presumably it’s too new of a .doc format for my old Pages app). In the end, I just uploaded it to Google Drive and viewed it in Google Docs (and then I deleted it), it was all very cumbersome.

    1. Randy Banks

      Found the same post and (Word) document (there’s high security for you. Haven’t tried the manual instructions yet, but can’t get the automatic removal patch to work on my vulnerable Win 8.1 box.

      Note that my vulnerable Dell was shipped (by Dell UK) in late May, way before August 2015.

  4. Wladimir Palant

    Assuming that the certificate is only installed on the system level (which the screenshot seems to imply) Firefox shouldn’t be affected because it uses its own certificate store and its own set of root certificates. Internet Explorer and Chrome on the other hand would be affected…

    Yes, how come that nobody recognized this as a problem? That’s some major demonstration of incompetence.

    1. timeless

      and Microsoft Edge.

      The answer is because they clearly didn’t ask anyone in the security space what the ramifications were.

      At the very least, they could have been told to limit the scope of the certificate (so that it couldn’t be used for Code-Signing or various other things)…

    1. timeless

      See http://krebsonsecurity.com/2015/11/security-bug-in-dell-pcs-shipped-since-815/comment-page-1/#comment-395817

      If you’re having problems, you should give feedback to the extension author.

      … or you could disable it:
      «Google Chrome:
      1. Click the menu icon “≡” at the top right of the browser window, choose “Tools” and choose “Extensions” to open a new “Options” tab.
      2. Uncheck “Enabled” to disable an extension, or click “Remove” to delete it completely.
      3. Click the “Disable” link under the plugin you want to disable.»
      … from https://support.box.com/hc/en-us/articles/200523808-How-To-Disable-Plugins-Add-ons-Extensions-in-Multiple-Browsers

    1. Trevor Bavoom

      Since the private key was included as well, everyone with a copy can now issue a revocation certificate.

      I would recommend that you not follow Dell’s advice to remove the certificate from any certificate store. Rather you need to mark the certificate as untrusted. You should even add the certificate to Firefox, marked as untrusted. Eventually, the browsers themselves will include the untrusted certificate and associated revocation.

    2. timeless

      Revoking a certificate breaks whatever things are using it.

      In this case, Dell Foundation Software.

      If you’re going to make a fix that you want people to download, you probably only want to make *one* fix — otherwise, you risk download fatigue. Thus, they’re better off making a fixed version of that software.

      In the interim, you can release workaround instructions (which they’ve done, they seem OK, except that there are additional certificates that their systems shipped which have the some problem, and thus the instructions are woefully incomplete).

      Also, I fully expect the next Microsoft Windows security update to include a revocation or similar, so it really isn’t the most useful use of Dell’s time (they need to identify all similar mistakes and fix their software, which will take time).

      Note: I’m not strictly opposed to revocation, I’m just saying they have more important things to do… Also, I don’t really trust them to get the revocation right– I’d rather Microsoft, Google, and Mozilla do it.

  5. Mike

    Most computers are infected right out of the box. It’s been that way for a long time now. This is one of the biggest reasons to build and NOT to buy (it’s not so much about the price).

    You’re saying since 8/15…….this goes back alot further than that. It involves alot more than just Dell.

    Not everything needs to be HTTPS. This idea that everything must move in that direction is delusional. Wifi is NOT safe, secure, or even all that stable. Particularly when it’s open and public. If security is even a concern, then connecting to the local coffee shop is already an instant ‘no’ regardless of any kind of questionable certificate.

    1. timeless

      Notably, Wi-Fi does support certificate based encryption. But it generally isn’t pinned to a CA (you can optionally do so, but Pinning has its own headaches, so people rarely do). And the Certificate here wasn’t restricted to a given purpose, so it could sign Wi-Fi certificates, which means that a computer visiting a “secure” Wi-Fi system could be MITM’d too.

  6. Daniel B Martin

    I bought a Dell recon desktop PC last week. Would it contain the same malware? It has a seemingly “clean” Windows 7 Professional.

    1. timeless

      Probably.

      The reports say Windows 7 was affected.

      Checking is easy. Use MSIE or Google Chrome and visit https://zmap.io/dell/

      If it says you’re affected, you’re definitely affected. If it doesn’t say you’re affected, you might still be (assuming that you use Chrome and Google ships an update…).

      If in doubt, follow the removal instructions:
      “To address this, we are providing our customers with instructions [1] to permanently remove the certificate from their systems via direct email, on our support site and Technical Support.”

      [1] https://dellupdater.dell.com/Downloads/APP009/eDellRootCertRemovalInstructions.docx

      * Feel free to use Brian’s link, he’s more trustworthy than a random commenter on his blog.
      ** Keep in mind that there may be at least two other similar certificates that you’ll need to deal with, unfortunately, this was the tip of the Dell iceberg, we know it’s deeper than 1, but we don’t know how deep, and their first instructions only cover the first one (3 are known…).

  7. Sharon Lee

    Your article states “Dell says the eDellRoot certificate was installed on all new desktop and laptops shipped from August 2015 to the present day.” I bought a Dell desktop in March of this year and mine has the eDellRoot certificate. Obviously, more are affected than they’re admitting to.

  8. Jay Libove

    Brian, I wonder if two something elses I’ve seen relating to Dell support represent different, related security risks.

    My Dell Inspiron does NOT run Dell’s own image, and I do NOT have the Dell Foundation software installed. However, I do have some of the standard Dell support/maintenance software added (as downloaded from the Dell support website for my model, Inspiron 15 3541).

    1:
    In the Windows Trusted Root CAs Store, I found a self-signed certificate, with private key, marked for all purposes, with an expiration date 1000 years into the future (yes, really, in the year 3000 something, RSA 4096) in the exact name of my computer’s internal Domain FQDN (i.e. MyNotebooksName.MySub.MyDomain.ORG).
    I think it was created on the same date when I allowed some company’s support to perform a remote session to look at a problem (I don’t recall if it was Dell, Microsoft or another; a legitimate company, in any case). [I’ve removed this cert, so I don’t still have a copy; my mistake, I should have exported it for further analysis].

    2:
    I notice, in the Windows Certificate Store, that a custom Store called “PC-Doctor, Inc.” is created, containing a pair of certs issued to “Dell Inc” and another pair to “PC-Doctor, Inc.”. We know that Dell repackages PC-Doctor for remote support. All four of these certs are marked for Code Signing. Three of the four are already expired; a fourth (one of the two “PC-Doctor, Inc.” certs) is still valid (until Aug 19, 2016, serial number ‎19 46 ba 8c 8a e2 38 e1 78 3d 25 2c ba 48 b4 6e, thumbprint ‎1e ac c5 ad 8d 01 4a c1 cf 0e 9e 40 9d 7f 69 dc 10 71 ca a3).
    I’m not familiar enough with how Windows treats certificates stored in these various Stores – that is, whether a Code Signing cert in the custom PC-Doctor, Inc. Store would be trusted by Windows to verify a signature on any code, or only on code somehow cryptographically linked to this customer PC-Doctor, Inc. Store.
    I note that none of these certs have private keys, so the risk is somewhat limited (possibly execution of untrustworthy code, in case someone manages to get a copy of the matching signing cert private key). [These I’ve not (yet) removed].

    Any thoughts on these two further somewhat-weird certificates present? (Note that I’m not in the habit of installing questionable software, but I can’t claim to be perfect nor clairvoyant!)

    thanks,
    Jay Libove, CISSP, CIPP/US, CIPT, CISM

    1. timeless

      On 1, I really wish you had kept it and marked it untrustworthy.

      It certainly /sounds/ like the kind of thing that a centralized IT group would do. But without knowing what restrictions the certificate has (is it encryption only? can it do Wi-Fi? can it issue code-signing?), or other attributes, I really don’t know how to search for it.

      If you have peers at work, try to see if they have a similar one (I bet they do), you should be able to get enough information their in order to help someone here track it down…

      On 2, shouldIremoveIt [1] is thumbs down on the PC Doctor software.

      [1] http://www.shouldiremoveit.com/Dell-Support-Center-8089-program.aspx
      * I’ve never seen their site, but its UX reminds me of a service I used years ago, I suspect it’s the same group…

  9. William Warren

    This is even more suspect as Dell owns Sonicwall and now VMWare. This was no accident. Dell knows exactly what they were doing. this should rightfully undermine the trust in their ENTIRE umbrella of product offerings.

  10. Mahhn

    “It’s unclear why nobody at Dell saw this as a potential problem,”
    I’m not surprised. I worked at Dell as a Sr Desktop Support for a few years. They short staff internal IT departments so that there is low motivation for quality work beyond what is required to keep your job. Getting internal recourses (besides the great parts shipping dept) was always a pain in the arse. It took months to get some departments to do 5 min of work for critical systems – when those systems weren’t needed by the dept that you needed support from. I would not recommend purchasing Dell products/services to anyone. At my current job we replaced dells with HPs as other techs had problems with Dell over the years also. I would love to site specifics, but it was years ago now. On the plus side -Most of the people (like all people) were great, unfortunately policy was/is not for people, but for profit. Like the shortcut that by passes security for easy support options reported on here.
    I did not sign any non disparagement agreement when I left.

  11. IA Eng

    During my 6 o’clock coffee, I romped around a few websites. One of them mentioned – somewhere – maybe PC World that the Cert issue only applied to XPS 15 laptops. Time will tell if others are discovered.

    One interesting note about removal of this Cert is what Salted Hash found out in an interview. If it is covered here, cool. I’m in one of those not much into reading modes today…..

    “Many people have indicated that removing the eDellRoot certificates from the root and personal certificate stores is sufficient to protect users,” the Duo Labs report explains.

    “This is not entirely accurate; you must remove the eDell plugin entirely or the certificate will be reinstalled whenever it is loaded. This can be accomplished by deleting the ‘Dell.Foundation.Agent.Plugins.eDell.dll’ module from the system. Failure to do so may result in continued exposure to this security flaw.”

  12. Mel

    I can sort of see this happening. A few years ago I did an IoT thing where the product featured the ability to call home for support. Thus it needed a private key to authenticate itself in the VPN. The product was fairly expensive and shipped in low volume, so it was easy to justify five bucks to generate and install a unique key for each unit. Shipping tens of millions, somebody would be sure to notice those five buck expenses adding up.

    Definitely though, Dell hit on a stupid policy. Installing a private key that everybody knows is a waste of money, no matter how cheap.

  13. Kopecky

    Thank you Brian. You are my home page for over 2 yrs. My new Dell model#3847 (1 week new) had cert. your links showed this & auto removed this huge flaw. You da man!

  14. Robert

    I called Dell Support last night about the eDellRoot problem, and they knew nothing about it and denied that there was a problem. They offered to check my computer for computer problems, and I said it isn’t an operational problem, it’s a security problem. Finally, I gave up. Unless a Dell person found a problem, it doesn’t exist.

  15. welcome lenovo

    welcome lenovo pc to united states, so bye bye michael dell and theirs deffective windows 10 pc and laptops brand

  16. Craig Young

    I have setup a site using a certificate signed with edellroot in case anyone wants to test their system: https://edellroot.secur3.us

    That should prompt with a browser warning (interstitial) unless the system is vulnerable. Keep in mind that Firefox does not use the Windows certificate store so use IE or Chrome to test.

  17. EmJay

    While this was only shipped since August, it affects ANYONE who has updated the Dell Foundation Services recently. I have a machine from 2013 and another from early 2015. Both have this certificate installed.

  18. SSpaz

    bought my dell direct from Microsoft no bloatware at all installed and upgraded from win 10 home to win 10 pro and no dell foundation services found on my computer

    1. timeless

      Not unless you install Wine and get the affected Dell Foundation Software to install and run there…

      And then you’d only be affected if you tried to run Chrome in Wine or something signed w/ the certificate.

      (The browser used by Wine is actually Gecko, which isn’t impacted.)

      Also, the odds of Dell Foundation Software working in Wine are pretty much nil.

  19. Joseph

    people should start using Linux more and more, and get more cultivated about computers and smartphones security.

  20. nicole a

    Please forgive my computer illiteracy as I continue with my comment. I have an Alienware laptop but it is from Dell. I had Dell Update, Dell System Detect, and another that allows techs access to my computer [which I had recently used for video card problems] installed. I have deleted all these though I am not sure if they contained any spyware. I use Reason Core to check for malware and spywareand nothing has yet
    come up. Even though my computer was shipped before August, could it be possible that more computers were infected? It feels to me that Dell just got caught is all. I think it may have been a deliberate install but, I am a cynic when it comes to computer security.
    The end.

Comments are closed.