September 25, 2014

As if consumers weren’t already suffering from breach fatigue: Experts warn that attackers are exploiting a critical, newly-disclosed security vulnerability present in countless networks and Web sites that rely on Unix and Linux operating systems. Experts say the flaw, dubbed “Shellshock,” is so intertwined with the modern Internet that it could prove challenging to fix, and in the short run is likely to put millions of networks and countless consumer records at risk of compromise.

The bug is being compared to the recent Heartbleed vulnerability because of its ubiquity and sheer potential for causing havoc on Internet-connected systems — particularly Web sites. Worse yet, experts say the official patch for the security hole is incomplete and could still let attackers seize control over vulnerable systems.

The problem resides with a weakness in the GNU Bourne Again Shell (Bash), the text-based, command-line utility on multiple Linux and Unix operating systems. Researchers discovered that if Bash is set up to be the default command line utility on these systems, it opens those systems up to specially crafted remote attacks via a range of network tools that rely on it to execute scripts, from telnet and secure shell (SSH) sessions to Web requests.

According to several security firms, attackers are already probing systems for the weakness, and that at least two computer worms are actively exploiting the flaw to install malware. Jaime Blasco, labs director at AlienVault, has been running a honeypot on the vulnerability since yesterday to emulate a vulnerable system.

“With the honeypot, we found several machines trying to exploit the Bash vulnerability,” Blasco said. “The majority of them are only probing to check if systems are vulnerable. On the other hand, we found two worms that are actively exploiting the vulnerability and installing a piece of malware on the system. This malware turns the systems into bots that connect to a C&C server where the attackers can send commands, and we have seen the main purpose of the bots is to perform distributed denial of service attacks.”

The vulnerability does not impact Microsoft Windows users, but there are patches available for Linux and Unix systems. In addition, Mac users are likely vulnerable, although there is no official patch for this flaw from Apple yet. I’ll update this post if we see any patches from Apple.

Update, Sept. 29 9:06 p.m. ET: Apple has released an update for this bug, available for OS X Mavericks, Mountain Lion, and Lion.

The U.S.-CERT’s advisory includes a simple command line script that Mac users can run to test for the vulnerability. To check your system from a command line, type or cut and paste this text:

env x='() { :;}; echo vulnerable' bash -c "echo this is a test"

If the system is vulnerable, the output will be:

vulnerable
 this is a test

An unaffected (or patched) system will output:

 bash: warning: x: ignoring function definition attempt
 bash: error importing function definition for `x'
 this is a test

US-CERT has a list of operating systems that are vulnerable. Red Hat and several other Linux distributions have released fixes for the bug, but according to US-CERT the patch has an issue that prevents it from fully addressing the problem.

The Shellshock bug is being compared to Heartbleed because it affects so many systems; determining which are vulnerable and developing and deploying fixes to them is likely to take time. However, unlike Heartbleed, which only allows attackers to read sensitive information from vulnerable Web servers, Shellshock potentially lets attackers take control over exposed systems.

“This is going to be one that’s with us for a long time, because it’s going to be in a lot of embedded systems that won’t get updated for a long time,” said Nicholas Weaver, a researcher at the International Computer Science Institute (ICSI) and at the University of California, Berkeley. “The target computer has to be accessible, but there are a lot of ways that this turns accessibility into full local code execution. For example, one could easily write a scanner that would basically scan every Web site on the planet for vulnerable (Web) pages.”

Stay tuned. This one could get interesting very soon.


138 thoughts on “‘Shellshock’ Bug Spells Trouble for Web Security

      1. John Germain

        P – Really, what is the point of that comment. It was a simple question.

        1. Bruce Hobbs

          Before calling customer support, reboot your system. Before asking a question, try a Google search first. Simple stuff.

      2. q

        You probably for got you’re not on 4chan anymore. Seriously, this is all you were able to come up with? This is the wrong place for people like you.

    1. Bob Brown

      I am no expert, and this vulnerability is still being analyzed, but it appears that simply commenting out mod_cgi in /etc/httpd/conf/httpd.conf gets rid of Linux web vulnerabilities.

      Of course, it also breaks anything that uses CGI in your web site, but that’s increasingly unlikely in 2014.

      PHP, if run through mod_php appears not to be vulnerable.

      The command line vulnerability remains, but of course, if I can issue commands at the command line, I don’t need a vulnerability. (Be sure to check sudo.)

      Please understand, this is what I understand, to the best of my knowledge. This stuff is still being figured out, and I could turn out to be very wrong.

      1. bob

        Might want to look into DHCP and SSH based exploits. Both are often publicly accessible and both will pass stuff to bash (if it’s the default shell) in such a way as to exploit this ‘sploit.

    2. icknay

      Here’s the one paragraph answer: bash does not respond to network probes itself, so by itself it is harmless. However, bash is very often used to glue several programs together. So if the web server calls over to a cgi program, bash is used to glue the connection together. So now the bad guys can hunt around for combinations of networked software that happen to use bash glue at some stage. The web-server/cgi combination is a common one, and there must be many more.

    3. skip

      Yes, CGI, however any application which hands off processing to bash could be suspect. One way, with CGI, is to pass the bash command exploits in the browser header variables. If there’s a chance the CGI application is running these variables through a bash interpreter, your smoked.

      According to a security geek at Google, the latest patch is easily defeated with an alternative:
      https://twitter.com/taviso/status/514887394294652929

  1. Tommy

    So many outward facing computers that may control electric/oil/gas/water/nuclear/telephone facilities etc could easily be hackable by bad actors!… oh dear

  2. Rick

    “In addition, Mac users are likely vulnerable”

    Even Yosemite Beta 3 is vulnerable.

    1. Aaron

      Looking at some of comments below, it seems mac are most vulnerable?

      1. timeless

        No. Macs just have more users likely to say “me too”.

        And, offhand, I’d bet there are more Apple readers of this blog than Linux readers (sorry Bryan, although if you’re willing to share analytics, I’d love to read them).

        * Ubuntu users aren’t vulnerable (since Ubuntu switched to dash ages ago).
        * Most Linux users (who haven’t updated; or, as w/ Heartbleed who updated before a complete fix…) are probably vulnerable (and if you have a Linux laptop, you’re probably at relatively high-risk).

        Note: while Apple hasn’t posted a public update, there isn’t an obvious exploit path for Mac OS X users. Since the native DHCP client doesn’t appear to trigger the system shell and most OS X systems don’t have many services running. Deciding whether the out-of-the-box apache modules on OS X might be vulnerable would take me too much effort. Users should disable their webserver if they don’t want to patch Bash. In theory one could have a /cgi-bin which parsed its own input into env vars and then spawned a child. That could be vulnerable.

  3. Sfer

    So,

    Q1:
    what are the commands
    to test for vulnerability
    in a fully-patched Ubuntu Linux? (v. 12.04-14.04+)

    The recipe in the article above,
    is for Mac pcs…

    Q2:
    What are other
    effective pre-emptive steps
    Ubuntu home users can take,
    **after** applying the “official” Ubuntu BASH patch?
    (it was made available already, yesterday…).

      1. SFer

        Thanks!

        Just used in Ubuntu 12.04 LTS-32bit,
        the test line in Kreb’s article, (above).

        Result:
        Not affected.

        Also did:
        bash –version
        from Terminal (Ctrl-Alt-T).

        Result: (after patching/updates from Ubuntu)
        my Bash is version 4.2.25(1).

        *** Apparently ***
        not updated Bash versions 3.2.25 or older
        could be vulnerable for sure.

        So _best advice_
        for the time being,
        seems to update NOW
        from your Linux distro site!

        Ubuntu has updated BASH twice
        in the last 2 hours…a first in my experience!

        Hope this helps..

  4. Henrik

    MS Windows is actually potentially vulnerable, just not by default. All versions of bash can be hit by this, which includes cygwin bash and other versions integrated with software like git for windows.

    1. Steve

      Glad you mentioned it. It’s doubtful many Cygwin installations are vulnerable, because few of us have web servers that can access Cygwin’s bash. But still… Cygwin has already issued a patch, so update your installations pronto.

  5. T

    This vulnerability is pretty ubiquitous in connected appliances/devices since almost all of them use Linux. The challenge is to find the network service that connects to the vulnerable command shell in order to conduct an exploit. So far CGI, DHCP, and a number of other services offer this linkage. As days, weeks, months, and years progress we will continue to find new areas where this vulnerability can be exploited. At least Heartbleed was easy to find. Good luck.

    1. LonelyBit

      Not every linux appliance uses bash. Most Lnux appliances with limited storage would prefer busybox because it is smaller. BusyBox is not affected by this vulnerability.

  6. jim miller

    Since most consumer routers run a version of Linux I wonder if these are vulnerable as well?

    jim

    1. Henrik

      @Jim: No, most consumer routers run busybox to conserve memory.

  7. Kurt

    on MAC running 10.9.5 running from terminal prompt

    env x='() { :;}; echo vulnerable’ bash -c “echo this is a test”
    vulnerable
    this is a test

  8. TheOreganoRouter.onion.it

    This article is not in my wheel-house, even though I’ve played around with Fedora Red-Hat years ago and basically understand the concept of the security vulnerability

  9. Vito

    On a Mac Pro running OS X 10.8.5:

    env x='() { :;}; echo vulnerable’ bash -c “echo this is a test”
    vulnerable
    this is a test

    Poop.

  10. Denver Cyber

    Question – why was this bug released so quickly before giving companies a chance to patch or perform other remediation? I haven’t seen much discussion on that point anywhere.

    1. Coolac

      I think the redhat guy caught a teenager exploiting it already. I could be wrong.

  11. B_Brodie

    So, what did teh NSA know, and when did they know it (and how did they use it)?

    After all, they have massive systems to scan for this kind of vulnerability. They had to be aware. Who did they warn? US DOD? Seems not – DOD systems are being attacked as we read this story.

    So, we can conclude that the NSA had no problem exploiting such a glaring flaw while leaving vast swaths of the world’s IT infrastructure vulnerable to intrusion and damage.

    Not very patriotic. Wonder what the NSA will have to say before Congress, etc, about this.

    Either they knew and failed to warn us, or they didn’t know and all those billions we’re giving the intelligence community are as useful as screen doors on a submarine…

  12. Steve Maher

    Workaround for anyone who cannot patch.
    Test it first – don’t come complaining it breaks on your X****. Why this works? Read the source code like I did.

    Add a bashshell account with no privs.

    chown bashshell /bin/bash
    chmod 4755 /bin/bash

    A couple of downsides to this, 1 being that a user will have to source their our profile ie . /etc/bash.bashrc and the other being that some rather complicated frameworks in bash may not function as desired.

  13. Coolac

    Nice, I’ve been waiting for this article.

    I think, ABCD='() { echo this is abcd; }’ bash -c ‘ABCD’ = Im’ still vulnerable i believe. I could be wrong.

    3 bash updates, and still incomplete?

  14. Coolac

    Hey look windows is safer then linux haha. i don’t run a webserver, but my god, my router, my wireless printer. When scanned they don’t seem to have telnets open, but this is pretty nuts, i’m not sure if they are still vulnerable or not?

    Cups, dhclient, i think are vulnerable.

  15. john

    The bug is not that dangerous. You either have to have direct access to the box to run anything in the shell or run some vulnerable configurations of Apache. CGI scripts went out of fashion 10+ years ago, smarter people have Nginx but not Apache on their servers and FastCGI is not affected. Far from being as dangerous as Heartbleed, which could be easily remotely exploited.

    1. bob

      Large amounts of software still have CGI chunks in, eg cPanel. Any machine that exposes SSH or DHCP is vulnerable. The real risk is embedded hardware – tends to be very old and never updates. Your servers are possibly at risk, your consumer routers almost certainly are. Attacks built around shellshock have already been seen for servers, I haven’t seen any for routers, cars, fridges, etc yet but I’m sure there’ll be some. Don’t be complacent.

    2. Henrik

      @john: From what I’ve read, php, python, java, and C++ through websites are also vulnerable because popen() and system() are mapped to /bin/sh, which is actually bash on most linux boxes. Beyond websites, it also affects a lot of other services like loadbalancers and DHCP clients, the latter of which has a public proof of concept out already. It’s a mistake to dismiss this because you don’t run archaic CGI-bin bash scripts on your webserver.

      1. john

        Python, Java etc could be affected only if they run shell via popen AND only if they are behind web server which injects HTTP headers into environment. Nginx does not inject, Apache sometimes does. Many sane people have been running such services behind nginx for quite a few time.
        About dhcp clients – first and foremost such client should point to the malicious dhcp server. Atacker should compromise dhcp server to be able to attack clients.

        Sure, the attack can be dangerous, but the target will be mostly cheaper web hostings and vps’s with panels. Heartbleed made vulnerable banks, large shops and web-scale services.

        I’m a webmaster myself and have a few VPS’s and a dedi server, Neither of them are vulnerable to this attack as they don’t run apache and it is impossible to run anything in the shell without having ssh private key. With Heartbleed all the services with ssl were vulnerable and I had to upgrade them hastily. And I’m not alone with this.

      2. Coolac

        /bin/sh is DASH on debian, not bash.

        For a login shell, if extra paranoid you can switch zsh.

  16. Rolling

    http://www.smh.com.au/it-pro/security-it/stephane-chazelas-the-man-who-found-the-webs-most-dangerous-internet-security-bug-20140926-10mixr.html

    He said he found the bug after reflecting on an earlier bug he found in Bash a few months ago.

    “After some thorough investigation, I reported it with as much information as possible to a few select Linux distribution security lists and Chet Ramey, the Bash maintainer, on September 9,” he said.

    “They’ve all worked hard to make sure a patch was ready on as many systems as possible [and select infrastructure providers were notified] by the time the vulnerability was disclosed on the agreed date yesterday [about 2am Thursday AEST]. That was very professionally handled. I believe the impact was about as minimised as could be, and I’m proud to have contributed to that.”

    1. Coolac

      I’m a little confused because the article says he found it the morning of September 12th, but reported it on september 9th? umm.

      And ya i ‘m sure this has been exploited for years. And what weird timing this is…

  17. Robert B.

    A simple WAF will stop the web attacks, you can even use a free one like mod_security. Its really not hard to protect against. Yes its a big deal if you don’t use protection but overall if people don’t protect their selves then what can you do.

    1. Mike

      WAF’s are not simple. They are not easy to implement, especially if you have a custom web application that’s customer facing. It takes days and sometimes weeks to weed out all the false positives. That’s too long in a reactive sense.

      Also, many embedded systems are running vulnerable bash. This may very well include firewalls, switches, routers, etc. Can’t exactly put a WAF in front of those…..

      1. Robert B.

        Actually they are easy to implement, I have been doing it full time for over 6 years now, this is my expertise. And you can simply put this single signature in blocking mode with the rest disabled or log only. If you have your routers or switches directly exposed to the internet then you have more serious issues. And yes you can put a WAF in front of those devices for their management ports.

        1. Robert B.

          And let me clarify before you respond, you should not directly expose your management ports directly to the entire internet. At minimum lock management ports to a subnet or single IP.

        2. Mike

          Right, and if someone is inside your network, then a WAF at the edge is useless.

          This is a very serious flaw. Simply saying that you should and should’ve always had a WAF is not an answer and does not negate the severity of this flaw.

          1. Robert B.

            Inside your network is a complete and total different story and situation. You can’t compare the two situations. Inside the network again when it comes to routers, switches and firewalls you should be using out of band or a segmented VLAN for access of administrative ports. Now if you can’t trust your admins then you have other issues. WAFs can be implemented on the inside as well, they are not constrained to the outside. This vulnerability is bad but again you can protect yourself with proper setup and configuration.

            1. Mike

              Yes, you can absolutely compare the situation. How many intrusions have we heard about in recent times where bad actors were inside large corporate networks for MONTHS? What could they do then?

              What about employees with malicious intent? Employees who know they’re going to get fired for some reason, who have the technical know-how to exploit this?

              I’m sorry, but dismissing this and saying people should “protect” themselves is the same sort of complacent thinking that got us and keeps us in this sad state of bad security.

              1. Robert B.

                So you are saying don’t protect yourself and just throw your hands in the air and complain? You have to protect yourself, inside or outside. You can just sit there and complain because you didn’t setup admin port ACLs on your devices or implement mitigation tactics like WAFs. There will always be vulnerabilities and always be attackers, to complain about them and not put owness on yourself for not protecting your network is a dereliction of duties if you admin a network. MitM attacks are also very dangerous but how many people enable arp spoofing protection on their switches? If someone is already in your network and wants to attack your systems there are a myriad of ways to do it besides just a single vulnerability. You have to have defense in depth, not just complain that there are dangerous vulnerabilities in software. And by the way, those bad actors came in through other means and gaps in security, they didn’t just walk in and sit down at a system and start shellshocking away.

                1. Mike

                  Yes, *I* protect myself and my network. But I am thinking outside of my own world and I know that most companies and people don’t take the same precautions that you mention. I know that many companies don’t have the resources to implement anything even close to what you mention.

                  If anything, I’m saying we as a community need to take more responsibility, be more vigilant and less complacent. Just because *I’m* “fine”, doesn’t mean everyone else is.

                  We can’t just take the stance of blaming people for not taking adequate measures to protect their network and call it their fault when they get exploited. Before Heartbleed, we ALL thought we were secure using OpenSSL. Look at how that turned out. We could have spent thousands, tens or hundreds of thousands of dollars protecting our networks, only to be undermined by a flaw that wouldn’t have even logged the exploit anywhere.

                  1. Robert B.

                    I can’t disagree with that! It takes everyone to make a difference.

            2. timeless

              http://allthingsd.com/20131010/google-cio-ben-fried-on-how-google-works/

              Google’s most recent big internal technology operations project is something it calls Beyond Corp. It’s a way to make the company more secure by treating internal networks as if they were just as risky as the big, bad Internet. Analysts have previously labeled this technique a “zero trust network,” where all traffic is scrutinized.

              Fried described Beyond Corp as an alternative to the way many large companies give their employees digital tokens and use encryption to allow them to securely tunnel into “virtual private networks.” ”It’s what comes after VPN,” he said.

              “The fallacy is believing that you should actually trust any network,” Fried explained. “This idea that one network is more secure than the other was completely false, because most of the computers that were on this network were actually connecting to the Internet at some other point in the day.”

  18. NotMe

    As usual this site has the best information, thanks Brian and thanks to all your readers for the extra info and insights.

  19. Mike

    When Hearbleed happened, my first thought was “What’s next?”. Well, here’s my answer.

    Has the open source community failed? Have we failed as sysadmins, technicians, developers and testers? How are these extremely critical, yet seemingly easy to exploit vulnerabilities going unnoticed for YEARS? Heartbleed gave up information by just sending simple null commands to openssl. Shellshock can get bash to run any command simply by entering the commands after an environment variable string. How has the community not tested or picked up on this for DECADES in this case? We are all partly to blame. We are all responsible for not noticing these things.

    The internet security world needs a call to action. We as sysadmins, testers, coders, web developers, managers, we all need to spend time out of our day and solely focus on security. It’s a new world. We need to find these holes before they go unnoticed for decades. We need to contribute back to the open source community, and not just assume that someone else is looking for these vulnerabilities because they’re obviously not.

    What’s next?

    1. Coolac

      Its like you said Mike…. complacency, and the tendency to arrogantly blame incompetent users. The mentality needs to change.

  20. JimH

    Time to get some info from ‘the group’.. does this/will this/can this also affect mobile devices (read: phones/tablets)?

    Per at least one of the statements on the National Vulnerability Database page (http://nvd.nist.gov/cwe.cfm#cwes), it appears to me as at least a possibility.. possibly not discovered at this point.

    Thanks for the info..

    1. timeless

      Some? probably.

      Most? probably not.

      The one exception I can find:
      * Tizen seems to use bash as its default shell (If you have a Tizen device, I’m sorry. You should have stuck w/ the n900/n9).
      * Mer (SDK) uses bash as its default shell. But this is just the SDK. I don’t know about the related platform (Sailfish OS).

      Undecided:
      * Sailfish OS by Jolla (I can’t find a VM).

      The probably-not-vulnerable-to-this-disaster-out-of-the-box:
      * Android doesn’t include bash.
      * BlackBerry 10 uses “ksh” (it doesn’t have Bash).
      * Firefox OS uses mksh (a ksh relative).
      * iOS apparently doesn’t include Bash (I’m not sure what it actually uses — if anything).
      * Maemo (Nokia’s n900 and related products) used BusyBox.
      * MeeGo (Nokia’s n9) used BusyBox.
      * Symbian didn’t have a shell (you could add zsh if you wanted to).
      * Ubuntu’s Touch should use dash (since Ubuntu uses dash).
      * WebOS uses Bourne shell (not Bash).
      * Windows Phone wouldn’t use bash (it presumably uses cmd32).

      Note:
      Jailbroken Android and iOS phones are apparently vulnerable since most jailbreaks include bash to help their jailbreakers do stuff w/ their newly jail-broken devices.

      All things considers, phones seem to be about the only appliance that may be safe from this. Other embedded devices:
      * Routers
      * NAS
      * Firewalls
      * VPNs
      * Load-Balancers

      seem to be better candidates for being vulnerable.

      1. timeless

        *Oops*, move MeeGo (Nokia’s n9) to the Vulnerable column.

        It uses bash (I never got an n9 because I left before the product shipped).

      2. timeless

        (I can’t tell if this posted)

        Move to the Vulnerable set:
        * MeeGo (Nokia’s n9) used bash.

        And change my note about Tizen to indicate that people should have stuck with Maemo / n900 instead of upgrading to the n9 :).

        1. timeless

          Sigh, how embarrassing. I was right the first time.

          it uses “busybox ash” just as I thought — unfortunately I misread someone’s chat response of “bb ash” as “bash”.

  21. John

    Question for Brian:

    How do you decide when/whether to break a story like this? I just read this morning that the news of it is expected to create a flood of attacks as those criminal hackers who might have been unaware rush to exploit it. This isn’t intended as criticism; I appreciate your reporting. Without your reporting in this area, I’d be unaware of the risks, or unaware until it was way too late to do anything about them. But it must be tough in some cases. Or maybe not. I’m not a student of journalism ethics.

  22. Alvin

    Couldn’t this still affect windows machines that use unix-like environments like cygwin?

    1. timeless

      Sure. But if you replace a LAMP stack with a WAMP system, you should expect to suffer all the penalties of maintaining a LAMP stack.

      Here’s the cygwin update for bash:
      https://cygwin.com/ml/cygwin-announce/2014-09/msg00033.html

      Also vulnerable: MSYS, and Git for Windows (which includes Bash).

      But if you’re running a git server using Git for Windows, you won’t get any sympathy for me. (Run Mercurial instead, or use code.google.com or github or bitbucket.org)

Comments are closed.