December 6, 2010

Researchers have discovered that dozens of Web sites are using simple Javascript tricks to snoop into visitors’ Web browsing history. While these tricks are nothing new, they are in the news again, so it’s a good time to remind readers about ways to combat this sneaky behavior.

The news is based on a study released by University of California, San Diego researchers who found that a number of sites were “sniffing” the browsing history of visitors to record where they’d been.

This reconnaissance works because browsers display links to sites you’ve visited differently than ones you haven’t: By default, visited links are purple and unvisited links are blue. History-sniffing code running on a Web page simply checks to see if your browser displays links to specific URLs as purple or blue.

These are not new discoveries, but the fact that sites are using this technique to gather information from visitors seems to have caught many by surprise: A lawyer for two California residents said they filed suit against one of the sites named in the report — YouPorn — alleging that it violated consumer-protection laws by using the method.

As has been broadly reported for months, Web analytics companies are starting to market products that directly take advantage of this hack.  Eric Peterson reported on an Israeli firm named Beencounter that openly sells a tool to Web  site developers to query whether site visitors had previously visited up to 50 specific URLs.

The Center for Democracy & Technology noted in March that another company called Tealium has been marketing a product taking advantage of this exploit for nearly two years.  “Tealium’s “Social Media” service runs daily searches of a customer’s name for news and blog postings mentioning the customers, and then runs a JavaScript application on the customer’s site to determine whether visitors had previously read any of those stories,” CDT wrote. “The service allows Tealium customers a unique insight into what sites visitors had previously read about the company that may have driven them to the company’s Web site.”

If you’d like see this history sniffing technique in action, check out this blog post (from 2008) and click the “Start Analyzing My Browsing History” button about halfway down the page. That site also will try to guess whether you’re a man or a woman by indexing the sites it finds against the Quantcast Top 10,000 sites. It guessed that there was a 99 percent likelihood I was male (phew!), but your mileage may vary.

Fortunately, the browser makers (most of them) have responded. These sniffing attacks — such as the proof-of-concept I linked to above — do not appear to work against the latest versions of Chrome and Safari.  Within Mozilla Firefox, these script attacks can be blocked quite easily using a script-blocking browser plugin, such as the Noscript add-on.

Mozilla addressed this history-sniffing weakness in a bug report that persisted for eight years and was only recently corrected, but the changes won’t be rolled into Firefox until version 4 is released. As a result, current Firefox users still need to rely on script blocking to stop this. Internet Explorer currently does not have a simple way to block scripts from within the browser (yes, users can block Javascript across the board and add sites to a whitelist, but that whitelist lives several clicks inside of the IE options panel).


32 thoughts on “What You Should Know About History Sniffing

  1. Russ

    I’m pretty sure you can achieve the same thing using a Flash cookie, but I guess this helps tag stuff like iphones.

    If I choose “Start Private Browsing” would that keep this from sniffing my history for sites I have browsed outside that private browsing session?

    1. F-3000

      Assuming that “private browsing” works as supposed to, yes, it should work against the sniffing, since there should be no browsing history available.

  2. JBV

    IE has an option to clear browser history on exiting; enabling this will limit available history for snoops, especially if you exit IE frequently.

    Chrome has “Tools” (under the wrench) to clear browsing history. Second line of defense here.

    1. stvs

      Google Panopticlick, whattheinternetknowsaboutyou, and supercookies to see what information you’re leaking and all the various exploits for it. The Wall Street Journal recently ran an excellent series of articles on internet privacy and the business and technology of browser snooping.

      One of the most relevant articles for this discussion is ” Microsoft Quashed Effort to Boost Online Privacy“, which describes an internal dispute at MS between developers who wanted to provide MS customers with privacy tools, and MS ad-men who wanted to monitor and track them. The ad-men won the argument, and customer tracking capabilities are essentially built into IE. Not a great foundation to trust any MS products to protect any of your personal information. But it’s not just MS, all major players are in a race for the bottom of personal privacy to capture the most targeted advertising revenue.

      The counters range from simple NoScript and CSS control via layout.css.visited_links_enabled as mentioned, but more importantly and effectively TACO, BetterPrivacy, and Adobe Flash Settings to control and delete Flash supercookies, and disabling “Third Party Cookies”.

      Even better to use a local Squid->Privoxy or Privoxy->Polipo proxy chain for everything, configured for privacy.

      1. Russ

        SEO already seems to be having fun with some of those. I googled “whattheinternetknowsaboutyou” and the first three non-sponsored results are for wtikay.com. The fifth and sixth results are for whattheinternetknowsaboutyou.com and I have no idea which one to trust.

        I guess now the Internet knows I hate clicking tricky links.

  3. John Stenger

    John STENGER (damn hard to remember how to spell that)likes this!

  4. Anon E. Moose

    Check out the BetterPrivacy Extension for Firefox. It covers the removal of Browser History, cookies, Flash Cookies, LSO, and SuperCookies. It is worth the time to check for the privacy advocate.

    1. AlphaCentauri

      But if I have gone to the trouble of modifying my browser to increase privacy, that tells a lot about me, too. We need an add-on that makes *everything* positive, so it’s no longer a useful technique at all.

      1. F-3000

        “History-sniffing code running on a Web page simply checks to see if your browser displays links to specific URLs as purple or blue.”

        Thus, one way could be that you turn both colours purple – or reverse them. Or change the colours…

    2. Heron

      Thanks for the heads-up about this add-on. I’m going to look into it.

  5. David

    Yikes, not sure how I should take this:

    Likelihood of you being FEMALE is 50%
    Likelihood of you being MALE is 50%

    Site Male-Female Ratio

    1. timeless

      that generally means that the browser you’re using isn’t compatible w/ the information disclosure technique being [ab]used by the site.

    2. Hunter S Tomboy

      I think it means I’d like to get to know you a bit better ;^)

      1. Dr. Odz

        Pretty pathetic if you need this test to figure out your gender: you’re supposed to be checking your browsing history.

    3. Alex Liy

      Got the same result as well. But then again, I *can* attribute that to the fact that I am using the latest version of Chrome when I gave the site a try.

  6. AlphaCentauri

    I came out 68/32. Interesting that most of the sites I’ve visited that had low male:female ratios were sites I had visited in the course of researching spam. But the vast majority of people who research and report spam are male.

  7. Richard

    Likelihood of you being FEMALE is 47%
    Likelihood of you being MALE is 53%

    Site Male-Female Ratio
    washingtonpost.com
    1.15

    —————————————

    You da man, Brian. What web sites were you browsing before you took the test at MikeOnAds and scored 99% probability of being male? Signed, a worried man.

    Washington Post apparently has low machismo rating. But I also had tabs open at El Paso Times, which did not seem to register/ Maybe the web site has no gender rating for El Paso Times? Using Mozilla Firefox 3.6.12, with no-script add-on. No-script did not block the test.

  8. Jim Bengtson

    A weakness of the history sniffing program is that it relies on you using the default colors for visited and unvisited. “By default, visited links are purple and unvisited links are blue. History-sniffing code running on a Web page simply checks to see if your browser displays links to specific URLs as purple or blue.”

    So change the colors that your browser uses to display links.

    In IE, select Tools/Internet Options, General tab, Colors button. In Firefox use Tools/Options, Content tab, Colors button. Change Visited URLs to display in Orange and Unvisited URLs to display in Green, or whichever colors appeal to you.

    1. timeless

      i haven’t looked at how the test actually works, but assuming that it’s able to do color checks, it’s trivial to calibrate by providing a link you can’t possibly visit and a page it forces you to have visited (via a redirect or something).

      please don’t rely on changing colors as protection.

  9. xAdmin

    I use multiple layers for privacy:

    1. MVPS Blocking hosts file (blocks ad sites and known malicious ones at the OS level regardless of application used to access the Internet)
    2. Created a list of over 150 blocked cookies (mainly ad sites) in IE
    3. After closing browser or before and after using any sensitive website, use a batch file that clears browser cache (rundll32.exe InetCpl.cpl,ClearMyTracksByProcess 255) and deletes Flash Player, Windows Media Player, and Microsoft Silverlight cache directories, and general temp folder files.
    4. Every few days (sometimes more often), I log off the limited user account, log in as administrator and run the same batch file to delete the index.dat files that IE uses (need to log off as the files are locked by Windows when that user is logged in). Index.dat files store all the websites you visit and can grow in size to the point they can affect browser performance.

    I really don’t rely on the browser history for anything, but instead bookmark any website I may want to go back to again. I especially bookmark the secure login pages of any sensitive websites so I’m going directly to them before going anywhere else to minimize any privacy issues or cross site scripting that could lead to stealing credentials. I also disable the browsers auto complete function and NEVER have it remember usernames or passwords.

    It sounds paranoid, but it ensures privacy (and security).

    On a side note, I’d use Firefox, but it lacks a command line function to clear the browser cache (to use in a batch file) and its bookmark functionality doesn’t compare to IE’s Favorites Center and how Favorites are stored as single files, whereas Firefox stores them in a single file that can’t be manipulated without using the browser. Not to mention IE’s accelerators.

    1. PaulJ

      @Xadmin – ‘2. Created a list of over 150 blocked cookies (mainly ad sites) in IE’

      Can you describe what you’ve done? Locked cookie files? Manual ‘optout’ flags?
      The Network Advertising Initiative site will create dozens of opt-out cookies for many of the major networks; I’m curious which others you are hitting.

      1. xAdmin

        As I’ve browsed the web, I periodically check what cookies are saved (before clearing them). If in question, I’ll research online what they are for and if needed add them via IE’s, Tools, Internet Options, Privacy tab, Sites, enter site (ex. *.doubleclick.com), click “Block”. All the entries are stored in the registry (HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings\P3P\History) which I export for backup purposes and to import to another system or to restore when needed.

        I prefer this method as it doesn’t require any extra software (ex. Maxa Cookie Manager), but uses the browsers built-in functionality.

        A few more entries (2o7.com, atdmt.com, fastclick.com). I’d post more, but it will probably get flagged by the commenting system and have to be approved before it gets posted.

        1. PaulJ

          @xAdmin, thanks, I see the same capability is in Firefox under Tools -> Privacy -> Accept Cookies from Sites {Exceptions…}. I had never clicked the Exceptions button.

          These settings are stored in the permissions.sqlite database under your Firefox profile. Import/export would appear to rely on some SQL fun.

          I learn something new (and useful) every day!

  10. AndyfromTucson

    I use Firefox and years ago I turned off showing links I have visited in a different color simply because I thought it was ugly to have links display in different colors. I also turned off my history years ago just because I saw no need for it; if I want to remember a site I bookmark it.

    When I tried the site mentioned above it came back 50% male 50% female. Does that mean that it didn’t work? Presumably because of my Firefox settings?

  11. Michael

    There is a hidden configuration property in Firefox 3.5 and newer that will protect against link color sniffing. The trade-off is that visited links won’t change color at all.

    In the configuration editor (about:config) pull up “layout.css.visited_links_enabled” and set it to “false”.

    Some information on the attack and info on that configuration change is at http://dbaron.org/mozilla/visited-privacy

    1. AndyfromTucson

      The demo didn’t work on my Firefox. I think it didn’t work because I have show visited links in a different color turned off.

Comments are closed.