February 21, 2017

Amid the hustle and bustle of the RSA Security Conference in San Francisco last week, researchers at RSA released a startling report that received very little press coverage relative to its overall importance. The report detailed a malware campaign that piggybacked on a popular piece of software used by system administrators at some of the nation’s largest companies. Incredibly, the report did not name the affected software, and the vendor in question has apparently chosen to bury its breach disclosure. This post is an attempt to remedy that.

The RSA report detailed the threat from a malware operation the company dubbed “Kingslayer.” According to RSA, the attackers compromised the Web site of a company that sells software to help Windows system administrators better parse and understand Windows event logs. RSA said the site hosting the event log management software was only compromised for two weeks — from April 9, 2015 to April 25, 2015 — but that the intrusion was likely far more severe than the short duration of the intrusion suggests.

That’s because in addition to compromising the download page for this software package, the attackers also hacked the company’s software update server, meaning any company that already had the software installed prior to the site compromise would likely have automatically downloaded the compromised version when the software regularly checked for available updates (as it was designed to do).

Image: RSA

Image: RSA

RSA said that in April 2016 it “sinkholed” or took control over the Web site that the malware used as a control server — oraclesoft[dot]net — and from there they were able to see indicators of which organizations might still be running the backdoored software. According to RSA, the victims included five major defense contractors; four major telecommunications providers; 10+ western military organizations; more than two dozen Fortune 500 companies; 24 banks and financial institutions; and at least 45 higher educational institutions.

RSA declined to name the software vendor whose site was compromised, but said the company issued a security notification on its Web site on June 30, 2016 and updated the notice on July 17, 2016 at RSA’s request following findings from further investigation into a defense contractor’s network. RSA also noted that the victim software firm had a domain name ending in “.net,” and that the product in question was installed as a Windows installer package file (.msi).

Using that information, it wasn’t super difficult to find the product in question. An Internet search for the terms “event log security notification april 2015” turns up a breach notification from June 30, 2016 about a software package called EVlog, produced by an Altair Technologies Ltd. in Mississauga, Ontario. The timeline mentioned in the breach notification exactly matches the timeline laid out in the RSA report.

As far as breach disclosures go, this one is about the lamest I’ve ever seen given the sheer number of companies that Altair Technologies lists on its site as subscribers to eventid.net, an online service tied to EVlog. I could not locate a single link to this advisory anywhere on the company’s site, nor could I find evidence that Altair Technologies had made any effort via social media or elsewhere to call attention to the security advisory; it is simply buried in the site. A screenshot of the original, much shorter, version of that notice is here.

Just some of the customers of Eventid.

Just some of the customers of Eventid.

Perhaps the company emailed its subscribers about the breach, but that seems doubtful. The owner of Altair Technologies, a programmer named Adrian Grigorof, did not respond to multiple requests for comment.

“This attack is unique in that it appears to have specifically targeted Windows system administrators of large and, perhaps, sensitive organizations,” RSA said in its report. “These organizations appeared on a list of customers still displayed on the formerly subverted software vendor’s Web site. This is likely not coincidence, but unfortunately, nearly two years after the Kingslayer campaign was initiated, we still do not know how many of the customers listed on the website may have been breached, or possibly are still compromised by the Kingslayer perpetrators.”

It’s perhaps worth noting that this isn’t the only software package sold by Altair Technologies. An analysis of Eventid.net shows that the site is hosted on a server along with three other domains, eventreader.com, firegen.com and grigorof.com (the latter being a vanity domain of the software developer). The other two domains — eventreader.com and firegen.com — correspond to different software products sold by Altair.

The fact that those software titles appear to have been sold and downloadable from the same server as eventid.net (going back as far as 2010) suggests that those products may have been similarly compromised. However, I could find no breach notification mentioning those products. Here is a list of companies that Altair says are customers of Firegen; they include 3M, DirecTV, Dole Food Company, EDS, FedEx, Ingram Micro, Northrop Grumman, Symantec and the U.S. Marshals Service.

RSA calls these types of intrusions “supply chain attacks,” in that they provide one compromise vector to multiple targets. It’s not difficult to see from the customer lists of the software titles mentioned above why an attacker might salivate over the idea of hacking an entire suite of software designed for corporate system administrators.

“Supply chain exploitation attacks, by their very nature, are stealthy and have the potential to provide the attacker access to their targets for a much longer period than malware delivered by other common means, by evading traditional network analysis and detection tools,” wrote RSA’s Kent Backman and Kevin Stear. “Software supply chain attacks offer considerable ‘bang for the buck’ against otherwise hardened targets. In the case of Kingslayer, this especially rings true because the specific system-administrator-related systems most likely to be infected offer the ideal beachhead and operational staging environment for system exploitation of a large enterprise.”

A copy of the RSA report is available here (PDF).

Update, 3:35 p.m. ET: I first contacted Altair Technologies’ Grigorof on Feb. 9. I heard back from him today, post-publication. Here is his statement:

“Rest assured that the EvLog incident has been reviewed by a high-level security research company and the relevant information circulated to the interested parties, including antivirus companies. We are under an NDA regarding their internal research though the attack has already been categorized as a supply chain attack.”

“The notification that you’ve seen was based on their recommendations and they had our full cooperation on tracking down the perpetrators. It’s obviously not as spectacular as a high visibility, major company breach and surely there wasn’t anything in the news – we are not that famous.”

“I’m sure a DDoS against our site would remain unnoticed while the attack against your blog site made headlines all over the world. We also don’t expect that a large organization would use EvLog to monitor their servers – it is a very simple tool. We identified the problem within a couple of weeks (vs months or years that it takes for a typical breach) and imposed several layers of extra security in order prevent this type of problem.”

“To answer your direct question about notifications, we don’t keep track on who downloads and tries this software, therefore there is no master list of users to notify. Any anonymous user can download it and install it. I’m not sure what you mean by ‘you still haven’t disclosed this breach’ – it is obviously disclosed and the notification is on our website. The notification is quite explicit in my opinion – the user is warned that even if EvLog is removed, there may still be other malware that used EvLog as a bridgehead.”

My take on this statement? I find it to be wholly incongruent. Altair Technologies obviously went to great lengths to publish who its major customers were on the same sites it was using to the sell the software in question. Now the owner says he has no idea who uses the software. That he would say it was never intended to be used in major organizations seems odd in contrast.

Finally, publishing a statement somewhere in the thick of your site and not calling attention to it on any other portion of your site isn’t exactly a disclosure. If nobody knows that there’s a breach notice there to find, how should they be expected to find it? Answer: They’re not, because it wasn’t intended for that purpose. This statement hasn’t convinced me to the contrary.

Update, 11:13 p.m. ET: Altair Technologies now has a link to the breach notification on the homepage for Evlog: http://www.eventid.net/evlog/

88 thoughts on “How to Bury a Major Breach Notification

  1. I.T. Guy

    What kind of lazy admins aren’t writing their own PowerShell scripts to parse the event log?

    1. Code Monkey

      Is reinventing the wheel not looked down on in the sysadmin world?

    2. Al

      Obviously the kind who doesn’t have as much free time as you do.

      1. Someone

        Your response leads me to believe you don’t know PowerShell. I think it’s quite the opposite of what you believe. PowerShell event viewer scripts can consist of as little as a single line and a where statement. It’s not very complicated, and it takes less time to write for a well-versed administrator to write than this comment has taken me.

    3. SeymourB

      Probably the lazy admins who have been using this service since before powershell existed.

    1. Emile

      They appear to control (directly or through partnerships willing to alter responses) enough public name servers (ideally top level resolvers) that requests from some impacted organizations used to resolve the name. I may leave it to someone else to confirm what the DNS networks are within RSA’s reach/partnerships.

  2. J. Tate

    Well done, shedding the light in a way that most will get. Some will stay focused on the insignificant things- but your article was well done. The world has little clue how unfair the playing field is when it comes to breach or data protection laws.

    Idea: Lets talk about how “dead data” is recycled and becuse of its untouched legal identification, Im sure your readers would be in AWE if we were to reveal just what happens to stolen data, how its used, and just how little LEO can do to help.

  3. Jeff C

    Interesting, the link tot eh list now contains a note indicating removal of the list on 2/23/17 with instructions of how to request the list. The attempts to cover up remain lame.

      1. web

        As I mentioned in a previous comment, a number of those “customers” have been defunct for many years. I’m not sure I could fault Altair for (finally) taking it down when attention has been brought to it.

    1. B

      They say you can request the information that was removed, but if I wanted that information I’d just use archive.org…

  4. Bugbear

    Looks like list of customers has been removed due to “abuse”. Google cache is your friend. 🙂

    1. Emile

      Hit this link for details of loss of life as a consequence of cyber intrusions.

  5. NystagmusE

    Thanks for this informative article.
    Some of those affected victims organizations are VERY significant, of course.

    Ouch, ouch, ouch!
    All this during the chaos-worshipping-Trump election year.
    Very, ouch!

    I really wish the average US citizen was more educated about security issues by default. Our educational systems of both children and adults needs to be fortified.

    Snowden said we need more basic defensive security, not more offensiveness and attack stuff. Good articles like this bolster his honorable statements.

    1. Mike

      You talk like this is new. Like the world never had an issue until Trump.

      Have you been listening at all? This has been going on for a very long time. These problems were happening all the Obama admin. It remains one of the primary points of focus for this website.

      It isn’t just about ‘American’ understanding either. Few people across the planet seem interested in security. The entire human race seems to think that all security issues can be corrected, mitigated, and solved through updates, patches, and antivirus/antiwalware software.

  6. Jill S Duxbury

    There is no perfect security and many do not have the availability to address these issues in the real world or real time. It has nothing to do with being educated about security or not caring. The damage is already done. Most of the population of the planet cannot do anything about these incidents, not even the ones causing them. We all get the default method.
    Correct about not having the time and can only address the issues of our everyday lives. Not even unplugging can protect us, or our information and or business.
    I would love to hear some real constructive criticism going on and give credit as the system is still up and working or you would not be able to write or read these articles. The fact that we are still connected and can communicate no matter the annoyances should be noted. Also noting that there has been a lot of quick response by some to alert and correct the damage caused. Correct we still have a long way to go on retrieval.
    Remember that some times keeping things in house while tracking the culprits is necessary, cost effective for both service and customers to minimize the damage.

  7. John Rosengarten

    In today’s world of huge databases, how hard would it be to maintain an integrity check of web sites? One where we can see the history, not just what is happening today.

    As a sysadmin, if I know I patched last month, it would be ever so assuring to verify that last month’s patch download had no compromises inside.

    Someone should be keeping a history of these lapses so we can make use of the data.

    You can’t analyze data unless you collect it in an organized manner in the first place.

Comments are closed.