Most anti-virus products designed for use in businesses do a poor job of detecting the exploits that hacked and malicious Web sites use to foist malware, a new report concludes.
Independent testing firm NSS Labs looked at the performance of 10 commercial anti-virus products to see how well they detected 123 client-side exploits, those typically used to attack vulnerabilities in Web browsers including Internet Explorer and Firefox, as well as common desktop applications, such as Adobe Flash, Reader, and Apple QuickTime.
Roughly half of the exploits tested were exact copies of the first exploit code to be made public against the vulnerability. NSS also tested detection for an equal number of exploit variants, those which exploit the same vulnerability but use slightly different entry points in the targeted system’s memory. None of the exploits used evasion techniques commonly employed by real-life exploits to disguise themselves or hide from intrusion detection systems.
Among all ten products, NSS found that the average detection rate against original exploits was 76 percent, and that only three out of ten products stopped all of the original exploits. The average detection against exploits variants was even lower at 58 percent, NSS found.
NSS President Rick Moy said most vendors appear to have chosen to focus on detecting the malicious software variants delivered by these exploits than on blocking the exploits themselves. Moy notes that while the anti-virus vendors state they are now processing more than 50,000 malware samples every day, it appears the majority of vendors still fail to block the most widely-used methods of delivering those malware samples.
“When you’re talking about exploits that have been published on a government funded web site for months on end, there’s really no good excuse as to why you’re not covering that,” Moy said. “Since there are far fewer exploits than malware, it is imperative that attacks be defeated in the earliest possible stage.”
The NSS tests revealed that certain exploits were consistently missed by the anti-virus products, particularly those that attacked the IE peers and MS VBscript help Internet Explorer vulnerabilities that Microsoft first disclosed in March 2010.
Moy shared a copy of the report on the condition that I refrain from disclosing how each individual product performed, as his company plans to sell the report. But as with the last NSS report I wrote about — which looked at how long it takes anti-virus products to block malicious Web sites — this study focuses on testing individual aspects of anti-virus product performance, including some areas that are glossed over in industry tests.
Even without information about which products earned the highest marks in exploit blocking, one takeaway from the report is the importance of patching as soon as possible after a vendor releases a fix, Moy said.
“There is not a lot of focus on stopping exploits, is what we’re finding, even though certainly at least against the older exploits these security products should act as a virtual patch,” Moy said, adding that organizations should consider developing custom exploit signatures for high-value systems, either at the host or network layer. “The ‘patch immediately’ approach probably works for smaller organizations, but larger companies tend to wait quite a while to make sure patches don’t conflict with homegrown apps.”
Still, NSS doesn’t make a lot of information available about its methods, and this omission has driven much of the criticism of previous NSS Labs reports.
“It would be nice if at least some information about the way the figures were arrived at were available for scrutiny, so that an interested party would have more than just a rather spectacular but otherwise context-free chart to gauge the relative value of the report,” wrote Kurt Wismer, an anti-virus industry watcher and blogger. “As it stands, the information they make available on their site is worse than useless – figures without adequate context are precisely where the idiom of ‘lies, damn lies, and statistics’ comes from. Posting the context-free chart the way they have only serves to sensationalize the report.”
Wismer said the study highlights an area where many products have room for improvement, and that having more anti-virus products blocking the exploitation stage would be a very advantageous improvement. But he said the report itself doesn’t provide a full picture of the performance of these products.
“It just doesn’t tell the customer whether or not they’d actually be protected in the real world,” Wismer wrote in an e-mail to KrebsOnSecurity.com. “The more links in the chain of events leading to compromise that can be used to a defenders advantage. a chain is only as strong as it’s weakest link and so only one stage of a multi-stage attack needs to be blocked in order for the final intended outcome to be thwarted. A test that doesn’t include all the stages therefore necessarily omits information that could be important in determining which products provide the best assistance at protection.”
Interestingly, a series of reports released earlier this month by anti-virus testing lab AV-Test comes to similar conclusions as the NSS report about the exploit-blocking abilities of the major anti-virus products. According to AV-Test, the industry average in protecting against exploits (both known and unknown) was 75 percent.
Thanks for including another perspective on the NSS tests (from Kurt Wismer). As an enterprise customer who has spent time going deep on one of their prior reports, I think their approach is questionable (vendors have almost no opportunity to respond or suggest tweaks to product settings, despite NSS’ claims to the contrary), and the real-world applicability is doubtful.
Those who do this for a living know that among the top 3-5 AV vendors, they’ll all provide coverage against ~85% of what’s out there, and the remaining 15% is a toss-up. Sometimes vendor X gets coverage first, and sometimes it’s vendor Y. And when we run “mystery” malware against VirusTotal, there’s never one vendor that consistently rises to the top. Determining the “best” vendor in the real world is more about manageability, support, and the intangibles than it is about finding one AV vendor who will catch everything. That’s a pipe dream…
Also, the first rule of evaluating these reports: if any tool rates 100% on malware detection (e.g., detecting all of NSS’ arbitrary 123 samples), then question the methodology. There’s no such thing as 100% in the real world.
AVs are probably thinking they don’t want to involve deeply with exploits as it is a problem of a particular software vendor. As NSS report said in case of exploit variants detection is lower. So if AVs are deeply involved, they have to invest a lot of research which will take a lot of time compared to create signature definitions of malware sample spread through that expliot!
I’m sorry, I guess I missed the part in the term “anti-virus” that included exploit detection.
“…most vendors appear to have chosen to focus on detecting the malicious software variants delivered by these exploits than on blocking the exploits themselves…”
Hence the name “anti-virus”.
Those of us who are less sophisticated rely on our AV providers to block malicious stuff. We don’t know enough to distinguish between “exploits” and viruses. We do know that we don’t want either one ultimately to be used to attack our computers.
Considering the humongous revenue received by AV services’ subscription fees, it would seem reasonable to expect them to have “detected 123 client-side exploits, those typically used to attack vulnerabilities in Web browsers including Internet Explorer and Firefox, as well as common desktop applications, such as Adobe Flash, Reader, and Apple QuickTime.”
That is nonsensical. It is as if, given that I own a car, which has a very expensive maintenance, was expecting it to fly because I don’t distinguish between transportation by road or air.
If you don’t believe your AV product is worth it’s price, move to a free one, drop it on the floor or negotiate a better price with your vendor/reseller, but don’t ask for more functionality than what it implements: AVs are slow and intrusive enough today.
Your software vendor is the one responsible for flaws in its software. You’re barking at the wrong tree if you ask AV vendors to fix them for you.
Your analogy is way off-course. I know what a car is expected to do, and I rely on the manufacturer/producer and dealer/vendor to fix any hidden flaws. I don’t want to have to wait until I have an accident to find out that there’s a defect in my car and I don’t want to wait for a successful attack to find out that there’s a defect in my AV program. If the auto industry average in protecting against car malfunctions was 75 percent, would you suggest getting a different car or would launching an investigation be more appropriate?
I’m not sure that I understand your distinction between an AV “product” and a “vendor,” but I do know the difference between protected and not protected. I discover the latter when my computer(s) are successfully infiltrated by malware of any kind. I expect an AV product to be able to identify and deal with a mere 123 already-detected exploits, just as it deals with a bazillion other forms of malware.
Most (unsophisticated) AV users expect “protection,” and believe that they’re getting it. Norton, for example, says of it’s AV program: “Winning protection against viruses, spyware and more.” That’s what I expect.
You don’t understand even the simple analogy: the car is your AV, and what it does is protect you against known malware. It doesn’t fix your software (including your OS) because doing that is a) a different problem and b) the responsibility of that software vendor. No matter if you think it should fix ypur software, you got it wrong.
I’ll also throw in my two cents with JBV. There’s both signature-based and anomaly-based or behavioral detection. Most “legitimate” usage won’t take advantage of known exploits, but a lot of viruses will.
If “Anti-Virus” marketers want to start advertising their “Signature Analysis Products” instead, then I’ll agree that exploits won’t be too interesting for customers trying to decide which to buy.
the use of the term “anti-virus” in this day and age is a bit of an anachronism. anti-malware software covers a much wider array of threats and that includes the delivery vehicles for those threats (which is why many internet security suites include anti-spam and some sort of ‘web guard’), using a much wider array of techniques than what people traditionally think of when they think of AV.
and frankly, for most intents and purposes, exploits are malware (albeit admittedly a bit of a special case).
The most important message is that having a top rated AV product doesn’t mean you can start clicking links and visiting shady websites willy-nilly. As obvious as that seems to everyone here, I run into that attitude so often — “I have an antivirus program, therefore I can bank online/download from file sharing sites/visit porn sites/gamble online,” etc. without worrying about malware.
Which product performed best in these tests probably doesn’t fully represent real world performance, but getting hung up on comparative performance distracts from the message that NO product is good enough to absolve the user from responsibility for using his/her common sense.
On the other hand, I do agree that it’s disappointing they don’t look for the known exploits. Yes, they may be very good at detecting the payload. But they’re never going to be 100% effective.
You have to put it in the context of what the consequences of failure are. If getting a trojan means a couple annoying hours on the phone with tech support, okay, maybe you’re willing to accept an occasional infection. But when the consequences could be hundreds of thousands of dollars gone from your company’s bank accounts with no chance of recovering it, when you’re not just worrying about making a mistake yourself but also must worry about one of your clerical employees getting snookered by a phishing email, you want a better safety net.
We use the terms “virus” and “infection” for good reason. There are significant correlations with biologic infections. If you go to the doctor with a minor infection, you should get an antibiotic chosen to be the most likely to work and the least likely to cause side effects. But if you show up in the emergency room with high fever and low blood pressure and no idea where the infection started, the doctors will usually choose a cocktail of several antibiotics, making sure that the strengths of one cover the weaknesses of another. When it’s really important to stop the infection, when the consequences of failure are dire, you don’t want everything to depend on a single weak link.
If I’m a small business owner and my retirement savings depend on whether my employee opens the wrong email, I want my AV to block the exploit AND detect the malware.
I tend to think of AV as detection and response, not prevention. Use a OS with a secure design if you want the later.
Instead of relying on a piece of software to protect you from the malicious payload, try a product that protects you from the exploit itself.
eEye Digital Security’s Blink Endpoint Protection.
http://forums.eeye.com/forums/t/998.aspx?PageIndex=1