Most anti-virus products designed for use in businesses do a poor job of detecting the exploits that hacked and malicious Web sites use to foist malware, a new report concludes.
Independent testing firm NSS Labs looked at the performance of 10 commercial anti-virus products to see how well they detected 123 client-side exploits, those typically used to attack vulnerabilities in Web browsers including Internet Explorer and Firefox, as well as common desktop applications, such as Adobe Flash, Reader, and Apple QuickTime.
Roughly half of the exploits tested were exact copies of the first exploit code to be made public against the vulnerability. NSS also tested detection for an equal number of exploit variants, those which exploit the same vulnerability but use slightly different entry points in the targeted system’s memory. None of the exploits used evasion techniques commonly employed by real-life exploits to disguise themselves or hide from intrusion detection systems.
Among all ten products, NSS found that the average detection rate against original exploits was 76 percent, and that only three out of ten products stopped all of the original exploits. The average detection against exploits variants was even lower at 58 percent, NSS found.
NSS President Rick Moy said most vendors appear to have chosen to focus on detecting the malicious software variants delivered by these exploits than on blocking the exploits themselves. Moy notes that while the anti-virus vendors state they are now processing more than 50,000 malware samples every day, it appears the majority of vendors still fail to block the most widely-used methods of delivering those malware samples.
“When you’re talking about exploits that have been published on a government funded web site for months on end, there’s really no good excuse as to why you’re not covering that,” Moy said. “Since there are far fewer exploits than malware, it is imperative that attacks be defeated in the earliest possible stage.”
The NSS tests revealed that certain exploits were consistently missed by the anti-virus products, particularly those that attacked the IE peers and MS VBscript help Internet Explorer vulnerabilities that Microsoft first disclosed in March 2010.
Moy shared a copy of the report on the condition that I refrain from disclosing how each individual product performed, as his company plans to sell the report. But as with the last NSS report I wrote about — which looked at how long it takes anti-virus products to block malicious Web sites — this study focuses on testing individual aspects of anti-virus product performance, including some areas that are glossed over in industry tests.
Even without information about which products earned the highest marks in exploit blocking, one takeaway from the report is the importance of patching as soon as possible after a vendor releases a fix, Moy said.
“There is not a lot of focus on stopping exploits, is what we’re finding, even though certainly at least against the older exploits these security products should act as a virtual patch,” Moy said, adding that organizations should consider developing custom exploit signatures for high-value systems, either at the host or network layer. “The ‘patch immediately’ approach probably works for smaller organizations, but larger companies tend to wait quite a while to make sure patches don’t conflict with homegrown apps.”
Still, NSS doesn’t make a lot of information available about its methods, and this omission has driven much of the criticism of previous NSS Labs reports.
“It would be nice if at least some information about the way the figures were arrived at were available for scrutiny, so that an interested party would have more than just a rather spectacular but otherwise context-free chart to gauge the relative value of the report,” wrote Kurt Wismer, an anti-virus industry watcher and blogger. “As it stands, the information they make available on their site is worse than useless – figures without adequate context are precisely where the idiom of ‘lies, damn lies, and statistics’ comes from. Posting the context-free chart the way they have only serves to sensationalize the report.”
Wismer said the study highlights an area where many products have room for improvement, and that having more anti-virus products blocking the exploitation stage would be a very advantageous improvement. But he said the report itself doesn’t provide a full picture of the performance of these products.
“It just doesn’t tell the customer whether or not they’d actually be protected in the real world,” Wismer wrote in an e-mail to KrebsOnSecurity.com. “The more links in the chain of events leading to compromise that can be used to a defenders advantage. a chain is only as strong as it’s weakest link and so only one stage of a multi-stage attack needs to be blocked in order for the final intended outcome to be thwarted. A test that doesn’t include all the stages therefore necessarily omits information that could be important in determining which products provide the best assistance at protection.”
Interestingly, a series of reports released earlier this month by anti-virus testing lab AV-Test comes to similar conclusions as the NSS report about the exploit-blocking abilities of the major anti-virus products. According to AV-Test, the industry average in protecting against exploits (both known and unknown) was 75 percent.