Webroot Bulletin Regarding AV-Comparatives Results

By Grayson Milbourne and Joe Jaroch

If there is one thing that can be observed about the AV industry, it is that no solution is ever 100% effective at blocking malware. With this in mind, Webroot SecureAnywhere (WSA) was designed to protect users even in cases where undetected malicious software has made it onto the system.

AV-Comparatives recently published results for June’s “Real World” Protection Test. This test aims to replicate a real world experience for how malware would infect a PC. The scores indicate how many threats were detected vs. missed.

In June’s test, Webroot SecureAnywhere detected 93.4% of samples while 6.4% were able to “compromise” the test PC. I put compromise in quotes because WSA functions very differently from the other products in this test. The key difference is in how WSA protects its users when a malicious piece of software slips by. Unfortunately this test doesn’t demonstrate how well a user is protected in the event of an infection, or how long it takes before an unidentified infection is detected and removed. These are two very important factors which should be considered when judging WSA’s efficacy. Because of these layers and because of how our threat research process works, we’re constantly aware of how effective our solution is for our users.  Since the release of SecureAnywhere, we’ve seen a massive improvement in user security based on the dramatic decrease in the number of threats we’ve had to manually remove from user systems. In addition, our support cases have decreased while our satisfaction scores have climbed well above others in the industry.

To better understand how WSA provides protection in these areas it is first necessary to take a step back and understand how WSA is fundamentally different from a traditional definition-based antivirus solution. One primary difference is WSA uses a cloud-based architecture which is populated in real time with files seen by all WSA endpoints around the globe. A major benefit of this approach is there are no definition updates to burden the system, rather WSA checks in with the cloud whenever a new file is discovered. Another major benefit is that malware researchers are able to focus on and classify new files exclusively seen by our user base, also in real time. This ability is especially important because it drastically reduces the turnaround time from discovery of a new malicious application to the classification and protection of all WSA users.

After looking at the 68 misses from June’s AV-Comparatives test, we found 65 of the samples had been classified within a few hours of their test, with the remaining three being classified minutes after receiving the samples. Of the 68 misses, 34 of the files were seen for the very first time during the test; none of our users were ever affected by them and we had never encountered those components across our entire user base. The other 932 samples were blocked automatically during the test.

So this begs the question, how did WSA protect these infected endpoints while the infections were still unknown to the cloud user base? There are two pieces to this puzzle. The first piece focuses on ensuring WSA is able to reverse all system changes made by a new unknown file and to prevent any irreversible changes from taking place. For example, if a newly discovered program makes file system, disk, registry, or memory changes, these are recorded and analyzed in real time. WSA then checks frequently with the cloud while the program runs to see if an updated classification is available for the unknown files on a system. During this period, the program is able to change the system, but it is under a transparent sandbox where all of the changes taking place are not only being analyzed for behavior correlation, but are also being recorded to see the before-and-after view of every modification to the system. If at any point the cloud comes back and indicates a file is malicious, WSA will automatically remove the infection and restore the system perfectly to a pre-infection state. WSA effectively creates its own system imaging feature that works on a per-application basis, allowing it to generically and completely revert out any threat without needing humans to write signatures. Pretty neat, eh?

The other piece to protecting an infected system is to prevent sensitive data leakage. Most often, malware is after credentials to various websites, whether that is personal email, banking sites or social networking sites. To prevent this from happening, WSA has an innovative combination of security components that are used to create a safe browsing environment which works without any requirement of user interaction. This is done by blocking the various methods used to lift keystrokes from secured browser sessions as well as the numerous new methods that threats are using today: information stealing attacks running in the browser, screen-grabber threats, man-in-the-middle attacks, and various other forms of covert information gathering.

With this layer of protection, WSA will block threats even if it doesn’t know that a file is malicious. While it is definitely best to remove any threats from your system for performance reasons, with WSA enabled, you could install an undetected, zero-day Zeus infection and continue safely banking online even with it active on your system. This layered defense allows WSA to close the gap from what it finds with pure signatures to what it is able to actually protect the user against. While detection is certainly very important, actually blocking the vectors of attack used by malware is what the goal of security software should really be.

Currently most “Real World” tests rely on automation and AV scanners are only given a single chance at detection before the test system is reverted for the next round of testing. Unfortunately this testing model doesn’t give WSA a chance to leverage our unique cloud approach as it has a very static view of the files being tested. If another scan had taken place a short amount of time later, nearly all samples would have been detected from background rules running in our cloud and all system changes would have been reversed automatically.

Webroot continues to drive innovation in our products though this isn’t always met with equal innovation in the testing process. We are actively working with various 3rd party testers to build better real world testing environments which gauge how well security software is able to mitigate the risks of infected systems along with how rapidly vendors are able to update protection to their user base after the discovery of a new malicious threat.

6 thoughts on “Webroot Bulletin Regarding AV-Comparatives Results

  1. Very informative piece, helps explain how WSA applies unique techniques to protect the client. While I no longer use Webroot, I am intrigued by your advanced products and continue to follow your development. In light of your article, I am curious if you may have further information or clarification to a circumstance experienced on a client running WSA. Having used Webroot for years, I uninstalled WSA on this client some months ago after I was getting redirected to fake AV download requests. When the redirection recurred during separate randow browsing sessions, I did not notice any response from WSA in the browser protection or auto-protection. WSA was monitoring select processes but no infection was blocked even after a couple weeks. I installed a different security product on the client in addition to WSA. Promptly this product’s firewall blocked outbound malicious network traffic, which may have been a MD5 phoning out for the fake AV to intrude the system. WSA scans and other scans still revealed no rogues. My question in light of reading your article, could it be that I was actually being protected by WSA even though I was repeatedly redirected to rogue AV website? It would be helpful if you can walk through an example. I’m not sure how to address the other firewall blocking outbound traffic, though.

    • It’s hard to say at this point as redirection threats can take place at multiple levels, many of which don’t require an active infection. We’d be happy to help if you ever see something like this again, even if you aren’t using a Webroot product. Just write into our support inbox and we’ll help you diagnose the issue directly.

      – Joe Jaroch

    • Thank you, Joe, for your generous offer and willingness to help above expectations. At the time, Webroot researchers worked to troubleshoot but nothing was found. Whether it was something that redirected DNS traffic or a simple http cookie with an imbedded iframe, never know for certain. Keep up the groundbreaking work; it’s clear the product has improved over the past several months and may pave the way for the future of internet security. And some feedback if you’re willing. If Webroot built a dual layered product that offered cloud-based protection and client-based definitions engine, I’d be interested even at a premium. Don’t mind the resource usage and system memory, traditional machines are built to accomodate those needs.

  2. So letting malicious code run and upload files is fine, as long as you revert all the changes in a few hours?

    • The vast majority of threats are still blocked automatically – just in the case where we don’t block a threat, we have the backup measures in place to continue protecting the system, unlike other security products which do nothing and allow threats to roam free. WSA applies a transparent sandbox which will limit the infections ability to affect the system and will automatically undo its changes even if they are complex and wide reaching. The generic information stealing trojan protection will prevent it from stealing user data or logging keystrokes. So, while our goal is to certainly block everything before it runs, you’re still safe even if we miss it initially.

      Hope that helps!

      -Joe Jaroch

  3. I cannot tell you how much I appreciate the people at Webroot (I have no connection to them except that I use their AV product SecureAnywhere). I have used a number of the top AV products (I own four, and once a month I switch to see what is going on on my machine when the others check it). Not only does Webroot beat them all in detection and with fewer false positives, it has hardly any noticeable affect on my CPU and memory usage. I love it!

Join the Conversation

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s