The Rise of the Rogue AV Testers

By Costin Raiu

Recently, I was sitting around with a number of colleagues from Kaspersky Lab, discussing everybody’s favorite subject: the state of anti-virus testing these days. During the talks, somebody brought up the name of a new, obscure testing organization in the Far East. Nobody else had ever heard of them and so my colleague Aleks Gostev jokingly called them a “rogue Andreas Marx."

It then occurred to us that some of these new testing labs that have recently appeared mimic the tactics of Rogue AV products. What exactly do I mean? Well, as we know the rogue AV business model is based on selling a false sense of security; we professionals know it is fake, but the victims don’t. People buy a Rogue AV hoping it will solve their security problems, but the products don’t do anything at best, and at worst, install additional malware.

Rogue AV Testers are somehow similar in behavior. In their case, the business model is no longer based on a false sense of security but instead, on a false sense of insecurity. So, how do they operate? Well, it seems to start with a number of tests which look legitimate, and mimic real world conditions. Then, the tests slowly become more “complicated” and security products do worse and worse. Sometimes, the product that did best in the previous test suddenly becomes the worst in the group. In other cases, all products fail miserably. Finally, the main idea emerges: that all security products are bad and utterly useless. Hence, the false sense of insecurity is promoted through the tests: you are insecure, your money was misspent – beware! 

Going further, the rogue AV testers use various techniques such as not disclosing product names in published test results and attempting to sell theses results for exorbitant fees.
Here are some characteristics we identified as being specific to rogue AV testers, that can help you spot them:

• They are not affiliated with any serious testing organization, such as AMTSO. Sometimes, the Rogue AV Testers could also show fake affiliations or even falsely display (say) the AMTSO logo on their website, in order to remove suspicion and doubt.
• They publish free public reports, but charge money for the “full” reports. In general, the public reports should look as bad as possible for all the tested products, to maximize the profits from selling the full reports.
• The public reports are full of charts that look complicated and intelligent, but sometimes reveal amusing mistakes.
• They claim all AV (or security) products are useless. This is the foundation stone of any business based on the “false sense of insecurity”.
• They charge for samples and methodology, usually very large sums of money, to make sure the flawed methodology and samples cannot be reviewed externally.Reputable testers will make samples and methodology available for free to the developers of the products they test, instead charge for the rights to publish the results in magazines or for the permission to use the results in marketing materials. Charging money for samples is a clear indication that something wrong is going on.

There are other characteristics, but I think everybody gets the point.

Just as Rogue AV (scareware) products exploded and became one of the most profitable categories of crimeware, I suspect Rogue AV testers will follow. In the process, they will also become an extremely profitable category. And of course, the worst of all, they will provide a strong, negative value to the entire IT security industry.

So, if you are trying to compare security solutions, I recommend sticking to established testing organizations such as Virus Bulletin, AV-TEST.ORG and AV-COMPARATIVES or reputable magazines, with a good history behind them. If in doubt, ask for AMTSO affiliations and finally, do not forget about the list of hints that can help you spot Rogue AV Testing behavior.

Do not become a victim of the Rogue AV Testers!

* Costin Raiu is the Director of Kaspersky Lab's Global Research & Analysis Team (GReAT) . This essay was first published in the current issue of Virus Bulletin magazine.   

Commenting on this Article is closed.