Most of us have been in Estonia for the past few days for a couple of conferences. You may hear more about that later, when Normal Service is resumed. One thing I wanted to remark on now, though (partly because it relates directly to some presentations I’ve been doing) is a spike in the use of VirusTotal as a tool for comparing detection performance. This is a topic we (and the guys at VirusTotal/Hispasec themselves, who are a really good bunch) are rather sensitive about.
I’ll probably come back to this in the near future, but the gist of the problem is this. VirusTotal is a tool many people find very useful as a shortcut to checking a possibly malicious file, but it isn’t a detection test. Most importantly, it submits the files you submit to a battery of command-line scanners. This gives you a good chance of identifying a known malicious program, but the fact that a scanner doesn’t identify a file as malware does not mean it isn’t malicious, obviously. However, if a file is identified as malicious by one group of scanners but not another, it doesn’t necessarily mean that the second group is less competent at detection, either. Scanners that use sophisticated behaviour analysis, active heuristics and so on are disadvantaged by this misuse as a comparative test tool, since there is no behaviour to analyse. Generally, command-line scanners simply look at the code passively, rather than running it in a safe environment to see what it does in practice, so products that are heavily dependent on signature detection may seem to do better than products with advanced heuristics. In the real world, however, where on-access scanning is the first line of defence for most people, the advantage tends to swing the other way.
You might want to check out what Hispasec/VirusTotal have to say themselves at http://blog.hispasec.com/virustotal/22. Alas, I’m sure I’ll be back to this topic sooner rather than later, and in appreciably more detail.
David Harley BA CISSP FBCS CITP
Director of Malware Intelligence
Visit ESET on Facebook
Follow ESET on Twitter
September 12th, 2008 at 10:46 am
Yes,Virustotal is not a Analysis tool,I find many people very depend on virustotal’s result to analyze if the files which they uploaded are malware,if many avs can detect them,they think the files must be malware,but in fact,I often see many avs detect normal files as malware.
Some organizations and individuals use virustotal’s result to do some av tests,but they can’t analyze if these samples are real malware or normal files before,so I can’t trust these tests which based on virustotal.
September 19th, 2008 at 3:25 am
Nod 32,I have nod 32 since Jan. or Feb.2007 and been satisfied but now you want to sell me something else or what I have.You downloaded my version and if I have to buy it again,I hope you will do the same again because I`not too technical qhen it comes to computer or any electronique
because I am mostly visual.So please reply and tell me what is my situation here o.k?I havealso a hotmail addres:pumpkenhead@hotmail.com,thanks for reading me.
Emilienne Morais Lebrun
September 19th, 2008 at 3:28 am
May I have more info on buying nod 32?
Emilienne
September 19th, 2008 at 3:45 am
Hello, Emilienne.
If you go to http://www.eset.com/purchase, you should be directed from there to the appropriate web page. If you need information on installation and such, you can also try http://www.eset.com/support.
David Harley
Malware Intelligence Team
October 3rd, 2008 at 2:57 am
You’re correct, VirusTotal is for testing files, not AV products. I interviewed Julio Canto from VirusTotal for a blogpost about this subject:
http://blog.didierstevens.com/2008/04/21/only-x-out-of-32-antivirus-products-detect-this/
October 3rd, 2008 at 3:32 am
Thanks for that, Didier. Good blog post. Julio Canto is a good guy for sure, very knowledgeable and unfailingly helpful.
David Harley
ESET Research Team
October 4th, 2008 at 8:12 am
VirusTotal is just a small step in your virus analysis process. You shouldn’t relay 100 % on the results, because sooner or later, virus writers will figure out a way to trick virustotal to give you missleaded results.
http://extremesecurity.blogspot.com
November 20th, 2008 at 10:16 pm
I think that http://www.filterbit.com is much much faster
November 23rd, 2008 at 5:58 am
Filterbit aslo is not a testing tool. Filterbit is much faster because it uses far less scanners and does not have the traffic that Virus total has.
online virus scannign services are uselss for testing in terms of comparing scanners. Online sevices fail to discriminate against false positives. If I write program that says every file is infected, then my useless program will be the one that filterscan and otehr services say is the best. It really is that easy.
Randy Abrams
Director of Technical Education
November 25th, 2008 at 10:35 pm
Well Filterbit is faster because it is using Metascan
I’d bet you that if Virus total would be using it will be much faster!!
November 26th, 2008 at 9:04 am
Perhaps. (He said diplomatically.) But speed of submission to multiscanner sites isn’t the issue. The point is that multiscanner sites aren’t an appropriate way to rank scanner performance as an alternative to detection testing, because they don’t constitute a full test of a scanner’s detection ability.