ESET Threat Blog

The anti-malware industry isn't a suitable environment for the thin-skinned. We get used to receiving "more kicks than ha'pence" (see http://www.virusbtn.com/spambulletin/archive/2006/11/vb200611-OK)..

In particular, I've grown accustomed to the fact that many people expect all the following from an AV product:

• Absolute Protection
• Absolute Convenience
• Absolutely no  False Positives
• Absolutely no charge

False positives (FPs) are not a minor issue: my experience is that many people (especially in corporate environments) are more infuriated by an FP than by a false negative (i.e. where security software fails to detect real malware).

So it was a pleasant surprise to come across a blog from a source not usually associated with fulsome praise of the AV industry that was actually rather positive, in a backhanded sort of way. The author states "…while I rant quite a bit about the AV industry, you have to give this one to them: the number of false positives is really low. For example, in the AV-Comparatives test 20 false positives is considered many, even though the collection is over 1 500 000 samples (so the acceptable FP rate is below 0.0015%!)." [Sadly, this calculation doesn't altogether make sense, either to us or to AV-Comparatives. In fact, it seems to be based on comparing the number of "acceptable" false positives to the AV-Comparatives malware test set. It isn't based on the AV-Comparatives clean set: AV-C don't publish that information, as they consider FP ratios in percentages to be potentially misleading.]

The blog isn't actually about us, I should make clear: it's about the importance of false positives in intrusion detection, an area with just enough methodology and terminology shared with anti-malware to be confusing – for instance, both technologies talk about FPs and signatures, but often mean something slightly different by them. This led me to another blog that makes a useful distinction between "real" false positives – misdetection of innocent objects as being malicious – and "noise" such as "inappropriate traffic that is legitimately detected that we just don't care about." Actually, because modern anti-malware has a much wider remit than traditional virus detection, that is a concern we share with IDS and IPS vendors – of course, there's an overlap – most AV products now include some IPS (Intrusion Protection System) functionality. But it also has an analogue in more traditional anti-malware detection, which tends to cover "noise" programs such as adware and "possibly unwanted" code as well as out-and-out malware. 

Then there's the miscellaneous detritus that may be left behind when malware is removed: it may be harmless in itself, but what happens when two security programs are used on the same system and one is more cautious about what it removes than the other? Which is "correct" is a subjective judgement rather than a right/wrong issue.

To take another common case, no-one wants to see those misidentifications of innocent system files as malicious that affect even the most cautious products from time to time, but what about files that are detected as malicious because they're packed with a run-time packer that's often (but not necessarily always) used by malware authors? Well, that's not an issue I'm going to be able to give a short and simple answer to here. But I will say this (and often have before): given the complexity of the malware problem, not to mention the sheer volume of samples, the wonder is that mainstream anti-malware generates so few FPs. And it's a real pleasure to find someone agreeing on that point who can't be accused of undue bias in favour of this industry.

By the way, these blogs reminded me of an old but still relevant paper on "The Base-Rate Fallacy and its Implications for the Difficulty of Intrusion Detection" by Stefan Axelsson. If you're interested in the false positive conundrum and  the words "Bayes Theorem" don't strike terror to your heart, you might find it well worth reading.

David Harley BA CISSP FBCS CITP
Director of Malware Intelligence

ESET Threatblog (TinyURL with preview enabled): http://preview.tinyurl.com/esetblog
ESET Threatblog notifications on Twitter: http://twitter.com/esetresearch
ESET White Papers Page: http://www.eset.com/download/whitepapers.php

Securing Our eCity community initiative: http://www.securingourecity.org/

Nowadays we see lots of malicious software that is designed to steal money and information. A new virus was recently discovered that seems to be all about proving a concept rather than blatant maliciousness.

The Win32/Induc.A virus does not infect like most viruses do. Delphi is a programming language. Induc infected the Delphi IDE so that when the programmers compile their programs the programs are already infected.

As far as we are able to determine at this time, this virus went undetected since April 2009. Most of the samples of infected files we have seen are other trojans, mainly those that steal bank information. So, we detected the Trojan, but didn’t know that it was also infected.

For the average user the virus is essentially harmless. The problem is that some software development companies use Delphi, got infected, and when we added detection for Win32/Induc.A their programs were detected. Some of these companies accused ESET of having false positives when their programs were actually infected!

In reviewing our internal malware collections our researchers have found over 4,000 infected samples. Our Threatsense.Net network has identified over 30,000 unique infected samples in the first 24 hours after we added detection.

For a write up about this virus you can visit http://www.eset.eu/encyclopaedia/win32-induc-a-virus?lng=en

Ironically, some other malicious software that was previously undetected by antivirus vendors will now be detected because it is infected with Induc.A!

It’s pretty rare now to be able to talk about a widespread virus that probably won’t cause you any harm.

Randy Abrams
Director of Technical Education

From time to time the discussion of whether or not there are (or can be) good worms comes up, usually specifically in the context of program maintenance, updates and upgrades.

In fact, the idea of maintenance viruses goes back at least as far as Dr. Fred Cohen, who pretty much "wrote the book" on early viruses, almost literally. In fact, looking at his book "A Short Course on Computer Viruses" (in which he specifically discusses "benevolent viruses") again, I was struck by how far malware and anti-malware technologies today still reflect his ideas.

However, the idea of the benevolent maintenance worm, which last year resurfaced in a paper in which Microsoft researchers had an interest, has tended to meet with a chilly reception from the mainstream researcher community.

An event today really drove home the point about how bad a good worm could really be.

 We had an unfortunate false positive event this morning at ESET. A synchronization error allowed a new heuristic module to be released along with a new signature update. Each worked great on its own, but together they created false positives on some Windows operating system files. Both components went through QC, but not together. We’ve got that process fixed now.

The problem was spotted almost immediately, and within ten minutes the faulty update was removed from the download site, thereby preventing 95% of our users from encountering a problem. As for the other 5%, many of those people never knew there was a problem because the next update replaced the quarantined files on all versions of our products except for version 2.7. We have also issued a new tool to help anyone who needs assistance, and that also works on all versions, including 2.7. Our people in tech support can provide the tool upon request, and, of course, tech support is free for all ESET customers.

So, what does it really mean, when we say that 95% of our users were unaffected? Unfortunately, It means that there were still a couple of million users who did encounter the problem. A couple of million users in 10 minutes, and this was done without the help of a worm. The Slammer worm infected 90% of all vulnerable connected computers in 10 minutes. No matter how good the intent, there will be problems with software from time to time. With a faulty signature we can turn off the source almost instantly and it goes no farther. With a worm, well, you know what breaks loose. Don’t just think Slammer, think Code Red, Blaster, and many, many mass mailers (some of which are still very much current).

 No matter how good the intent, self-replicating code is not a good idea for a distribution mechanism. There is always a better alternative for distribution.

So, you may wonder, how do we get our updates out to a couple of million users in ten minutes or less without the benefit of a tame worm? Ironically, by a mechanism analogous to that used to update zombie PCs in a botnet. But we’d like to think that’s a case of the malware community taking ideas from us, rather than vice versa.

Randy Abrams & David Harley
ESET Research Team

False positives. Every anti-malware vendor’s worst nightmare.

The European publisher Heise, apparently recently reinvented as The H, has pointed out that both GData and Bitdefender were inaccurately flagging winlogon.exe as Trojan.Generic.1423603. In case you were wondering, this doesn’t mean the whole anti-malware industry has gone mad: GData’s product uses two engines, one of which is  Bitdefender’s.

In fact, it doesn’t mean that Bitdefender have gone crazy, either. False positives – diagnosing an infection where none actually exists – is one of those regrettable issues from which no vendor (or AV customer) is safe. Sometimes it’s a major inconvenience for customers, and that’s why Heise have been characteristically abrasive on the issue: "This latest case gives rise to the assumption that false alarms caused by anti-virus software will continue to unnerve users. " But the point is well taken: FPs can be alarming, confusing, inconvenient and downright expensive for customers (especially where innocent but essential files are quarantined or deleted), which is why we go to considerable lengths to minimize the risk (but even ESET can’t eliminate it altogether, and it really is an issue we work very hard on!)

In fact, high profile misidentifications are surprisingly rare, given the complexity and sophistication of the technology. There are probably quite a few instances of low-profile FP, affecting applications less commonly used, that are hardly noticed. Despite the anger that antivirus and antispam FPs sometimes inspire, much of the security industry is founded on the idea that for some organizations and individuals, a degree of inconvenience caused by sensitive filters (in the broadest sense of the word filter) is acceptable.

For example, it’s common for mail filters to block whole swathes of executable filetypes. It’s unlikely that a filename like something.lnk would ever be attached to a legitimate email file attachment (the suffix denotes a shortcut rather than a "real" executable file), so blocking that suffix hardly ever results in misidentification. However, there are a great many innocent files that have the .EXE suffix, yet we often accept the filtering of .EXE attachments because we consider the risk from malicious attachments worth the risk of not receiving a legitimate .EXE through the mail, or the inconvenience of finding an alternative mode of transportation. In fact, some organizations have been doing this for at least ten years (and similar considerations apply in the world of spam management, as I discussed more recently in a Virus Bulletin article).

The fact is that the industry has had to move on from the early days of exact identification and one sighature to one threat. Nowadays, products use combinations of approaches that are nearer to what we used to call generic detection like change detection: whitelisting, filtering by filetype, change detection, heuristics, behaviour analysis, and so on. (Of course, products vary widely in terms of which techniques they use, how they are implemented, and so on.)

Our labs are now seeing over 100,000 unique samples a day. Other vendors may use different ways of counting, but they all face the glut problem, which is why there is so much emphasis in the industry on automation, distributed processing, and dynamic analysis. The miracle is, in the face of these difficulties, that significant false positives are so rarely encountered.

But there you go: problems attract far more attention than a product doing pretty much what it was designed to do, especially an antimalware product…

David Harley BA CISSP FBCS CITP
Director of Malware Intelligence

One of the security best practices is to back up your data regularly. This is sound advice as it helps mitigate the damages from many different threats. Lots of people think of data loss when they think of viruses, but very few viruses actually tried to cause data loss. There have been a few that encrypt data in an attempt to extort ransom money so the user can get their data back, but this is relatively rare. Today, most of the threats are not about destroying data, they want to collect your data so they can steal your money or identity. Still, backing up your data may help reduce losses if you are wit with some malicious programs.

What else causes data loss? Sometimes improperly configured antivirus products themselves cause data loss. A couple of years ago one of the largest antivirus vendors in the world had a false positive problem that deleted many Microsoft office files. False positives happen to all of us from time to time, but files should be quarantined so that if it is a false positive the data can be recovered. The product in question was revised to make quarantine the default setting after so many of their customers who did not perform proper backups lost a lot of data.

Fingers are high up on the list. Did you ever accidentally delete something and not have a back up?

Hard drive failures can also cause massive data loss. A recent event that our own Aryeh Goretsky brought to my attention is what made me decide to write this blog entry. Hard drive failures are typically fairly rare, but there is a biggie out there now.

It seems that Seagate has released droves of 1 terabyte hard drives that have a problem causing them to die. Having good backups may be the only way to recover your data without spending a few thousand dollars in such a situation.  

There are about 18 pages of comments on a Seagate forum (as of this writing). The drives may be working fine for 3 months and then die instantly.

Below are a few links about the issue. The last link is the Seagate forum I mentioned.

http://www.techreport.com/discussions.x/16232

http://www.theinquirer.net/inquirer/news/374/1050374/seagate-barracudas-7200-11-failing 

http://www.dslreports.com/forum/r21737309-AVOID-seagate-ST31000340ASSD15-drives 

http://forums.seagate.com/stx/board/message?board.id=ata_drives&thread.id=3668&view=by_date_ascending&page=1

It is unfortunate that Seagate refuses to comment on the situation. Late last year it was discovered that Seagate had shipped about 1,800 brand new drives with malware on them.

If your data is important enough to put on a hard drive, be sure you back it up. There are many threats to your data, and viruses are the least of those threats.

Randy Abrams
Director of Technical Education

I’ve just picked up a comment to a previous blog that pointed to what I presumed to be a malicious URL. We’re grateful for all such information, but for obvious reasons, we won’t approve comments that point to malicious code!

You can find information in our knowledgebase here about how to forward malware samples or false positive samples to our labs: it doesn’t specifically mention malicious URLs, but you can send those to the same address.

It’s not that we’re not willing to pass on information to the labs ourselves (and I’ve already forwarded the link), but it’s much more effective for you to send the information direct. As a case in point, it turns out that by the time I saw this comment and forwarded the link, our products had already been updated to detect this particular binary.

Thanks!

David Harley CISSP FBCS CITP
Director of Malware Intelligence