WATeR 2013 - The First Workshop on Anti-malware Testing Research

Anti malware products have become a key element of information system protection against current threats. Testing how effective these products are at protecting the end-users and their computers is therefore crucial, whether it is to improve product quality (engineering testing) or to aid users in making better decisions about their acquisition and operation (certification and comparative testing). However designing and conducting relevant anti-malware tests producing significant and accurate results is a complex question. First of all, anti-malware products must be tested not only in varying operating conditions and environment, but also against a constantly evolving threat, driven by malware creators whose goal is to make them perform poorly. Second, the evolving complexity and variety of malware, both in terms of propagation methods and ultimate uses has rendered the problem of proper testing much more difficult with time. Long gone are the days when anti-malware performance could be accurately measured by the traditional method of exposing them to a list of malware file samples containing most if not all of the known malware variants. Today, the sheer number of known malware samples, their extensive use of the Internet and the Web, and in many cases the necessity for human intervention for propagation make this approach much less satisfactory. On the other hand, many traditional anti-malware products have in fact migrated towards dynamic protection services, with anti-malware vendors constantly updating threat databases and signatures, which are being consulted on-demand by locally installed products; the so-called anti-malware â?œin-the-cloudâ?? solutions. Testing the efficacy of such new types of products and services under relevant and repeatable conditions constitutes a formidable technical, operational and even economical challenge.
The Anti-Malware Testing Standards Organization (AMTSO), regrouping key players from the anti-malware industry including product vendors, testers, security experts and technology publishers, fully recognizes the technical challenges posed by testing in the current context. It also acknowledges the need to re-evaluate traditional testing approaches in the current context, to potentially develop new approaches to testing, while providing an open forum for discussion on these issues including the academic community, regulatory bodies and government, and groups representing the interests of end-users. To that effect the AMTSO in collaboration with, the IEEE (appoval pending) are co-sponsoring and organizing the First Worshop on Anti-malware Testing Research. The aim of this workshop is to bring together experts from the industry in contact with academic researchers in order to identify and define the important technical problems associated with anti-malware testing methodologies, and hopefully helping to establish collaborations on potential research projects attempting to find solutions to these problems.