Enhancing collaborative spam detection with bloom filters

Yan, Jeff and Cho, Pook Leong (2006) Enhancing collaborative spam detection with bloom filters. In: Computer Security Applications Conference, 2006. ACSAC '06. 22nd Annual :. IEEE, USA, pp. 414-428. ISBN 0769527167

Abstract

Signature-based collaborative spam detection (SCSD) systems provide a promising solution addressing many problems facing statistical spam filters, the most widely adopted technology for detecting junk emails. In particular, some SCSD systems can identify previously unseen spam messages as such, although intuitively this would appear to be impossible. However, the SCSD approach usually relies on huge databases of email signatures, demanding lots of resource in signature lookup, storage, transmission and merging. In this paper, we report our enhancements to two representative SCSD systems. In our enhancements, signature lookups can be performed in constant time, independent of the number of signatures in the database. Space-efficient representation can significantly reduce signature database size. A simple but fast algorithm for merging different signature databases is also supported. We use the Bloom filter technique and a novel variant of this technique to achieve all this.