Wed, 20 Aug 2003 13:30:11 GMT

The Day Email Died?. Between 10:45pm last night and 6am this morning, I received 1,470 pieces of spam (a run rate of nearly 5,000 per day). Most of them were from the SoBig worm, which seems to be the worst yet. And as far as I can tell, it's still getting worse.

I wonder if this is the last straw that will convince people to move in droves to challenge/response or whitelist mechanisms, as I argued last year. And I'd be surprised if these volumes don't overload mail servers at major ISPs. We're seeing the closest thing yet to the Morris worm that shut down the Net in 1988.

The good news is that my lovingly tuned filtering guantlet, including the rule-based SpamAssassin, the Bayesian POPfile, and a couple dozen hand-coded rules, has held up well. All 1,470 of the messages were auto-routed to the trash. The bad news is that, with this many spams, it's impractical to check manually for false positives (good email accidentally deleted as spam). My overall filter accuracy is well above 99%, but I'm still seeing occasional false positives (maybe two per week) when I search for them.

We have to confront the reality: either email is broken, Microsoft's email software is broken, or those two statements are the same. If it's the middle statement, Microsoft and other vendors can close holes and improve filtering in their products. Email itself isn't going to change. It's too widely deployed. I still think a combination of steps will tame the spam epidemic, but we're not there yet. [Werblog]