Thursday, 11 August 2005

10:54pm: New project "postprox"
Today I released version 0.1.0 of postprox, a small SMTP proxy for use as a content filter.

As part of the ongoing attempts to cut down on spam, I decided I wanted to implement an additional filter. This one scans the content of a message for URLs, and any that are found then have their hostnames resolved to IP addresses and those addresses are looked up in the Spamhaus SBL+XBL blacklist. In other words, it rejects messages that advertise web sites hosted by known spammers.

However, rather than accept each message then filter it at the point of delivery, I wanted to check the messages at the point of entry (as the server receives them), and reject them outright instead of accepting them first. To do this with my MTA - Postfix - I needed to set up a "before-queue content filter", which means passing incoming email through an SMTP proxy which then feeds back into the mail server again.

The idea is that the SMTP proxy takes the message and runs the filter program on it, and based on that it decides whether to pass the message on to the mail server's queue or reject it there and then.

This is more efficient than accepting the message and scanning it on delivery because if you want to bounce it (i.e. inform the sender that their message wasn't delivered), if you've accepted the message already you have to generate a bounce message and send it out. If, instead, you reject it before it is queued, it's the responsibility of the mail server that sent you the message to generate the bounce. Since spam and viruses are usually sent with forged "sender" addresses, this drastically cuts down on the amount of junk bounces cluttering up the Internet, but still allows legitimate senders to be informed that their message didn't get through.

Initially I tried using ProxSMTP to do the proxying, but on a medium-to-high volume mail server I was getting one or two "double-bounce" messages per hour due to proxy failures, and it just generally seemed unstable.

So I wrote postprox, which has fewer features than ProxSMTP but is smaller and in theory less buggy as a result. Ironically though, when I was trialling the first (unreleased) version it exhibited similar problems - after some discussion on the postfix-users mailing list it turns out that I'd probably programmed the same bug as was in ProxSMTP. At least it's fixed now.

At the moment there's only one example content filter included, which uses ClamAV to reject any email containing a virus. I have the URL scanner working, but there is no OSS version yet; if there is one, it will probably be in C for speed, and it will be a separate project.

The postprox manual contains details of how to set it up with Postfix.