I had a backup. Really.

What is "stopfaxnow.com"?

My fax machine gets one interesting/useful fax for every 20 or 30 ones it actually receives (the rest are spam), and today I got a curious one:

---

Help us put an end to junk faxes once and for all. It's time to get paid back for the paper and wasted supplies that junk faxes have cost you. Did you know that you oculd get $25 to $100 for each junk fax advertisement sent to your fax machine?

This is a public announcement from StopFaxNow.com, a group dedicated to informing people about the recent changes taking place in the law. Our goal is to help people like you put a STOP to the companies behind unsolicited faxes.

If you were sent COMMERCIAL FAX ADVERTISING, like a product of some kind, a hot stock or travel offering, or some locak service, without your prior consent, the advertiser's behind those faxes may be liable and you could be receiving a settlement for each fax.

As of August 1, 2006, we have completed our new national database of collection organizations that will legally persue junk fax offenders and send you a portion of the money they receive. All you do is mail them your unwanted junk faxes, up to four years old, and they do the rest. These companies are starting to make a real difference and their hard work is paying off.

We offer this list free of charge so take advantage of it. Now join the crusade to stop junk faxes and help make it un profitable for them to continue business!

Log on to our website at www.stopfaxnow.com and register today. Our site also contains free resources, articles, tools and tips that will help you stop fax spammers. And while you are there, be sure to sign up for our quarterly newslatter designed to keep you informed of all the latest news and events as they happen.

WWW.STOPFAXNOW.COM

---

OK, so what's this about? Google shows nothing, and the website itself is kinda thin on details, but my overall impression is that this borders on a scam, but is not quite one (though this is subject to change).

The law in some areas provide for damages if one gets unsoliciated faxes - and I get at least a dozen a week: grrr - and this looks to me like a front for collection agencies or attorneys who plan to make a business out of going after the senders.

By getting people to send in their fax spam, they crank up the damages, then share a portion of the fine/penalty with those who were part of the "class" (term not used in the legal sense).

Superficially this has some minor appeal - sue the spammers! - but the fact that the fax itself had no header or removal information seems to put it in the same category as the objectional material.

The website itself contains very thin information, and their "resources" are really just a few links to the common places (the FTC, the FCC). It looks contrived to me, not a real "resource center".

We do see some hint about what they have in mind:

---

In the month of October, we will focus on retiring your old fax machine and show you how to receive all your fax documents over the internet, or by using a fax modem right inside your computer. This will give you the ability to print only the documents you want, delete the ones you don't and minimize the use of your supplies, saving you money!

---

Hmmm, so they'll offer some kind of e-fax service perhaps? I think we have those already, and the other proprietors don't resort to spamming to push their business.

Looking into stopfaxnow.com domain, we find it registered in Hong Kong:

---

Registrant:
Ling Corporation
35 Central Plaza
18 Harbour Road
Hong Kong, Hong Kong 142587

---

The webservers are located there too, and this all just looks really suspicious considering they're shooting for targets in the United States.

Until I get more information, I'm giving this a thumbs down. Anybody else know anything about these guys?

SPF Tip: only one TXT record

A customer reported that large numbers of automated in-response-to-users emails were not being received by users at the big email providers (Yahoo!, Hotmail, AOL, etc.), and this was killing their customer service. "I forgot my password" recovery, "you have new messages", "confirm your email address" were getting lost by the thousands into the ether.

Digging in, I found the problem to be with SPF (Sender Policy Framework) records which the customer provided in DNS. He had a subdomain allocated to the mailsender machines ("mailout.example.com"), and SPF lets you specify what amounts to a list of IP addresses which are allowed to send email with that domain.

Mailservers which check SPF look up a TXT record in DNS from the sender's domain and see if the IP address is listed. If so, it's passed: otherwise it's a forgery. Sites which publish no SPF data don't get any SPF verification or treatment.

The problem is that the customer published two TXT records, thinking they were cumulative. They had a long list of individual IP addresses of mailservers, so they were spread across two records. But this doesn't seem to be allowed, so sites asking for SPF data got only one of them at random. So half the time their email server's IP was in the TXT record which was discarded, and email was getting dumped in massive volumes.

Fortunately, the fix was easy: consolidate two records into one, and convert the list of individaul IP addresses into a smaller number of /24 ranges. Now it all fits in one record, is easier to read, and email stops getting bounced.

Moral of this story: bad SPF is worse than no SPF

"What happened? Did you kill all the spammers?"

The amount of spam received has been on the rise nearly everywhere, much of it due to malware spambots harbored by those who can't properly secure their computers. SpamAssassin and the various blocklists can only go so far in stemming the tide of this onslaught, so I looked into greylisting: it's been a huge hit on my own mailserver and at a customer.

Greylisting relies on the fact that spambots are not real mail transfer agents: they "fire and forget" to the target mailserver, and if the message is not accepted for whatever reason, they just drop it. Real mailservers queue the messages and retry later, spambots don't.

With greylisting, the first inbound connection from a remote is turned away with a 4xx temporary failure, and the sender/recipient/IP is recorded into a local database: the message is not accepted for delivery. If the remote tries again more than 5 minutes later, the previous attempt is remembered, and the connection permitted.

Temporary failure conditions have always been part of the SMTP protocol (example: 452 = Insufficient system storage), and real mailservers simply know how to deal with. They'll be back, but malware won't.

Issues to think about

I have always understood what greylisting did in the technical sense, but avoided it for some time because I didn't really have a real-world sense for what issues might arise. Of course it should mean less spam, but does it require babysitting? Are there bad effects? I just didn't know, but now I have a better handle on it.

This is in the context of the wonderful Postfix mailserver using the postgrey module. My mailserver runs on Linux.

• There is a one-time delay the first time a particular sender/recip/IP of a remote mailserver tries to deliver to you: postgrey's default window is 5 minutes, but you're still at the mercy of the other end to retry - if its queue-retry interval is an hour, you'll wait an hour. You can whitelist particular recipients (support@, sales@) if you simply can't stand this delay in the general case.
• You can whitelist friendly mailservers so that they never receive a delay. Though a delay is only really applied to the sender/recipient combination, two large enterprises which talk to each other a lot have a very large number of permutations: whitelisting the server this off.
• Some mailservers have a very long retry interval (say, 12 hours), and this is substantial enough that postgrey provides a mechanism to whitelist some servers known to be problematic. Likewise, some don't retry at all, and can be whitelisted on the same basis.
• Greylisting tests are done on a class-C block level, not at the individual IP level: many of the larger mail systems - Hotmail, Yahoo!, AOL - have large farms of sending servers, and each connection could come from a different IP address. This would play havoc with greylisting.
• In this context, whitelisting doesn't mean "is trusted not to send spam", but "it's a real mailserver": there is no benefit in turning away a real mailserver if it's just going to try again shortly thereafter, so we may as well just let it through. The use of greylisting doesn't obviate the use of other antispam mechanisms such as blocklists or content-based filtering.
• Postgrey uses Postfix's clever policy delegation mechanism. Rather than build in custom support for greylisting, one can create a "policy server" which is handed a sender/recipient/IP triplet at the start of an SMTP conversation, and this policy server tells Postfix what to do about it. This is really nice because you can handle policy any way you like.
• Postgrey comes with a small program which can scan the maillog and produce a report of blocked mailservers and all the various parameters: this can be reviewed periodically to look for systems to whitelist, but I've not found anything yet worth taking action on (nor have I looked much).

I perform backup mail relay for several domains, and a few of them have been the subject of relentless brute-force attacks via email. This was many thousands of messages a day which had to run through the blacklists, SpamAssassin, and be bounced due to bad recipients. It made for a very busy mailserver.

Once I implemented greylisting, this shut down to nothing. In the last six days, more than 70,000 connections have been turned away on my little mailserver, and I get just a few spams a day now.

One of my customer's users asked "What happened? Did you kill all the spammers?". If wishing made it so :-)

The spammers will adapt, of course, but I don't think I've ever done anything which made as big a difference in my receives spam as greylisting (the previous winner was using the Spamhaus blocklist).

This is highly recommended for anybody running a real mailserver.

Keyword blacklisting for MT Trackback spam

Like many others, I've gotten really tired of trackback spam. Though comment moderation helps on that front, version 3.01D of Movable Type doesn't seem to have anything corresponding for trackbacks. Because these things are indexed by Google, I have long felt an obligation to delete them on a very timely basis. I'm sick of it.

Because I use MT with mod_perl, I understand that the standard add-ons for this don't work, so I decided to hack up my own. I am very weak with the whole MT infrastructure, so elected for something simple that I could understand. This means "It's a hack". But it's been working for me quite well.

I found the code in mt/lib/MT/App/Trackback.pm that seems to process the trackbacks in realtime, and it seemed clear just how an entry could be rejected. So I added my own: the idea is that we know the URL and excerpt from the entry, and if either one contains certain keywords, we'll reject the trackback.

Adding this code to the start of the file:

---

use File::Spec;
use MT::TBPing;
use MT::Trackback;
use MT::Util qw( first_n_words ...);
use MT::App;
@MT::App::Trackback::ISA = qw( MT::App );

#START ADDING HERE
my $ban_patternfile = "/tmp/banpat";
my $ban_patternlog  = "/tmp/banlog";

# THIS IS A HACK
sub check_banned {
    my $url     = shift;
    my $excerpt = shift;

    return 0    if not open(XX, $ban_patternfile);

    my @PATS = <XX>;

    close XX;

    my $banned = 0;

    foreach my $pat ( @PATS )
    {
        $pat =~ s/#.*$//;    # dump comments
        $pat =~ s/\s+$//;    # dump trailing whitespace
        next if not $pat;

        if ( $url =~ m/$pat/i  or  $excerpt =~ m/$pat/i )
        {
            $banned = 1;
            last;
        }
    }

    # log if possible
    if ( open (XX, ">>$ban_patternlog") )
    {
        printf XX "%s %s\n", $banned ? "Banned" : "Passed", $url;
        close XX;
    }

    return $banned;
}
#END ADDITION

sub init {
    my $app = shift;
    $app->SUPER::init(@_) or return;
    $app->add_methods(
...

---

... defines the function that does the banning and logging, and it must then be actually called by adding two lines to the ping sub:

---

    no_utf8($tb_id, $title, $excerpt, $url, $blog_name);

    return $app->_response(Error=> $app->translate("Need a Source URL (url)."))
        unless $url;

    # SJF HACK
    return $app->_response(Error=> $app->translate("Banned trackback."))  # NEW
        if check_banned($url, $excerpt); # NEW 

    if (my $fixed = MT::Util::is_valid_url($url || "")) {
        $url = $fixed;
    } else {

---

The file /tmp/banpat - which you can put anywhere - should contain a one-per-line list of words, phrases, or URLs that should be banned, and this list usually comes together relatively quickly. The file /tmp/banlog shows which trackbacks have been blocked or passed, though it shows only the URLs, not the excerpts.

There is nothing about this that is not a hack, and those who actually know the internals of MT can surely do better. I don't know MT internals, and every time I touch MT (including routine upgrades) it's a huge cluster, so the only thing time permitted was an ugly hack. Perhaps it will be useful to others.

... and am I the only one who finds that MT's "preview" doesn't actually do a real "preview"? Ugh.

Update: - it's been some days now, and a quick check of the banlog shows 234 entries, with only four of them being Passed. The offending entries were easy to add to the ban pattern file (and delete in MT). I think that catching 98% of trackback spam has been worth the effort.

Spam blacklists as denial-of-service attacks?

I'm a big believer in spam blocklists - I use several myself - as well as other proactive efforts to limit spam. This is a really big problem, and there is inevitable collateral damage too, but a new one is emerging: blacklisting email forwarders.

AOL members can mark any email as spam, and this black mark gets attached to - among other things - the source IP address. If too much "spam" comes from one IP address, it can get blacklisted for a time. For direct injection or open-relay spam, this is fine, but what about email that is merely forwarded through a mail server at the user's request?

I host a small number of email accounts on my servers, with a few being forwarded elsewhere. If these go to AOL accounts and are reported as spam, it gets credited to my server. Eventually my server could get blacklisted, and I've not done anything even remotely wrong.

This hasn't happened to me, but it does happen to others: at one email forwarder I know of positively forbids any forwarded email from being reported to Spamcop (which apparently considers the last IP address in the header as the "source", much like AOL does). They have been blacklisted several times, and they will now cancel the accounts of anybody who does reports them.

I guess the moral of the story is to be careful how you're reporting spam: if you're forwarding email through somebody else's server, improper spam reporting can very well lead to the whole server being considered "a spam source" (it's not hard to construct a denial-of-service attack on this basis).