Using IPv4 for Outbound SMTP in Postfix

I got a rejection notice from Microsoft Office 365.  My email couldn’t be delivered, they said, because it was on the Spamhaus malware-source list.  Sure enough, the /64 was on it… which, I guess, might be the prefix for the entire data center? at my VPS provider.  All I get is a measly /128.

Point being, I wanted to send outbound mail over IPv4 only, while keeping inbound mail available on IPv6.  Thanks to slop, the internet didn’t know how to do it conditionally.  I ended up modifying my /etc/postfix/master.cf to include the configuration on the smtp client only:

smtp    unix    -   -   y   -   -   smtp
    -o inet_protocols=ipv4

The change here is adding the inet_protocols configuration line under the “smtp” UNIX service (first on line), handled by the “smtp” client (last on line).

I worked this out from scattered hints in various documentation, but in a broader search, I managed to find Postfix’s own corroboration of what I did.

I can no longer deliver to IPv6-only MX hosts, but in the short run, that’s far less of a concern than being unable to deliver to dualstack hosts which reject my IPv6 but not my IPv4.  The long-term solution is some sort of smtp_protocol_map that I could specify IPv4-only “for Microsoft” and let the rest of the Internet use IPv6 if it wanted to, but I don’t think Postfix supports that.

The information about my specific server applies to Postfix 3.8 shipped with Ubuntu 24.04 LTS.

Spammers Get Confused About Temporary Errors

When I wrote Everything Needs Rate Limits, I mentioned in passing that the disk-full state prevented receiving email.  The MTA was returning a temporary “try again later” code, but the clients weren’t responding.  I got several session transcripts emailed to me that were of the form:

> EHLO some-host-name
< 200 OK + capabilities listing
> MAIL FROM sender-address
< 200 OK
> RCPT TO recipient-address
< 415 Resource unavailable, try again later
> RCPT TO recipient-address
< 415 Resource unavailable, try again later
> RCPT TO recipient-address
< 415 Resource unavailable, try again later
> DATA …
< 500 No recipient given
X Connection lost

The client is recognizing that they’re getting some sort of error, but their idea of “later” was milliseconds later, and the disk space problem was not being cleared at CPU speeds.  After enough rejections of their recipient-address, they YOLO’d it and sent the email body, to no avail.

(I also think it’s pretty interesting that the MTA is happy to tell everyone else ENOSPC, yet deliver these error emails through to me.  I suppose it purposely stops accepting email with some reserve disk space, so that it can continue to deliver critical errors for a while.)

Scattered Notes on Dovecot’s userdb, passdb, and passwd-file

Dovecot can authenticate users using a passwd-like file.  This happens in two phases.  First, users are looked up in the passdb.  If the user is found and authenticated, then the user is looked up again in the userdb to get things like their UID/GID and home directory.

Now, this doesn’t allow for aliasing users in Dovecot.  If the login is user@example.com, then the defaults will lead to trying to find “user@example.com” in the passdb, then the userdb.  Failure to have these configured correctly can result in different errors:

1. User not found in the passdb: authentication fails.  (Beware of fail2ban here.)
2. User not found in the userdb: user can authenticate, but appears to have no mail!

For my own system, the virtual address needs to be resolved to a particular system user (aka Unix account.)  I also want to share the password files with Postfix for outbound email authentication.  This made Dovecot complicated: I want to log in as user@domain, then have that processed as user for both lookups in a file that is specific to the domain. I put the shortened user in the passwd-file, and now I have to configure passdb carefully:

# /etc/dovecot/local.conf snippet
passdb {
    args username_format=%n /local/auth/%d/passwd
    override_fields user=%n
    driver = passwd-file
}
userdb {
    args /local/auth/%d/passwd
    driver = passwd-file
}

This makes passdb do the first lookup using the short username, %n, with the args setting.  Then, that short username is returned by override_fields for use in later lookups.  After that, userdb can continue with no special settings; it will use the overridden user to look up the short name, and nothing special needs to happen.

I believe that the passwd-file can’t return a different username, because there’s only one username field (the first field), and it is also the lookup key.  This is what requires us to use override_fields for this scenario.

Because I Can: SMTPS + MIME::Lite monkeypatch

I had an app that calls to MIME::Lite->send() to send an email, which until recently was using an SMTP server slated for decommissioning Real Soon Now.  It was my job to convert it to Amazon SES, and I figured it would be easier to tell MIME::Lite to use SES's SMTP interface instead of importing the web side's full Perl library tree just for one module out of it.

Ha ha!  SES requires SSL, and neither MIME::Lite nor Net::SMTP have any idea about that.  They were both written before the days of dependency injection, so I had to go to some length to achieve it.  And now, I golfed it a bit for you:

package MyApp::Monkey::SMTPS;
use warnings;
use strict;
use parent 'IO::Socket::SSL';

# Substitute us for the vanilla INET socket
require Net::SMTP;
@Net::SMTP::ISA = map {
  s/IO::Socket::INET/MyApp::Monkey::SMTPS/; $_
}  @Net::SMTP::ISA;

our %INET_OPTS = qw(
  PeerPort smtps(465)
  SSL_version TLSv1
); # and more options, probably

# Override new() to provide SSL etc. parameters
sub new {
  my ($cls, %opts) = @_;
  $opts{$_} = $INET_OPTS{$_} foreach keys %INET_OPTS;
  $cls->SUPER::new(%opts);
}

PeerPort overrides the default of smtp(25) built in to Net::SMTP; I needed a port where the whole connection is wrapped in SSL instead of using STARTTLS, and 465 is the one suitable choice of the three that SES-SMTP supports.

The main caveat about this is that it breaks Net::SMTP for anyone else in-process who wants to send mail to a server that lacks a functional port 465.  But as you may have guessed, that's not a problem for my script, today.

Unintended Consequences (an API Design Rant)

tl;dr

• Your package needs to understand encoding. It can’t just throw away structure and hope for the best.
• If a package is overly simple, it’s likely to be too simple for real-world use; and more likely to be reimplemented with a “slightly less awful” hack because it wasn’t that big in the first place.
• MIME won.  Email packages should understand/generate MIME by default where necessary, and only avoid MIME processing at the caller’s option.  If that.
• Perl’s Unicode implementation adds too much complexity (and it doesn’t help that people don’t agree on terminology).  Now package authors get to write two APIs, one for Unicode and one for octets.

Exposition

Consider the simple program:
use Email::MIME;
$m = Email::MIME->create(
header_str => [
To => join(', ', qw(
  foobar2000@example.com
  spaceball99@other.example.com
  break@domain.example.net
))]);
print $m->as_string;

What happens if you run this on Perl 5.10?  The final email address is “break@domain.example .net”—note the space between “example” and “.net”!

As it turns out, such a thing may actually be legal per RFC 2822: the obs-domain syntax allows for embedded CFWS around the dots of the domain.  However, Amazon SES doesn’t support it, so some email was bouncing with the error message, “Domain contains control or whitespace.”

The rogue space explains the SES failure, but how did it get there in the first place?

Keep the Pieces

When a low-level function is going to write out a string, for instance the To header of an email, I often find myself tempted to “just” pass down strings.  Often, I find later that I would rather have the header in array form at some intermediate level, so that I can add recipients only if they’re not already present.  I’m then forced to parse the string in some manner, with that choice requiring some balance of performance and correctness.  (It’s tempting to make code deal only with the subset of the RFC you think you’ll need.)

If this happens more than once on the way down (“some errors should email us admins if we’re not already involved in this transaction”), it gets even worse: build original array, reify to string, {parse, modify, reify} × 2.  Whereas just handing the array down through looks more like build, modify × 3, reify and send.

Letting the lowest layer put the data on the wire in the format the wire requires can also be more robust: if there are only Bcc recipients and the To address is “Undisclosed-recipients: ;” then checking whether To is empty loses its simplicity: it can have a non-empty value and yet not have real recipients.  Also, nobody at the higher layers has to care whether your addresses are actually separated by comma or by semicolon.

Finally, this lets you push down basic cleaning like calling array_unique() into the lowest level, meaning each modification along the way can quickly append and trust the result will be safe on the wire.  All those layers become more concise and readable.

Observations on SPF: Sender Policy Framework

Recently at work, I updated our SPF policy to something accurate.  Along the way, to understand the policy I was deploying and what the previous version actually meant, I had to understand the various rules and types involved.