Step 1 – upstream SMTP authentication using TLS with postfix.
This document describes how to set up a postfix installation on linux to authenticate to an upstream SMTP server for sending mail out. It does not cover authentication of email clients to the postfix server itself. The document is part of a set of four which collectively describe:
Step 1 – upstream SMTP authentication using TLS with postfix;
Step 2 – getting a mailman listserver running with postfix;
Step 3 – configuring lighttpd with SSL for mailman;
Step 4 – putting it all together and letting the world in.
Because I use debian, the installation instructions tend to assume apt as the package manager, but this is really about software configuration rather than package installation so I have tried to write this and the subsequent documents in a way which is distro agnostic. It shouldn’t matter what package manager you use – you can install everything from source and run the usual ./configure; make; sudo make install for each package if that is your way of doing things. I simply prefer debian package management because it makes keeping my systems patched and up to date easy. If you have a package manager that you are happy with, then use that.
The first hurdle is to get postfix installed and running. This is relatively easy using apt – simply “apt-get install postfix” and we have a basic installation of the MTA ready to go. The apt-get installation process asks you about how you want to configure your installation. Because I wanted to run the MTA as a local relay host, I chose “install postfix for internet with upstream relay”. But the actual installation configuration didn’t really matter because I modified the configuration manually after installation.
Installing an MTA on the end of a domestic ADSL connection and expecting it to work these days is overly optimistic. Few commercial mail servers will accept mail from a server obviously sited on a DSL IP block – it is far too likely to be a source of spam or other unwanted mail. The only way you are likely to get mail out is via an authenticated connection through a commercial upstream provider (either your ISP, or a “free” provider such as Gmail). This means that we have to configure postfix to relay through our upstream. There are one or two good descriptions of how to do this on-line. I found Mark Sanborn’s description of setting up postfix with Ubuntu (which is itself based on Mike Chirico’s Gmail on Home Linux Box using Postfix and Fetchmail) helpful. Redwall firewall have also published a rather nice description of implementing upstream authentication for postfix. Postfix’s own documentation on SASL authentication and relayhost configuration are also required reading. However, none of the on-line resources exactly reflected my particular situation and requirements and some of the information is way too detailed (Mike Chirico’s otherwise excellent paper for example) so I modified my local install accordingly.
An anonymised copy of my complete postfix main.cf is here. As is normal for my local modifications of standard installations I chose to group all the local configuration details together at the end of the file for ease of future reference.
Note that what we are doing here is configuring postfix as an SMTP client to a remote SMTP server (our upstream). We are not configuring postfix to authenticate remote SMTP clients to our server. There is a distinct difference, but the postfix configuration lines look very similar and it is easy to get confused. In particular, lines referring to “smtpd_sasl_[something]” refer to the postfix MTA acting as a server, whilst “smtp_sasl_[something]” refer to us acting as a client. So for example:
“smtpd_sasl_auth_enable = no” – means we do not want to act as an authenticating server (it is not necessary in our scenario);
whilst
“smtp_sasl_auth_enable = yes” – means that we do want to act as an authenticating client (i.e. passing our UID and password to the upstream).
The relevant postfix main.cf configuration lines are:
# SASL details
#
# no for the server
smtpd_sasl_auth_enable = no
# but yes when we act as a client
smtp_sasl_auth_enable = yes
# where we store the hashed UID and password
smtp_sasl_password_maps = hash:/etc/postfix/sasl/saslpw
# security options set to disallow anon auth (but allow plaintext within the TLS session)
smtp_sasl_security_options = noanonymous
# our upstream relay is here
relayhost = [auth.smtp.upstream.com]
#
# end SASL
Now we have to create the saslpw file containing the uid and password. I chose to store all sasl related files in /etc/postfix/sasl and all tls files in /etc/postfix/tls so create those directories first. Now in sasl, create the password file containing the single line required in the form: host uid:pwd. So if your upstream relay is called “auth.smtp.upstream.com” and your normal email client credentials are “you@yourdomain.org” with password “weakpassword” your file would look like this:
[auth.smtp.upstream.com] you@yourdomain.org:weakpassword
Save this file and create the db hash by running “postmap saslpw“. You should now have a new file called saslpw.db in the sasl subdirectory. Note that if you share your linux PC with others (or even if you don’t) it is a good idea to ensure that the saslpw files are owned root:root and set mode 600 to protect your email password from prying eyes.
Note the form of the relayhost name. The square brackets prevent postfix from looking up the MX records for the enclosed name. If you are specifying the actual relay hostname, then an MX lookup is redundant (and possibly unhelpful). It is also possible to force postfix to use the TCP network port 587, which is normally reserved for email clients, when talking to the upstream relay. This can be useful if your ISP blocks outbound connections to the standard port 25 (which in my view is a particularly pointless action). If you wish to do this, then the appropriate form of the relayhost line would be:
relayhost = [auth.smtp.upstream.com]:submission
Note that in this case the contents of the saslpw file would have to reflect that form so the file would contain:
[auth.smtp.upstream.com]:submission you@yourdomain.org:weakpassword
We now have sufficient information in our configuration to allow postfix to authenticate to our upstream and send mail through it. You could test this now by reloading postfix and sending an email through postfix to yourself, but doing so would send your credentials across the net in clear. This may be fine for you if that is what you do normally with your POP3 email client, but I prefer to encrypt my credentials with TLS. I don’t want the world and its dog knowing my passwords. So we come to the next stage, which is to enable TLS.
Here we are going to enable postfix to connect to the upstream server over an SSL (TLS) encrypted connection. This will protect our credentials in transit. It also allows us to gain some (small) confidence that we are actually talking to our genuine upstream server because we will only talk to the server which presents us with a signed certificate from a trusted third party. Ironically, the upstream server does not care that the certificate we use locally is created by us because it does not rely on the certificate for authentication.
Again the relevant postfix main.cf configuration lines are:
# TLS parameters
#
# where we get our entropy
tls_random_source = dev:/dev/urandom
# first server side details – i.e. when we accept TLS connections – not needed so
smtpd_use_tls=no
# now the important bit – our client TLS details for the upstream
smtp_use_tls = yes
#
# note that the cert below was generated locally using openssl thusly:
# “openssl req -new -x509 -nodes -out smtp-client.pem -keyout smtp-client.pem -days 10000”
#
smtp_tls_cert_file = /etc/postfix/tls/smtp-client-cert.pem
# use the cert file as the key file as well
smtp_tls_key_file = $smtp_tls_cert_file
# now list the certs of the servers we trust (actually, we only need the Thawte cert used by our upstream).
smtp_tls_CAfile = /etc/postfix/tls/cacert.pem
# where is the cache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
# increase the logging
smtp_tls_loglevel = 2
# and set the per site details (force TLS with specific hosts)
smtp_tls_per_site = hash:/etc/postfix/tls/tls_sites
#
# end TLS
Now we need to create our own local certificate and key file and store it in /etc/postfix/tls/smtp-client-cert.pem (the actual name doesn’t matter, just make sure that the reference in the postfix main.cf points to the correct file). There are a variety of different ways of creating this file, I chose to use openssl in the form of the single command:
openssl req -new -x509 -nodes -out smtp-client.pem -keyout smtp-client.pem -days 10000
You will be asked to answer a few questions to provide details to be included in the certicate (Country code, state, locality, organisation, name, email adddress etc). Give answers which make sense to you – the exact details do not really matter because you will not be submitting the certificate to a third party for verification. What does matter, is that we get hold of, and store locally, a copy of the certificate issued by the certificate authority which verifies the authenticity of our upstream server. This certificate needs to be stored in our smtp_tls_CAfile. There are two steps here. Firstly we need to find out which CA our upstream uses (Thawte, Verisign or whatever), then we need to get hold of the CA’s cert and store it for verification of the upstream certificate. This is not too difficult.
There are a couple of ways of finding out which CA our upstream uses. I found that the easiest was simply to run wireshark (aka ethereal) whilst sending a test email and then examining the exchange. Go to the start of the exchange, right click then “follow tcp stream” and wireshark will pop up a window showing the entire SMTP exchange. The bit we are interested in will be after the SMTP handshake and the “STARTTLS” command. You should see some TLS details then some clear text stating the CA name and its certificate (in my case I found “Thawte” and “Thawte Premium Server CA”). If you don’t have wireshark, or don’t feel comfortable using it, then try an alternative method using openssl to connect. This method actually connects to the POP3 server rather than the SMTP server, but here I am assuming that the upstream provider will use the same CA for all its certificates.
Try the following command:
openssl s_client -connect pop.provider.co.uk:995 -cert /etc/postfix/tls/smtp-client-cert.pem
where pop.provider.co.uk:995 should be the name and port number used by your POP client to connect (POP3 uses port 110, POP3S – SSL encrypted – uses port 995) and the -cert switch points to the client certificate file you created in the step above.
This should result in a connection to the server which will give you details of the CA used. You need to look for the section which starts “Certificate chain” which will look something like this:
Certificate chain
0 s:/C=GB/ST=England/L=wherever/O=ISP Ltd/CN=pop.provider.co.uk
i:/C=ZA/ST=Western Cape/L=Cape Town/O=Thawte Consulting cc/OU=Certification Services Division/CN=Thawte Premium Server CA/emailAddress=premium-server@thawte.com
which again tells you that the server uses Thawte’s Premium Server CA services.
Having established that Thawte is the CA, we need to get hold of a copy of the appropriate certificate. This can be obtained from Thawte itself. You will be asked to provide some details about yourself and will need to agree to Thawte’s licence terms before being allowed to download a zip of their certificates. Once you have that file, you need to extract a copy of the relevant certificate (in this case, a copy of the “Thawte Premium Server CA” which can be found in the file called “ThawtePremiumServerCA_b64.txt”).
The actual certificate looks like this:
—–BEGIN CERTIFICATE—–
MIIDJzCCApCgAwIBAgIBATANBgkqhkiG9w0BAQQFADCBzjELMAkGA1UEBhMCWkEx
FTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMR0wGwYD
VQQKExRUaGF3dGUgQ29uc3VsdGluZyBjYzEoMCYGA1UECxMfQ2VydGlmaWNhdGlv
biBTZXJ2aWNlcyBEaXZpc2lvbjEhMB8GA1UEAxMYVGhhd3RlIFByZW1pdW0gU2Vy
dmVyIENBMSgwJgYJKoZIhvcNAQkBFhlwcmVtaXVtLXNlcnZlckB0aGF3dGUuY29t
MB4XDTk2MDgwMTAwMDAwMFoXDTIwMTIzMTIzNTk1OVowgc4xCzAJBgNVBAYTAlpB
MRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEdMBsG
A1UEChMUVGhhd3RlIENvbnN1bHRpbmcgY2MxKDAmBgNVBAsTH0NlcnRpZmljYXRp
b24gU2VydmljZXMgRGl2aXNpb24xITAfBgNVBAMTGFRoYXd0ZSBQcmVtaXVtIFNl
cnZlciBDQTEoMCYGCSqGSIb3DQEJARYZcHJlbWl1bS1zZXJ2ZXJAdGhhd3RlLmNv
bTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA0jY2aovXwlue2oFBYo847kkE
VdbQ7xwblRZH7xhINTpS9CtqBo87L+pW46+GjZ4X9560ZXUCTe/LCaIhUdib0GfQ
ug2SBhRz1JPLlyoAnFxODLz6FVL88kRu2hFKbgifLy3j+ao6hnO2RlNYyIkFvYMR
uHM/qgeN9EJN50CdHDcCAwEAAaMTMBEwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG
9w0BAQQFAAOBgQAmSCwWwlj66BZ0DKqqX1Q/8tfJeGBeXm43YyJ3Nn6yF8Q0ufUI
hfzJATj/Tb7yFkJD57taRvvBxhEf8UqwKEbJw8RCfbz6q1lu1bdRiBHjpIUZa4JM
pAwSremkrj/xw0llmozFyD4lt5SZu5IycQfwhl7tUCemDaYj+bvLpgcUQg==
—–END CERTIFICATE—–
We now simply need to create the file /etc/postfix/tls/cacert.pem with this certificate as its contents.
It is perfectly possible to force postfix to use TLS for any and all remote mail servers. The configuration file line would look like this:
smtp_enforce_tls = yes
But I prefer to give myself the flexibility to use TLS only with specified servers, which is where the “smtp_tls_per_site = hash:/etc/postfix/tls/tls_sites” line comes in useful. This configuration option points to a text file which contains details of which form of authentication to use with which server (see the postfix configuration documentation for details). Using this form of configuration gives us the ability to limit connection to a server to a form which: requires TLS encryption; requires that the remote SMTP server hostname matches the information in the remote SMTP server certificate; and also requires that the remote SMTP server certificate was issued by a trusted CA. This is the bit that gives me the extra confidence that I may actually be talking to the appropriate upstream when I hand it my mail.
An appropriate tls_sites file to be stored in the tls subdirectory would look like this:
auth.smtp.upstream.com MUST
some-server-with-no-tls.provider.com NONE
where the keyword “MUST” forces TLS and also forces the requirements we stipulate above. If we don’t care whether the remote SMTP server hostname matches the information in the remote SMTP server certificate, or that the server certificate was issued by a trusted CA, then we could use the keyword “MUST_NOPEERMATCH”.
Again. once we have created the text file we must create the db hash by running the command “postmap tls_sites“.
We can now test the server by sending some mail through it and watching the log file for any errors. Fire up a couple of terminal sessions and run a tail -f /var/log/mail.info in one. In the other, try sending mail manually as follows. Note that lines in bold are what you type, everything else is a server response.
telnet localhost 25
Trying 127.0.0.1…
Connected to localhost.
Escape character is ‘^]’.
220 smtp.ourhostname.org ESMTP – whatever your banner says.
mail from: you@yourdomain.org
250 2.1.0 Ok
rcpt to: your-test-recipient@yourdomain.org
250 2.1.5 Ok
data
354 End data with
Subject: Test message
Hello world.
.
250 2.0.0 Ok: queued as 204868BBAA
quit
221 2.0.0 Bye
If everything works, you should see a bunch of details in the logfile describing a successful connection. But in particular we want to see a line which shows that we have successfully connnected to the correctly authenticated upstream server. It will look something like:
[details omitted]
Jul 16 21:23:08 host postfix/smtp[3526]: Verified TLS connection established to auth.smtp.upstream.com [111.222.111.222]:25: TLSv1 with cipher AES256-SHA (256/256 bits)
[further details omitted]
If you have problems here you may need to add some sasl modules. I initially had authentication failures with the error message “SASL authentication failure: No worthy mechs found”. This is caused by the omission of some required sasl libraries. The cure (in debian) is to run “apt-get install libsasl2-modules” then try again.
Finally there are some postfix configuration options which are necessary to protect our MTA when we open it to the world at Step 4 below. Firstly, limit the networks which are are allowed to relay mail through Postfix to our local net. Again, the postfix documentation may be useful here. An appropriate configuration limiting trust to localhost and the RFC1918 network 192.168.1.0 would would look something like this:
mynetworks = 127.0.0.0/8, 192.168.1.0/24
Failure to do this properly could leave you operating an open relay which will make you very unpopular. And you will be astonished at how quickly your mailer will receive unwanted attention.
Now only listen on localhost and specific local network address
inet_interfaces = $myhostname, localhost
And since we are running on a host behind a DSL NAT firewall (you are running a NAT firewall aren’t you?) we need to specify the external IP address which will be visible to the outside world:
proxy_interfaces = 123.123.123.123
Note here that it is perfectly possible (and indeed may be preferable) to specify a symbolic hostname in place of the IP address. So if your ISP does not give you a fixed IP address for your NAT firewall you can still get this to work by subscribing to a Dynamic DNS service such as DynDNS and entering your dynamic dns hostname here. So, if your dynamic hostname were mypostfix.homelinux.net then your config line would be
proxy_interfaces = mypostfix.homelinux.net
The next stage is to get postfix configured for mailman. This leads us to Step 2.