I have been a subscriber to Hanno Böck’s Feisty Duck TLS Newsletter for some time. Böck’s newsletters provide a useful service to TLS users. I am also a big fan of Ivan Ristić’s “Openssl cookbook” which is available as a free download from the Feistyduck website.
A couple of days ago the latest Feistyduck newsletter hit my email inbox. That newsletter pointed to a paper published at the USENIX security conference which detailed a large number of vulnerabilities in STARTTLS implementations.
STARTTLS is a mechanism that allows for upgrading plaintext protocol connections to TLS. The research focused on STARTTLS in the communication between email clients and servers (SMTP, IMAP, POP3). Apparently, it turns out this upgrading mechanism is fragile and can lead to a number of security problems.
TLS is used widely to protect what used to be predominantly plaintext data transmissions. It is probably most commonly seen in webservers. HTTP (the protocol used to transport web data) was originally only provided in plaintext over port 80. With the introduction of SSL (and later TLS) web traffic moved mainly (but still not entirely) over to encrypted transmission on port 443. This is a good example of the way encryption is currently implemented on the ‘net – plaintext is offered on one port (in this case 80) whilst encrypted traffic is offered on an entirely separate port (in this case 443). However, there are many “legacy” protocols in use which have traditionally been offered on well known, and standardised, ports. A good example of this is SMTP mail transmission on port 25.
The SMTP protocol was first defined by Jon Postel back in 1982 in RFC 821. As well as defining the transactions involved in the protocol, that RFC also specified port 25 as the standard port to be used in the network. Appendix A of the RFC says:
“The SMTP transmission channel is a TCP connection established between the sender process port U and the receiver process port L. This single full duplex connection is used as the transmission channel. This protocol is assigned the service port 25 (31 octal), that is L=25.”
Port 25 has been reserved for email transmission ever since.
Back in 1982, all TCP connections were in plaintext. The world was a much simpler, and more trusting place then than is now the case. SMTP does not even have to be authenticated (i.e. you do not have to specify a user identity or password) in order to pass email to a mail server. As RFC 2821 from 2001 (which obsoleted the original RFC821) notes:
“SMTP mail is inherently insecure in that it is feasible for even fairly casual users to negotiate directly with receiving and relaying SMTP servers and create messages that will trick a naive recipient into believing that they came from somewhere else. Constructing such a message so that the “spoofed” behavior cannot be detected by an expert is somewhat more difficult, but not sufficiently so as to be a deterrent to someone who is determined and knowledgeable. Consequently, as knowledge of Internet mail increases, so does the knowledge that SMTP mail inherently cannot be authenticated, or integrity checks provided, at the transport level. Real mail security lies only in end-to-end methods involving the message bodies, such as those which use digital signatures.”
it even goes on to say:
“This specification does not further address the authentication issues associated with SMTP other than to advocate that useful functionality not be disabled in the hope of providing some small margin of protection against an ignorant user who is trying to fake mail.”
The SMTP protocol makes no distinction between mail from another MTA (i.e. mail coming from a sending server and mail from an MUA i.e. mail from a client system typically used directly by an end user such as Thunderbird or Claws on a Linux system, K9 or Gmail on an Android mobile phone, iOS mail on Apple or Outlook on MS Systems. Worse, email can now be sent/received using (shudder) a web browser if the mail provider offers a webmail interface (a true bastardisation of the standards if ever there was one. HTML email is an abomination that is probably responsible for more security problems than any other form of transaction on the ‘net).
So, it is theoretically possible (if in practice somewhat difficult these days) to send mail directly from a desktop client through port 25 on an MTA with no authentication and no encryption of the connection. In practice however, connections to port 25 are reserved for inter MTA (i.e. mail server to mail server) transactions where no authentication is required. Since RFC 2487 in 1999 encryption has been encouraged by the use of STARTTLS extension to the transaction but because even now, not all MTAs support that, any receiving MTA MUST fall back to an unencrypted connection if mail is to be successfully delivered. RFC 2487 (and its successor RFC 3207) specifically state:
“A publicly-referenced SMTP server MUST NOT require use of the STARTTLS extension in order to deliver mail locally. This rule prevents the STARTTLS extension from damaging the interoperability of the Internet’s SMTP infrastructure. A publicly-referenced SMTP server is an SMTP server which runs on port 25 of an Internet host listed in the MX record (or A record if an MX record is not present) for the domain name on the right hand side of an Internet mail address.”
STARTTLS is therefore seen as “opportunistic” rather than mandatory encryption. Personally I’d like to see a new RFC which mandates TLS encryption on all public mail servers. The fact that it is not means that I cannot be sure that my mail is not transmitted in clear at some stage even if I have ensured it is encrypted between my desktop (or ‘phone) and my own mailserver.
As long ago as 1998, it was recognised that there was some conflict between the requirements of message transfer (i.e. transmission of a complete message across the internet) and message submission (i.e. introduction of a new message from a client into the network). This led to RFC 2476 which says:
“SMTP was defined as a message *transfer* protocol, that is, a means to route (if needed) and deliver finished (complete) messages. Message Transfer Agents (MTAs) are not supposed to alter the message text, except to add ‘Received’, ‘Return-Path’, and other header fields as required by [SMTP-MTA].
However, SMTP is now also widely used as a message *submission* protocol, that is, a means for message user agents (MUAs) to introduce new messages into the MTA routing network. The process which accepts message submissions from MUAs is termed a Message Submission Agent (MSA).”
That RFC went on to say:
“Separating messages into submissions and transfers allows developers and network administrators to more easily:
* Implement security policies and guard against unauthorized mail relaying or injection of unsolicited bulk mail.
* Implement authenticated submission, including off-site submission by authorized users such as travelers.
* Separate the relevant software code differences, thereby making each code base more straightforward and allowing for different programs for relay and submission.
* Detect configuration problems with a site’s mail clients.
* Provide a basis for adding enhanced submission services in the future.”
Port 587 was reserved for mail submission by this RFC and that port is widely used for that purpose to this day. As noted above, RFC3207 introduced the SMTP extension to offer STARTTLS on port 25. That RFC also noted that:
“STARTTLS is a valid ESMTP extension when used on the Submission port, as defined in [RFC2476]. In fact, since the submission port is by definition not a publicly referenced SMTP server, the STARTTLS extension can be particularly useful by providing security and authentication for this service.”
So, as is widely the case across the ‘net today, my email server has been configured to allow only MTA to MTA connections on port 25 (but offering and NOT requiring encryption through use of STARTTLS “smtpd_tls_security_level = may” in postfix’s main.cf configuration file). Authenticated (using SASL) connections from MUAs have hitherto been accepted on port 587 offering STARTTLS. I insist on authentication only being accepted over an encrypted channel though (“smtpd_tls_auth_only = yes” in main.cf). Unfortunately, the default in postfix is to allow authorisation in clear if “smtpd_tls_security_level = may” is set (as is required by the SMTP RFCs. The postfix manual says:
“Sending AUTH data over an unencrypted channel poses a security risk. When TLS layer encryption is required (“smtpd_tls_security_level = encrypt”), the Postfix SMTP server will announce and accept AUTH only after the TLS layer has been activated with STARTTLS. When TLS layer encryption is optional (“smtpd_tls_security_level = may”), it may however still be useful to only offer AUTH when TLS is active. To maintain compatibility with non-TLS clients, the default is to accept AUTH without encryption. In order to change this behavior, set “smtpd_tls_auth_only = yes”.”
Given the vulnerabilities outlined above however, I have now decided to drop STARTTLS enirely from my mail submission configuration and ONLY offer it on MTA to MTA transactions. Fortunately this is simple if I move submission to port 465 (SMTPS) and insist on TLS connections before any transaction can take place.
In postfix this is accomplished by modifications to the “master.cf” configuration file. Options to the specified daemons (e.g. smtpd) here override the defaults set in the “main.cf” configuration. The relevant portion of my master.cf now looks like this:
#
# Postfix master process configuration file. For details on the format
# of the file, see the master(5) manual page (command: “man 5 master” or
# on-line: http://www.postfix.org/master.5.html).
#
# Do not forget to execute “postfix reload” after editing this file.
#
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (no) (never) (100)
# ==========================================================================
smtp inet n – y – – smtpd
# changed by MBM to stop SASL AUTH on port 25 and only allow it on 587
# See also main.cf
-o smtpd_sasl_auth_enable=no
#
#smtp inet n – y – 1 postscreen
#smtpd pass – – y – – smtpd
#dnsblog unix – – y – 0 dnsblog
#tlsproxy unix – – y – 0 tlsproxy
# Submission on port 587 activated by MBM
# But DISABLED on 31 August 2021 following STARTTLS vulnerability disclosure.
# See SMTPS below
#submission inet n – y – – smtpd
# -o syslog_name=postfix/submission
# -o smtpd_tls_security_level=encrypt
# SMTPS (on port 465) enabled by MBM and STARTTLS on submission disabled on
# 31 August 2021 after STARTTLS vulnerability disclosure
# See https://nostarttls.secvuln.info/
smtps inet n – y – – smtpd
-o syslog_name=postfix/smtps
-o smtpd_tls_wrappermode=yes
-o smtpd_tls_security_level=encrypt
# end SMTPS config
It is worth noting here that RFC8314 (“Cleartext Considered Obsolete: Use of Transport Layer Security (TLS) for Email Submission and Access”) from 2018 said:
“This specification outlines current recommendations for the use of Transport Layer Security (TLS) to provide confidentiality of email traffic between a Mail User Agent (MUA) and a Mail Submission Server or Mail Access Server.”
This RFC (which until this recent problem arose I had entirely missed) was clearly aimed at getting mail admins to move away from offering opportunistic encryption and on to mandatory encryption on the submission ports.
That RFC says at section 3.3
“When a TCP connection is established for the “submissions” service (default port 465), a TLS handshake begins immediately. Clients MUST implement the certificate validation mechanism described in [RFC7817]. Once the TLS session is established, Message Submission protocol data [RFC6409] is exchanged as TLS application data for the remainder of the TCP connection. (Note: The “submissions” service name is defined in Section 7.3 of this document and follows the usual convention that the name of a service layered on top of Implicit TLS consists of the name of the service as used without TLS, with an “s” appended.)
The STARTTLS mechanism on port 587 is relatively widely deployed due to the situation with port 465 (discussed in Section 7.3). This differs from IMAP and POP services where Implicit TLS is more widely deployed on servers than STARTTLS. It is desirable to migrate core protocols used by MUA software to Implicit TLS over time, for consistency as well as for the additional reasons discussed in Appendix A. However, to maximize the use of encryption for submission, it is desirable to support both mechanisms for Message Submission over TLS for a transition period of several years. As a result, clients and servers SHOULD implement both STARTTLS on port 587 and Implicit TLS on port 465 for this transition period.
Note that there is no significant difference between the security properties of STARTTLS on port 587 and Implicit TLS on port 465 if the implementations are correct and if both the client and the server are configured to require successful negotiation of TLS prior to Message Submission.”
Back in the day when I managed internet systems for a living I’d like to think that I would have been on top of current thinking as outlined in RFC8314. These days I am a (long retired) hobbyist managing only my own systems and I no longer automatically read all CERT alerts so I forgive myself for missing this recommendation. But I hope that current professional mail admins have NOT missed this.
I strongly recommend readers of this post take a look at Appendix A of RFC8314 for a good discussion of the reasoning behind the decision to prefer implicit TLS in all email transactions.