My mail setup is postfix/dovecot with SASL authentication and TLS protecting the mail authentication exchange. My X509 certs are self generated (and so not signed by any CA). I pick up mail over IMAPS (when mobile) and POP3S (at home – for perverse reasons of history I like to actually download mail to my main desktop over POP3 and archive it to two separate NAS backups). I send via the standard SMTP port 25 but require authentication and protect the exchange with TLS.

My mail had been working fine ever since I set it up some years ago, but as I said, back in October my wife complained that she could no longer send email from her HTC mobile (we both use t-mobile as the network provider). She was at work at the time so away from my home network. Both our phones are setup to use use wifi for connectivity where it is available (as it is at home of course). When my wife complained I checked my phone and it could send and receive without problem. But when I switched wifi off, thus forcing the data connection though the mobile network, I got the same problem as my wife reported. On checking my mail server logs I read this:

postfix/smtpd[28089]: connect from unknown[149.254.186.120]
postfix/smtpd[28089]: warning: network_biopair_interop: error reading 11 bytes from the network: Connection reset by peer
postfix/smtpd[28089]: SSL_accept error from unknown[149.254.186.120]:-1
postfix/smtpd[28089]: lost connection after STARTTLS from unknown[149.254.186.120]
postfix/smtpd[28089]: disconnect from unknown[149.254.186.120]

(the ip address is one of t-mobile’s servers on their “TMUK-WBR-N2″ network)

Everything I could find about that sort of message suggested that the client was tearing down the connection because there was something wrong with the TLS handshake and it was not trusted. Checking earlier logs, I found that t-mobile’s address had apparently changed (to the address above) recently. So I assumed that some recent network change following the Orange/T-mobile merger had been badly managed and all would be well again as soon as the problem was spotted. Wrong. It persisted. So I had to investigate further. As part of my investigation of the error, I tried moving mail from port 25 to 587 (submission) because that sometimes gets around the problem of ISPs blocking, or otherwise interfering, with outbound connections from their networks to port 25, No deal. In fact it looked as if t-mobile were blocking all connections to port 587 (I assumed a whitelisting policy block, or again, a cockup).

So, the scenario was: mail works when connecting over wifi and using my domestic ISP’s network, but doesn’t when using t-mobile’s 3G network. Symptoms point to a lack of trust in the TLS handshake. Tentative conclusion? There is an SSL/TLS proxy somewhere in the mobile operator’s chain. That proxy sucessfully negotiates with our phones, but when it gets my self certified X509 cert from the server. it can’t authenticate it and decides that the connection is untrusted so tears it down. My server sees this as the client (my phone) tearing down the connection. [As it turns out, this conclusion was completely wrong, but hey].

I said in an email at the time to a friend whose advice I was seeking, “I suspect cockup rather than outright conspiracy, but if my telco is dumb enough to stick a MITM ssl proxy in my mail chain, they really ought to have thought about handling self signed certs a little better. Otherwise it sort of gives the game away.”

In response, he very sensibly suggested that I should run a sniffer on the server and check what was going on. At that time, I was busy doing something else so I didn’t. And because the problem was intermittent (and my wife stopped complaining) I never got around to properly investigating further. (I should explain that I rarely send mail from my mobile nowadays. I just read mail there and wait until I get home to a decent keyboard and can reply to whatever needs handling from there. My wife just gave up bothering to try).

I should have persisted because of course I wasn’t the only one to experience this problem.

Back in November, a member of the t-mobile discussion forum called “dpg” posted a message complaining that he could not connect to port 587 over t-mobile’s 3G network. In response, a member of the t-mobile forum team suggested that dpg might reconfigure his email so that it was relayed via t-mobile’s own SMTP server. Not unreasonably, dpg didn’t think this was an acceptable response – not least because he would then have to send his email in clear. He then posted again saying that “the TLS handshake fails when the mail client receives a TCP packet with the reset (RST) flag set.” (This is a bad thing (TM). Further, he posted again saying that he had set up his own mail server and repeated earlier tests so that he could see both ends of the connection. At the client side he posted mail from his laptop tethered to his phone which was connected to the t-mobile 3G network. By running sniffers at both ends of the connection he was able to prove to his own satisfaction that something in the t-mobile network was sending a RST and tearing down any connection when a STARTTLS was seen. Again, in a later post in response to one from another poster who apparently manages several mail servers and had been looking at the same issue for a client, dpg says:

“I must say I’m not too pleased to discover that T-Mobile may be snooping all traffic to check for SMTP messages. I have demonstrated that they may be doing this by running a SMTP server on a non-standard port and finding that they still sent TCP reset packets during TLS negotiation – so they must be examining all packets and not just those destined for TCP ports 25 and 587.

I’m also not that keen on T-Mobile spoofing/forging TCP resets. This is the sort of tactic resorted to by the Great Firewall of China (http://www.lightbluetouchpaper.org/2006/06/27/ignoring-the-great-firewall-of-china/) and also by Comcast back in 2007 (https://www.eff.org/wp/packet-forgery-isps-report-comcast-affair) until the US FCC told them to stop (http://hraunfoss.fcc.gov/edocs_public/attachmatch/FCC-08-183A1.pdf).”

Then 9 days ago, dpg posted this message:

“I finally got to the bottom of this. I was contacted by T-Mobile technical support today and was told that they are now actively looking for and blocking any TLS-secured SMTP sessions. So, it is a deliberate policy after all, despite what the support staff have been saying on here, twitter and on 150. They told me it is something they have been rolling out over the last three months – which explains why it was intermittent and dependent on IP address and APN to begin with.

So, the only options for sending email over T-Mobile’s network are:
- unencrypted but authenticated SMTP (usually on port 25)
- SSL-encrypted SMTP (usually on port 465)
- unauthenticated and unencrypted email to smtp.t-email.co.uk

TLS-encrypted SMTP sessions are always blocked whether or not they are on the default port of 587.”

(As an aside, there is, of course, another alternative. You can ditch t-mobile as your provider and pick one which doesn’t use DPI to screw your connections. You pays your money….)

Following this, a new poster called “mickeyc” said this:

“I’ve been experiencing this exact same problem. I run my own mail server which has SSL on port 465 and also uses TLS on port 587. I used wireshark to confirm that the RST packets are being spoofed. This is the exact same technology used by “The Great Firewall of China”. I have two t-mobile sims. One is about a year old and doesn’t experience this problem (yet), one is a few weeks old and does.”

He went to say that he had also experienced problems with his OpenVPN connections and would be blogging about the problem (damned bloggers get everywhere) and sure enough, Mike Cardwell did so at grepular.com. That blog post is worth reading because it has an interesting set of comments and responses from Mike appended.

Mike’s post seems to have been picked up by a few others (El Reg has one, and as Mike himself has pointed out, boingboing.net has a particularly OTT post which seems to say that he is accusing t-mobile of something he clearly isn’t.

Finally, two days ago, dpg posted this:

“I’m pleased to report that T-Mobile is no longer blocking TLS-secured email on port 587. As a follow-up to an email exchange over the Christmas period I was contacted today to say that, contrary to what I had been told previously, it was never a deliberate policy to block TLS-secured outgoing email. There was a problem with some equipment after all, which was resolved yesterday.”

I tried again myself today. Initially, I got the same old symptoms (“lost connection after STARTTLS”) then I rebooted my ‘phone and lo and behold I could send email.

Like Mike, I tend to the cockup over conspiracy theory, it’s more likely for one thing. IANAL, but it seems to me that it would be in breach of RIPA part I, Unlawful Interception, for the telco to intercept my SMTP traffic in the way it seems to have been doing. That is not likely to be a deliberate act by a major UK mobile network provider.

But I’ll still keep an eye on things.

Firstly, X is not exactly a secure protocol, nor is it easy to filter at NAT firewalls, so the ability to tunnel it over ssh is hugely welcome. In fact, ssh can be used to tunnel practically any other protocol you care to name, so it should be your first port of call should you wish to connect to a remote system using an insecure protocol. (I use it to wrap rsync for example).

I don’t run X on my VMs (there is no need, they don’t run desktop software) and I had not previously seen the need to run X based graphical programs on those servers. However, a couple of days ago I thought it would be really useful to run etherape on one particular remote server so that I could watch the traffic patterns. Normally I use iptraf (which is ncurses based) when I want to monitor network traffic in real time, but etherape is pretty cool and gives a nice graphical view of your network connections. But it runs on an X based gui.

So. I changed the remote server’s sshd_config to enable X forwarding (“X11Forwarding no” becomes “X11Forwarding yes”) and restarted sshd. On my desktop I similarly changed my local ssh_config file to allow X forwarding (“ForwardX11 no” becomes “ForwardX11 yes”) to obviate the need to use the -X switch on the command line. I then installed etherape on the remote server and fired it up only to get the message “Error: no display specified”. Sure enough “echo $DISPLAY” showed nothing. But I had thought (and everything I had read confirmed) that ssh should take care of setting the appropriate display when X11 forwarding was set.

So I then tried setting a display manually (export DISPLAY=localhost:10.0 on the remote server) and then got the response “Error: cannot open display: localhost:10.0″. So, still no deal. I spent some time scratching my head (and reading man pages) and sent off a query to my local Linux User group in parallel asking for advice. They were gentle with me.

The first, and rapid, response, said:

On the server:

sudo apt-get install xauth

Then disconnect and reconnect the client.

Jobs a good un.

Thank you Brett.

So the moral is, make sure that you have X authorisation working properly on the remote system (check for the existence of $HOME/.Xauthority) if you experience the same symptoms I did.

But I got phished. Damn.

Here’s what happened.

I advertised an unwanted mobile phone on gumtree. I chose gumtree in preference to ebay because a) adverts are free, and b) gumtree allows you to target the advertising to a specific location. I like this idea because it means you can say “I’ve got a doohickey for sale in South London. Come and see it and pay cash if you want it”. My ad gave details of the item for sale and, as is recommended, I chose to have responses emailed to me. Here I made mistake number one – I used my normal email address rather than a disposable one. To be fair, gumtree don’t expose any of your private details, they just forward any responses to the address you give. Here’s where I made mistake number two, I responded to queries about the ad from the address given to gumtree. Damn. Idiot. So stupid.

So why do I think the responses weren’t kosher? Well there were a number of giveaways. Firstly the requests were for information already in the ad (“how much do you want?”); secondly, there were a suspiciously high number of “spilling misteaks” in the emails; thirdly, the correspondent wanted me to mail the ‘phone to a location outside the UK (“Thanks for your quick respond actually i will love to buy the Ad for my Daughter who is currently studying at British international college (BIC) in West Africa so am willing to pay you additional £48.76 for the shipping via Express Air Mail.” (sic)); fourthly, the respondents all seemed desperately keen for me to accept paypal as the preferred payment option. I’m normally quicker on the uptake than this, but sadly it took me four or five emails to realise that there was a pattern here and that the people after the phone seemed to be following a script and were completely ignoring my responses. Here’s a sample:

Someone calling him or her self “Janet Mason”:

“Hello Seller,
Can I know the condition of the item? I think you will accept PayPal. And I will pay the postage and packing cost for the item. If you can send me paypal payment request now and I will make the payment straight away without any delay. Hope to hear from you very soon.”

My response:

“Janet

As the ad say, the phone is in “as new condition”. This means what it says. The phone is completely clean and has no visible markings or scratches.”

“Janet’s” reply:

“Ok send me your paypal Payment request now so that I can make the payment now.”

My response:

“I’d like a bit more detail first please.

Where are you? (Full address and telephone mumber so that I can confirm that I am sending to “Janet Mason”.

Details of your confirmed paypal account (so that I know that Paypal have verifed you).

If you want to know why I am concerned please read the paypal guidance for sellers – particularly the bit about sending only to UK or US based addresses and getting signatures on receipt of goods.

I have received several requests to send the phone to “my daughter/son/nephew” or whatever in various Countries outside the EU. I am naturally suspicious.”

“Janet” then says:

“Hello, 
Am in London right now but due to the nature of my work here in London I will not be able to post the item to my Business Partner Daughter in Nigeria as a New Year Gift. But I will pay for the postage and packing cost via FedEX. Get back to me with your paypal payment request now so that I can make the payment now and get the item posted out tomorrow Morning.”

Correspondence ends…..

Now whilst I have not lost the ‘phone, I have verified a usable email address to a bunch of scammers. I expect my spam volume to that address to increase dramatically. Never mind though, I’m not alone in losing out to the bad guys, and at least I haven’t lost any passwords in the process.

Still, I’m pretty pissed off.

On 29 December last, Matt Mullenweg posted a notice to the wordpress security blog announcing a very important update which he recommnded be applied as soon as possible because it fixes a “core security bug in [wordpress'] HTML sanitation library, KSES”. Mullenweg rated this [3.04] release as “critical.”

I have just updated my installation. I recommend you do the same.

Just wondering.

I have no idea why my earlier experiments failed.

I fund fast exit nodes as a way of paying something back to the community – but as I have pointed out before, they also allow me to have a permanent entry point to the tor network which I can tunnel to over ssh, thus protecting my own tor usage from snoopers. For some time I used a configuration based on tyranix’s notes documented on the tor project wiki. But eventually I found that to be rather limiting because it meant that I had to remember to run an ssh listener on each machine I used around the house (my laptop, netbook, two desktops, my wife’s machine etc) and to configure the browser settings as necessary. Then I hit upon the notion of centralising the ssh listener on one machine (I used the plug) in a sort of ssh proxy configuration. This meant that I only had to configure the local browsers to use the central ssh listener as a proxy and everything else could be left untouched. It also has the distinct advantage that my wife no longer has to worry about anything more complex than switching browser when she wants to use tor.

But I hit a snag when initially setting up ssh on the plug. For some reason (which I have never successfully bottomed out) the ssh process dies after an hour of inactivity. This is not helpful. Enter autossh. Using autossh means that the listener is restarted automagically whenever it dies so I can be confident that my proxy will always be there when I need it.

Here’s the command used on the plug to fire up the proxy:

autossh -M 0 -N -C -g -l user -f -L 192.168.57.200:8000:127.0.0.1:8118 tornode

That says:

- M 0 – turn off monitoring so that autossh will only restart ssh when it dies.
- N – do not execute a command at the remote end (i.e. we are simply tunneling)
- C – compress traffic
- g – allow remote hosts to connect to this listener (I limit this to the local network through iptables on the plug)
- l user – login as this user at the remote end
- f – background the process
- L 192.168.57.200:8000 – listen on port 8000 on the given IP address (rather than the more usual localhost address)
- 127.0.0.1:8118 tornode – and forward the traffic to localhost port 8118 on the remote machine called tornode

Of course “tornode” must be running ssh on a port reachable by the proxy. Again, I use iptables on tornode to limit ssh connections to my fixed IP address – don’t want random bad guys knocking on the door.

Now on tornode I have polipo listening on port 8118 on localhost. I used to use privoxy for this, but I have found polipo to be much faster, and speed matters when you are using tor. My polipo configuration forwards its traffic to the tor socks listener on localhost 9050. I also disabled the local polipo cache (diskCacheRoot = “”) because leaving it enabled means that the cache (by default /var/cache/polipo) directory will contain a copy of your browsed destinations in an easily identifable form – not smart if you really want your browsing to be anonymous (besides, my wife deserves as much privacy as do I).

The final bit of configuration needed is simple. Set your chosen browser to use the proxy on port 8000 on address 192.168.57.200. Since I use firefox for most of my browsing, I simply use opera for tor and I have that browser stripped to its basics and locked down as much as possible. Using tor is then simply a matter of firing up opera in place of firefox. This means that I always know when I am using tor or not (and just to reassure myself, the opera homepage is the torcheck page).

You can’t be too careful.

If you visit the scroogle SSL interface, you get a redirect to a notice which explains why some changes made at google mean that scroogle can no longer work properly. Scroogle managed to get a workaround in place for a few days, but it seems that another google change has finally killed that too unless google can be convinced to help out – unlikely in my view. The scroogle redirect page (dated 1 July 2010) has the following line from Daniel Brandt:

“Thank you for your support during these past five years. Check back in a week or so; if we don’t hear from Google by next week, I think we can all assume that Google would rather have no Scroogle, and no privacy for searchers.”

That in itself is bad enough, but as a separate new posting explains, scroogle now seems to be the target of a botnet aimed at swamping its servers. As Brandt goes on to say:

“Google has a few hundred thousand servers, while Scroogle has six. They can put up with sites that spread malware, but our bandwidth is limited. Even if Google relents and the output=ie interface returns, this Scroogle malware problem could still be increasing at that point. Eventually it alone might shut down Scroogle.”

Sad. I hate to see the little guy lose out.

In early January we discovered that two of the seven directory authorities were compromised (moria1 and gabelmoo), along with metrics.torproject.org, a new server we’d recently set up to serve metrics data and graphs. The three servers have since been reinstalled with service migrated to other servers.

Whilst the direrctory servers apparently also hosted the tor project’s svn and git source code repositories, Dingledine is confident that the source code has not been tampered with – and nor has there been any possible compromise of user anonymity. Neverthless, the project recommends that tor users and operators upgrade to the latest version. Good advice I’d say – I’ve just upgraded mine.