Archive for the ‘net tools’ Category

moxie’s proxy

Sunday, January 22nd, 2012

Moxie Marlinspike, a security researcher probably best known for his SSL proxy tool, likes google even less than I do. His googlesharing website says:

“Google thrives where privacy does not. If you’re like most internet users, Google knows more about you than you might be comfortable with. Whether you were logged in to a Google account or not, they know everything you’ve ever searched for, what search results you clicked on, what news you read, and every place you’ve ever gotten directions to. Most of the time, thanks to things like Google Analytics, they even know which websites you visited that you didn’t reach through Google. If you use Gmail, they know the content of every email you’ve ever sent or received, whether you’ve deleted it or not.

They know who your friends are, where you live, where you work, and where you spend your free time. They know about your health, your love life, and your political leanings. These days they are even branching out into collecting your realtime GPS location and your DNS lookups. In short, not only do they know a lot about what you’re doing, they also have significant insight into what you’re thinking.”

His solution to this problem was interesting. He came up with the idea of a proxy system which would intercept all google queries, strip off identifying material (such as cookies and UserAgent strings and other HTTP headers) substitute new identifiers and mix the requests up with those from other users before forwarding to google. Implementation depended upon a Firefox addon (nothing for other browsers) which identified google queries and forwarded them to the proxy. All other traffic was untouched.

image of googlesharing proxy

I stopped using google (except via scoogle) some time ago, and when Moxie’s new proxy first surfaced I thought it interesting but susceptible to the same problem I discussed in mid 2009 when writing about Hal Roberts’ experience of GIFC – all you are doing is shifting knowledge of your searches from google to a new intermediary. However, Moxie later addressed this problem with the release of version 0.20 of his addon so I thought I’d take another look at it. Unfortunately the addon won’t work with FF 9 (which I am using). Moxie’s proxy is not the only one out there however. Because he released the code under an open source licence, others have picked it up. I found one at gs.netsend.nl. They also provide an updated FF addon which will work with versions up to 15 (i.e. probably around next wednesday given the speed with which Mozilla is currently shipping new FF releases).

Once the addon is installed, it gives you two proxy options in the preferences settings – one is the original proxy.googlesharing.net, the other is gs.netsend.nl itself. In testing I found that the original googlesharing proxy seemed to be off-line, but when using the netsend.nl proxy I was reassured to see the message “Search results anonymized by GoogleSharing” added to the google homepage. I was even more reassured that my sniffer showed a connection to vps1101.pcextreme.nl on 31.21.98.201 and not to any known google network.

So, will I use it? Maybe. But the proxy mechanism seems to be unreliable. In many tests, the proxy connection seemed to be bypassed and the connection was obviously made direct to google (as evidenced by my sniffer). I think this failure is doubly unfortunate because it does not fail safe (i.e. the connection does not simply fail with an error message, it passes you direct through to google). This could lead the unwary to think that they are protected when in fact they are not.

I prefer not to use google at all. And in those cases where I do want to compare results with another search engine I prefer to do so via tor. But it is one more option in my toolkit if used carefully. And if using it pisses off google, then it is worth it occasionally.

tails in a spin

Thursday, January 12th, 2012

When I first tested running a tails mirror on one of my VMs, the traffic level reported by vnstat ran at around 20-30 GiB per day. I figured I could live with that because it meant that my total monthly traffic would be unlikely to exceed my monthly 1TB allowance. However, when I checked the stats on that server last week (around the 9th of Jan) I found that I was shipping out around 150 GiB per day and vnstat was predicting a monthly total of close to 3 TB. As the tails admins said when I told them that I would have to shut off the mirror on that VM while I sorted something, “Ooops”. Ooops indeed. I couldn’t chance a massive bill for exceeding my bandwidth allowance by quite that much. The actual stats for 4, 5, 6, 7, 8 and 9 January before I pulled the plug were: 34.23 GiB, 69.14 GiB, 178.31 GiB, 131.68 GiB, 99.05 GiB and 133.27 Gib. It turns out that tails 0.10 was released on 4 January and I hadn’t been prepared. A lesson learned.

Having shut down and had the DNS round robin amended, I attended to finding some way of throttling my traffic so that I could live within my allowance whilst still providing a useful mirror. I scratched my head for a while before stumbling on the obvious, I should be throttling at application level. (Sometimes I find that I miss simple answers because I am looking for complicated ones).

I started out by assuming that I should be using tc and iptables mangling, or something like the userspace tool trickle, all of which looked horribly more complicated than the approach taken by tor (which allows you to simply set the acceptable bandwidth rate to some limit, plus set an accounting period maximum of some total transfer limit per day/week whatever). And of course it turns out that my webserver (lighttpd) allows something similar. Just set the server limit to some chosen max transfer rate and, if necessary, also impose a per IP max rate. The magic configuration file options are:

# limit server throughput to 3000 kbytes/sec (~30000 kbits/sec)
server.kbytes-per-second = 3000
#
# and limit individual connections to 50 kbytes (~500 kbits/sec) – NB. I don’t actually use this
# connection.kbytes-per-second = 50

I tested this by pulling a copy of the tails iso from one of my other VMs which has a high bandwidth connection and got acceptable (and expected) results. So now I can go back on-line later this month safe in the knowledge that I’m not going to blow all my bandwidth in one week.