Archive for the ‘tips, tricks and howtos’ Category

plug instability

Sunday, March 7th, 2010

I’m still having a variety of problems with my sheevaplug. Not least of which is the fact that SDHC cards don’t seem to be the best choice of boot medium. I have had failures with two cards now and some searching of the various on-line fora suggests that I am not alone here. In particular, SD cards seem to suffer badly under the read/write load that is routine for an OS writing log files – let alone one running a file or web server. I have also had several failures with my external USB drive. It seems that the plug boots too quickly for the USB subsystem to initialise properly. This means that there is not enough time for the relevant device file (/dev/sda1 in my case) to appear before /etc/fstab is read to mount the drive. A posting on the plugcomputer.org forum suggested a useful workaround (essentially introducing a wait), but even that was only partially sucessful. Sometimes it worked, sometimes it didn’t. In fact, the USB drive still often fails after a random (and short) time and then remounts read-only. Attempts to then remount the drive manually (after a umount) result in failure with the error message “mount: special device /dev/sda1 does not exist”.

In my attempts to cure both the booting problems and the USB connection failures I have installed the latest uboot (3.4.27 with pingtoo patches linked to from Martin’s site) and updated my lenny kernel to Martin’s 2.6.32-2-kirkwood in the (vain as it turns out) hope that the latest software would help. Here I also discovered another annoying problem – installing the latest kernel does not result in a new kernel image, the plug still boots into the old kernel until you run “flash-kernel”. Fortunately this is reasonably well known and is covered in Martin’s troubleshooting page.

I will persevere for perhaps another week with the current plug configuration. If I can’t get a stable system though I will try installing to USB drive (perverse as that may seem) and changing the uboot to boot from that rather than the flaky SD card. Most on-line advice suggests that USB support in uboot is rather “immature”, but it can’t be any worse than the current setup. My thinking is that if I can introduce a delay iin the boot process by uboot so that I can successfully boot from an external HDD, the drive connection might then be stable enough to be usable.

Of course I could be completely wrong.

from slug to plug

Sunday, February 28th, 2010

Well this took rather longer than expected. I intended to write about my latest toy much earlier than this, but several things got in the way – more of which later.

About three or four weeks ago I bought myself a new sheevaplug.

image of sheevaplug

The plug has been on sale in the US for some time, but UK shipping costs added significantly to $99 US retail price. Recently however, a UK supplier (Newit) has started stocking and selling the plugs over here – and at very good prices too. My plug arrived within three days of order and I can thoroughly recommend Newit. The owner, one Jason King no less (fans of 1970’s TV will recognise the name), kept me informed of progress from the time I placed the order to the time it was shipped. He even took the trouble to email me after shipping to check that I had received it OK. Nice touch, even if it was automated.

Looking much like a standard “wall wart” power supply typically attached to an external disk, the plug is actually quite chunky, but it will still fit comfortably in the palm of your hand. Inside that little box though there is enough computing power to make a slug owner more than happy. The processor is a 1.2 GHz Marvell Kirkwood ARM-compatible device and it is coupled with 512MB SDRAM and 512MB Flash memory. Compare that to the poor old slug’s 266 MHz processor and 32 MB of flash and you can see why I’d be interested – particularly since the plug can run debian (and Martin Michlmayr has again provided a tarball and instructions to help you out.

The plugs come in a variety of flavours, but all offer at least one USB 2.0 port, a mini usb serial port, gigabit ethernet and an SDHC slot. This means that debian (or another debian based OS such as Ubuntu) can be installed either to the internal flash or to one of the external storage media available. Newit ship the plugs in various configurations and will happily sell you a device fully prepared with debian (either Lenny or Squeeze according to your taste) on SD card to go with the standard Ubuntu 9.04 in flash. Personally I chose to install debian myself, so I bought the base model. (No, I’m not a cheapskate, I just prefer to play. Where’s the fun in buying stuff that “just works”?)

Given that Martin’s instructions suggest that installing to USB disk can be problematic, and that I have debian lenny on my slugs (and had a spare 4 Gig SDHC card lying around) I chose to use his tarball to install lenny to my SDHC card. Firstly I formatted the card (via a a USB mounted card reader) as below:

/dev/sdb1 512 Meg bootable
/dev/sdb2 2.25 Gig
/dev/sdb3 1024 Meg swap

(note that the plug will see these devices as “/dev/mmcblk0pX” when the card is loaded. The “/dev/sdbX” layout simply reflects the fact that I was using a USB mounted card reader on my PC. )

I then downloaded and installed Martin’s lenny tarball to the newly formatted card and as instructed edited the /etc/fstab to match my installation. Martin’s fstab file is below:

# /etc/fstab: static file system information.
#
# proc /proc proc defaults 0 0
# Boot from USB:
/dev/sda2 / ext2 errors=remount-ro 0 1
/dev/sda1 /boot ext2 defaults 0 1
/dev/sda3 none swap sw 0 0
# Boot from SD/MMC:
#/dev/mmcblk0p2 / ext2 errors=remount-ro 0 1
#/dev/mmcblk0p1 boot ext2 defaults 0 1
#/dev/mmcblk0p3 none swap sw 0 0

As you can see it defaults to assuming a USB attached device. You need to comment out the USB boot entries and uncomment the SD/MMC entries if. like me, you are intending to boot from SD card. At this stage I also edited “/etc/network/interfaces” to change the eth0 interface from dhcp to static (to suit my network) and I changed “/etc/resolv.conf” because the default includes references to cyrius.com and a local IP address for DNS.

Before we can boot from the SD card, we have to make a few changes to the uboot boot loader configuration to stop it using the default OS on internal flash (where the factory installed Ubuntu resides). Again, Martin’s instructions are helpful here but he points to the openplug.org wiki for instructions in setting up the necessary serial connection to the plug. On my PC (running Ubuntu 8.04 LTS) I got no ttyUSB devices by default and “modprobe usbserial” did not work but “modprobe ftdi_sio vendor=0×9e88 product=0×9e8f” did work for me.

Now open a TTY session using cu thusly “cu -s 115200 -l /dev/ttyUSB1” – don’t use putty on linux, it doesn’t allow cut and paste which can be very useful if you are following on-line instructions (of course it helps if you cut and paste the right instructions). I found that booting is too fast if you have to switch on the plug and then return to a keyboard so I recommend simply leaving the terminal session open and resetting the plug with a pin or paper clip. Hit any key to interrupt the boot session, then follow Martin’s instructions for editing the uboot environment.

My plug was running v 3.4.16 of uboot, so at first I used version 3.4.27 (downloaded from plugcomputer.org) and loaded that via tftp as described by Martin, But this turmed out to be a mistake because my plug failed to boot thereafter. I got the following error message via the serial console:

## Booting image at 00400000 …
Image Name: Debian kernel
Created: 2009-11-23 17:25:02 UTC
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 1820320 Bytes = 1.7 MB
Load Address: 00008000
Entry Point: 00008000
Verifying Checksum … Bad Data CRC

Some searching suggested that the uboot image was probably the problem and that reverting to v3.4.19 would solve this. So I downloaded 3.4.19 from “vioan’s” post “#6 on: November 16, 2009, 03:21:34 PM” at the plugcomputer.org forum and reflashed the plug with that image. Success – my plug now booted into debian lenny. Tidy up, update the OS and add a normal user as recommended and we’re ready to go.

My plug was intended to replace the slug I was using as my local apt-mirror. That mirror is now fairly large because I have a mix of 32 and 64 bit ubuntus (of varying vintages) and 386 and ARM versions of debian. I therefore recycled an unused 500 gig lacie USB disk and mounted that as /home2 (originally as /home, but I soon changed that when I wanted to unmount it frequently and then lost my home directory….) Copying the apt-mirror (175 Gig) over the network from my old slug was clearly going to take forever – high speed networking is not the slug’s forte, so I mounted both the slug and the plug’s disks locally on my PC and copied the files over USB – much faster. It was here that I discovered why the old lacie disk (a “designed by porsche” aluminium coated beast) was lying idle. I’d forgotten that it sounded like a harrier jump jet on take off when in use. I put up with that for a week – just long enough to get me to a free weekend when I could rebuild the old slug (now used as just an NTP server and the webcam) to boot from a 4 gig USB stick so that I could recycle its disk onto the plug. I’ve just finished doing that.

One other problem I found with the plug which caused me much head scratching (and delayed my writing this as I noted above) was that it consistently failed to boot back into my debian install after a “reboot” or “shutdown -r” – I had to power cycle the device to get it to boot properly. I spent some time this weekend with the serial port connected before I noticed (using “printenv” at the uboot prompt) that I had mixed up the uboot environment variables printed on Martin’s site. I had actually copied part of the instructions for the USB boot variant instead of the correct ones for the SD card boot. Sometimes “cut and paste” can be a mistake.

life is too short to use horde

Saturday, January 23rd, 2010

I own a bunch of different domains and run a mail service on all of them. In the past I have used a variety of different ways of providing mail, from simple pop/imap using dovecot and postfix, through to using the database driven mail service in egroupware.

Recently I have consolidated mail for several of my domains onto one of my VPSs. I don’t have a lot of mail users so at first I stuck with the simple approach available to all dovecot/postfix installations, i.e. – using dovecot as the local delivery mechanism and simply telling postfix to hand off incoming mail to dovecot. Dovecot then has to figure out where to deliver mail. I also used a simple password file for the dovecot password mechanism. This mechanism worked fine for a small number of users, but it rapidly becomes a pain if you have multiple users across multiple domains and you wish to allow those users to change their passwords remotely. The solution is to move user management to a MySQL backend and change the postfix and dovecot configurations to use that backend database.

Now to allow (virtual) users to change their mail passwords, most on-line documentation points to the sork password module for horde. But have you /seen/ horde? Sheesh, what a dog’s breakfast of overengineered complexity. I flatter myself that I can find may away around most sysadmin problems. but after most of a day one weekend trying to install and configure the entire horde suite just so that I could use the remote password changing facility I gave up in disgust and went searching for an easier mechanism. Sure enough I found just what I wanted in the shape of postfixadmin. This is a php application which provides a web based interface for managing mailboxes, virtual domains and aliases on a postfix mail server.

Postfixadmin is easy to install and has few dependencies (beyond the obvious php/postfix/mysql). There are even ubuntu/debian packages available for users of those distributions. I also found an excellent installation howto at rimuhosting which I can recommend.

I can now manage all my virtual domains, user mailboxes and aliases from one single point – and the users can manage their passwords and vacation messages from a simple web interface.

image of postfixadmin page

postfixadmin domain creation

Whilst I currently only provide pop3s/imaps mail access through dovecot, postfixadmin offers a squirrelmail plugin to integrate webmail should I wish to do that in future.

Simple, elegant and above all, usable. And it didn’t take all day to install either.

using scroogle

Saturday, January 2nd, 2010

For completeness, my post below should have pointed to the scroogle search engine which purportedly allows you to search google without google being able to profile you. Neat idea if you must use google (why?) but it still fails the Hal Roberts test of what to do if the intermediate search engine is prepared to sell your data. I actually quite like the scroogle proxy though, particularly in its ssl version because anything that upsets google profiling has to be a good thing. Besides, the really paranoid can simply connect to scroogle via tor.

(Odd that google seem not to have tried to grab the scroogle domain name. If they do, let’s just hope that they get the groovle answer.)

system monitoring with munin

Sunday, November 15th, 2009

A while back a friend and colleague of mine introduced me to the server monitoring tool called munin which he had installed on one of the servers he maintains. It looked interesting enough for me to stick it on my “to do” list for my VPSs. Having a bunch of relevant stats presented in graphical form all in one place would be useful. So this weekend I decided to install it on both my mail and web VPS and my tor node.

Munin can be installed in a master/slave configuration where one server acts as the main monitoring station and periodically polls the others for updated stats. This is the setup I chose, and now this server (my web and mail host) acts as the master and my tor node is a slave. Each server in the cluster must be set to run the munin-node monitor (which listens by default on port 4949) to allow munin itself to connect and gather stats for display. The configuration file allows you to restrict connections to specific IP addresses. On the main node I limit this to local loopback whilst on the tor node I allow the master to connect in addition to local loopback. And just to be on the safe side, I reinforced this policy in my iptables rules.

The graphs are drawn using RRDtool, which can be a little heavy on CPU usage, certainly too heavy for the slugs which ruled out my installing the master locally rather than on one of the VPSs. But the impact on my bytemark host looks perfectly acceptable so far.

One of the neatest things about munin is its open architecture. Statistics are all collected via a series of plugins. These plugins can be written in practically any scripting language you care to name. In the plugins which came by default with the standard debian install of munin I found plugins mostly written as shell scripts with the occasional perl script. However, a couple of the additional scripts I installed were written in php and python. The standard set of plugins covers most of what you would expect to monitor on a linux server (cpu, memory i/o, process stats, mail traffic etc). but there were two omissions which were quite important to me. One was for lighttpd, the other for tor. I found suitable candidates on-line pretty quickly though. The tor monitor plugin can be found on the munin exchange site (a repository of third party plugins). I couldn’t find a lighttpd plugin there but eventually picked one up from here (thomas is clearly not a perl fan).

Most plugins (at least those supplied by default in the the debian package) “just work”, but some do need a little extra customisation. For example the “ip_ ” plugin (which monitors network traffic on specified IP addresses) gets its stats from iptables and assumes that you have an entry of the form:

-A INPUT -d 192.168.1.1
-A OUTPUT -s 192.168.1.1

at the top of your iptables config file. You also need to ensure that the “ip_” plugin is correctly named with the suffix formed of the IP address to be monitored (e.g. “ip_” becomes “ip_192.168.1.1″). The simplest way to do this (and certainly the best way if you wish to monitor multiple addrresses) is to ensure that the symlink from “/etc/munin/plugins/ip_” to “/usr/share/munin/plugins/ip_” is named correctly. Thus (in directory /etc/munin/plugins):

ln -s /usr/share/munin/plugins/ip_ ip_192.168.1.1

The lighttpd plugin I found also needs a little bit of work before you can see any useful stats. The plugin connects to lighty’s “server status” URL to gather its information. So you need to ensure that you have loaded the mod_status module in your lighty config file and you have specified the URL correctly (any name will do, it just has to be connsistent in both the lighty config and the plugin). It is also worth restricting access to the URL to local loopback if you are not going to access the stats directly from a browser from elsewhere. This sort of entry in your config file should do:

server.modules += ( “mod_status” )

$HTTP["remoteip"] == “127.0.0.1″ {
status.status-url = “/server-status”
}

The tor plugin connects to the tor control port (9051 by default) but this port is normally not configured because it poses a security risk if configured incorrectly. Unless you also specify one of “HashedControlPassword” or “CookieAuthentication”, in the tor config file, then setting this option will cause tor to allow any process on the local host to control it. This is a “bad thing” (TM). If you choose to use the tor plugin, then you should ensure that access to the control port is locked down. The tor plugin assumes that you will use “CookieAuthentication”, but the path to the cookie is set incorrectly for the standard debian install (which sets the tor data directory to /var/lib/tor rather than the standard /etc/tor).

So far it all looks good, but I may add further plugins (or remove less useful ones) as I experiment with munin over the next few weeks.

logrotate weirdness in debian etch

Sunday, October 18th, 2009

I have two VPSs, both running debian. One runs lenny, the other runs etch. The older etch install runs fine, and is much as the supplier delivered it. Until now I have not had cause to consider the need to upgrade the etch install to lenny because it “just worked”. But today I noticed for the first time a very odd difference between the two machines. A difference which had me scratching my head, and reading too many man entries, for some long time before I found the answer.

For reasons I don’t need to go into, I log all policy drops in my iptables config to a file called “/var/log/firewall”. This file is (supposedly) rotated weekly. The logrotate and cron entries on both machines are identical.. The entry in “/etc/logrotate.d/firewall” looks like this:

/var/log/firewall {
rotate 6
weekly
mail logger@baldric.net
compress
missingok
notifempty
}

The (standard) file “/etc/logrotate.conf” simply calls the firewall logrotate file out of the included directory “/etc/logrotate.d”. The “/etc/cron.daily/logrotate” file (which calls the logrotate script) is also standard and simply says:

#!/bin/sh

test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate /etc/logrotate.conf

and the (again standard) crontab file says:

# /etc/crontab: system-wide crontab
# Unlike any other crontab you don’t have to run the `crontab’
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user command
54 * * * * root cd / && run-parts –report /etc/cron.hourly
55 4 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.daily )
36 5 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.weekly )
3 3 5 * * root test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.monthly )
#

So far so simple, and you would expect the file “/var/log.firewall” to be rotated once a week at 04.55 on a sunday morning. Wouldn’t you.

Well, on lenny, you’d be right. But on the etch machine the file was rotated daily at a time completely unrelated to the crontab entry. It turns out that there is a bug in the way etch handles logrotation because syslog doesn’t use logrotate and overrides the logrotatiion entries run out of cron. I found this after much searching (and swearing).

See the bug report at bugs.debian.org and this entry which pointed me there.

I love standards.

debian on a DNS-313

Saturday, October 3rd, 2009

I bought another new toy last week – a D-Link DNS 313 NAS.

D-Link DNS-313

D-Link DNS-313

Actually, this was a mistake because what I really wanted was the DNS-323. I just wasn’t careful enough at the time. Quite apart from having space for two 3.5″ SATA hard drives instead of just one, the 323 is a very different beast to its smaller (and much cheaper) sibling.

Martin Michlmayr has a nice guide to installing debian on a 323. Given that the 323 has a faster processor and more RAM than a slug and it can take two internal SATA disks rather than just the external USBs it looks like an attractive option for a new debian based server. Pity then that I bought the wrong one. My excuse is that I thought the only difference was in the disk capacity and I was prepared to settle for just 1TB of store. The (normally reliable) owner of the shop where I bought the beast was also adamant that the disk capacity was the only difference between the two devices. I should have known better than to succumb to what was essentially an impulse buy when I wasn’t really intending to buy a new NAS at the time (I was in the shop for something else and picked up the 313 because I recalled reading Martin’s pages recently).

Once I got the 313 home of course and checked the specs it looked as if I would be stuck with the D-Link supplied OS. However, a bit of searching turned up the DSM-G600, DNS-323 and TS-I300 Hack Forum which has a series of articles on installing debian on a 313 (despite the forum title). A forum contributor called “CharminBaer” has put together a nice tarball of debian lenny which allows the user to replace the D-Link OS without actually reflashing the device. This means that the original bootloader is retained but the device boots into the replacement system on disk. The nicest part of this installation method is that there is almost no risk of bricking the device because the installation simply entails copying the tarball onto the disk over a USB connection, extracting the files and then booting into a shiny new Lenny install. Result.

The tarball can be found here, and the installation instructions are here.

Many thanks to “CharminBaer”.

wordpress on lighttpd

Saturday, September 12th, 2009

I have commented in the past how I prefer lighttpd to apache, particularly on low powered machines such as the slug. I used to be a big apache fan, in fact I think I first used it at version 1.3.0 or maybe 1.3.1, having migrated from NCSA 1.5.1 (and before that Cern 3.0) back in the day when I ran web servers for a living. However, those days are long gone and my web server requirements are now limited to my home network and VPSs so I don’t need, nor do I want, the power of an industrial strength apache installation. In fact, my primary home web server platform (the slugs) struggles with a standard apache install. Lighttpd works very well on machines which are low on memory.

Having got used to lighttpd, it seemed a natural platform to use on my VPSs. And it performs very well on those machines for the kind of traffic I see. Moving trivia to my bytemark VPS meant that I had to take care of some minor configuration issues myself – most notably the form of permalinks I use. Most of the documentation about running your own wordpress blog assumes that you will be using apache (since that is the most popular web server software provided by shell account providers). For those of you who, like me, want to use lighttp instead, the configuration details from my vhosts config file are below. Lighttpd is remarkably easy to configure for both virtual hosting in general, and for wordpress in particular. Note that I also choose to restrict access to wp-admin to my home IP address, this helps to keep the bad guys out.

Extract of “conf-enabled/10-simple-vhost.conf” file:

# redirect www. to domain (assumes that “mod_redirect” is set in server.modules in lighttpd.conf)

$HTTP["host"] =~ “www.baldric.net” {
url.redirect = ( “.*” => “http://baldric.net”)
}

#
# config for the blog
#
$HTTP["host"] == “baldric.net” {
# turn off dir listing (you can do this globally of course, but I choose not to.)
server.dir-listing = “disable”
#
# do the rewrite for permalinks (it really is that simple)
#
server.error-handler-404 = “/index.php”
#
# reserve accesss to wp-admin directory and wp-login to our ip address
#
$HTTP["remoteip"] !~ “123.123.123.123″ {
$HTTP["url"] =~ “^/wp-admin/” {
url.access-deny =(“”)
}
$HTTP["url"] =~ “^/wp-login.php” {
url.access-deny =(“”)
}
}

}

# end

Enjoy.

aspire one bios updates

Sunday, July 5th, 2009

This post is partly for my own benefit. It records some of the most useful references to bios updates for the AAO. My own AAO is actually running a fairly early bios (3114) and deliberately so. I upgraded to 3309 in (yet another futile) attempt to get sony memory sticks to work but found that screen brightness suffered for no perceptible gain in any other functionality. So I reverted. That aside, the references here may be useful to others.

Firstly, there is considerable discussion of the relative merits of various bios versions on the Aspire One User Forum. That site also has a lot of other useful (and sometimes not so useful) discussion points. Broadly, though, I have found that forum a good starting point for searching for AAO related information.

Next up is the very useful blog site run by Macles. Macles gave a very good set of instructions on flashing the bios in this post, which has been added to. and improved with the addition of lots of step by step pictures, by “flung” at netbooktech in this posting. Note however that the link to the acerpanam site supposedly hosting the bios images is broken. The images, along with many other driver downloads can currently be found by following links on the gd.panam.acer.com pages. Alternatively, I have found the images on the Acer Europe ftp site a good source (though it has yet to show the 3310 release).

No – my sony memory sticks still don’t work.

tor on a vps

Sunday, July 5th, 2009

I value my privacy – and I dislike the increasing tendency of every commercial website under the sun to attempt to track and/or profile me. Yes, I know all the arguments in favour of advertising, and well targeted advertising at that, but I get tired of the Amazon style approach which assumes that just because I once bought a book about subject X, I would also like another book about almost the same subject. I don’t much like commercial organisations profiling me (and, incidentally, I find it highly ironic that we in the UK seem to make a much bigger fuss about potential “big brother” Government than we do about commercial data aggregation, but hey).

Sure, I routinely bin cookies, block adware and irritating pop up scripts, and use all the, now almost essential, firefox privacy plugins, but even there we still have a problem. I don’t know who wrote those plugins, I just have to trust them. That worries me. Some of the best known search engines are even more scary if you think carefully about the aggregate information they have about you.

Sometimes I care about the footprint I leave, sometimes I don’t, but the point is that I should be in control of that footprint. Increasingly that is becoming difficult. Besides being tracked by sites I visit, last year’s controversy about BT’s use of phorm is also worrying. If my ISP can track everything I do, then I face another level of difficulty in protecting my fast vanishing privacy.

Besides using a locked down browser, DNS filtering which blocks adware, cutting cookies and all the other tedious precautions I now feel are necessary to make me feel comfortable, I often use anonymous proxies when I don’t want the end site to know where I came from. But even that now looks problematic. If you use a single anonymising proxy, all you are doing is shifting the knowledge about your browing from the end site to an interrmediary. That intermediary may (indeed should) have a very strict security policy. Ideally, it should log absolutely nothing about transit traffic. But if that intermediary does log traffic data and then sells that data to a third party, you may be in an even worse position than if you had not attempted to become anonymous. Back in january of this year, Hal Roberts of Harvard University, posted a blog item about GIFC selling user data. If sites such as Dynaweb are prepared to sell user data, then the future for true anonymity looks problematic. As Doc Searle said in this blog posting,

We live in a time when personalized advertising is legitimized on the supply side. (It has no demand side, other than the media who get paid to place it.) Worse, there’s a kind of gold rush going on. Even in a crapped economy, a torrent of money is flowing into online advertising of all kinds, including the “personalized” sort. No surprise that companies in the business of fighting great evils rationalize the committing of lesser ones. I’m sure they do it it the usual way: It’s just advertsing! And it’s personalized, so it’s good for you!

No, as Searle well knows, it is not good for you.

What to do? Enter tor and privoxy.

I first used tor some years ago in its earlier incarnation as “the onion router” (hence its name) and until recently had used it only sporadically since. The main drawback of the early tor network was its speed, or lack of it. Tor gets is strength (anonymity) from the way it routes traffic.

how tor works

Tor traffic passes through a series of nodes before exiting at a node which cannot be linked back to the original source. So tor performance depends on a large number of both fast intermediate relays and a large number of exit nodes. Since not all tor users are prepared to run relays, let alone an exit node (it can be bandwidth expensive and in the case of an exit node can lead to your system being mistaken for a hostile, or compromised, site) tor can be slow, at times painfully slow. But recently tor has been getting faster as more relay and exit nodes are added. It is now at a state which is probably usable most of the time, so long as you are prepared to wait a little longer than is customary for some web pages to load (and you don’t use youtube…..).

When using tor recently I have tended to follow the well trodden path of local installation alongside privoxy. Because I believe in giving something back to the community if I am gaining benefit, I also set my local configuration to run as a relay. But that caused some difficulty. If we assume that my tor usage was fairly representative of the majority of tor users out there, then the fact that my relay was only operational when my client system was up and running meant that the relay would be seen by the tor network as unstable and probably slow, Indeed, the fact that I had to throttle tor usage to the minimum to stop the network from impacting unduly on my ADSL bandwidth, meant that I was not entirely happy with the setup. So I stopped relaying. But that leaves me feeling that I am taking advantage of a free good when I could be contributing to that good.

Some while back I bought myself a VPS from Bytemark (an excellent, technically savvy, UK based hosting company) to run a couple of webs and an MTA. I use it now largely as a mail server (running postfix and dovecot) and the traffic is relatively low volume. That VPS is pretty small (though actually way better specced than some real servers I have run in the past) but I reasoned that I could easily run a tor relay on that machine and then connect to it remotely from my client system. I did, and it worked fine. But I soon found that the tor network seems to have a voracious appettite for bandwidth, Even with a fairly strict exit policy (no torrents allowed!) and some tight bandwidth shaping, I still found that I was using about 2 Gig of traffic per day (vnstat is useful here). Any more than that would start to encroach on my bandwidth allowance for my VPS and possibly impact on the main business use of that server. Monthly rates for VPSs are now less than I pay for my mobile phone contract (and arguably more useful than a phone contract too) so I decided to specialise and buy another VPS just for tor. I now run an exit node on a VPS with 384 MB of RAM and 150 Gig monthly traffic allowance. That server is currently throttled to about 2 Gig of traffic per day, but I will double that very shortly.

Now one of the nicest things about running a tor relay is the fact that your own tor usage is masked and you may get better anonymity. I therefore run privoxy on my tor relay and proxy through from my client to that proxy which in turn chains to tor internally on my relay. However, if you simply configure your local client to proxy through to your relay in clear you are allowing your ISP (and anyone else who cares to look) to see your tor requests – not smart. So I tunnel my requests to the tor relay through ssh. My local client has an ssh listener which tunnels tor requests through to the relay and connects to privoxy on port 8118 bound to localhost on the relay. I also have a separate browser on my desktop which has as its proxy the ssh listener on my client system. For a good description of how to do this see tyranix’s howto on the tor wiki site. Now whenever I want to use tor myself I simply switch browser (and that browser is particularly dumb and stripped, and has no plugins or toolbars which could leak information). Of course, should I get really paranoid, I could always run the local browser in a VM on my desktop and reload the VM after each session.

But I’m not that paranoid.