Continue reading »]]> Some time ago my wife bought me a Sansa Sandisk Clip+ music player. When she asked me “what kind of MP3 player” I would like, I specifically specified the Clip+ because it could handle ogg vorbis encoded audio files. All my audio disks are encoded in this format. Picky I know, but there you go.

The version she bought me was the 8 GB Black which comes with (you guessed it) 8 GB of internal storage – sufficient for a fair number of audio tracks. But this version also has a microSDHC slot which will take a maximum of another 32 GB. That should enable me to carry most of my music collection (which currently runs to around 47 GB) if I cut out some of the obvious duplicates and exclude some of my more embarrassing ’70s choices. The device is small, neat and light and also has a pretty good battery life.

But Sansa have not been entirely honest in their advertising. Sure, the device will accept additional storage, but it is largely unusable. Once you get past around 6 or 7 GB on the internal storage and even as little as 3 or 4 GB on the additional card, the device is not capable of building its database of the collection. The symptom is pretty obvious. As soon as you disconnect the Clip+ from the USB connection used to transfer files (and incidentally to charge the device) the display shows “Refreshing your media” and a progress bar which slowly fills from left to right. If you only use the internal storage, there is no problem, but as soon as you get past the 3 or 4 GB additional store on the external card, the clip+ will sit there, refreshing away, for hour after hour if you let it.

Even after a firmware upgrade, the device wouldn’t do what it was supposed to, so I turned to the FLOSS community yet again. This time in the shape of rockbox.

Rockbox is a free, open source jukebox utility which runs on a wide variety of devices. The website provides detailed instructions on how to install and use rockbox and even comes with an installer for most operating systems so that you don’t have to get your hands dirty installing it manually. Most impressive of all however, is that the rockbox firmware can be installed alongside the original player’s firmware without danger of bricking the device. If you find that you don’t like rockbox (and what’s not to like about a free product that outperforms the paid for original?) you can still boot into the original firmware because rockbox provides a dual boot facility. And if you really don’t like it, then you can simply remove it and go back to using the original.

Thoroughly recommended.

Continue reading »]]> In January of this year I wrote about t-mobile’s apparent policy of actively looking for and blocking any TLS-secured SMTP sessions over their network. At the time I believed this to be a cockup rather than a deliberate policy. I still prefer to believe that, but the episode left a rather sour taste in my mouth. So this month I took the opportunity presented by the end of my contract to shift to another provider. Of course, in doing so I gained a nice shiny new ‘phone which meant that I could spend a fun few hours setting it up the way I wanted it and nailing it down as much as possible so that it didn’t leak all my data to google. This is unnecessarily difficult, and much harder than it should be (and I know that people like Peter H will simply tell me that I shouldn’t be using an android ‘phone in the first place). But that is not the point of this post.

Like most people these days, I use my ‘phone to pick up email. The standard email client on my last ‘phone was pretty uninspiring so I used K-9 mail in its place. K-9 is a pretty good application, but it has a silly little bug in it which is still not sorted properly. This bug manifests itself in a rather odd, and unpredictable way – K-9 seems to “forget” the X509 certificate used to protect the authentication process if that certificate is self-signed, or otherwise not verifiable by an external CA. The cure, such as it is, is to simply refresh the certificate by reloading the account settings and accepting the cert when K-9 warns you that “TrustAnchor found but validation failed”. The length of time between accepting the cert and K-9 “forgetting” it again seemed random to me, so I got into the habit of refreshing my account settings whenever I noticed that I hadn’t received any mail for a while. Annoying, but not ultimately a deal breaker for using what was otherwise a pretty good application.

So, the first application I looked at on my new mobile was, of course, email. The default mail client on this new phone looks a lot slicker than the old one on my previous phone, but then it is a much newer ‘phone, from a different manufacturer and the android version is much newer too, so no real surprise there. The setup seemed to have no problem with my self signed certs so I thought I might stay with the default to see if it would solve my annoying little problem with K-9.

Unfortunately not.

Whilst I had no problem with incoming mail over my IMAPS connection, all attempts to send mail failed. On checking my server logs I found the following (real details changed or obfuscated):

Mar 20 20:45:57 pipe postfix/smtpd[7594]: NOQUEUE: reject: RCPT from home.baldric.net[12.34.56.78]: 504 5.5.2 <localhost>: Helo command rejected: need fully-qualified hostname; from=<null@baldric.net> to=<noone@baldric.net> proto=ESMTP helo=<localhost>

Aha! My postfix configuration is set up to reject hosts which do not have valid hostnames or do not announce themselves with fully qualified domain names (i.e. names of the form “host.domain”). Now since I use SASL authentication in my postfix configuration the fix is relatively easy; just ensure that the stanza “permit_sasl_authenticated” appears in both “smtpd_sender_restrictions” and “smtpd_helo_restrictions” before “reject_non_fqdn_hostname” – thusly:

smtpd_helo_required = yes
smtpd_helo_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_non_fqdn_hostname, reject_invalid_hostname
smtpd_sender_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_non_fqdn_sender, reject_unknown_sender_domain

(In fact, this episode highlighted an error in my postfx configuration because my helo restriction was inadequate. By now checking the authentication before the helo restriction kicks in I am still well protected, but mail from valid authenticated users is permitted.)

I am in that (very) small minority of people who run their own mail servers and are able to change server side configurations. But, and this is a big but. I should not have to change the server side configuration to accommodate a broken client and the vast majority of people will not be able to do so anyway. Almost all well set up mail servers will reject mail where the client connection announces itself in the helo exchange as “localhost”. That is normally an indication of a spammer, indeed spamassassin will allocate a high score to any mail which is so flagged. This means that there will be a huge, and growing, number of people who cannot send mail from their android ‘phones.

If this is the default android mail behaviour, then google need to fix it now. Meanwhile, K-9 is looking attractive again.

Continue reading »]]> Moxie Marlinspike, a security researcher probably best known for his SSL proxy tool, likes google even less than I do. His googlesharing website says:

“Google thrives where privacy does not. If you’re like most internet users, Google knows more about you than you might be comfortable with. Whether you were logged in to a Google account or not, they know everything you’ve ever searched for, what search results you clicked on, what news you read, and every place you’ve ever gotten directions to. Most of the time, thanks to things like Google Analytics, they even know which websites you visited that you didn’t reach through Google. If you use Gmail, they know the content of every email you’ve ever sent or received, whether you’ve deleted it or not.

They know who your friends are, where you live, where you work, and where you spend your free time. They know about your health, your love life, and your political leanings. These days they are even branching out into collecting your realtime GPS location and your DNS lookups. In short, not only do they know a lot about what you’re doing, they also have significant insight into what you’re thinking.”

His solution to this problem was interesting. He came up with the idea of a proxy system which would intercept all google queries, strip off identifying material (such as cookies and UserAgent strings and other HTTP headers) substitute new identifiers and mix the requests up with those from other users before forwarding to google. Implementation depended upon a Firefox addon (nothing for other browsers) which identified google queries and forwarded them to the proxy. All other traffic was untouched.

I stopped using google (except via scoogle) some time ago, and when Moxie’s new proxy first surfaced I thought it interesting but susceptible to the same problem I discussed in mid 2009 when writing about Hal Roberts’ experience of GIFC – all you are doing is shifting knowledge of your searches from google to a new intermediary. However, Moxie later addressed this problem with the release of version 0.20 of his addon so I thought I’d take another look at it. Unfortunately the addon won’t work with FF 9 (which I am using). Moxie’s proxy is not the only one out there however. Because he released the code under an open source licence, others have picked it up. I found one at gs.netsend.nl. They also provide an updated FF addon which will work with versions up to 15 (i.e. probably around next wednesday given the speed with which Mozilla is currently shipping new FF releases).

Once the addon is installed, it gives you two proxy options in the preferences settings – one is the original proxy.googlesharing.net, the other is gs.netsend.nl itself. In testing I found that the original googlesharing proxy seemed to be off-line, but when using the netsend.nl proxy I was reassured to see the message “Search results anonymized by GoogleSharing” added to the google homepage. I was even more reassured that my sniffer showed a connection to vps1101.pcextreme.nl on 31.21.98.201 and not to any known google network.

So, will I use it? Maybe. But the proxy mechanism seems to be unreliable. In many tests, the proxy connection seemed to be bypassed and the connection was obviously made direct to google (as evidenced by my sniffer). I think this failure is doubly unfortunate because it does not fail safe (i.e. the connection does not simply fail with an error message, it passes you direct through to google). This could lead the unwary to think that they are protected when in fact they are not.

I prefer not to use google at all. And in those cases where I do want to compare results with another search engine I prefer to do so via tor. But it is one more option in my toolkit if used carefully. And if using it pisses off google, then it is worth it occasionally.

Continue reading »]]> When I first tested running a tails mirror on one of my VMs, the traffic level reported by vnstat ran at around 20-30 GiB per day. I figured I could live with that because it meant that my total monthly traffic would be unlikely to exceed my monthly 1TB allowance. However, when I checked the stats on that server last week (around the 9th of Jan) I found that I was shipping out around 150 GiB per day and vnstat was predicting a monthly total of close to 3 TB. As the tails admins said when I told them that I would have to shut off the mirror on that VM while I sorted something, “Ooops”. Ooops indeed. I couldn’t chance a massive bill for exceeding my bandwidth allowance by quite that much. The actual stats for 4, 5, 6, 7, 8 and 9 January before I pulled the plug were: 34.23 GiB, 69.14 GiB, 178.31 GiB, 131.68 GiB, 99.05 GiB and 133.27 Gib. It turns out that tails 0.10 was released on 4 January and I hadn’t been prepared. A lesson learned.

Having shut down and had the DNS round robin amended, I attended to finding some way of throttling my traffic so that I could live within my allowance whilst still providing a useful mirror. I scratched my head for a while before stumbling on the obvious, I should be throttling at application level. (Sometimes I find that I miss simple answers because I am looking for complicated ones).

I started out by assuming that I should be using tc and iptables mangling, or something like the userspace tool trickle, all of which looked horribly more complicated than the approach taken by tor (which allows you to simply set the acceptable bandwidth rate to some limit, plus set an accounting period maximum of some total transfer limit per day/week whatever). And of course it turns out that my webserver (lighttpd) allows something similar. Just set the server limit to some chosen max transfer rate and, if necessary, also impose a per IP max rate. The magic configuration file options are:

# limit server throughput to 3000 kbytes/sec (~30000 kbits/sec)
server.kbytes-per-second = 3000
#
# and limit individual connections to 50 kbytes (~500 kbits/sec) – NB. I don’t actually use this
# connection.kbytes-per-second = 50

I tested this by pulling a copy of the tails iso from one of my other VMs which has a high bandwidth connection and got acceptable (and expected) results. So now I can go back on-line later this month safe in the knowledge that I’m not going to blow all my bandwidth in one week.

Continue reading »]]> OK, yes, I know there are probably already a gazillion web pages on the ‘net explaining exactly how to do this, but I got caught out by a silly gotcha when I tried to do this a couple of days ago, so I thought I’d post a note.

Firstly, X is not exactly a secure protocol, nor is it easy to filter at NAT firewalls, so the ability to tunnel it over ssh is hugely welcome. In fact, ssh can be used to tunnel practically any other protocol you care to name, so it should be your first port of call should you wish to connect to a remote system using an insecure protocol. (I use it to wrap rsync for example).

I don’t run X on my VMs (there is no need, they don’t run desktop software) and I had not previously seen the need to run X based graphical programs on those servers. However, a couple of days ago I thought it would be really useful to run etherape on one particular remote server so that I could watch the traffic patterns. Normally I use iptraf (which is ncurses based) when I want to monitor network traffic in real time, but etherape is pretty cool and gives a nice graphical view of your network connections. But it runs on an X based gui.

So. I changed the remote server’s sshd_config to enable X forwarding (“X11Forwarding no” becomes “X11Forwarding yes”) and restarted sshd. On my desktop I similarly changed my local ssh_config file to allow X forwarding (“ForwardX11 no” becomes “ForwardX11 yes”) to obviate the need to use the -X switch on the command line. I then installed etherape on the remote server and fired it up only to get the message “Error: no display specified”. Sure enough “echo $DISPLAY” showed nothing. But I had thought (and everything I had read confirmed) that ssh should take care of setting the appropriate display when X11 forwarding was set.

So I then tried setting a display manually (export DISPLAY=localhost:10.0 on the remote server) and then got the response “Error: cannot open display: localhost:10.0″. So, still no deal. I spent some time scratching my head (and reading man pages) and sent off a query to my local Linux User group in parallel asking for advice. They were gentle with me.

The first, and rapid, response, said:

On the server:

sudo apt-get install xauth

Then disconnect and reconnect the client.

Jobs a good un.

Thank you Brett.

So the moral is, make sure that you have X authorisation working properly on the remote system (check for the existence of $HOME/.Xauthority) if you experience the same symptoms I did.

Continue reading »]]> Upgrading the slugs to squeeze killed the webcam. At first I thought that squeeze was missing the necessary gspca drivers, but no, a quick look in /dev revealed an entry for video0 and “lsmod” reported “gspca_zc3xx” loaded correctly. This is a different driver to that which my camera loaded in lenny (spca5xx) but a quick search around the web confirmed that the camera should be happy with the new driver. So clearly the webcam program itself was at fault (confirmed by checking the output when running the program by hand). Time to find an alternative.

A scan of the debian repositories turned up a bunch of possibilities, some of which I had looked at in the past when first installing the camera. I eventually plumped for fswebcam because it was quick and easy to install and configure, it seemed to be actively supported (unlike Gerd Knorr’s old program) and it didn’t need a GUI. More to the point, it works with my old Logitech camera.

The program can be run from the command line with option switches or configured to read its options from a config file. And despite being advertised as a tiny, it has quite a rich feature set. It can even perform simple manipulation of the captured image, such as resizing, averaging multiple frames or overlaying a caption on the image. Nifty.

My current config file is shown below as an example. I may change this because I’m not quite happy with the brightness/contrast mixture and I’m still playing with the options available. Still, a recommended package.

# /etc/fswebcam.conf
#
# config file for fswebcam utility
#
# device and input are defaults
device /dev/video0
input 0
# log messages – only use this is debugging setup
# log /var/log/fswebcam.log
# repeat image capture every 30 seconds
loop 30
# set number of frames to skip if the camera sends bad frames on startup.
# Note – this is only necessary when using the camera for video capture
# skip 10
# background the webcam process
background
# set resolution of image (default is 384 x 288)
resolution 320X240
# set the palette to use in the output format (here JPEG) and the jpeg quality
# (my camera doesn’t like this option so it is commented out)
# palette JPEG
jpeg 95
# set some additional controls
# (use “fswebcam -d v4l2:/dev/video0 –list-controls” to see options)
set brightness=55%
set contrast=40%
# place the banner at the bottom of the image (default)
bottom-banner
# and colour it black
banner-colour #000000
# set the font to use in the banner title and timestamp
font /usr/share/fonts/truetype/msttcorefonts/arialbd.ttf
# set the title and timestamp to display in the banner – (timestamp uses strftime format)
title “Webcam on the Slug”
timestamp “%d %B %Y at %H:%M:%S”
# where we save the image
save /home/web/webcam.baldric.net/images/webcam.jpeg
#
# end

Continue reading »]]> Debian 6 (squeeze) has been the current stable version since February 2011. The latest version (6.02) was released in late June. I have put off updating my slugs from lenny (old stable) for a while because I wanted to see how others faired before committing myself. Indeed, initial reports on the debian arm list indicated that the upgrade could be problematic. Worse, a completely clean install of squeeze turns out to be impossible because the debian installer uses more memory than is physically available on the slugs. So the only way to go, even for a clean new installation, is to install lenny first, then upgrade.

Given that both my slugs are operational, and are now an integral part of my network, I decided to invest in a new one as a development machine to test the upgrade rather than risk fritzing a perfectly good setup. (Back in the day I would have been happy to “fix it ’till it broke”, but these days I don’t really need to experiment that much and I’d rather keep a working system, well, working).

Second hand slugs go for around £25 on ebay, and there are still plenty about, so I bid for one that had only about a day to go and was successful. Unfortunately, when it turned up I found that the power supply was fsckd and so I had to switch off one of my operational slugs in order to test the new one. Happily it appeared to boot up OK so I fired off a disgruntled email to the seller and then ordered a new PSU. The seller claimed that it “worked OK when I boxed it” and didn’t offer to pay for the replacement PSU so I wasn’t too happy with him. I became even less happy when the new PSU arrived and I booted up the slug in preparation for reconfiguration to match my network before installing debian.

The debian installation process is handled via an SSH shell. You need to know the address of the slug in order to connect and install. The installer also needs the addresses of a local DNS server and the default route to the outside world (so it can find the servers containing the installation packages). Now the default, factory settings, for slugs includes a fixed IP address of 192.168.1.77. If this does not match your requirements, it must be changed before reflashing with debian. Guess what? The default address didn’t work, so the previous owner must have reconfigured the slug to match his network and he had not bothered to reset to factory default before selling. Nor had he been considerate enough to let me know the new configuration. Needless to say I won’t be buying anything else from him. Nor did he get decent feedback.

I couldn’t reconfigure the new slug until I could connect so I needed to find out what address it was using. A quick nmap scan of the 192.168.1.0/24 netblock showed that it wasn’t even on the default network range so I fired up wireshark and etherape on one of my machines in the hope of catching the slug arping and getting the address from the request. In the event, etherape proved to be quicker (and easier) in providing the answer since the slug quickly popped up and disclosed its IP address as 192.168.2.10. Adding a route to the 192.168.2.0/24 net then allowed me to finally connect and reconfigure the new beast to suit my network. I then rebooted and started a fresh installation of lenny (as previously described in one of my earlier posts). About four hours later I had a nice new clean slug running lenny.

However, since the main purpose of getting the new slug was to allow me to test the upgrade to squeeze in safety I really needed to make it look like my main operational slug. That slug is my DNS and DHCP server, and primary rsync backup for my desktops. It also runs a webserver. Like most (lazy) sysadmins my system documentation tends to lag somewhat behind reality so I can’t rely on the various readme files I routinely create on my boxes to be completely up to date (or even accurate). Fortunately for me though, debian provides a neat way of snapshotting installed packages on a system. You can then use this snaphot to create a mirror of that system which will include all the same packages. Here’s how:

On the source system:

dpkg –get-selections | grep -v deinstall > packages.txt

This lists all active packages, except those deinstalled, and sticks the list in a text file.

Now copy that file to the target system, ensure that the target system’s “sources.list” file matches that on the source, and then run:

dpkg –clear-selections
dpkg –set-selections < packages.txt
apt-get dselect-upgrade

This will download and install all the packages necessary to get the target system matching the source.

All that is now left to do is copy across any relevant configuration files so that the two systems fully match and then rebooot the target to check that everything looks OK.

New slug now finally matching old slug it was time to upgrade to squeeze. Martin Michalmayr’s excellent website documents the upgrade process in meticulous detail. The key points to note here are his recommendation that you read the release notes for debian 6.0. In particular, note and follow the chapter on upgrades from debian 5.0 before attempting an actual upgrade. One of the main differences between 5.0 and 6.0 is the use of UUIDs to reference disks. In my case this meant changing my /etc/fstab from this:

# /etc/fstab: static file system information.
#
#
proc /proc proc defaults 0 0
/dev/sda2 / ext3 errors=remount-ro 0 1
/dev/sda1 /boot ext2 defaults 0 2
/dev/sda5 none swap sw 0 0

to this:

# /etc/fstab: static file system information.
#
#
proc /proc proc defaults 0 0
UUID=db57451a-e3e5-4d8a-95b9-494c48bb5e8d / ext3 errors=remount-ro 0 1
UUID=022bc211-1c52-4848-9ee1-e211e72b28e4 /boot ext2 defaults 0 2
/dev/sda5 none swap sw 0 0

Before finally starting the upgrade I opened two separate SSH sessions to the slug. In one I ran the upgrade process as below:

first a partial upgrade as recommended at Section 4.4.4, “Minimal system upgrade”

apt-get update
apt-get upgrade

then install the required linux kernel image and udev as outlined in Section 4.4.5. “Upgrading the kernel and udev”

apt-get install linux-image-2.6-ixp4xx
apt-get install udev

followed by

reboot

and

apt-get upgrade
apt-get dist-upgrade

to complete the system upgrade.

Now here is where the second SSH session is most useful. The final upgrade and distribution upgrade installs the file indexing package “apt-xapian-index”. Correspondents on the debian arm list have noted that this package consumes more memory than is available on the poor old slug and it starts swapping itself to death. The process must be killed immediately and the package removed. If you leave it too long after the upgrade has completed you will find it impossible to log in until the initial indexation has completed (in excess of 24 hours or more has been reported) because the system is too busy. I ran “top” in the second shell during the dist-upgrade process and kept an eye on the load averages. As soon as they started climbing above 3 I knew that it was time to watch out for the apt-xapian-indexer and kill it. Once the system load returned to normal I was then able to finalise the upgrade with:

apt-get purge apt-xapian-index
apt-get autoremove

to remove the offending indexer and clean up any residual unneeded packages. A final reboot to check all was well was sufficient to convince me that it was safe to upgrade my two operational slugs using the same process. Testing the upgrade on the new slug in the way I did also meant that I now had a backup slug configured exactly like my main DNS server but running squeeze. Any failure on the remaining upgrade would not then be critical.

I must be getting old. I never used to be this cautious.

Continue reading »]]> I have just added a couple of new plugins to this blog and tidied up some old cruft that I had been meaning to get around to for a while. One of the plugins I have added is a really rather good statistcs tool called Counterize II. It provides a very quick (and impressively comprehensive) set of stats about page hits, browser types, referring URLs etc. all readily accessible from the wordpress admin dashboard – so no need to trawl through web logs to find out where your visitors are coming from or which is your most popular post or page. Thoroughly recommended.

In the process of searching for such a plugin I also came across Angie Bowen’s posting about the top ten things she always does when setting up a new blog. Whilst I don’t agree with all her recommendations (identifying the ones I disagree with is left as an exercise for the class) I think this is a very handy aide-memoire. Her pointers about blog security and discussion settings in particular are sensible and worth reading.

Continue reading »]]> I recently stumbled upon sig2dot, a gpg/pgp keyring graph generator. In fact this seems to have been around for some time, but I’d never come across it before. It can be used to generate a graph of all of the signature relationships in a GPG/PGP keyring, and, like other visualisation tools, this graphical image producing program can give new insight into relationships between objects.

The sig2dot program itself is available in the debian/ubuntu repositories in the package called “signing-party”. But unless you want to install a shed load of other unnecessary cruft along with it (exim? for god’s sake, why?), I recommend you simply pull the perl code direct from the author’s site. Along with the sig2dot program itself, you will need “neato” from the graphviz package and “convert” from the wondrous imagemagick package suite. If you don’t already have those installed then it is pretty safe to pull them from your distro’s package repository.

That done, try the following:

first create an ascii graphviz dot file ready for neato

$ gpg –list-sigs –keyring ~/.gnupg./pubring.gpg | sig2dot.pl > ~/.gnupg/pubring.dot

now convert to a postscipt file

$ neato -Tps ~/.gnupg/pubring.dot > ~/.gnupg/pubring.ps

before using imagmagick to convert to a png graphic

$ convert ~/.gnupg/pubring.ps ~/.gnupg/pubring.png

Those of you with gpg keyrings may wish to try it out (and no. I’m not going to show you mine).

Continue reading »]]> I had a couple of comments on my earlier post about autossh which suggested that I should look at alternative mechanisms for keeping my ssh tunnel up. Rob in particular suggested that setting “ServerAliveInterval” should work. Oddly I had tried this in the past whilst trying out various configuration options and I swear it didn’t work for me. But since the autossh mechanism felt inelegant I thought I’d revisit my ssh_config file as Rob suggested. And indeed setting ServerAliveInterval to 300 (i.e. 5 minutes) solved my tunnel drop problem. I’d guess that other intervals of less than 1 hour would equally work but I haven’t checked.

I have no idea why my earlier experiments failed.