Debian 6 (squeeze) has been the current stable version since February 2011. The latest version (6.02) was released in late June. I have put off updating my slugs from lenny (old stable) for a while because I wanted to see how others faired before committing myself. Indeed, initial reports on the debian arm list indicated that the upgrade could be problematic. Worse, a completely clean install of squeeze turns out to be impossible because the debian installer uses more memory than is physically available on the slugs. So the only way to go, even for a clean new installation, is to install lenny first, then upgrade.
Given that both my slugs are operational, and are now an integral part of my network, I decided to invest in a new one as a development machine to test the upgrade rather than risk fritzing a perfectly good setup. (Back in the day I would have been happy to “fix it ’till it broke”, but these days I don’t really need to experiment that much and I’d rather keep a working system, well, working).
Second hand slugs go for around £25 on ebay, and there are still plenty about, so I bid for one that had only about a day to go and was successful. Unfortunately, when it turned up I found that the power supply was fsckd and so I had to switch off one of my operational slugs in order to test the new one. Happily it appeared to boot up OK so I fired off a disgruntled email to the seller and then ordered a new PSU. The seller claimed that it “worked OK when I boxed it” and didn’t offer to pay for the replacement PSU so I wasn’t too happy with him. I became even less happy when the new PSU arrived and I booted up the slug in preparation for reconfiguration to match my network before installing debian.
The debian installation process is handled via an SSH shell. You need to know the address of the slug in order to connect and install. The installer also needs the addresses of a local DNS server and the default route to the outside world (so it can find the servers containing the installation packages). Now the default, factory settings, for slugs includes a fixed IP address of 192.168.1.77. If this does not match your requirements, it must be changed before reflashing with debian. Guess what? The default address didn’t work, so the previous owner must have reconfigured the slug to match his network and he had not bothered to reset to factory default before selling. Nor had he been considerate enough to let me know the new configuration. Needless to say I won’t be buying anything else from him. Nor did he get decent feedback.
I couldn’t reconfigure the new slug until I could connect so I needed to find out what address it was using. A quick nmap scan of the 192.168.1.0/24 netblock showed that it wasn’t even on the default network range so I fired up wireshark and etherape on one of my machines in the hope of catching the slug arping and getting the address from the request. In the event, etherape proved to be quicker (and easier) in providing the answer since the slug quickly popped up and disclosed its IP address as 192.168.2.10. Adding a route to the 192.168.2.0/24 net then allowed me to finally connect and reconfigure the new beast to suit my network. I then rebooted and started a fresh installation of lenny (as previously described in one of my earlier posts). About four hours later I had a nice new clean slug running lenny.
However, since the main purpose of getting the new slug was to allow me to test the upgrade to squeeze in safety I really needed to make it look like my main operational slug. That slug is my DNS and DHCP server, and primary rsync backup for my desktops. It also runs a webserver. Like most (lazy) sysadmins my system documentation tends to lag somewhat behind reality so I can’t rely on the various readme files I routinely create on my boxes to be completely up to date (or even accurate). Fortunately for me though, debian provides a neat way of snapshotting installed packages on a system. You can then use this snaphot to create a mirror of that system which will include all the same packages. Here’s how:
On the source system:
dpkg –get-selections | grep -v deinstall > packages.txt
This lists all active packages, except those deinstalled, and sticks the list in a text file.
Now copy that file to the target system, ensure that the target system’s “sources.list” file matches that on the source, and then run:
dpkg –clear-selections
dpkg –set-selections < packages.txt apt-get dselect-upgrade
This will download and install all the packages necessary to get the target system matching the source.
All that is now left to do is copy across any relevant configuration files so that the two systems fully match and then rebooot the target to check that everything looks OK.
New slug now finally matching old slug it was time to upgrade to squeeze. Martin Michalmayr’s excellent website documents the upgrade process in meticulous detail. The key points to note here are his recommendation that you read the release notes for debian 6.0. In particular, note and follow the chapter on upgrades from debian 5.0 before attempting an actual upgrade. One of the main differences between 5.0 and 6.0 is the use of UUIDs to reference disks. In my case this meant changing my /etc/fstab from this:
# /etc/fstab: static file system information.
#
#proc /proc proc defaults 0 0
/dev/sda2 / ext3 errors=remount-ro 0 1
/dev/sda1 /boot ext2 defaults 0 2
/dev/sda5 none swap sw 0 0
to this:
# /etc/fstab: static file system information.
#
#proc /proc proc defaults 0 0
UUID=db57451a-e3e5-4d8a-95b9-494c48bb5e8d / ext3 errors=remount-ro 0 1
UUID=022bc211-1c52-4848-9ee1-e211e72b28e4 /boot ext2 defaults 0 2
/dev/sda5 none swap sw 0 0
Before finally starting the upgrade I opened two separate SSH sessions to the slug. In one I ran the upgrade process as below:
first a partial upgrade as recommended at Section 4.4.4, “Minimal system upgrade”
apt-get update
apt-get upgrade
then install the required linux kernel image and udev as outlined in Section 4.4.5. “Upgrading the kernel and udev”
apt-get install linux-image-2.6-ixp4xx
apt-get install udev
followed by
reboot
and
apt-get upgrade
apt-get dist-upgrade
to complete the system upgrade.
Now here is where the second SSH session is most useful. The final upgrade and distribution upgrade installs the file indexing package “apt-xapian-index”. Correspondents on the debian arm list have noted that this package consumes more memory than is available on the poor old slug and it starts swapping itself to death. The process must be killed immediately and the package removed. If you leave it too long after the upgrade has completed you will find it impossible to log in until the initial indexation has completed (in excess of 24 hours or more has been reported) because the system is too busy. I ran “top” in the second shell during the dist-upgrade process and kept an eye on the load averages. As soon as they started climbing above 3 I knew that it was time to watch out for the apt-xapian-indexer and kill it. Once the system load returned to normal I was then able to finalise the upgrade with:
apt-get purge apt-xapian-index
apt-get autoremove
to remove the offending indexer and clean up any residual unneeded packages. A final reboot to check all was well was sufficient to convince me that it was safe to upgrade my two operational slugs using the same process. Testing the upgrade on the new slug in the way I did also meant that I now had a backup slug configured exactly like my main DNS server but running squeeze. Any failure on the remaining upgrade would not then be critical.
I must be getting old. I never used to be this cautious.