I have recently been building a new NAS box (of which, possibly, more later). In fact the build is really a rebuild because I initially built the server about three years ago in order to consolidate a bunch of services I was running on assorted separate servers into one place. That first build was a RAID 1 array of two 2 TB disks (to give me a mirrored setup with a total of 2 TB store). At the time that was sufficient to hold all my important data (backed up both to other networked devices and to standalone USB disks for safety). But I have just upgraded my main desktop machine to a nice shiny new core i7 Skylake box with 16 GB of DDR4 and a 3 TB disk. That disk is already two thirds full (my old machine had a rather full 2 TB disk). This meant that my NAS backup storage requirements exceeded the capacity of my RAID 1 setup. Adding disks wouldn’t help of course because all that would do is add mirror capability rather than capacity. So I decided to upgrade the NAS and bought a bunch of new 2 TB disks with the intention of setting up a RAID 5 array of 4 disks, thus giving a total storage capacity of 6 TB (8 TB minus 2 TB for parity). Furthermore I initially looked at using FreeNAS rather than my usual debian or ubuntu server with software RAID simply because it looked interesting and, with plugins, could probably meet most of my requirements. But I could not get the software to install properly and after three abortive attempts I gave up and decided that I didn’t really like freeBSD anyway….
So I opted to go back to mdadm on linux – at least I know that works. Better still I would be able to retain all my old setup from the old RAID 1 system without having to worry about finding plugins to handle my media streaming requirements, or owncloud installation, for example.
My previous build was on debian (which is by far my preferred server OS) but ubuntu server has recently been released in a LTS version at 16.04 and I thought it might be fun to try that instead. So I did. (For any readers who have not tried installing linux on a RAID system there are plenty of sites offering advice, but the official ubuntu pages are pretty good). During the build I hit what I initially thought was a snag because the installation seemed to get stuck at around the 83% level when it was apparently installing the linux kernel image and headers. Indeed I confess that on the first such installation I pulled the plug after about three hours of no apparent activity because I was beginning to think that there might be something wrong with my hardware (the earlier FreeNAS failures worried me). My on-line searches for assistance were initially not particularly helpful since none of the huge number of sites advising on software RAID installation bothered to mention that initial RAID 5 build (or rebuild) using large capacity disks takes a very long time because of the need to calculate the parity data. Incidentally, it is this parity data and its layout that gives RAID 5 its write performance penalty.
One useful outcome of my research about RAID 5 build times (which in my case eventually took just over 6 hours) was my discovery of the wintelguy’s site providing an on-line calculator (and much more besides) for RAID performance and capacity. There is even a very useful page allowing you to compare two separate configurations side by side – thoroughly recommended. More worrying, and thought provoking, is the reclaime.com calculator for RAID failure. That site suggests that the probability of successfully rebuilding a RAID 5 array of 4 * 2 TB disks after a failure is only 52.8%.
That is why you need to keep backups…….