RAID on Linux

These are just a few notes on aspects of RAID that confused me when attempting to set it up on a computer for the first time. There are many sites that give instructions for setting up RAID, but most of them jump straight into the details without providing the really basic information I wanted before getting started. I am attempting here to answer at least some of the basic questions, and then point you to other sites that contain the rest of the information you'll need.

The information provided here was current as of June 2013. My system is running Linux as the sole operating system, so dual booting is not covered.

What is RAID?

RAID stands for Redundant Array of Independent Disks (originally it was "Inexpensive" disks). There are several different kinds of RAID.
RAID 0 does block level striping. In other words it splits the data between two or more disks. Since each disk only has to read half the data, the data is available twice as quickly. However there is no redundancy, so if either disk fails, you're in trouble.
RAID 1 is the type I'm interested in here. To quote from Wikipedia, "in RAID 1 data is written identically to two drives, thereby producing a mirrored set". If one drive fails your data is still available on the other drive. Furthermore, you can replace the faulty drive and rebuild the RAID array.

There are other types of RAID which I won't go into here in the interests of brevity. Some good links for further information...
Dedoimedo site
Linux home networking site
Linux RAID Wiki
Ubuntu installation - RAID

 

How is RAID implemented?

  (see also here).
RAID can be implemented in hardware, firmware or software.
Hardware RAID used to offer performance benefits (not so much these days with fast CPUs) but according to the sites I've read, if your hardware RAID card dies, you have to find an exact replacement card to be able to read your disks. Because everything is done in hardware, Linux sees the RAID array as a normal disk. For monitoring the disks to see if one of them has failed, you use software that came with the RAID card - well I think this is true... hard to find information on this topic.
***Since I don't have a hardware RAID card, the rest of this document deals only with software and firmware RAID.***
Firmware RAID a.k.a. fake FAID is where your computer's BIOS can be configured to present two installed disks as a RAID array to your operating system. It does not have a separate RAID card as in hardware RAID. Once booted, your operating system implements RAID in software. Why bother with firmware RAID then? As far as I can tell, the only reason is to allow dual booting, but I have yet to find a web site where this is explicitly spelled out. You should also be aware that if you go into your computer's BIOS and configure it for firmware RAID, all data on your disks will be lost, even if you go back and unconfigure firmware RAID. Some sites say fake RAID is terrible, others like it.
Software RAID does everything in the Linux kernel.

What about the disks?

Most sites say that the two (or more) disks should be identical. However the Dedoimedo site says that the size of the RAID array will be equal to the size of the smaller disk, implying that the disks do NOT have to be the same size. Furthermore, in the partitioning example on that site, the illustrations show that the root and home partitions on their disks are of slightly different sizes.

Does my computer have hardware RAID?

All the information I've found about hardware RAID talks about a dedicated card, so one way to answer this question would be to open up your computer and see if such a card is fitted. I have not found Linux software that will answer this question.

When I first started setting up RAID on our new computer, I was unaware of the distinction between hardware and firmware RAID. In June 2013 we acquired a Dell Optiplex 9010 with two 1 Tbyte disks. When I recieved the computer, I googled "dell optiplex 9010 raid" and found my way to a useful looking site with instructions for setting up RAID. Only problem was that they neglected to explicitly spell out that they were talking about BIOS RAID, not hardware RAID.

Do I set up the disks as a RAID array, and then install Linux, or do I install Linux and then convert the installation to RAID?

Instructions for either method can be found on the web.
The Dedoimedo site talks about installing RAID after Linux has been installed, ***CHECK - you later say it does the other way ***and suggests the interesting idea of using RAID 1 on your home and root partitions (where you want to preserve data) and RAID 0 on your swap partition (where you want speed and don't care about the data). I used the method described on an Ubuntu site which describes setting up the RAID array during the Linux installation. This site describes installation using the server edition of Ubuntu 12.04. It is important to note that the desktop edition does not have the necessary software to set up RAID during installation. I used the alternate edition, which does have the required software - as mentioned on the Dedoimedo site.

If I install software RAID on all partitions, including root, how will Linux boot?

I'm assuming here that you don't have a separate boot partition, and that the booting stuff is within /boot in the root partition. So the question comes down to - if I'm using Linux software to do RAID, but the system is booting and thus has not loaded Linux yet, how will the boot process be able to read the disks in order to load the Linux kernel?
The answer is simple - GRUB can handle RAID 1 disks, so it can load the Linux kernel. How does GRUB get loaded? In RAID 1, the disks are mirrored, so GRUB stage 1 is automatically installed in the master boot record (MBR) of each disk. So at boot time the BIOS reads the MBR of a disk (doesn't matter which one) as if RAID was not installed, and from then on, GRUB runs and understands RAID.
For other versions of RAID, things are more complicated - see the Dedoimedo site. (Mention /etc/mdadm.conf here).

Which to use - software RAID or fake RAID?

The general consensus on the web appears to be in favour of software RAID.
One web site expressed the opinion that fake RAID combines the bad bits of both software and hardware RAID, without either of their advantages. Another site suggested that if necessary, you could pull out a drive configured for software RAID and use it as a normal drive (with data intact) in another computer running Linux. You could not do that with fake RAID or hardware RAID. There's a good answer to this question here.  

What version of GRUB am I running?

(This applies to systems where Linux is already installed).
GRUB is the GRand Unified Boot loader which is responsible for loading the operating system.

In a terminal type

grub-install -v
If the command is not found, you might need to type
/sbin/grub-install -v
or
/usr/sbin/grub-install -v
If the old version of GRUB is running, you will see something like
grub-install (GNU GRUB 0.97)
If GRUB version 2 is running, you will see something like
grub-install.real (GRUB) 1.99-21ubuntu3.9
As you can see, old GRUB has a version number starting with zero, and GRUB 2 has a reported version number starting with "1".

I've been presented with a computer running RAID. How to I tell what kind of RAID it is running?

(hardware, raid 0, 1 etc.)  

Installing Software RAID

The real life example on the Dedoimedo site starts with two hard disks that are already partitioned, although the partitions are not exactly the same sizes. Of course if you're starting with blank disks, the first step is to partition each one as you would for a normal Linux installation. Even if your disks are already partitioned, if the partitions are not the same size on each disk, you should edit the partitions so that they are the same. Obviously of one disk has a 20 Gbyte root partition, and the other has an 18 Gbyte root partition, RAID will use the smaller size, and you'll be wasting 2 Gbytes on one of the disks.

blkid

This command (as root) shows the UUIDs of each disk (/dev/sd* and /dev/md*). The ones for sda and sdb are the same (sda1 = sdb1 etc) but the numbers do not agree with those in mdadm.conf

Update

Problems appear to be fixed as of 26 July 2015.
(new disk, apt-get dist-upgrade, reboot, plus I created 2 missing symbolic links in /dev/disk/by-uuid)

root@bloch:~# ll /dev/disk/by-uuid
lrwxrwxrwx 1 root root   9 Jul 28 08:53 2a8e0fbf-6b75-41e3-a01a-822779930812 -> ../../md2
lrwxrwxrwx 1 root root   9 Jul 28 08:53 46db580b-e2b6-4352-a35c-98dc54142184 -> ../../md0
lrwxrwxrwx 1 root root  11 Jul 28 08:53 d5168796-7758-427e-b6cd-b92f41c719bd -> ../../md127

root@bloch:~# cat /etc/mdadm/mdadm.conf 
# definitions of existing MD arrays
ARRAY /dev/md/0 metadata=1.2 UUID=9b1b529c:a3542f6c:539bbd7c:880c9bc2 name=gorter:0
ARRAY /dev/md/1 metadata=1.2 UUID=d828c5fd:409d491f:e395ecfb:c65d75e7 name=gorter:1
ARRAY /dev/md/2 metadata=1.2 UUID=e8c6a464:7536fe7b:963f149d:96ca1e21 name=gorter:2

root@bloch:~# blkid
/dev/sda1: UUID="9b1b529c-a354-2f6c-539b-bd7c880c9bc2" UUID_SUB="d718c78b-7e64-089b-89c2-e52c2403b430" LABEL="gorter:0" TYPE="linux_raid_member" 
/dev/sda2: UUID="d828c5fd-409d-491f-e395-ecfbc65d75e7" UUID_SUB="31dff0f6-29b7-3daa-5100-24365d83b958" LABEL="bloch:1" TYPE="linux_raid_member" 
/dev/sda3: UUID="e8c6a464-7536-fe7b-963f-149d96ca1e21" UUID_SUB="ce8de045-eefe-3fbf-eb4e-602ade4e8d26" LABEL="gorter:2" TYPE="linux_raid_member" 
/dev/sdb1: UUID="9b1b529c-a354-2f6c-539b-bd7c880c9bc2" UUID_SUB="e7c27528-43c7-5439-2e06-bac0ff2f5a1f" LABEL="gorter:0" TYPE="linux_raid_member" 
/dev/sdb2: UUID="d828c5fd-409d-491f-e395-ecfbc65d75e7" UUID_SUB="2ea436a5-5c62-5ecc-063d-4344a3f875eb" LABEL="bloch:1" TYPE="linux_raid_member" 
/dev/sdb3: UUID="e8c6a464-7536-fe7b-963f-149d96ca1e21" UUID_SUB="3a4a8681-2b1e-e874-045a-6fc67467615b" LABEL="gorter:2" TYPE="linux_raid_member" 
/dev/md2: UUID="2a8e0fbf-6b75-41e3-a01a-822779930812" TYPE="ext4" 
/dev/md127: UUID="d5168796-7758-427e-b6cd-b92f41c719bd" TYPE="swap" 
/dev/md0: UUID="46db580b-e2b6-4352-a35c-98dc54142184" TYPE="ext4" 


root@bloch:~# cat /proc/mdstat 
Personalities : [raid0] [raid1] [linear] [multipath] [raid6] [raid5] [raid4] [raid10] 
md0 : active raid1 sdb1[1] sda1[2]
      39028608 blocks super 1.2 [2/2] [UU]
      
md127 : active raid0 sdb2[1] sda2[0]
      23436288 blocks super 1.2 512k chunks
      
md2 : active raid1 sdb3[1] sda3[2]
      925849408 blocks super 1.2 [2/2] [UU]
      
unused devices: 

Note that /dev/md1 has been automatically replaced by /dev/md127 for some reason. At least swap space is now working...

root@bloch:~# swapon -s
Filename                                Type            Size    Used    Priority
/dev/md127                              partition       23436284        0       -1

Australian National University NMR Centre home page   |   Setting Up WebEx