Next
Previous
Contents
There can be many good reasons for using RAID. A few are; the ability
to combine several physical disks into one larger ``virtual'' device,
performance improvements, and redundancy.
Linux RAID can work on most block devices. It doesn't matter whether
you use IDE or SCSI devices, or a mixture. Some people have also used
the Network Block Device (NBD) with more or less success.
Since a Linux Software RAID device is itself a block device, the above
implies that you can actually create a RAID of other RAID
devices. This in turn makes it possible to support RAID-10
(RAID-0 of multiple RAID-1 devices), simply by using the RAID-0 and
RAID-1 functionality together. Other more exotic configurations, such
a RAID-5 over RAID-5 "matrix" configurations are equally
supported.
The RAID layer has absolutely nothing to do with the filesystem
layer. You can put any filesystem on a RAID device, just like any
other block device.
Often RAID is employed as a solution to performance problems. While
RAID can indeed often be the solution you are looking for, it is not a
silver bullet. There can be many reasons for performance problems, and
RAID is only the solution to a few of them.
In the description of The RAID levels, there will be a
mention of the performance characteristics of each level.
The word ``RAID'' means ``Linux Software RAID''. This HOWTO does not
treat any aspects of Hardware RAID. Furthermore, it does not treat any
aspects of Software RAID in other operating system kernels.
When describing RAID setups, it is useful to refer to the number of
disks and their sizes. At all times the letter N is used to
denote the number of active disks in the array (not counting
spare-disks). The letter S is the size of the smallest drive
in the array, unless otherwise mentioned. The letter P is
used as the performance of one disk in the array, in MB/s. When used,
we assume that the disks are equally fast, which may not always be
true in real-world scenarios.
Note that the words ``device'' and ``disk'' are supposed to mean about
the same thing. Usually the devices that are used to build a RAID
device are partitions on disks, not necessarily entire disks. But
combining several partitions on one disk usually does not make sense,
so the words devices and disks just mean ``partitions on different
disks''.
Here's a short description of what is supported in the Linux RAID
patches. Some of this information is absolutely basic RAID info, but
I've added a few notices about what's special in the Linux
implementation of the levels. Just skip this section if you know
RAID.
The current RAID patches for Linux supports the following
levels:
- Linear mode
- Two or more disks are combined into one physical device. The
disks are ``appended'' to each other, so writing linearly to the RAID
device will fill up disk 0 first, then disk 1 and so on. The disks
does not have to be of the same size. In fact, size doesn't matter at
all here :)
- There is no redundancy in this level. If one disk crashes you
will most probably lose all your data. You can however be lucky to
recover some data, since the filesystem will just be missing one large
consecutive chunk of data.
- The read and write performance will not increase for single
reads/writes. But if several users use the device, you may be lucky
that one user effectively is using the first disk, and the other user
is accessing files which happen to reside on the second disk. If that
happens, you will see a performance gain.
- RAID-0
- Also called ``stripe'' mode. The devices should (but need not)
have the same size. Operations on the array will be split on the
devices; for example, a large write could be split up as 4 kB to disk
0, 4 kB to disk 1, 4 kB to disk 2, then 4 kB to disk 0 again, and so
on. If one device is much larger than the other devices, that extra
space is still utilized in the RAID device, but you will be accessing
this larger disk alone, during writes in the high end of your RAID
device. This of course hurts performance.
- Like linear, there is no redundancy in this level either. Unlike
linear mode, you will not be able to rescue any data if a drive
fails. If you remove a drive from a RAID-0 set, the RAID device will
not just miss one consecutive block of data, it will be filled with
small holes all over the device. e2fsck or other filesystem recovery
tools will probably not be able to recover much from such a device.
- The read and write performance will increase, because reads and
writes are done in parallel on the devices. This is usually the main
reason for running RAID-0. If the busses to the disks are fast enough,
you can get very close to N*P MB/sec.
- RAID-1
- This is the first mode which actually has redundancy. RAID-1 can be
used on two or more disks with zero or more spare-disks. This mode maintains
an exact mirror of the information on one disk on the other
disk(s). Of Course, the disks must be of equal size. If one disk is
larger than another, your RAID device will be the size of the
smallest disk.
- If up to N-1 disks are removed (or crashes), all data are still intact. If
there are spare disks available, and if the system (eg. SCSI drivers
or IDE chipset etc.) survived the crash, reconstruction of the mirror
will immediately begin on one of the spare disks, after detection of
the drive fault.
- Write performance is often worse than on a single
device, because identical copies of the data written must be sent to
every disk in the array. With large RAID-1 arrays this can be a real
problem, as you may saturate the PCI bus with these extra copies. This
is in fact one of the very few places where Hardware RAID solutions
can have an edge over Software solutions - if you use a hardware RAID
card, the extra write copies of the data will not have to go over the
PCI bus, since it is the RAID controller that will generate the extra
copy. Read performance is good, especially if you have multiple
readers or seek-intensive workloads. The RAID code employs a rather
good read-balancing algorithm, that will simply let the disk whose
heads are closest to the wanted disk position perform the read
operation. Since seek operations are relatively expensive on modern
disks (a seek time of 6 ms equals a read of 123 kB at 20 MB/sec),
picking the disk that will have the shortest seek time does actually
give a noticeable performance improvement.
- RAID-4
- This RAID level is not used very often. It can be used on three
or more disks. Instead of completely mirroring the information, it
keeps parity information on one drive, and writes data to the other
disks in a RAID-0 like way. Because one disk is reserved for parity
information, the size of the array will be (N-1)*S, where S is the
size of the smallest drive in the array. As in RAID-1, the disks should either
be of equal size, or you will just have to accept that the S in the
(N-1)*S formula above will be the size of the smallest drive in the
array.
- If one drive fails, the parity
information can be used to reconstruct all data. If two drives fail,
all data is lost.
- The reason this level is not more frequently used, is because
the parity information is kept on one drive. This information must be
updated every time one of the other disks are written
to. Thus, the parity disk will become a bottleneck, if it is not a lot
faster than the other disks. However, if you just happen to have a
lot of slow disks and a very fast one, this RAID level can be very useful.
- RAID-5
- This is perhaps the most useful RAID mode when one wishes to combine
a larger number of physical disks, and still maintain some
redundancy. RAID-5 can be used on three or more disks, with zero or
more spare-disks. The resulting RAID-5 device size will be (N-1)*S,
just like RAID-4. The big difference between RAID-5 and -4 is, that
the parity information is distributed evenly among the participating
drives, avoiding the bottleneck problem in RAID-4.
- If one of the disks fail, all data are still intact, thanks to the
parity information. If spare disks are available, reconstruction will
begin immediately after the device failure. If two disks fail
simultaneously, all data are lost. RAID-5 can survive one disk
failure, but not two or more.
- Both read and write performance usually increase, but can be hard to
predict how much. Reads are similar to RAID-0 reads, writes can be
either rather expensive (requiring read-in prior to write, in order to
be able to calculate the correct parity information), or similar to
RAID-1 writes. The write efficiency depends heavily on the amount of
memory in the machine, and the usage pattern of the array. Heavily
scattered writes are bound to be more expensive.
Spare disks
Spare disks are disks that do not take part in the RAID set until one
of the active disks fail. When a device failure is detected, that
device is marked as ``bad'' and reconstruction is immediately started
on the first spare-disk available.
Thus, spare disks add a nice extra safety to especially RAID-5 systems
that perhaps are hard to get to (physically). One can allow the system
to run for some time, with a faulty device, since all redundancy is
preserved by means of the spare disk.
You cannot be sure that your system will keep running after a disk
crash though. The RAID layer should handle device failures just fine,
but SCSI drivers could be broken on error handling, or the IDE chipset
could lock up, or a lot of other things could happen.
Also, once reconstruction to a hot-spare begins, the RAID layer will
start reading from all the other disks to re-create the redundant
information. If multiple disks have built up bad blocks over time, the
reconstruction itself can actually trigger a failure on one of the
"good" disks. This will lead to a complete RAID failure. If you do
frequent backups of the entire filesystem on the RAID array, then it
is highly unlikely that you would ever get in this situation - this is
another very good reason for taking frequent backups. Remember, RAID
is not a substitute for backups.
There's no reason to use RAID for swap performance reasons. The kernel
itself can stripe swapping on several devices, if you just give them
the same priority in the fstab file.
A nice fstab looks like:
/dev/sda2 swap swap defaults,pri=1 0 0
/dev/sdb2 swap swap defaults,pri=1 0 0
/dev/sdc2 swap swap defaults,pri=1 0 0
/dev/sdd2 swap swap defaults,pri=1 0 0
/dev/sde2 swap swap defaults,pri=1 0 0
/dev/sdf2 swap swap defaults,pri=1 0 0
/dev/sdg2 swap swap defaults,pri=1 0 0
This setup lets the machine swap in parallel on seven SCSI devices. No
need for RAID, since this has been a kernel feature for a long time.
Another reason to use RAID for swap is high availability. If you set
up a system to boot on eg. a RAID-1 device, the system should be able
to survive a disk crash. But if the system has been swapping on the
now faulty device, you will for sure be going down. Swapping on a
RAID-1 device would solve this problem.
There has been a lot of discussion about whether swap was stable on
RAID devices. This is a continuing debate, because it depends highly
on other aspects of the kernel as well. As of this writing, it seems
that swapping on RAID should be perfectly stable, you should however
stress-test the system yourself until you are satisfied with the
stability.
You can set up RAID in a swap file on a filesystem on your RAID
device, or you can set up a RAID device as a swap partition, as you
see fit. As usual, the RAID device is just a block device.
Next
Previous
Contents