RAID

I had a lot of trouble getting a RAID card to work. It finally worked because 3ware RAID cards are supported in Linux kernel 2.6.19 or later. I had to switch the distribution I was using from Debian Etch to openSUSE 10.3 because Debian Etch uses 2.6.18.

I finally got RAID to work using openSUSE 10.3 and a 3ware 9650EX-4LPML RAID card. I am using four 400 GB SATA hard drives, one from Western Digital and three from Seagate. If you are using larger hard drives you will need to mount the array after you have installed the operating systems as the installer does not support volumes larger than 2.0 GB. Check out the section on AutoCarving in the user manaul for details. While I detail my difficulties below, I will start with the success. This description assumes that you have a separate hard drive for the OS and will mount the RAID array as /home.

The simplest way to do this is to set up RAID when you install the operating system. Install the card and 4 hard drives. When the RAID BIOS is recognized onscreen, hit alt+3 to enter RAID setup. Follow the prompts to create a RAID 5 logical volume using all four drives. You will then need to initialize the array. This will take a long time.

When the array is initialized, save and exit the BIOS screen and continue booting from the installation disk. Follow the prompts and when when it gets the disc usage part select the single hard drive to install / and pick the RAID array to install /home. Complete the OS installation and you are done. Your array is then seamlessly integrated as /home which is where you store all your data.

The array will appear in /dev as /dev/sdb and the volume as /dev/sdb1. The hard drive with the OS is /dev/sda and the volume is /dev/sda1.

If you do not set up the array during installation, you can add it after the fact using the command line. I was not able to create a volume of format the array using GParted. I had to use fdisk and mkfs. It went like this.

#fdisk /dev/sdb
type n for a new volume. accept the defaults for the start and finish. This will use the entire array as a single volume.
type t to change the type and select 83, which is a Linux partition.
p will print the partition table to the screen. It should look like this

Device Boot Start End Blocks ID System
/dev/sdb1 1 4088 2060320 83 Linux

w will save the changes. Note that you will need to reboot before the new partition table will be used so don't try to mount the new volume until after you have rebooted. But don't reboot yet.

#mkfs.ext3 /dev/sdb1

this will format the array with ext3. This will take a while
When it is done, reboot then

#gedit /etc/fstab

add the following line
/dev/sdb1 /home ext3 defaults 1 2

then

#mount -a

you're done.

Some of the references here came from this article on software RAID.

*********************************************

The rest of this describes the long string of failures I had

*********************************************

I bought a HighPoint RocketRAID 2300 and downloaded the latest BIOS/Driver from their website. I am using kernel 2.6.18.5. The website states that you can use 2.4.x or 2.6.x kernels, but when I try to "make", I get the error message

expr: syntax error
../../../inc/linux/Makefile.def:85: *** Only kernel 2.4/2.6 is supported but you use 2.. Stop.

Doing some research at I came across this list of RAID cards with comments. Based on the reference to the Promise EX8350, I bought the Promise EX4350 which is supposed to be the same thing but with 4 ports in stead of 8.

A discussion of RAID levels can be found at Wikipedia.

Debian x386 device database Promise EX4350
uses stex module
PCI ID = 105a:4302

13 January 2008

The basic steps for setting up the Promise RAID array are:
1. Install the controller card and hard drives
2. Create a logical volume.
3. Compile the driver.
4. Set up WebPAM

I installed the card and 3 hard drives and created a logical volume. Each disk is 400 GB, so using RAID 5 the volume has 800 GB capacity. I plan to add a fourth drive later, but I need to get a 5.25 to 3.5 bay adapter. The case claims to hold 6 3.5 drives, but two of them are floppy sized. The floppy bays are smaller and provide less room for airflow, so I decided to stick the fifth drive in the unused 5.25 in bay.

I ran into a problem when I tried to compile the driver however. The driver is compiled as a kernel patch and this required me to download the kernel source code (luckily it is in Synaptic (linux-source-2.6.18)). It then uses the kernel configuration information to configure the driver. I got an error that stated that the configuration file did not match my kernel configuration.

Next step. Try and find out what my kernel configuration is. I wonder if I would be having this problem if I installed from a full set of disks instead of the NetInstall.

When in installed the linux source code from Synaptic, it but a tar ball on my hard drive at
/usr/src/linux-source-2.6.18.tar.bz2

next run the command
# bzip2 -cd linux-source-2.6.18.tar.bz2 | tar xvf -
this will make a folder
/usr/src/linux-source-2.6.18
with all the source code files in it

Next, go to that directory as root
#cd /usr/src/linux-source-2.6.18
#make mrproper
#make oldconfig
#make prepare
#make modules_prepare

Next, download the driver source file from www.promise.com
(http://www.promise.com/upload/Support/Driver/shasta_linux_src_2.9.0.22.tgz)
and unzip it to a directory
/home/user/Desktop/Downloads/2.9.0.22
This directory will contain 3 files, Makefile, Readme and stex.c
Make a backup copy of Makefile then open it with a Linux based text editor (do not use notepad) and change the kernel source directory line from...

################# kernel configuration ###############

ifeq ("x", "x$(KERNEL_SOURCE_DIR)")
KERNEL_SRC_DIR=$(shell if ls /lib/modules/`uname -r`/build >/dev/null 2>&1; then echo /lib/modules/`uname -r`/build; elif ls /lib/modules/`uname -r`/source >/dev/null 2>&1; then echo /lib/modules/`uname -r`/source ;elif ls /usr/src/linux-2.6 >/dev/null 2>&1; then echo /usr/src/linux-2.6; else echo /usr/src/linux; fi)
else
KERNEL_SRC_DIR=$(KERNEL_SOURCE_DIR)
endif

to...

################# kernel configuration ###############

ifeq ("x", "x$(KERNEL_SOURCE_DIR)")
KERNEL_SRC_DIR=$(shell if ls /lib/modules/`uname -r`/build >/dev/null 2>&1; then echo /lib/modules/`uname -r`/build; elif ls /lib/modules/`uname -r`/source >/dev/null 2>&1; then echo /lib/modules/`uname -r`/source ;elif ls /usr/src/linux-2.6 >/dev/null 2>&1; then echo /usr/src/linux-2.6; else echo /usr/src/linux-source-2.6.18; fi)
else
KERNEL_SRC_DIR=$(KERNEL_SOURCE_DIR)
endif


Next, go to the directory with the driver source
#cd /home/user/Desktop/Downloads/2.9.0.22
#make clean
#make KERNEL_SOURCE_DIR=$KERN_SRC_DIR

if everything worked, this will take a while. It took about 20 minutes for me.

This will make a file called stex.ko. The final step is to install this driver file.
#insmod stex.ko

I did all of this but could not get the array to show up anywhere I could do anything with it. In retrospect maybe all I needed to do was mount the array, but I did not think of it at the time and don't know if it would actually have solved all my problems. At any rate I tried following the directions to get WebPAM working and got nowhere. At this point I gave up on hardware RAID and decided to use software RAID.

Following the instructions at this website, Setting up Linux software RAID, I made four volumes from one hard drive since I only had two SATA ports on the motherboard. I was able to do everything successfully so I went and bought a PCIe to SATA controller card only to discover that an amateur Linux user such as myself couldn't make that work either. I needed something that would work out of the box with minimal configuration. Compiling kernel drivers was stretching my knowledge and my patience.

My research led me to 3ware. I bought the 3ware 9650SE-4lpl card. It turns out that 3ware drivers part of the Linux kernel version 2.6.19 or later. Since Debian Etch is on 2.6.18 I went looking for a way to get 3ware drivers on Debian. I found a Debian Etch ISO created by 3ware that incorporated the drivers I needed, but could not get it to install. I tried Debian Lenny (testing) but could not get it to install either. My patience had grown short and I did not give either of the options much time. I had recieved a copy of Windows server 2003 and was getting closer and closer to ditching Linux and using that. Only my contempt for windows kept me from jumping ship.

I finally decided to abandon Debian and go to one of the distributions that was explicitly supported in the driver and had a 2.6.19 or greater kernel. The answer ended up being openSUSE 10.3. I installed the 3ware card and 3 drives, then installed the openSUSE OS. It recognized the array right away and allowed me to mount it as /home during installation. I was finally on my way. Details of this successful installation are in the RAID that did work chapter.