Our website would like to use cookies to store information on your computer. You may delete and block all cookies from this site, but parts of the site will not work as a result. Find out more about how we use cookies.

Login or Register

Powered by
Powered by Novacaster
 
CentOS 4, 3Ware controllers etc
by Simon at 14:51 21/06/06 (Blogs::Simon)
Musings on the theme of trying to install CentOS 4.3 (2.6.9-34.ELsmp) on a dual Opteron 2.4GHz 250 Supermicro with a 3Ware 9550SX-8LP, 4 250GB SATA 7V250F0 Maxtor disks and 4GB of RAM.
So let's get started.

First off, we need a driver diskette for the 3Ware card so we can boot the installer with linux dd otherwise we won't be able to see the hardware RAID1 mirror (with two spare disks) that is configured. There is a driver for 3Ware cards in the kernel of Centos 4.3, but the 9550SX is fairly new and isn't recognised by it.

3Ware provide an updated driver for CentOS 4.3 as a downloadable ZIP file, the contents of which go on a 1.44MB floppy. The universe was kind to me, and it only took two attempts to find and format a usable floppy out of the ancient stack, so it's bye bye original MacRascol 35mm file recorder driver software (14 years old).

Kick off the installation process, feeding it the driver for the 3Ware card in the process and off it goes. I'm doing a minimal install for speed (ha ha).

First problem - the machine hangs while formatting /boot as ext3. I chose to use DiskDruid and set up a 100MB /boot partition, the rest of the space being split between 2GB swap and /. Why? Because I'm suspicious of LVM, and it's the default if you use the installer's Autopartition feature.

So why has it hung? Much Googling provides no answers, so I (as Hugo says) "push it back up the hill" for another go.

This time, I hit alt-F5 to watch what's going on when the formatting step is running, instead of leaving the X-based installer screen up (alt-F7) with its pretty (but uninformative) progress bar.

Surprisingly, this time round the format completes successfully even though writing the inode tables on / takes 15 minutes with long pauses every couple of hundred or so. Not being used to formatting such luxuriously expansive amounts of disk space, I don't know whether this is typical or not.

Package installation also takes what seems a long time (6 minutes), it's only 679MB or so after all, from local CDROM. I guess I've forgotten how long an installation usually takes.

Reboot, success!

In my enthusiasm, I immediately run yum update and blithely install a newer version of the kernel (2.6.9-34.0.1.ELsmp) along with various other updates.

Sadly, this then breaks on reboot with the error mkrootdev: label /1 not found, after which we get a few reports relating to the inability to mount a root device and finally collapsing with Kernel panic - not syncing: Attempted to kill init!

Drat.

Booting the original kernel works fine, so what's gone wrong? I begin to suspect that the new kernel's initrd is at fault[1] and that I've overlooked the fact that the new kernel will have the old driver built into its initrd, and I probably need to re-make it before trying to boot with the new kernel.

Therefore, having first nuked the box from orbit and reinstalled to get me back to the stage prior to my original yum update so I can proceed in a slightly more orderly fashion, I do the update again and this time take a look at the relevant 3Ware .ko files immediately afterwards:

[root@dyn-102 /]# find /lib -name 3w-9\* | xargs ls -l
-rwxr--r-- 1 root root 38528 May 24 15:26 /lib/modules/2.6.9-34.0.1.EL/kernel/drivers/scsi/3w-9xxx.ko
-rwxr--r-- 1 root root 40900 May 24 15:26 /lib/modules/2.6.9-34.0.1.ELsmp/kernel/drivers/scsi/3w-9xxx.ko
-rwxr--r-- 1 root root 38528 Mar 8 07:07 /lib/modules/2.6.9-34.EL/kernel/drivers/scsi/3w-9xxx.ko
-rwxr--r-- 1 root root 40900 Mar 8 07:07 /lib/modules/2.6.9-34.ELsmp/kernel/drivers/scsi/3w-9xxx.ko
-rw-r--r-- 1 root root 241176 Apr 4 15:26 /lib/modules/2.6.9-34.ELsmp/updates/3w-9xxx.ko
-rw-r--r-- 1 root root 236731 Apr 4 15:26 /lib/modules/2.6.9-34.EL/updates/3w-9xxx.ko

I replace the older 3w-9xxx.ko files with the file from the relevant updates directory, to end up with this:

[root@dyn-102 log]# find /lib -name 3w-\9* | xargs ls -l
-rwxr--r-- 1 root root 236731 Jun 21 13:07 /lib/modules/2.6.9-34.0.1.EL/kernel/drivers/scsi/3w-9xxx.ko
-rwxr--r-- 1 root root 241176 Jun 21 13:05 /lib/modules/2.6.9-34.0.1.ELsmp/kernel/drivers/scsi/3w-9xxx.ko
-rwxr--r-- 1 root root 236731 Jun 21 10:35 /lib/modules/2.6.9-34.EL/kernel/drivers/scsi/3w-9xxx.ko
-rwxr--r-- 1 root root 241176 Jun 21 10:33 /lib/modules/2.6.9-34.ELsmp/kernel/drivers/scsi/3w-9xxx.ko
-rw-r--r-- 1 root root 241176 Apr 4 15:26 /lib/modules/2.6.9-34.ELsmp/updates/3w-9xxx.ko
-rw-r--r-- 1 root root 236731 Apr 4 15:26 /lib/modules/2.6.9-34.EL/updates/3w-9xxx.ko

Time to build a test initrd for the new kernel in a temporary work area:

[root@dyn-102 3401newinitrd]# /sbin/mkinitrd -v -f initrd-3401-t1.img.gz 2.6.9-34.0.1.ELsmp

Now to unpack it and see what's inside. I've previously unpacked both the original kernel's initrd (containing the correct 3Ware driver it was fed at the outset) and the initrd from the new kernel installed as part of the yum update in a similar fashion into parallel directories (/34/ and /3401/) so I'll be able to compare them all against each other:

[root@dyn-102 3401newinitrd]# gunzip initrd-3401-t1.img.gz
[root@dyn-102 3401newinitrd]# cpio -i < initrd-3401-t1.img
2010 blocks
[root@dyn-102 3401newinitrd]# cd ../
[root@dyn-102 kernelstuff]# find . -name 3w-\* | xargs ls -l
-rw-r--r-- 1 root root 40900 Jun 21 12:18 ./3401/lib/3w-9xxx.ko
-rw-r--r-- 1 root root 41888 Jun 21 13:30 ./3401newinitrd/lib/3w-9xxx.ko
-rw-r--r-- 1 root root 41888 Jun 21 12:10 ./34/lib/3w-9xxx.ko

That looks a bit more promising - the new initrd's copy of the 3Ware driver is the same size as the one that's working and both are bigger than the driver that came along by default in the updated kernel via yum.

This time, let's create the new initrd with a proper filename in the /boot directory. No point keeping the old ones - they don't work anyway.

[root@dyn-102 kernelstuff]# /sbin/mkinitrd -v -f /boot/initrd-2.6.9-34.0.1.ELsmp.img 2.6.9-34.0.1.ELsmp

Might as well do the non-smp one at the same time:

[root@dyn-102 kernelstuff]# /sbin/mkinitrd -v -f /boot/initrd-2.6.9-34.0.1.EL.img 2.6.9-34.0.1.EL

Fingers crossed - reboot using the 34.0.1 kernel and its new initrd.... success!

Now to work out how to get all that into a kickstart %post section....
--
simon

[1] It isn't at fault, of course, I am.

<< Sunny Solstice 2006? Maximum Southerly Lunar Declin... >>
View Comments (Flat Mode) Printer Version
CentOS 4, 3Ware controllers et... Simon - 21/06
    Re: CentOS 4, 3Ware controller... Jon Maidment - 11/07
       Re: CentOS 4, 3Ware controller... Simon - 11/07
          Re: CentOS 4, 3Ware controller... There is an attachment here Jon Maidment - 11/07
             Re: CentOS 4, 3Ware controller... Simon - 12/07