We had to install a new cluster of servers this week, and during the planning I discovered a notion bubbling into my head:

"Ya know, CentOS 5 is going to take forever to install across all these servers if I have to keep swapping CDs."

Okay, a little back story: This hadn't been an issue in the past.  CentOS offered a single ServerCD install on 4.4 that could be easily burned and deployed without much headache.  Worst case we had an installable image on a 1gig USB Pen Drive that made this incredibly easy.

Not any more.  CentOS went with either the monolithic 3.5 gigabyte DVD image, or forcing you to download 6 CDs in order to get the same effect.   With some tuning and package removal at install, we managed to get it down to 2 cds.. Remove Dialup Networking and the command line web browsing packages and you're down to 2 cds -- the second CD it needs exactly TWO packages from.. now why exactly they couldn't put all of 'em on the first CD is beyond me.. different issue.  Let's get back to it.

I REALLY didn't find it effective to sit there for multiple hours swapping CDs before being able to configure my new server cluster.

Enter PXE booting.

For the uninitiated: PXE stands for "Pre-eXecution-Environment," a techonlogy introduced by intel some years ago that is surprisingly underutilized despite every server we own supporting it.  PXE allows one to boot servers (disked or diskless) off of a common server.  This can be used to boot individual servers off of a common install, or to 'bootstrap' servers and install off of a remote server without the need for CDs or pen drives. 

This second case is what I'm covering here.

The pieces you'll need:

• a server available on the network.  This doesn't have to be a beefy workhorse.. It just needs to run some variant of linux (I used Gentoo), have a network connection, and that's about it.
• The following software packages:
• atftp - a tftp server with a small footprint and braindead configuration file
• udhcpd - dhcp server
• apache or another httpd server
• the CentOS 5 ISO CD images
• syslinux (for the pxelinux.0 bootstrap file)

Step 1: Set up atftp

This is incredibly easy.  in Gentoo, you can install atftp by doing this:

% USE="$USE pcre readline tcpd" emerge atftp

Add (or replace existing contenst with) the following in /etc/conf.d/atftp:

TFTPD_ROOT="/tftpboot"
TFTPD_OPTS="--daemon --user nobody --group nobody"

Create the directory for tftp to serve files from:

% mkdir /tftpboot

Set up a default config for servers accessing tftp to use:

% mkdir /tftpboot/pxelinux.cfg

Create the file /tftpboot/pxelinux.cfg/default:

prompt 1
default linux
timeout 100

label linux
kernel vmlinuz
append initrd=initrd.img ramdisk_size=9216 noapic acpi=off

Save that file.

Next, Start aftpd (as root):

% /etc/init.d/atftp start

Step 2: Set up udhcp:

Some of the options for udhcp will vary depending on your network.  But above all, MAKE SURE YOU DON'T HAVE ANOTHER DHCP SERVER RUNNING ON YOUR NETWORK! If you do, then adding another will gum up the works and could cause your existing servers to screech to a halt (or at the very least, lose network connectivity).

If you _do_ have another dhcp server running, then I would suggest picking up another network switch at your nearest Frys Electronics and plugging all your servers (including your new dhcp server) into that and completely isolate it until all of your servers are installed.

Install udchp (as root):

% emerge udhcp

udhcpd's config is in /etc/udhcpd.conf.  Have at least the following options:

# replace these with your own ip address range as appropriate
start 10.1.1.120
end 10.1.1.150

interface eth0 # replace this if your network runs on eth1, eth2, ath0, etc.

# these options are for bootp.. PXE uses this to figure out which server to 
# go to for tftp.  This will be the ip address of your dhcp/tftp server
siaddr 10.1.1.254
sname dhcp_server_name # the name of your dhcp server
boot_file /tftpboot/pxelinux.0

#set up dns, routes and other info
opt dns 10.1.1.22 10.1.1.23 # (replace with your DNS servers)
opt subnet 255.255.255.0 # replace with your subnet if it's different
opt router 10.1.1.1 # replace with your gateway
option domain local
option lease 864000 # 10 days of seconds
opt tftp 10.1.1.254 # the IP address of your tftp/dhcp server

Save that file, then start udhcp:

% /etc/init.d/udhcp start

Step 3: Install syslinux

We only need one file from syslinux, but it's necessary, so install the whole package:

% emerge syslinux

Copy /usr/lib/syslinux/pxelinux.0 to /tftpboot

Step 4: Install and expand ISOs

The first part of this is pretty trivial: Download all of the ISOs (or the whole DVD if you'd like) into /usr/local/src/isos (or wherever you like.. I'll put them here for the purposes of this howto.

Now you'll need to expand the contents of each of these isos into /tftpboot/i386/CENTOS-5 (again, arbitrary, but I used it, and so should you to keep things simple).

A simple script that will do this for you:

cd /usr/local/src/isos
mkdir -p /mnt/iso
mkdir -p /tftpboot/i386/CENTOS-5
for i in `ls *.iso`; do
  mount -o loop $i /mnt/iso
  cp -a /mnt/iso/* /tftpboot/i386/CENTOS-5
  umount /mnt/iso
done

Next, copy initrd.img and vmlinuz from the newly created /tftpboot/i386/CENTOS-5/images/pxeboot into /tftpboot.

Step 5: Configure Apache

Apache should work out of the box, but we need to tell it where to load the CDs from.

Edit /etc/apache2/httpd.conf, and add the following somewere:

<Directory /tftpboot/i386/CENTOS-5>
  Options Indexes
  AllowOverride None
</Directory>
Alias /linux /tftpboot/i386/CENTOS-5

Start/Restart apache:

% /etc/init.d/apache2 restart

That's it on our dhcp server.

Step 6: Install

Make sure your server is configured to PXE boot. Should be in the bios somewhere.. On many systems, you can hit "F12" to boot from the network.  Assuming you followed the directions above, you should get a bunch of information in /var/log/messages from udhcp and tftp during your new system boot.

If you see a boot: prompt, it means you were successful.  Hit enter and it should load anaconda after a bunch of lines of debugging.

After selecting language and locale, Anaconda will ask you what media you want to install from.  Select HTTP and click ok.

In the server field, enter the dhcp/tftp server address (above, it's 10.1.1.254).  For path, enter 'linux' (without the quotes).  Select OK and the install should proceed as normal.

Congratulations.  You now have a network install server!