Pumpkin

Pumpkin

Pumpkin is our new 8 CPU 24 disk monster machine. It is really, really nice.

Basic Setup

We run Xen on this so that it has two RHEL5 personalies: Pumpkin, 64-bit, and Corn, 32-bit. More information is at our Xen page.

Current Xen domU's:

Domain0

Pumpkin

Red Hat EL5 - 64-bit 8-CPUs 24 GB of memory.

DomU

Corn
Red Hat EL5 - 32-bit 2 CPUs, 4 GB of memory. Para-virtualized, boots from /dev/sdb

DomU

Landau
Red Hat EL4 - 64-bit 2 CPUs, 1 to 2 GB of memory, Fully virtualized (this may change), boots from /dev/sdf (for now).

DomU

Compton
Red Hat EL4 - 32-bit 2 CPUs, 1 to 2 GB of memory, Para-virtualized, doesn't boot yet cause I messed up.

RAID Setup'

The RAID is currently split. This allows for much easier maintenance and, in the future, possible upgrades.

Disk 1 to 11

RAID Set 0, which holds the RAID Volumes: System (300GB, RAID6, SCSI:0.0.0), System1(300GB, RAID6, SCSI:0.0.1), Data1 (6833GB, RAID5, SCSI:0.0.2)

Disk 11 to 22

RAID Set 1, which holds the RAID Volume: Data2 (7499GB, RAID5, SCSI:0.0.3)

Disk 23 and 24

Passthrough (single disks) at SCSI:0.0.6 and SCSI:0.0.7. These can be used as spares, as backup, or to expand the other RAID sets later on. Currently they are seen as /dev/sde* and /dev/sdf*. /dev/sdf and /dev/sde are currently used for Virtual Systems.

The RAID card can be monitored at http://10.0.0.99/ login as "admin" with a password that is the same as the door combo.

• To use this card with Linux you need a driver: arcmsr. This must be part of the initrd for the kernel, else you cannot boot from the RAID. You can also install from the CDs, if you have a driver floppy. It will then add the arcmsr driver into the initrd for you. You will still always need to have this driver!
• The kernel module can be built from the sources located on /dev/sdf in /usr/src/kernels/Acera_RAID. Just run make.

There exists a temporary drive which holds a RHEL5 distro and the original RHEL4 distro from the manufacturer. It is currently disconnected from pumpkin.

Virtual Host: Corn

We run a 32-bit personality "corn" using the Xen virtualization on pumpkin's /dev/sdb. Corn is a para-virtualized RHEL5 system, with pumpkin as the master host, or "domain0". It is a fully separate system (that could be booted as the main system with a few modifications to config files. Hint: don't do that!). This means that any system stuff installed on Pumpkin needs to be installed on corn separately.

Subscription Issue: A virtual host needs to be setup special. See the RedHat documentation. Both host and guest need rhn-virtualization-common and rhn-virtualization-host installed. This is now fixed.

The virtual host needs to have both ethernets bridged. According to Xen wiki, this is done by modifying the /etc/xen/scripts/network-bridge script, which is now network-bridge-two which calls the original twice. For the host, create two interfaces: first the one to xenbr1 and then the one to xenbr0, so that the first one ends up being eth1 and the second one eth0. Yes, it seems backwards, but it now works. The key is to have the lines

alias eth0 xennet
alias eth1 xennet

in the /etc/modprobe.conf file. This is now working.

To Do

• Setup SNMP for cacti monitoring.
• There must be other things....
• Setup sensors so that we can monitor the system. Will have to wait for a kernel that supports it

Done

• Sane iptables using ldap.
• Setup ethernet.
• Setup RAID volumes.
• Setup partitions and create file systems.
• Move the system to System drive and remove the current temp drive.
• Setup mount points for the data drives.
• Setup LDAP for users to log in.
• Setup Exports, so other systems can see the drives. There were issues with firewall, so I modeled the firewall after taro's. Seems to be working, I can successfully ls /net/data/pumpkin1 and ls /net/data/pumpkin2 on einstein.
• Setup autofs so that it can see other drives. What other drives? It's working for einstein:/home Other drives such as data drives
• Setup smartd so we will know when a disk is going bad. This can be done inside the RAID card using a system to send SNMP and EMAIL. but it needs to be done. E-mail seems to be set up, let's see if we get any through npg-admins
• Restrict access (/etc/security/access.conf)
• Setup sudo on both pumpkin and corn.
• Add the new systems to the lentil backup script. They're on there; lentil just needs the right SSH keys to rsync them.