Build a new server

Revision as of 18:16, 10 September 2012

Details on all the steps necessary to take a new (or old repurposed) piece of hardware and turn it into a happy functional server

Before you begin

• find the machine's MAC address (racadm getsysinfo)
• decide on private / public IP address, and if it will need to exist in some special range
• decide how the disks should be arranged (raid, partitioning, etc.)

Initial hardware setup

• Get the hardware racked and cabled
• Get an IP and name (RobH, Mark, Leslie are comfortable doing this.)

• follow DNS How-To section to add the name/ip to DNS
• set up $name.mgmt.$loc.wmnet as well to access the management interface
• set up $assettag.mgmt.$loc.wmnet to the same IP as $name.mgmt.$loc.wmnet.

• Set up DHCP with the MAC address / name info
  • if it's a Dell, get MAC address from the mgmt console, run racadm getsysinfo; we use the first interface
  • in git, edit files/dhcpd/* (any new uses server linux-host-entries.ttyS1-115200)
  • after editing, merge and then run puppet on brewster (dhcp server)
• Get the switch set up to pass traffic to the host (Mark)
• set up the hardware raid (if it has it)

PXE boot and the initial OS

• if it's a cluster host, set up netboot to partition the disk
  • You need to edit netboot.cfg from the puppet configuration
  • it's a bash case statement. Make sure your hostname is matched by a regex in there.
• if it's a misc host, you'll have to partition by hand (it will prompt you)
  • lvm over raid 1 is a decent config if you don't have anything more specific you need or you can simply NOT specify it in netboot.cfg and do the partition interactively in the install shell.
• ssh to root@$servername.mgmt.$loc.wmnet, force a restart and pxe boot (if it's a dell C2100, use IPMI; if it's a different dell, copy/paste this: dell pxe boot)
  • get the password from someone in ops if you don't have it

Quick and Easy DRAC commands for Dell DRAC

• Run the following in the DRAC to set it to PXE for ONLY the next boot, and to connect to the console once that is set and system is rebooted:

 racadm config -g cfgServerInfo -o cfgServerBootOnce 1
 racadm config -g cfgServerInfo -o cfgServerFirstBootDevice PXE
 racadm serveraction powercycle
 console com2

Normal Commands

• • powercycle the host: racadm serveraction powercycle
    • this takes about 15s
  • connect to the cosole: console com2
    • there might be no output for a little bit immediately after the powercycle. wait at least 30s or so
  • during boot, force netboot: F12
    • when connecting from OSX via the Terminal, escape-shift-2 sends F12

Resume split instructions

• • you can leave this running to watch it complete
    • This is where you'll be prompted for partition info
  • when you're done, ctrl-\ will disconnect you from the console
    • exit will disconnect from the mgmt interface

Check your partitioning

If you used a cluster host, check the partitioning

• "df -h" will tell you what the mounted partitions are
• "fdisk -l | grep Disk" will tell you the physical size of the disk
• "sfdisk -l /dev/sdX" will tell you the size and type of the partitions in blocks

Make sure most of the disk is used

Get puppet running

Warning: if you are rebuilding a pre-existing server (rather than a brand new name), on sockpuppet, run puppetca --clean $server_fqdn to clear out the old certificate before beginning this process. If you already began, also run (on the server you're building, not sockpuppet) find /var/lib/puppet/ssl -type f -exec rm {} \; to clean out the client.

• get a shell on both $server and the puppet master (sockpuppet)
  • Only one key has access to new installs.
  • from sockpuppet, ssh -o StrictHostKeyChecking=no -i ~/.ssh/new_install root@$server-fqdn to log into $server
  • If the server is an existing server: from sockpuppet ssh -i ~/.ssh/new_install root@$server-fqdn
• on $server, run puppetd --test --ca_server sockpuppet.pmtpa.wmnet

It will fail.

• on sockpuppet, run puppetca -s $server_fqdn
• on $server, run puppetd --test --ca_server sockpuppet.pmtpa.wmnet

It will fail once again

• on $server, run puppetd --test It should now succeed.

After your first couple of successful puppet runs, you should reboot just to make sure it comes up clean.

Set up puppet

• add $server to site.pp, either by hostname or within a regex if it's part of a class (eg srv\d\d*)
• do whatever puppet goodies you want to get the server to do what you want it to.

Related pages

• DNS
• An older version of the preseeding docs: Automated installation
• Another old page: Automated upgrade (reinstallation)
• More detail around working with the console: Remote_management
• PartMan