Automating vSphere ESX4 host installations
Automating ESX 4 installations is a great way to save time and to provide a method of server recovery in the event of hardware or software failure. It creates consistent high quality repeatable installations that can be quickly modified to handle new and changing hardware. The process can also provide some detailed levels of VMware ESX server instance documentation. This blog will discuss how the process works and how-to create the required elements for you to implement your own automated process.
The vSphere ESX 4 install process uses an updated linux boot release commonly referred to as Syslinux. This Syslinux release version 3.63 supports a variety of popular protocols to facilitate a remote central install repository. FTP, HTTP, NFS and gPXE are all available options for provisioning network attachment to a remote install repository.
From the Syslinux boot process vSphere launches it’s initrd.img kernel instance which is a custom VMware/Linux kernel containing a multitude of VMware ESX 4 drivers and components. The custom drivers allow for a more closely integrated VMware ESX 4 install process that targets an improved ESX 4 server configuration result.
The custom VMware kernel incorporates Linux kickstart scripting functionality to invoke automated installations. The script location is defined as part of the Syslinux functionality and is available as a menu at boot time. A control file located on the boot media provides these variable control elements. Depending on the media type Syslinux uses a respective cfg file to implement this function. The various available Syslinux boot methods that I am aware of are USB, CD, DVD, PXE and gPXE. In this blog I will demonstrate an ISO CDROM method to perform the automated boot cycle. Any of the boot methods mentioned will all work and have varying levels of complexity to achieve.
The ISO CD and DVD based Syslinux configuration uses a config file named isolinux.cfg, USB boot images would use syslinux.cfg as well as gPXE based boot services can use either depending on the final gPXE target image.
Here is a example and description of the boot time menu functional elements for the isolinux.cfg ISO based file in this demonstration.
default Default
gfxboot bootlogo
prompt 1
timeout 3000
label Default
menu default
kernel vmlinuz
append initrd=initrd.img mem=512M quiet ks=http://vc.laspina.ca:8088/esx/4.0/default.cfg
label vh0
kernel vmlinuz
append initrd=initrd.img mem=512M quiet ks=http://vc.laspina.ca:8088/esx/4.0/vh0.cfg
label vh1
kernel vmlinuz
append initrd=initrd.img mem=512M quiet ks=http://vc.laspina.ca:8088/esx/4.0/vh1.cfg
This cfg file provides three menu choices of default, vh0 and vh1. It will invoke the default after 300 seconds (timeout 3000 is not a typo) or you can manually select the other menu items. The “ks=” append option entry can also be one of file://… cdrom://… ftp://… nfs://… usb and UUID:ID/… The initrd.img element is an ESX initial ram disk image and it needs to version match your repository for a successful install process.
This isolinux.cfg file relies on DHCP to provide IP services. If DHCP is not an option you can use static methods to provision the same by passing the IP specific info into the initial ram disk image.
Here is an example of static IP parameters within the isolinux.cfg file.
label vh1
kernel vmlinuz
append initrd=initrd.img mem=512M quiet ksdevice=eth0 ip=10.10.0.1 netmask=255.255.255.0 gateway=10.10.0.254 nameserver=10.10.0.253 ks=http://vc.laspina.ca:8088/esx/4.0/vh1.cfg
Static IP parameters can also be defined in the kickstart ks file but then the ks file would need to be locally available in order for it to work.
Let’s now look at how we can create a remote repository based, automated ESX 4 ISO CD installation boot image. My tool of choice for this process was ISOMagic of which you can make images 300MB or less in size for free. Of course you can use others like PowerISO which is also one of my favourites. The first step in the process is to open the vSphere installation ISO and delete all but the highlighted files show in the graphic. While you could leave them intact I prefer to remove elements that are not required as this can be a template for USB or other boot images.
The ISO boot method is quite simple, all we need to do is create a text isolinux.cfg file based on the example show previously and drag it to the MagicISO window onto the isolinux folder. Once the file is replaced we can use SaveAs an ISO to the name of your choice then burn it or mount it on your server’s ILOM interface. I plan on making a USB based image later so stay tuned for that in another blog entry.
Be wary of using a windows based text editors as they do work well with Unix based text processing operations since it adds invisible characters to the edited files that will cause some of the unix processes to fail. I normally use Ultraedit to edit the Unix targeted files since it has a function that allows you to convert and save in Unix file format.
To provision an install repository is a matter of choice, you can use any of the many different hosts that can serve one of the supported protocols. If your going to use an http repo you will need to take note of Mime types that may not be defined on the web service of choice. In this blog example we are going to define an http based repo on a Virtual Center Server (VC).
Create a base directory on the VC to host the install repo and extract the ESX 4 ISO to an appropriately named subfolder.
e.g. My base is D:VMwareRepo and the subfolder is esx4.0
We need to setup an IIS service instance on the VC and create a WEB site on port 8088. Do not use the default port 80 as it will conflict with other VC services.
We assign our base repo directory to this site and allow directory browsing.
The addition of MIME type pkl is required, right mouse -> Properties on the IIS instance within your Computer Manager MSC
That’s all you need for provisioning an http repo with IIS on your VC. Once you have a repo defined and running make sure you can browse it using your favourite browser.
As an added layer of security I only allow ESX console IP interfaces of a specific subnet on the repo site. Here is an example screen shot. Remember to check browsing availability before you enable any subnet restrictions.
The next step is the most involved and interesting part of the process. Let’s use an example script named vh1.cfg to examine and discuss one of my scripted processes. The script is normally stored in your repo e.g. D:VMwareRepoesx4.0vh1.cfg.
##########################################################
# ESX 4 Kickstart installation script
# © Mike La Spina – Ubiquitous Talk
# File name: vh1.cfg
##########################################################
# Install or Upgrade
install url http://vc.laspina.ca:8088/esx/4.0
“We first define a source for our ESX 4 install files, the possible methods are file://, ftp:// nfs:// cdrom://, take note that you can use a custom port like 8088.”
##########################################################
#Network install type
network –bootproto=static –ip=10.20.0.1 –gateway=10.20.0.254 –netmask=255.255.255.0 –hostname=vh1.laspina.ca –nameserver=10.20.0.200 –device=vmnic0 –addvmportgroup=0
“This defines our final static IP on the vswif0 management interface of vmnic0 (aka the Service Console) and addvmportgroup=0 disables the default VM network creation.”
##########################################################
# root Password
rootpw changeme
“Obviously this sets a root password, however I do not recommend you use an encrypted password method as it can be reversed with simple tools. It is better to just immediately change it to a secured one.”
##########################################################
# Authconfig
authconfig –enableshadow –enablemd5
“Enables a local password shadow file and stores the passwords as MD5 hashes.”
##########################################################
# Regional Settings
keyboard us
timezone America/Winnipeg
“Obvious”
##########################################################
# Firewall settings
firewall –allowOutgoing
“Obvious”
##########################################################
# Enable reboot after script
reboot
“Obvious”
##########################################################
# Boot Config
bootloader –location=mbr
“Installs a master boot record on the firstdisk by default”
##########################################################
# Disk Partitioning
clearpart –firstdisk –overwritevmfs
part /boot –fstype=ext3 –size=250 –onfirstdisk –asprimary
part vh1-local0 –fstype=vmfs3 –size=16384 –grow –onfirstdisk
part None –fstype=vmkcore –size=100 –onfirstdisk
# Create the vmdk on the cos vmfs partition.
virtualdisk cos –size=8192 –onvmfs=vh1-local0
# Partition the virtual disk.
part / –fstype=ext3 –size=4096 –grow –onvirtualdisk=cos
part swap –fstype=swap –size=256 –onvirtualdisk=cos
“Creates the ESX boot, core dump and VMFS partitions as we would expect. We have new partition function available, we can now create our Console Operating System on a vmdk. Here we are defining a virtual disk vmdk named cos on VMFS volume vh1-local0. Very cool, the ESX kernel can now snapshot itself. Take note of the –overwritevmfs option, this can wipe out any perfectly healthy production VMFS volume, I recommend that you remove this option once your testing cycle is complete and only add it to destroy a confirmed existing targeted VMFS volume.”
##########################################################
# Accept the EULA
vmaccepteula
“Obvious”
##########################################################
#
%post –interpreter=bash
“In pre VMware ESX 4 releases it was not possible to directly configure most of the ESX config elements. ESX 4’s initrd.img contains all most everything to need to configure the host without creating any special first time startup scripts on the systems reboot cycle. Now if we can just use vimsh directly … it’s still very cool!”
##########################################################
# Allow hostd etc. some time to load
/usr/bin/sleep 90
“We need to create a delay to aloow the VMware processes some time to load, this is required in order to run vim commands. We could have grepped the output of ps but it still would not tell us its ready to accept work thus a simple delay will do. I am using 90 seconds here but some slower servers may require more. “
##########################################################
# Enable Kerberos Auth
/usr/sbin/esxcfg-auth –enablead –addomain=domain.local –addc=domain.local
“Sets up the Linux Plugable Authentication Module (PAM) to autheticate users against a Window Domain over Kerberos”
##########################################################
# Add Groups and Users
/usr/sbin/groupadd -g 5000 lg-esxsu
/usr/sbin/useradd -u 501 -G lg-esxsu super1
/usr/sbin/useradd -u 502 -G lg-esxsu super2
/usr/sbin/useradd -u 503 -G lg-esxsu super3
“Create a local group which will allow members full admin rights to the ESX ha-folder-root and create three user id’s that are members of the group. These user id’s will be authenticated against the domain.local Windows Domain”
##########################################################
# NTP time config
esxcfg-firewall -e ntpClient
echo restrict default kod nomodify notrap noquerynopeer > /etc/ntp.conf
echo restrict 127.0.0.1 >> /etc/ntp.conf
echo server 10.20.0.200 >> /etc/ntp.conf
echo driftfile /var/lib/ntp/drift >> /etc/ntp.conf
/sbin/chkconfig –level 345 ntpd on
/etc/init.d/ntpd start
“Enable outgoing NTP client port access and build the ntp.conf file to use a Windows Domain time service, this is important for kerberos authentication. Start the NTP client daemon.”
# Create vSwitch0, VMMotion1 and the Service Console port group
/usr/sbin/esxcfg-vswitch -a vSwitch0:64
/usr/sbin/esxcfg-vswitch -A “Service Console” vSwitch0
/usr/sbin/esxcfg-vswitch -A Network-00 vSwitch0
/usr/sbin/esxcfg-vswitch -A VMMotion1 vSwitch0
/usr/sbin/esxcfg-vswitch -p VMMotion1 -v 600 vSwitch0
/usr/sbin/esxcfg-vswitch -p Network-00 -v 700 vSwitch0
/usr/sbin/esxcfg-vswitch -L vmnic1 vSwitch0
/usr/sbin/esxcfg-vswitch -L vmnic0 vSwitch0
/usr/sbin/esxcfg-vswitch -M vmnic0 vSwitch0 -p “Service Console”
/usr/sbin/esxcfg-vswitch -M vmnic0 vSwitch0 -p VMMotion1
/usr/sbin/esxcfg-vswitch -M vmnic1 vSwitch0 -p Network-00
“Defines vSwitch0 for the Service Console and the default gateway IP, vMotion on VLAN 600 with a vMotion IP and create a VM Network-00. This vSwitch will be further configured for a custom standby adapter during the initial reboot with a post config script”
# Create vSwitch1 for iSCSI traffic
/usr/sbin/esxcfg-vswitch -a vSwitch1:64
/usr/sbin/esxcfg-vswitch -A iSCSI_Initiator vSwitch1
/usr/sbin/esxcfg-vswitch -p iSCSI_Initiator -v 500 vSwitch1
/usr/sbin/esxcfg-vswitch -L vmnic3 vSwitch1
/usr/sbin/esxcfg-vswitch -L vmnic2 vSwitch1
/usr/sbin/esxcfg-vswitch -M vmnic3 vSwitch1 -p iSCSI_Initiator
/usr/sbin/esxcfg-vswitch -M vmnic2 vSwitch1 -p iSCSI_Initiator
/usr/sbin/esxcfg-vmknic -a -i 10.20.10.64 -n 255.255.255.0 iSCSI_Initiator
“Defines vSwitch1 for iSCSI on VLAN 500”
##########################################################
#
%post –interpreter=bash
# Create post config script
cat << EOF > /etc/rc3.d/S99postconf
#!/bin/bash
“Some configuration elements still require a post run this statement sends the follow on text to the s99postconf file until the EOF marker is met.”
# Enable TCP outgoing kerberos, there are issues with udp and enable blockOutgoing
/usr/sbin/esxcfg-firewall -–openport 88,tcp,out,KerberosClientTCP
/usr/sbin/esxcfg-firewall –blockOutgoing
“Seriously, this is important, udp kerberos port 88 is defaulted with ‘esxcfg-auth –enablead ..’, bad default! Also the VWware Kerberos client default uses tcp first and this needs to be fixed as it does not comply with RFC 4120. Even if this works why would we risk dropping an auth packet since any fragged udp packet would be dropped over VPN’s etc. Also turn on the outgoing firewall rules that were disabled previously.”
# Enable VMotion on the VMKernel Interface
/usr/bin/vmware-vim-cmd hostsvc/vmotion/vnic_set vmk1
“Enables vMotion on vmkernel interface 1”
# Define Active and Standby failover for shared vSwitche0
/usr/bin/vmware-vim-cmd hostsvc/net/portgroup_set vSwitch0 ‘Service Console’ –nicorderpolicy-active vmnic0 –nicorderpolicy-standby vmnic1
/usr/bin/vmware-vim-cmd hostsvc/net/portgroup_set vSwitch0 VMMotion1 –nicorderpolicy-active vmnic0 –nicorderpolicy-standby vmnic1
/usr/bin/vmware-vim-cmd hostsvc/net/portgroup_set vSwitch0 Network-00 –nicorderpolicy-active vmnic1 –nicorderpolicy-standby vmnic0
“Here we are overriding our adapter team for vSwitch0 so that we can separate our active traffic on the two adapters while maintaining failover capability.”
# Grant the group named lg-esxsu admin permission to ha-folder-root
/usr/bin/vmware-vim-cmd vimsvc/auth/entity_permission_add vim.Folder:ha-folder-root lg-esxsu true Admin true
“Enables any member of the local group lg-esxsu Administrator permissions to the VMware host”
# Reset system to normal boot mode
echo “Removing automated post script.”
rm /etc/rc3.d/S99postconf
EOF
chmod +x /etc/rc3.d/S99postconf
“Obvious”
As you can see the process is quite involved, however the benefits are outstanding. I can build or recover an ESX 4 host in 10 minutes or less and I can reconfigure it to a different target with ease.
Hope you found the entry usefull and interesting.
Regards,
Mike
Tags: automation, central, ESX 4.0, iso, kickstart, repository, script, syslinux, vSphere
Site Contents: © 2009 Mike La Spina
Subscribe in a reader
Follow me on Twitter
What about the SAN connection. I am struggeling with the connection to the san. I don’t wan’t do disconnect the cables from the storage every time. The option onfirstdisk does not check if the disk is a LUN on the SAN.
Doe you have a solution for that?
Thanks.
Rob,
The onfirstdisk flag actually does check if the disk is a local or remote connection and you can specify the accepted type as follows.
–onfirstdisk=local
This will direct the part function to only use a locally defined disk. You can also spec a device string like mptspi, mptsas, qla2xxx etc.
for example
–onfirstdisk=mptsas,local
which is the first disk found on module mptsas and which is locally connected.
Regards,
Mike
Mike,
Thanks for your response. I looked it up at in the docs.
Very nice work! Unfortunately the /usr/bin/vmware-vim-cmd could not be executed by the S99postconf because it was missing dynamic libraries. In order to overcome this problem, you may add the following code:
#!/bin/bash
exec >/var/log/postInstallScript.log 2>&1
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/vmware/vmacore/
sleep 120
and name the startup script S99zpostconf
The sleep command allows time for the hostd process to start properly and accept commands from the CLI. Of course, there may be more elegant solutions but… this one works!
Cheers,
Dimitris.
Hi Dimitris,
I have not had issues running the vim commands, possibly there is a change in the current release. I will have to check that out. I do have a delay in my current running post script which is not detailed in the post.
Thanks for commenting on it. I will add it to the blog entry.
Regards,
Mike
Tried as suggested “append initrd=initrd.img mem=512M quiet ksdevice=eth0 ip=10.10.0.1 netmask=255.255.255.0 gateway=10.10.0.254 dns=10.10.0.253 ks=http://vc.laspina.ca:8088/esx/4.0”
Get install error “No nic found with name eth0”
Using ESX4u1
Try using ksdevice=vmnic0
also try using alt-f3 at the error screen and issue ifconfig -a to see what devices are available.
vmnic=0 does work, thanks. I found problem with IPAPPEND 2 which does not pass IP config from PXE (DHCP obtained IP) to the ESX installer and I have to add ip= to configure it manually.
Is there a way to make dhcpclass= parameter working with ESX installer? In Red Hat this configures DHCP client with vendor class string so dhcpd can filter everything else this ESX installer vendor class DHCP request.
Is it possible to supply vlan id for ksdevice to the installer?
Yes you can supply the vlanid, use the following format:
append initrd=initrd.img ksdevice=vmnic0 vlanid=5 ip=10.10.0.1 netmask=255.255.255.0 gateway=10.10.0.254 dns=10.10.0.253 ks=http://vc.laspina.ca:8088/esx/4.0
I believe IPAPPEND 2 needs to be on a line by itself other than that it only works with PXE loaded images.
I get a File not found error 404 returned from the server.
Sam,
Check the IIS logs, they should indicate what file is not found.
For the dns issue within the syslinix append replace the dns=x.x.x.x with nameserver=x.x.x.x and that should correct it.
Regards,
Mike
[…] on vSphere 4 (VI Tips) Creating USB based boot media for ESX 4 installs (Ubiquitous Talk) Automating vSphere ESX4 host installations (Ubiquitous Talk) Best Practices for vSphere (ESX 4) Service Console Partitions (VM/ETC) […]
Mike the boot process works up to the point of trying to download the files.And then the
system promtps me for the install media.
default Default
gfxboot bootlogo
prompt 1
timeout 3000
label sbxesx01
kernel vmlinuz
append initrd=initrd.img mem=512M quiet ksdevice=vmnic0 ip=192.168.1.97 netmask=255.255.255.0 gateway=192.168.1.2 dns=192.168.1.12 ks=http://192.168.1.92:8088/4.0/sbxesx01.cfg
I get file not Found?
I can browse to the Directory from a windows host running IE
I can use alt-f3 and Ping the IIS Server. If I use the server name inplace of the IP the
server will not resolve the name.
The server name resolves and is registered in DNS(192.168.1.12)
I used the config off the webpage to test with any ideas?
Hi,
I have been trying the kickstart install on ESX 4.0
The filed dns= is not working from my side.
I got to change it to nameserver=
Best regards
Souleyman
[…] I start, I would like to point everyone to this excellent blog post by Mike La Spina. If you are thinking about using kickstart for your ESX deployments, then this post is […]
Mike,
Awesome blog, but I have some questions:
1. You say to remove all directories but the highlighted ones and then add our custom isolinux.cfg file to the isolinux directory. Not sure what I am supposed to do here.
2. I did not remove any of the directories for the sake of testing but when I get to the boot screen, I just have the ESX 4 logo and no menue items to select. Any thoughts? I’m sure I missed something simple.
Thanks again for this great post, it saved me a lot of time for my deployment options.
Hi Steve,
While it’s not absolutely required to remove files that are not used, it does allow you to know exactly what is essential for correct operation.
The menu is defined in the isolinux.cfg file and you could simply start with this bare minimum isolinux.cfg and see if it works for you as this one works on my CD images.
default Default
prompt 1
timeout 3000
label Default
menu default
kernel vmlinuz
append initrd=initrd.img mem=512M quiet ks=http://vc.laspina.ca:8088/esx/4.0/default.cfg
label vh0
kernel vmlinuz
append initrd=initrd.img mem=512M quiet ks=http://vc.laspina.ca:8088/esx/4.0/vh0.cfg
Thanks for the prompt response, Mike. My isolinux.cfg file looks similar, but I will give yours a whirl. The only difference is I am assigning a stsic IP based on your information above.
As far as removing the unnecessary files from the installation media, I just need clarification. Do I remove the isolinux directory, because it is not highlighted in your screenshot and create an empty isolinux directory to put the isolinux.cfg file
OR
Do I leave the isolinux directory intact and just add my custom isolinux.cfg file to it?
Thanks
Hi Steve,
The isolinux directory is required. You would copy your modified custom isolinux.cfg into the isolinux directory.
Regards,
Mike
I am using ESX4.0U1 and using a static IP method to connect to an NFS server to read my .cfg file (also tried to use http). My problem is that the nfs mount appears to fail saying there is “No route to host” and the command exited with status 8192. If I run the exact same command from the Alt-F3 session it mounts the NFS drive without any issue (it is on the same subnet). /sbin/mount.nfs :/ /mnt/nfs -v -o nolock is the command and this works fine.
I have two NIC’s on the server and I am using vmnic0 and the ip is assigned to vswif0 correctly with the correct broadcast, netmask.
The esxcfg-route -l doesn’t return any routing information (even though it was added and shows up in the esx_install.log) Below is the isolinux.cfg entry I am using:
LABEL esx1
menu label esx1
kernel vmlinuz
append initrd=initrd.img vmkopts=debugLogToSerial:1 mem=512M ksdevice=vmnic0 ip=10.1.1.141 netmask=255.255.255.0 gateway=10.1.1.1 nameserver=10.1.1.230 ks=nfs://10.1.1.182/ks/esx1.cfg
Any suggestions as I have tried the http method also (which works fine from a web browser to get to the file). Same with NFS as I can mount it and see the files fine using the same command in the log file.
Regards,
Lindsay
Hi Lindsay,
Since it works when you boot a normal install process using the prompt then we know its in a syslinux to vmware transition point. You have two interfaces. I would try defining the ksdevice as vmnic1 and see where that takes you.
Regards,
Mike
I have tried using vmnic1 also.
Thanks,
Lindsay
In response to Dimitris above for a more elegant solution… Becuase some systems take forever for vimsh to come up, I use something like below rather than a simple sleep. This tries to run vmware-vim-cmd (just a refresh in the example) every minute for 30 minutes and will break out when it is finally successful. I’m no bash expert, but it seems to work for me.
for (( i=0; i<30; i+=1 )) {
/usr/bin/vmware-vim-cmd hostsvc/net/refresh && break || (echo "Vimsh components not ready, have waited $i minutes of maximum 30, waiting another minute..."; sleep 60)
}
For those interested, I believe we have tracked down the issue on the NFS mount. It turns out the ESX 4.0 Update 1 media doesn’t appear to work for mounting a .cfg from an NFS filesystem. The ESX 4.0 media and isolinux bundle does however. If you copy the isolinux from the 4.0 ISO distribution and use that (point your installation to the ESX 4.0 Update 1 in your cfg kickstart script then that works. VMware support are looking into this issue for us.
Thanks
Lindsay
[…] I prepared an installation script customized for each server with the IP address (it’s a pain to change it after the server is installed). I received inspiration from here. […]
I am trying to find a way of how to register the VMFS datastores automatically as part of the unattended ks.cfg deployment when the ESX host comes online.
Any ideas on how to achieve this as a %post config within a KS.cfg?
Thanks
Manoj
Hey Manoj,
Could you give an example of what you are trying to do. VMFS volumes are normally detected on scan when they’re exposed to a host.
Hi Mike,
Thank you for your prompt reply to my earlier post on your blog.
What I am trying to do is deploying a ESX4 host using ks.cfg unattended method and as part of the script I am providing the iSCSI target which has two VMFS Lun’s of which that I have already initialized with volume names by adding them once to the ESX host. When I deploy a new host and comes up it sucessfully maps to the iSCSI targets but the VMFS datastores do not appear for some reason I have to add them manually. Currently I am doing this on a VMware Workstation by deploying the ESX hosts as a VM and the iSCSI target I am using is the Starwind iSCSI v4 running on another Windows VM.
Hope this helps.
Thanks,
Manoj
Hi Mike,
Ignore my previous posts on this topic and I did some extra checking and found this is an issue with the Starwind iSCSI implementation that I have. For some reason the even after mapping the datastores manually on reboot the mappings are lost yet the ESX iscsi initiator successfully maps the targets.
I will have to revert to running something like openfiler. I am currently testing my build process before I introduce it into our non-production system which has both SAN and iSCSI storage LUNS.
One more thing on the ks.cfg script I wanted to know what I might be doing wrong for setting the console memory to 800MB. I am using the below:
mv -f /etc/vmware/esx.conf /etc/vmware/esx.conf.old
sed -e ‘s/boot/memSize = “300”/boot/memSize = “800”/g’ /etc/vmware/esx.conf.old >> /etc/vmware/esx.conf
Which used to work for me on ESX3 but in ESX4 for some reason it shows as 300.00MB (800.0 MB on restart) I have restarted the host 3 times but the change is not effected.
Thanks,
Manoj
Ok please help me out here Mike as I am about to lose my marbles over this :-D. I have been hacking away at this script for the better part of the day but every time I get to the post install section it sleeps for the designated 3 minute interval then seems to attempt to apply the post settings but always fails and eventually says something to the effect of “Failed to connect: Crypto Exception:error:02001002:system library:fopen:No such file or directory:/etc/vmware/ssl/rui.key. The virtual console f1 screen also spits out a ‘User-supplied script failed’ error message. It seems to apply my add group/users commands properly but not the network/svccon memory/vmotion checkbox tick commands. Strange thing is if I do ssh into the host post install and just drop the one liners in it applies without error. Any ideas?
%post –interpreter=bash
/bin/sleep 90
/usr/sbin/groupadd -g 5000 lg-esxsu
/usr/sbin/useradd -u 501 -G lg-esxsu super1
/usr/sbin/useradd -u 502 -G lg-esxsu super2
/usr/sbin/esxcfg-vswitch -A LAB_VMs vSwitch0
/usr/sbin/esxcfg-vswitch -A VMotion vSwitch0
/usr/sbin/esxcfg-vswitch -p VMotion -v 10 vSwitch0
/usr/sbin/esxcfg-vswitch -p LAB_VMs -v 11 vSwitch0
/usr/sbin/esxcfg-vswitch -M vmnic0 vSwitch0 -p VMotion
/usr/sbin/esxcfg-vswitch -M vmnic0 vSwitch0 -p LAB_VMs
/usr/sbin/esxcfg-vmknic -a -i 10.10.10.40 -n 255.255.255.0 VMotion
/usr/bin/vimsh -n -e “hostsvc/net/refresh”
/usr/bin/vimsh -n -e “internalsvc/refresh_network”
/usr/bin/vmware-vim-cmd hostsvc/memoryinfo 838860800
Kevin,
Have you tried a longer delay or added an additional delay after the last vswitch config.
Regards,
Mike
I did not get a chance to try that but did come to the understanding that I needed another post section which I was missing originally so the /usr/bin/vmware-vim-cmd hostsvc/vmotion/vnic_set vmk0
and /usr/bin/vmware-vim-cmd hostsvc/memoryinfo 838860800 statements were not applying and throwing that rui.key ssl crypto error.
Here is my current post section, everything applies now (vSwitch related and all so i am further now) except that last post EOF section. Thoughts/tips would be greatly appreciated.
%post –interpreter=bash
/bin/sleep 90
/usr/sbin/esxcfg-auth –enablead –addomain=domain.local –addc=domain.local
/usr/sbin/groupadd -g 5000 lg-esxsu
/usr/sbin/useradd -u 501 -G lg-esxsu super1
/usr/sbin/useradd -u 502 -G lg-esxsu super2
/usr/sbin/useradd -u 503 -G lg-esxsu super3
esxcfg-firewall -e ntpClient
echo restrict default kod nomodify notrap noquerynopeer > /etc/ntp.conf
echo restrict 127.0.0.1 >> /etc/ntp.conf
echo server 10.20.0.200 >> /etc/ntp.conf
echo driftfile /var/lib/ntp/drift >> /etc/ntp.conf
/sbin/chkconfig –level 345 ntpd on
/etc/init.d/ntpd start
/usr/sbin/esxcfg-vswitch -A LAB_VMs vSwitch0
/usr/sbin/esxcfg-vswitch -A VMotion vSwitch0
/usr/sbin/esxcfg-vswitch -p VMotion -v 10 vSwitch0
/usr/sbin/esxcfg-vswitch -p LAB_VMs -v 11 vSwitch0
/usr/sbin/esxcfg-vswitch -M vmnic0 vSwitch0 -p VMotion
/usr/sbin/esxcfg-vswitch -M vmnic0 vSwitch0 -p LAB_VMs
/usr/sbin/esxcfg-vmknic -a -i 10.10.10.40 -n 255.255.255.0 VMotion
/usr/bin/vimsh -n -e “hostsvc/net/refresh”
/usr/bin/vimsh -n -e “internalsvc/refresh_network”
%post –interpreter=bash
cat < /etc/rc3.d/S99postconf
#!/bin/bash
/usr/sbin/esxcfg-firewall –openport 88,tcp,out,KerberosClientTCP
/usr/sbin/esxcfg-firewall –blockOutgoing
/usr/bin/vmware-vim-cmd hostsvc/vmotion/vnic_set vmk0
/usr/bin/vmware-vim-cmd hostsvc/memoryinfo 838860800
/usr/bin/vmware-vim-cmd vimsvc/auth/entity_permission_add vim.Folder:ha-folder-root lg-esxsu true Admin true
echo “Removing automated post script.”
rm /etc/rc3.d/S99postconf
EOF
chmod +x /etc/rc3.d/S99postconf
Just thought I would provide an update and a sincere thanks for the hard work and time you put into documenting and sharing this with the community. I was able to resolve my issue with the my %post woes after initial kickstart reboot setting not sticking by adding a ‘/bin/sleep 120’ statement right after the #!/bin/bash line in the second %post section of the script. Everything is working perfectly and I am a dangerous man now ready to step my Virtual Infrastructure automation game up 😀
Well done Kevin and thanks for sharing your experiance as well.
Regards,
Mike
Hi Mike,
Sorry to trouble you, I have tried several things to try and get the console memory setting to 800MB but for some reason it still doesn’t make the change permanent even after 3-4 reboots. The message still indicates on next reboot.
Any pointers will be greatly appreciated.
Thanks
Manoj
Manoj,
The vimsh entry you are using appears to be a read only informational function. This configuration element is part of the /etc/vmware/esx.conf startup parameters.
You should have a line that resembles the following:
/boot/memSize = “640”
This value determines what the hypervisior will reserve for an esx console VM. It is this configuration element that must be set.
Your post script should find and replace the = memory value
You could use the following:
/bin/sed -i ‘s/memSize = “272”/memSize = “800”/g’ /etc/vmware/esx.conf
/usr/sbin/esxcfg-boot -g
/usr/sbin/esxcfg-boot -b
Regards,
Mike
Hello Mike,
I’d like to thank you very much for pointing out where I was going wrong. You are a really an expert at bash scripting. I have always been very envious of people who have this ability. I have always wondered how one could learn to do this?
Keep up the great work.
[…] Upgrading the existing ESX 3.5 nodes to ESX 4 is on the schedule for later this semester or early in the summer. To make the reinstall easier, I’ve scripted out the installs just about as much as possible using some excellent examples: Cylindric.net and Ubiquitous Talk. […]
Hi Mike,
I have moved my unattended ESX4 deployment setup into a test environment which has physical servers with multiple nic’s, previous i was testing my ks.cfg scripts using a VMware workstation environment by deploying virtualized ESX 4 VM’s.
The first physical server I tried to deploy using my previous working setup i keep getting the message “No COS Adapters specified in the Users settings”.
I understand the reason why this is happening i.e the error as some have already pointed out is that esx4 when it builds creates a virtual switch with a random mac and then tries to build off that.
The only thing that remains the same is that the PXE, DHCP and IIS (http path) are still running inside the Windows Virtual Machine that runs inside VMWare workstation setup with Bridged Network configuration.
I have tried various changes by using ksdevice=MAC Address and the IPAPPEND 1 options but to no avail.
Any ideas? where could I be going wrong?
Thanks,
Manoj
Hello Mike,
Thanks very much for your howto.
The configuration of my ESX are outside of KS.cfg. Do we have a way to mount the HTTP repo during the %post% phase?
Thanks and Regards
Smooky132
Hi Manoj,
Since you know the process worked in a closed virtual environment I would suggest that you only explore what is different from there. I would make use of the ESX install CD and manually examine what it detects for network adapters there. Specifically the enumerated order and the vmnic names are critical. You could also look at the logs while its running and possibly use F12 to see that vmkernel activity.
Regards,
Mike
Hi Smooky132,
Normally the custom vmware ks provisions the connectivity over http thus the only other ways are to use NFS or add davfs2 to the system image http://savannah.nongnu.org/projects/davfs2
Regards,
Mike
[…] « Automating vSphere ESX4 host installations Securing COMSTAR and VMware iSCSI connections […]
[…] Also, Mike La Spina has a great post for kickstart sample for vSphere ESX 4: http://blog.laspina.ca/ubiquitous/automating-vsphere-esx4-host-installations […]
i get the error unable to fine /dev/sda. There is no entry for /dev/sda in the script. IS the error coming because # Installation Method
cdrom
instead of # Installation Method
cdrom://
Hello Abhishek,
The format to specify a cdrom based install is as follows:
##########################################################
# Install or Upgrade
install cdrom
/dev/sda would be hardware probed via the busybox installation process, the issue is more likely a hardware problem. Boot the standard install CDROM to verify the hardware is working correctly. You can also examine what the process detected by using Alt-F2 when the error screen occurs.
Regards,
Mike
This is very interesting. I’m going to try out some variations to see how we can speed up and simplify our vSphere deployments.
In the early part of this post, you write “open the vSphere installation ISO and delete all but the highlighted files show in the graphic”.
It seems like the “but” should be removed. Is that right?
Hi Kevin,
It’s correct with the “but”, this method uses a network share as the repo so you don’t need the rest of the files to create a boot process. The reason for deleting the extra files is to keep the boot and install media small and clean.
Regards,
Mike
Hi all…i am experiencing a similar issue to that seen by Lindsay. I am running to create
an automated build for ESX 4 Update 1 using the http based repo method as described on this
site. As a test i first built the serve manually and then copyied the ks.cfg file
to my iis web server. however when i try the build i get the “user has added no COS nics”
error message. Aswell as the “no route to host” – i have tried pinging my iis server
and it works fine.
at the esx splash screen i have tried entering the usual ksdevice=vmnic2 and even
ksdevice=, ip= netmask= etc etc
Any help would be great!
The “no COS nics” message will not stop the install process from working so the problem must lie with the IP/Network/Nic assignment.