Friday, August 24, 2012

Installing and Configuring APC PowerChute Network Shutdown 3.0.1 for ESXi 5.0

I wrote this article original for ESXi 4.1 and PCNS 3.0.0, but enough has changed that I figured it warranted a new article... enjoy!

Shutting down windows servers with the APC network shutdown software is a no-brainer... but what about virtual machines? Sure, you could install the network shutdown software on every virtual machine, but that would be wasteful. Fortunately, there's a centralized way that you can shut down your virtual guests AND hosts... and the best part? It's (mostly) free! All you need is an APC battery backup unit, an APC network shutdown card, and a bunch of software. Here's what you need to do:

1) Download stuff. There's a bunch of it!
  1. Download PCNS 3.0.1 from the APC website. It's free now, and you do not even need to register for it (finally, somebody gets it!). You can find it here:
  2. While you are there, grab the latest firmware for the APC card you are using. Make sure you know the kind of network management card you have, as the older ones are not able to use the latest firmware.
  3. Download the latest VMware vMA from the website under 'tools'. As of this writing, it's
  4. Grab the free version of Veeam fastscp... it makes loading the APC installer on the vMA a snap. Note that fastscp is now bundled into the full backup and recovery tool... so grab that.

2) Install the latest network management card firmware first. It may take several shots at it... for some reason it likes to fail, but doing it over and over usually gets it going. Don't ask why.

3) Create one vMA virtual machine by extracting that zip file you downloaded and then launching the vSphere client and going to File -> Deploy OVF Template. It will guide you through what's required, but you basically need to pick a datastore for its 5GB volume. I usually make the volume 10GB for snapshot space. There are options to give it an IP, but i found that it didn't end up mattering due to the following error when trying to power on the newly created vMA:

  • The only workaround I found for this error was to edit the newly created vMA, go to the Options tab, and under vApp Options select "Disabled". You'll get a nag screen indicating it is removing properties, but the only options that I could find set were items about DHCP, which we aren't using...

4) Open the console of the vMA, power it on, and follow the initial setup instructions. I recommend assigning a static IP as well as a real hostname to the vMA for use later. For the hostname, specify a full domain name such as VMA.domain.local. Once you are completed, you should be dropped to a blue and grey login screen.

5) Install Veeam FastSCP, then open it. Click "Add Server" and add the vMA server as a "Linux Server". Be sure to use the vi-admin username and password you specified while setting up the vMA. Uncheck the box to "Elevate account to root" as we do not have root access - it's been disabled.

6) In FastSCP, browse to the pcns300ESXi.tar.gz file you downloaded earler and right click / copy it. Note that you have to right click / copy in FastSCP, not in windows explorer... FastSCP does not have windows clipboard access built in. Once copied, expand your vMA in FastSCP, browse to the tmp folder, and paste it in the root of tmp. You can close FastSCP now.

7) Back in the vMA console, choose the option to "login", enter the vi-admin credentials, and do the following:
  1. Browse to the tmp directory (for linux noobs, type cd /tmp) and run "gunzip pcns301ESXi.tar.gz" to unzip the file.
  2. In the same directory, run "tar -xf pcns300ESXi.tar" to extract the file.
  3. Browse into the newly created "ESXi" folder and run "sudo ./" to start the installation of pcns 3.0.0.
  4. You will likely get prompted with a warning (read it, it's funny) and need to enter your password. After that, installation starts.
  5. Accept the license agreement, and accept mostly defaults. When it gets to the part about java, be sure to let the installer install it's own bundled version of java.  
  6. When you get to the part about entering an IP, just press "q" to skip it.
  7. Make sure you get the message about Installation has completed, and the note about how to access it - this means installation was successful.
8) Now, you need to add your ESXi servers to the 'fast pass' access via the following command: "sudo vifp addserver hostIPaddress". You will be asked to enter the password for your host, so vMA will have it on file. Do this for each host in the order you would like them shut down.
  • IMPORTANT: Make sure that the vMA is located on the last host you add to the fastpass list. This makes sure that your vMA is one of the last things shut down, and that it is able to give your final host the sutdown command, which will in turn shut down the vMA. You'll probably want to disable vMotion for the vMA too, or only map it's storage to the final host, so that it cannot be moved.
9) Now, we need to configure PCNS. Open a web browser and go to https://IPofVMA:6547 and follow the configuration wizard. The only step worth menitoning is that you should NOT check the box for "Turn off the UPS after shutdown finishes." If you check it, there's a good chance the UPS will turn off while your hosts are still shutting down. The caveat to leaving this unchecked is that your ESXi servers most likely will not turn back on if power is restored before the UPS battery loses all of it's charge... you'll have to use iLO or DRAC to get in and turn your hosts on instead.

10) Next, you will want to configure shutdown events. Once the PCNS wizard finishes it forwards you to the PCNS configuration page. Click on "Configure Events" and configure some of the important events like "UPS: On Battery" and "Input Power: Restored". You'll most likely want to notify users for most of the events. Don't forget to check the box for "Shut Down System" next to "UPS: On Battery" and set it to go off after a reasonable amount of time on battery (this depends on how much runtime your battery has, but I usually set mine for 5 minutes. If the power is off for 5 minutes, it's most likely going to be off an awful lot longer.)

11) Also, you'll want to check the "Connected Servers" tab on the left under "UPS information" to make sure your vMA IP address is listed. If not, you'll want to add it as a client on your UPS's network management card's web page.

12) If you are using VMware HA... you have a few more steps to follow in order to get your VMs to shut down cleanly. See the following article for more details and a script that you should add to your battery event:

In short, it has you upload the following script to your vMA, then add it as a shutdown option:
IFS=$(echo -en "\n\b")
ups_vm="vSphere Management Assistant \(vMA\)"
for host in ${hosts[@]}; do
echo $host
source /opt/vmware/vma/bin/vifptarget -s $host
for i in `vmware-cmd -l --username xxxxx --password xxxxx`; do
if [ `vmware-cmd $i getstate | egrep -c "on"` -eq 1 ]; then
echo $i
if [ `echo $i | egrep -c $ups_vm` -eq 1 ]; then
echo "Skip shut down of VMA"
echo "Shutting down $i"
vmware-cmd "$i" stop soft
source /opt/vmware/vma/bin/vifptarget -c
I completed this by downloading the script file they had precreated, then used Veeam FastSCP to upload the file to the /tmp directory. Once there, I changed the permissions as indicated in the instructions (sudo chmod +x, and followed that up by moving the file to the /opt/APC folder, just so I didn't forget what it was for. After that, I just went into the "Configure Shutdown" page on the PowerChute network Shutdown web interface and entered the script and it's path in the "Run this command" box. I gave my system 300 seconds to shut everything down, which seems to be sufficient.

13) If you are not using HA, in the vSphere client, make sure all of your ESXi hosts have their "Virtual Machine Startup / Shutdown" options configured and they are set to "Enabled", otherwise your guests are likely to just be turned off rather than shut down! (Note, this doesn't apply to HA configurations, as HA will keep disabling your shutdown options.)

That should get the system going! You can test it by, well, pulling the plug! If that's too scary for you, you can also configure the shutdown option for "PowerChute cannot communicate with the NMC", then pull the network cable from the network management card. If your hosts and guests shut down correctly, all is well!

Note: If your hosts still refuse to shut down, check out this article for a probable fix: The long and short of the article is that you need to add the following line of code to the /opt/APC/PowerChute/group1/bin/shutdown file right after the line that says "export LD_LIBRARY_PATH":
For linux noobs... you can edit the file by giving the command "sudo vi pathAndFileName", then pressing "i" to "insert" text... then edit the file like normal. Once you are done, press escape, then type ":" and "wq" then hit enter. Simple fix.

Sunday, August 19, 2012

Installing Dell Openmanage on an ESXi Host

Found this when paroozing the VMware communities sites for more info on Dell Openmanage for ESX.

Section 1: Install the vSphere Client and put the target host in maintenance mode

1. Connect to your Host Server via Internet address.
2. Download the vSphere Client appropriate for your version of ESXi
3. Install the vSphere Client
4. Open the vSphere Client and connect to your vCenter or directly to the host
5. Shut down all the guests on the target host, or migrate them to another host.
6. Right click on the target host in the left hand pane and click "Enter Maintenance Mode."

Section 2: Install Dell OpenManage

1. Download and install ‘vSphere CLI’ from VMware website (you will need to create an account in order to login and download the file):

Most recent at time of this post is: VMware-vSphere-CLI-5.0.0-615813.exe (

2. Obtain the Dell OpenManage package for ESXi:


ESXi 4.0:
ESXi 4.1:
ESXi 5.0:

3. Launch ‘vSphere CLI’ via Start > VMware > VMware vSphere CLI > Command Prompt.
4a. On ESXi 4.X, do the following:
- In the CLI window, type "cd bin" and hit enter to change the directory to the "bin" subfolder
- type " --server IpAddressOfHost -i -b PathToVIBinstallPackage" and hit enter.
(eg: --server -i -b C:\

4b. On ESXi 5.X, do the following:

- Copy the zip file containing the VIB to a datastore that the host can access. I used the vSphere client to connect to the host, then hit the configuration tab, Storage, then right clicked on the default local storage and hit "browse datastore". From that window I created a new folder named "updates" and uploaded the file into it.

- Take note of the Datastore location and name that is shown under "Datastore Details" in the bottom pane while the datastore you uploaded the file to is selected. My local store was /vmfs/volumes/502e86dd-dfa6c7f0-9ce2-0022197a5bef ...

- In the CLI window, type "esxcli --server IpAddressOfHost software vib install -d PathToVIBinstallPackage" and hit enter.
(eg: C:\Program Files (x86)\VMware\VMware vSphere CLI>esxcli --server software vib install -d  /vmfs/volumes/502e86dd-dfa6c7f0-9ce2-0022197a5bef/updates/

5. The system will prompt for the username ( root - then press enter) and the root password (then press enter). The update will now be applied to the ESXi server. The update will take approx 5mins on ESXi 4, and only moments on ESXi 5. (You can view the progress on the vSphere Client screen if you have it open).

Note, you may need to enable CIM providers as well... 4.1 and 5.0 do this automatically, but I think certain builds of 4.0 require you do it manually. You can do this by opening up the CLI again and typing CIMoemProviderEnabled = 1 (thanks to a comment below for pointing this out!)

6. Once completed, close the command prompt screen and reboot the ESXi server. Once the server has come back from being rebooted, using the vSphere Client, right click on the server which is in (maintenance mode) and choose “Exit Maintenance Mode”.

Section 3: Connect to the server via Server Administrator

1. Download the latest version of Dell Open Manage Server Administrator by entering the following into your web browser:

(Version 6.3)

(Version 7.1)

and searching for the right package for your needs.

For windows it is:

(6.3) OM-SrvAdmin-Dell-Web-WIN-6.3.0-2075_A00.20.exe (125MB)
(7.1) OM-SrvAdmin-Dell-Web-WIN-7.1.0-5304_A00 (186MB)

2. Install Server Administrator
3. Run Server Manager and choose "Manage a Remote Node", connect via IP Address, enter in the root and root password, and ensure "Ignore Certificate Warnings" is ticked.

NOTE: If your OMSA is responding extremely slow, as mine was, and you are using the Dell Customized version of ESXi 5.0 update 1, look at this article, which details an update to fix some CIM issues. You'll need to use vSphere Update Manager to install a patch on your hosts.