Dell Server P2V stuck at loading screen

Almost every P2V I have done to date has gone flawlessly. I frankly expected more trouble, but have been very pleased with VMware's product thus far. Almost every virtualization that experienced trouble was a result of third party software, or in this case, drivers for third party hardware.

The specific server in question was a Dell PowerVault 700 series NAS... it was running Windows Server 2003 Appliance edition, an OEM only version of windows 2003. Not really supported for virtualization, but the intention was to upgrade it to Datacenter Edition anyway once all was said and done... it was just a matter of getting there. After the virtualization process finished, the server powered on, and would not get past that animated windows logo loading screen with the grey dots moving across the bottom. I let it sit for up to 30 minutes before I was fairly sure there was no hope. The processor spiked at 100% the entire time, so I knew it was some kind of driver or service misbehaving. The system booted into safe mode just fine, but wouldn't boot into normal mode, even in diagnostic startup mode.

I worked with Microsoft, they didn't have a clue either. Time for process of elimination! I opened up MSINFO32 and made an export of all the Software Environment > System Drivers section for safe keeping, then went to work disabling all of them, section by section, starting with the ones marked as 'Stopped' with 'Start Mode' Set to Boot. I knew it had to be a 'stopped' one as windows was starting fine in safe mode, but not when everything started up normally. You can change them by opening up regedit and going to HKLM\System\Services. Expand each service name, and under it is a key called 'Startup' with a value of 0-4. 0 is 'Boot', 1 is 'System', 2 is 'Automatic', 3 is 'Manual', and 4 is 'Disabled'. Make sure to write down the original value of each 'startup' item as you set it to 4, because you will need to revert the changes.


Either way, disabling all the 'Boot' startup items didn't fix it, so I moved on to disabling all of the ones with 'start mode' set to 'System'. With all Boot and System items set to disabled, bingo! It started up. Now I put them each back to their original startup mode, one by one, rebooting into normal mode each time to see if the server would start. It finally narrowed down to one service, called "Msdisp." The description? "DELL LED DRIVER". Go figure! It was a driver for the hardware chassis to control the LED lights on the front of it, and it apparently blew up when it couldn't find whatever LED controller it was looking for. I left the service disabled, returned all others to their original values, and I was off to the races!

Hopefully this process of elimination method helps someone else out there!

Performing a P2V on a Domain Controller

Yes, I know, it's not recommended. Heck, it's not even supported. But what if you want to throw caution to the wind, and do it anyway? Here's the method I used that worked successfully. A few notes though:

- This only works in environments with two or more domain controllers
- There may still be problems with clients authenticating should you be ballsy enough to do this during business hours, as I was. Rebooting usually fixes them.

1) You'll first of all want to transfer all of your FSMO roles to another DC in the environment. You can see which servers are hosting the FSMO roles in your environment by opening up a command prompt on your domain controller and typing "netdom query fsmo". You'll get back a neat list of which services are where. Transfer them to other domain controllers (I'll leave the specifics out of this article, google has the answers.)

2) Open a command prompt on the DC you wish to P2V and type dcpromo, then hit enter. Proceed through the process to remove active directory from this server. You may get an error the first time... it's okay... just run it again. It's complaining that services didn't stop or start in a timely fashion, which is normal for a DC. Reboot once you are done.

3) Uninstall DNS and WINS using Add/Remove programs. Also, if you are running DHCP, you may want to start up a split scope on another server, as you should disable the dhcp scope on this server for the time being.

4) Change the DNS on your server you wish to P2V so that it's primary DNS points to a VALID server other than itself... otherwise, your P2V will fail immediately. This is because the server being virtualized needs to be able to resolve the DNS names of the destination ESX server and VirtualCenter server. Change the DNS for your VirtualCenter server and ESX server so that they are all pointing to the same DNS server, just to be safe.

5) Run the P2V, it should go off without a hitch now that DNS is right.

6) Now that the server is back up and virtual, run dcpromo again to make it a domain controller once more. This should reinstall DNS for you, if you uninstalled it earlier. Fix up DHCP and WINS, and you are back in business. Don't forget to transfer back your FSMO roles, if you want them to be on your virtual server!

7) Readjust your DNS settings on the re-promoted domain controller to point to itself, and fix your VirtualCenter/ESX server DNS settings too, if you had to change them.

Ghost Network Cards after a P2V migration

I have learned many invaluable things in the past few days about P2Ving live production systems... but none so important as the following little gem. It seems that with most systems, when you P2V them you end up with Ghost network cards for each of your original physical NICs... that is, they do not show up in the device manager, yet they still use the static IP addresses you had assigned, do funny things to network communication and name resolution, etc. Here is how to get rid of them:

1) Open a command prompt and type in "SET DEVMGR_SHOW_NONPRESENT_DEVICES=1". You will not get a confirmation back, it will just drop back to the prompt. Now, in the SAME CMD window, without closing it...

2) Type in "DEVMGMT.MSC" to open the device manager from the command prompt. This is critical, if you dont open it from the same CMD window you typed the first step in, it wont work.

3) In device manager, go to View > Show hidden devices

4) Expand nic cards and right click on each fadded network card to remove it.

I tried following similar instructions over and over, and it was not until I launched the DEVMGMT.MSC from the command prompt that I set that first variable in that it worked.

Once I removed the ghost nic cards, set my static IP back to what it was supposed to be (as VMware converter gives the new vNIC a DHCP address) then rebooted, all was well.