Yes, I know, it's not recommended. Heck, it's not even supported. But what if you want to throw caution to the wind, and do it anyway? Here's the method I used that worked successfully. A few notes though:
- This only works in environments with two or more domain controllers
- There may still be problems with clients authenticating should you be ballsy enough to do this during business hours, as I was. Rebooting usually fixes them.
1) You'll first of all want to transfer all of your FSMO roles to another DC in the environment. You can see which servers are hosting the FSMO roles in your environment by opening up a command prompt on your domain controller and typing "netdom query fsmo". You'll get back a neat list of which services are where. Transfer them to other domain controllers (I'll leave the specifics out of this article, google has the answers.)
2) Open a command prompt on the DC you wish to P2V and type dcpromo, then hit enter. Proceed through the process to remove active directory from this server. You may get an error the first time... it's okay... just run it again. It's complaining that services didn't stop or start in a timely fashion, which is normal for a DC. Reboot once you are done.
3) Uninstall DNS and WINS using Add/Remove programs. Also, if you are running DHCP, you may want to start up a split scope on another server, as you should disable the dhcp scope on this server for the time being.
4) Change the DNS on your server you wish to P2V so that it's primary DNS points to a VALID server other than itself... otherwise, your P2V will fail immediately. This is because the server being virtualized needs to be able to resolve the DNS names of the destination ESX server and VirtualCenter server. Change the DNS for your VirtualCenter server and ESX server so that they are all pointing to the same DNS server, just to be safe.
5) Run the P2V, it should go off without a hitch now that DNS is right.
6) Now that the server is back up and virtual, run dcpromo again to make it a domain controller once more. This should reinstall DNS for you, if you uninstalled it earlier. Fix up DHCP and WINS, and you are back in business. Don't forget to transfer back your FSMO roles, if you want them to be on your virtual server!
7) Readjust your DNS settings on the re-promoted domain controller to point to itself, and fix your VirtualCenter/ESX server DNS settings too, if you had to change them.