VMM 2012 R2 – Bare-Metal Deployment Fails with Error 21161

Issue: Bare-Metal Deployments were failing. This was occurring after deep discovery had been completed and a build job had been initiated.
Exact Error Message: Error (21161) No available VHD resource was found within the host group.

The VHDs were present and builds had been working fine until recently. I checked all the logs and ran a VMM trace. The VMM trace indicated that there was a database inconsistency.

The VMM server had recently been updated to UR3.

Unfortunately the fix was to roll back to pre-UR3. This involved taking a fresh DB backup. Rolling back to the pre-UR3 backup in SQL, and uninstalling the updates from the VMM server.

Following this was the task of recreating any VMM objects that had been present, removing and adding hosts back (as the agent version was ahead-of-date).

This highlights how important regular SQL backups are for VMM servers as well as testing (as much as possible) after an Update Rollup.

EDIT: Niklas Akerlund has recently ran into the same error and has identified another fix for the issue, I can confirm that the fix outlined by Niklas works great! thanks Nicklas.

Error 21161 when doing Bare-Metal deployment on VMM 2012 R2

IPMI | VMM bare metal builds

A couple of tips when investigating VMM bare-metal IPMI issues
The VMM jobs will report the following if the connection failed:

Error 21212: Connecting to the BMC with address x.x.x.x failed.

This error indicates a problem with network connectivity, likely a firewall or routing issue (most IPMI communications go over port 623 UDP)

Error 21220: An out of band operation (IPMI) for the BMC x.x.x.x failed IPMI completion code 0x9 – Invalid Role.

This error indicates a problem with authentication. Check the VMM Run-As account, the username must have exactly the same case as the IPMI username. Also check the permissions required are sufficient (Full Admin Permissions are usually required)

SCVMM 2012 SP1 – Bare Metal Build Error 803d0008 A quota was exceeded

During a VMM bare-metal build error 803d0008 was showing up on the host console. The error occurred when the Host was attempting to send the discovered data back to the VMM server.

The logs showed the following:

 

Right at the bottom the error that stood out was “The numbers of bytes written exceeded the specified quota of 65536 bytes)

It looked as if there was perhaps to much information discovered and VMM was not able to deal with additional data over the 65536 byte limit.

This was a rebuild of an existing host, I had recently performed bare-metal builds successfully the day before.

The host happened to have a large number of LUNs presented to it from the SAN (23 to be exact).

The solution for me was to un-present the LUNs.

 

SCVMM 2012 Bare-Metal Failing on HP G7 Blade (IPMI Failure)

Found an issue with a blade yesterday where SCVMM 2012 was failing to power on the blade. The communication was occurring over IPMI. This was the first G7 that I had been testing bare-metal deployments on (having previously worked with Gen 8 Blades).

The iLO event logs would reveal the following entries:

IPMI/RMCP login by scvmm – x.x.x.x(DNS name not found)

IPMI/RMCP logout by scvmm – x.x.x.x(DNS name not found)

 

Whats more, after this occurred the ilo remote console would become unresponsive, requireing the blade to be reseated, or a reset of iLO.

 

Interestingly there was an option on the iLO Dedicated Network Port  that specified that iLO client applications (does this include IPMI?) should use IPv6 first

 

The fix for me was to disable IPv6 on the iLO dedicated network port, then SCVMM could communicate with the blade via IPMI.