ESXi 6.0 reports “error code 99” during Stage patches to entity VUM operation

As always when we have a maintenance window and need to do our work in the Virtual Environment, we always find new issues. This time was in one of the ESXi 6.0 hosts.

This is an HP DL360 G7 with the ESXi 6.0 build 3620759.

When trying to use VMware update manager in this host, scan run without any problem, but when trying to Stage the updates, we get:

Stage patches to entity esxi01.localdomain. The host returns esxupdate error code:99.
An unhandled exception was encountered. Check the Update Manager log files and esxupdate log files for more details.

Looking at the esxpudate.log(in ESXi var/log/) I see a lot of failed tries to upload VIB files to the repository. Also this:

2016-04-24T15:43:54Z esxupdate: root: ERROR: Traceback (most recent call last):
2016-04-24T15:43:54Z esxupdate: root: ERROR:   File “/usr/lib/vmware/esxcli-software”, line 462, in <module>
2016-04-24T15:43:54Z esxupdate: root: ERROR:     main()
2016-04-24T15:43:54Z esxupdate: root: ERROR:   File “/usr/lib/vmware/esxcli-software”, line 453, in main
2016-04-24T15:43:54Z esxupdate: root: ERROR:     ret = CMDTABLE
2016-04-24T15:43:54Z esxupdate: root: ERROR:   File “/usr/lib/vmware/esxcli-software”, line 173, in ProfileGetCmd
2016-04-24T15:43:54Z esxupdate: root: ERROR:     raise Exception(“No host image profile defined”)
2016-04-24T15:43:54Z esxupdate: root: ERROR: Exception: No host image profile defined

So need to check the Host Software profile(not the same as Host Profile) and check the issue.

Then I notice that somehow the ESXi did lose is software profile and was not possible to stage any update, or even install a VIB manually.

When troubleshooting the issue, I found out that there was no Software profile in this host.

Also checking the VIB list in this host, I get this:

Needed to check vibs folder and profiles folder in /var/db/esximg and then I notice the two folders were empty.

When checking the same information from a working ESXi host, I get this:

And also this:

Searching on the internet for the same type of issue, I cannot find anything similar. VMware KB has a solution for the “error code 99” by Initializing VUM without a reinstall. But since this is a problem in one host, this could not be the solution for this particular issue.

Then found an article with a similar issue in Patrick Terlisten blog.

Since I didn’t want at this moment reinstall the ESXi host, so I needed to try this solution, even was not 100% similar, was almost the same.

I start to follow Patrick recommendation to copy the esximg files/folders from a working host(in this case esxi-02) to the corrupted ESXi host(esxi-01).

First, if you don’t have your SSH Client enable in the host firewall, you need to enabled to do the next task using SCP command.

To enable SSH Client in the source ESXi host:

Note: Don’t forget to disable SSH Client after finish this tasks.

After you run SCP command, you will be prompted for the root password of the remote host and once you have successfully authenticated the file will copy.

After the copy, the files, check if the files are in the ESXi host(esxi01).

Host Software profile:

After checking that the files are copied into the destination, I rerun the VUM Scan into the host and after Stage, and no errors found. After that, I was able to remediate the ESXi host and is fully updated.

Again, thanks to Patrick for his solution that put me on the right track.

Hope this can help.

Note: Share this article, if you think is worth sharing.

©2016 ProVirtualzone. All Rights Reserved