— Important note: Incredible that I have published this blog post that I had written a couple of days ago and did some changes to publish and after publishing I check the vCenter 7 Update 3b Release Notes doesn’t exist anymore.
According to KB86191 the patch and also the ISO version was removed from VMware because had some issues with HA(vSphere HA fails to enable). This is exactly what I had before and had discussed in my previous blog post about this vSphere 7 updates.
Honestly, I don’t know what to say more about VMware updates and these continuous issues that are not been properly tested. Don’t they test this properly before they launch the updates or the patches?
Since the post is about how to apply a patch, 3b or other, I will leave the blog post published anyway and will update and VMware fix again and launch the proper patch with no issues. We hope.
But in the main vCenter had no issues for now. But it could be because I already did the workaround that I discussed in HERE and fix the issue that was wrong with the i46enu vib and they do not exist anymore in the ESXi hosts.
UPDATE 20/11/2021: VMware is announced that it will completely remove from their Download portal any vCenter/vSphere 7 Update 3b. At the moment only vCenter/vSphere 7 Update 3a is available.
Some, complement informational answers KB86398
End Note —
The vCenter update that we were waiting to fix some of the issues from previous versions is finally out. In this blog, New vCenter 7 Update 3b how to update, I will explain what is new, what is fixed in this version, and how to update your vCenter 7 since we need to do it manually when you have a previous version.
With this new vCenter 7 update 3b there is also new ESXi 7 update 3b patches. But if you try to stage, remediate any of your ESXi with these three new updates before updating your vCenter, it is not possible. You will get errors in the stage or when you try to remediate.
What are the main issues that this patch will address and fix?
- Upgrade to ESXi 7.0 Update 3 might fail due to changed name of the inbox i40enu network driver.
- You cannot enable vSphere HA on ESXi 7.0 Update 3 hosts.
- Rollup update by using a patch baseline to ESXi 7.0 Update 3 might fail.
- The hostd service might fail due to a time service event monitoring issue.
Some of the above issues I have already talked about in this blog, in HERE.
How to download and install the patch
Login to your VMware account and go to VMware Patch Portal
Select VC and 7.0.3 and click Search, then you would see the patch vCenter 7 update 3b. Click download and download the file VMware-vCenter-Server-Appliance-7.0.3.00200-18901211-patch-FP.iso
As you can notice, this is a patch file, not an ISO to upgrade. So to apply this type of file, we need to do it manually.
How to apply the patch.
First, you need to import the ISO to one of your ESXi datastores.
Then you can put the ISO in the vCenter Virtual CD/DVD drive and enable Connected and Boot At Power On(if you boot your vCenter before applying the patch).
Note: For some reason, when I try the above, I always get an error, and it was not possible to enable the ISO on vCenter. I needed to power off vCenter, login to the ESXi where vCenter VM was running, add the ISO again, enable, and power on.
So if you have the same issue, you need to do this with the vCenter power off, and of course, you need to it in the ESXi that the vCenter VM is running.
After the ISO is set on vCenter VM, we can start the update process.
Second, you need to login into your VCSA console(not the shell Linux line command).
If you log in and are automatically in the VCSA Linux shell, you need to change the chsh bash for the application. Run the following command chsh -s /bin/appliancesh root
Exit and login, and you will now be the VCSA Console. Do not use the command shell to go again to the VCSA Linux shell, the patch needs to run in the VCSA Console.
In the VCSA, we need to use three commands:
- software-packages stage -–iso
- software-packages list -–staged
- software-packages install –staged
First, we need to stage the ISO in vCenter.
Command> software-packages stage --iso
[2021-11-14 19:31:35,564] : Discovering updates...
[2021-11-14 19:31:41,005] : UpdateInfo: Using product version 7.0.3.00100 and build 18778458
[2021-11-14 19:31:41,006] : Target VCSA version = 7.0.3.00200
VMWARE END USER LICENSE AGREEMENT
12.11 Contact Information. Please direct legal notices or other
correspondence to VMware, Inc., 3401 Hillview Avenue, Palo Alto, California
94304, United States of America, Attention: Legal Department.
Do you accept the terms and conditions? [yes/no] yes
[2021-11-14 19:32:27,458] : Downloading RPM VMware-UpdateManager-7.0.3.00200-9361122.x86_64.rpm ....
[2021-11-14 19:32:29,464] : Downloading RPM VMware-certificatemanagement-7.0.3.00200-9356258.x86_64.rpm
[2021-11-14 19:32:31,472] : Downloading RPM VMware-commonjars-7.0.3.00200-9356258.x86_64.rpm ....
[2021-11-14 19:32:34,482] : Downloading RPM VMware-content-library-7.0.3.00200-9356258.x86_64.rpm ....
[2021-11-14 19:32:36,503] : Downloading RPM VMware-envoy-7.0.3.00200-9356258.x86_64.rpm
[2021-11-14 19:32:37,506] : Downloading RPM VMware-hvc-7.0.3.00200-9356258.x86_64.rpm
[2021-11-14 19:32:38,549] : Downloading RPM VMware-perfcharts-7.0.3.00200-9356258.x86_64.rpm
[2021-11-14 19:32:42,574] : Downloading RPM VMware-spherelet-solution-depot-7.0.3.00200-9361122.x86_64.rpm ....
[2021-11-14 19:32:44,583] : Downloading RPM VMware-spherelet_1_19-1.2.0-9361122.x86_64.rpm
[2021-11-14 19:32:46,592] : Downloading RPM VMware-spherelet_1_20-1.2.1-9361122.x86_64.rpm ....
[2021-11-14 19:32:48,597] : Downloading RPM VMware-sps-7.0.3.00200-9356258.x86_64.rpm
[2021-11-14 19:32:49,600] : Downloading RPM VMware-trustmanagement-7.0.3.00200-9356258.x86_64.rpm
[2021-11-14 19:32:50,610] : Downloading RPM VMware-vapi-7.0.3.00200-9356258.x86_64.rpm
[2021-11-14 19:32:52,618] : Downloading RPM VMware-vpxd-7.0.3.00200-9361122.x86_64.rpm ....
[2021-11-14 19:32:54,624] : Downloading RPM VMware-vpxd-agents-eesx-7.0.3.00200-9356258.x86_64.rpm
[2021-11-14 19:32:55,628] : Downloading RPM VMware-vpxd-svcs-7.0.3.00200-9356258.x86_64.rpm
[2021-11-14 19:32:56,632] : Downloading RPM VMware-vsan-health-7.0.3.00200-9356258.x86_64.rpm
[2021-11-14 19:32:57,636] : Downloading RPM VMware-vstats-7.0.3.00200-9350547.x86_64.rpm
[2021-11-14 19:32:59,643] : Downloading RPM cis-upgrade-runner-7.0.3.00200-9361122.x86_64.rpm ....
[2021-11-14 19:33:01,651] : Downloading RPM vmware-lookupsvc-7.0.3.00200-9356258.x86_64.rpm
[2021-11-14 19:33:03,660] : Downloading RPM vmware-vsm-7.0.3.00200-9356258.x86_64.rpm ....
[2021-11-14 19:33:05,672] : Staging completed successfully.
[2021-11-14 19:33:05,672] : Staging process completed successfully
List Stage files
Command> software-packages list --staged
[2021-11-14 19:37:12,917] :
leaf_services: ['vmware-pod', 'vsphere-ui', 'wcp', 'vlcm']
productname: VMware vCenter Server
releasedate: November 12, 2021
size in MB: 7084
summary: Patch for VMware vCenter Server 7.0
vendor: VMware, Inc.
version_upgrade_supported: ['184.108.40.20600', '220.127.116.1100', '18.104.22.16800', '22.214.171.12400', '126.96.36.19900', '188.8.131.5200', '184.108.40.20600', '220.127.116.1100', '18.104.22.16800', '22.214.171.12400', '126.96.36.19900', '188.8.131.5200', '184.108.40.20600', '220.127.116.1100', '18.104.22.16800', '22.214.171.12400']
After the ISO file is staged and all is ok, we can now install the patch.
Command> software-packages install --staged
[2021-11-14 19:46:28,259] : update is already staged. Proceeding to install.
[2021-11-14 19:46:53,053] : Installing version: 7.0.3.00200
[2021-11-14 19:47:11,868] : Running precheck ....
[2021-11-14 19:47:18,886] : Validating user input ....
[2021-11-14 19:47:25,906] : Preparing system for update ....
[2021-11-14 19:47:28,917] : Running test transaction ....
[2021-11-14 19:49:50,378] : Stopping services ....
[2021-11-14 19:50:01,411] : Extracting script information from rpms ....
[2021-11-14 19:50:04,420] : Installing RPM vmware-vlcm-7.0.3.00200-9350547.x86_64.rpm ....
[2021-11-14 19:50:15,455] : Installing RPM VMware-vpxd-7.0.3.00200-9361122.x86_64.rpm ....
[2021-11-14 19:50:22,477] : Installing RPM applmgmt-7.0.3.00200-9356258.x86_64.rpm ....
[2021-11-14 19:50:25,486] : Installing RPM VMware-vsan-health-7.0.3.00200-9356258.x86_64.rpm ....
[2021-11-14 19:52:12,842] : Installing RPM VMware-TlsReconfigurator-7.0.3.00200-9356258.x86_64.rpm
[2021-11-14 19:52:14,848] : Installing RPM VMware-Postgres-upgrade-96-13.3.0-18202630.x86_64.rpm ....
[2021-11-14 19:52:17,858] : Installing RPM VMware-Postgres-upgrade-11-13.3.0-18202630.x86_64.rpm ....
[2021-11-14 19:52:22,879] : Installing RPM VMware-Postgres-upgrade-10-13.3.0-18202630.x86_64.rpm ....
[2021-11-14 20:25:11,774] : Converting data as part of post install ....
[2021-11-14 20:25:13,788] : Installation completed successfully.
[2021-11-14 20:25:13,788] : Installation process completed successfully.
This last step will take a while, but after you see the Installation process is completed successfully, your patch has been installed correctly.
We now have a vCenter 7 update 3b
I thought all was ok when I noticed a warning in the VM Console, a vCenter warning the following message: “vmdir.password is not set; aborting installation”
I remember this error from previous versions, but old ones. I never had this, but I remember to read something about it.
This happened when the update/patch did not finish correctly. But looking at my /var/log/firstboot/rpmInstall.json file, I see success, so all should be ok.
I tested vCenter all seemed to work fine, but the warning was still there. So I followed the KB84259 to remove the warning and Rebooted, and the warning was gone.
Then I could update all the ESXi hosts with the new vSphere 7 update 3b without any issues.
Finally, VMware launched this patch to fix some of the issues we had from update 2 and update 3a. The HA issue had a significant impact on some of our vCenters/Clusters.
Since vSphere 7/vCenter 7 was launched, we have had so many issues that I can’t remember an ESXi version with so many bad bugs and significantly impacted VMware infrastructure. Like I already discussed HERE and HERE.
I hope now we have stable vCenter and ESXi hosts. We have been doing workarounds, fixes, updates, etc., since August. For companies with hundreds of ESXi hosts and thousands of VMs, this has a considerable impact and means many, many man-hours to handle all these issues.
Share this article if you think it is worth sharing. If you have any questions or comments, comment here, or contact me on Twitter.