I have vCenter 7.0.0.10700 installed and I want to update to 7.0.1.00200. When I run Update Planner > Interoperability it reports that all of my ESXi hosts are running ESXi 7.0.1. If I run the pre-update checks I get “No issues found”. When I go to the appliance to do the upgrade, both “Stage Only” and “Stage and Install” are greyed-out and unselectable.
I tried a dozen different tricks, including ssh-ing into the appliance as root and editing the /etc/applmgmt/appliance/software_update_state.conf file, but nothing could enable the “Stage Only” and “Stage and Install” buttons.
Use the command line
I finally decided to try upgrading via the command line. I have backups going back 30 days. I even double-checked and yes, my NFS server has files in the backup directory for each of the past 30 days and they have data in them. There’s probably even a way to restore one of those backups if something goes horribly wrong. Onwards!
I was already logged into the vCenter appliance shell as root. The next thing I needed to do was to figure out where the command line tools were hidden. I found them in /usr/lib/applmgmt/support/scripts.
Disclaimer: I work at VMware, but I have no idea if the following is an “acceptable practice” or not. If your production vCenter is broken and you have a support contract, call support. If you’re messing around on a home or test system and you don’t care how badly you screw it up, feel free to try the command line tools.
root@vcenter [ ~ ]# cd /usr/lib/applmgmt/support/scripts
root@vcenter [ /usr/lib/applmgmt/support/scripts ]# ls -al
total 108
drwxr-xr-x 4 root root 4096 Aug 30 18:18 .
drwxr-xr-x 4 root root 4096 Aug 30 18:18 ..
-r-xr-xr-x 1 root root 205 Aug 15 07:16 autogrow.sh
-r-xr-xr-x 1 root root 633 Aug 15 07:16 manifest-verification
-r-xr-xr-x 1 root root 286 Aug 15 07:16 mapping.sh
-r-xr-xr-x 1 root root 2056 Aug 15 07:16 pgtop.py
-r-xr-xr-x 1 root root 3396 Aug 15 07:16 port-accessible.py
drwxr-xr-x 2 root root 4096 Aug 30 18:18 postinstallscripts
-r-xr-xr-x 1 root root 5207 Aug 15 07:16 prestart-applmgmt.sh
-r-xr-xr-x 1 root root 4171 Aug 15 07:16 resize-root.py
-r-xr-xr-x 1 root root 251 Aug 15 07:16 setup-env.sh
-r-xr-xr-x 1 root root 4001 Aug 15 07:16 showlog.py
-r-xr-xr-x 1 root root 3910 Aug 15 07:16 shutdown.py
-r-xr-xr-x 1 root root 35773 Aug 15 07:16 software-packages.py
-r-xr-xr-x 1 root root 8085 Aug 15 07:16 support-bundle.py
drwxr-xr-x 2 root root 4096 Aug 30 18:18 testsThese are the Python scripts that are linked to the Command shell. I’m actually in the root shell. I can run these directly from the root shell, or exit back to the Command shell and use them in the “official” way. In case I need to pull in support let’s do this the official way.
The software-packages.py script is what does the upgrade. Let’s exit back to the Command shell and see what it says it supports.
root@vcenter [ /usr/lib/applmgmt/support/scripts ]# exit
Command> software-packages
usage: software-packages [-h] {stage,unstage,validate,install,list} ...
optional arguments:
-h, --help show this help message and exit
sub-commands:
{stage,unstage,validate,install,list}
stage Stage software update packages
unstage Purge staged software update packages
validate Validate software update packages
install Install software update packages
list List details of software update packagesStage the packages for the update
Since the appliance wasn’t letting me upgrade, I thought I’d first check to see if I already have upgrades staged.
Command> software-packages list --staged
[2021-01-22T21:45:41.022] : Packages not stagedOK. Nothing staged. How do I stage packages?
Command> software-packages stage --help
usage: software-packages stage [-h] [--url [URL]] [--iso] [--acceptEulas] [--thirdParty]
optional arguments:
-h, --help show this help message and exit
--url [URL] Download software update package from URL. If no url is specified, https://vapp-updates.vmware.com/vai-
catalog/valm/vmw/8dc0de9a-feedl-1337-be0a-6ddeadbeefa3/6.7.0.42000.latest/ is used.
--iso Load software update packages from CD/DVD drive attached to the appliance
--acceptEulas accept all Eulas
--thirdParty Stage third party packages.--thirdParty should only be usedwith --url.Sounds clear enough. I’ll try that:
Command> software-packages stage --url --acceptEulas
[2021-01-22T21:46:28.022] : Latest updates already installed on VCSA, Nothing to stageWell that’s not correct. There’s definitely an update available. Re-reading help again I notice that the default URL looks something like:
https://vapp-updates.vmware.com/vai-catalog/valm/vmw/8dc0de9a-feedl-1337-be0a-6ddeadbeefa3/6.7.0.42000.latest/I’ve obfuscated the actual URL, but that’s a vCenter 6.7.0 URL, I’m using 7.0.0, and I want 7.0.1.
I go back to the appliance web UI and click the Update > Settings button.
Settings shows a different URL for 7.0.1, so I copy and paste that into the command line:
Command> software-packages stage --acceptEulas --url https://vapp-updates.vmware.com/vai-catalog/valm/vmw/......
[2021-01-22T21:48:28.022] : Target VCSA version = 7.0.1.00200
[2021-01-22 21:48:28,781] : Running requirements script.....Trust but verify
A little while later everything was staged. I decided to validate everything.
Command> software-packages validate
[2021-01-22T21:50:11.022] : For the first instance of the identity domain, this is the password given to the Administrator account. Otherwise, this is the password of the Administrator account of the replication partner.
Enter Single Sign-On administrator password:
[2021-01-22T21:50:22.022] : Validating software update payload
[2021-01-22 21:50:22,327] : Running validate script.....
[2021-01-22T21:50:26.022] : Validation successful
[2021-01-22T21:50:26.022] : Validation process completed successfullyThen I check to see what’s staged:
Command> software-packages list --staged
[2021-01-22T21:50:45.022] :
category: Bugfix
kb: https://docs.vmware.com/en/VMware-vSphere/7.0/rn/vsphere-vcenter-server-70u1c-release-notes.html
leaf_services: ['vmware-pod', 'vsphere-ui', 'wcp']
vendor: VMware, Inc.
name: VC-7.0U1c
size in MB: 5107
tags: []
version_supported: []
productname: VMware vCenter Server
releasedate: December 17, 2020
executeurl: https://my.vmware.com/group/vmware/get-download?downloadGroup=VC70U1C
version: 7.0.1.00200
updateversion: True
allowedSourceVersions: [7.0.0.0,]
buildnumber: 17327517
rebootrequired: False
summary: {'id': 'patch.summary', 'translatable': 'In-place upgrade for vCenter appliances.', 'localized': 'In-place upgrade for vCenter appliances.'}
type: Update
severity: Critical
TPP_ISO: False
url: https://vapp-updates.vmware.com/vai-catalog/valm/vmw/8dc0de9a-feedl-1337-be0a-6ddeadbeefa3/7.0.0.10700.latest/
thirdPartyAvailable: False
nonThirdPartyAvailable: True
thirdPartyInstallation: False
timeToInstall: 0
requiredDiskSpace: {'/storage/core': 30.353511543273928, '/storage/seat': 32.21015625}
eulaAcceptTime: 2021-01-22 21:48:37 UTCWell, that shows:
version: 7.0.1.00200Which is the version I’ve been trying to upgrade to, so that looks good.
Did I mention that I have backup copies of vCenter going back 30 days? Well I do. If this goes really sideways I’m going to have to restore one of them.
Let’s do the update!
Command> software-packages install --staged
[2021-01-22T21:51:23.022] : For the first instance of the identity domain, this is the password given to the Administrator account. Otherwise, this is the password of the Administrator account of the replication partner.
Enter Single Sign-On administrator password:
[2021-01-22T21:51:43.022] : Validating software update payload
[2021-01-22 21:51:43,716] : Running validate script.....
[2021-01-22T21:51:47.022] : Validation successful
[2021-01-22 21:51:47,730] : Copying software packages 251/251
[2021-01-22 21:55:37,642] : Running system-prepare script.....
[2021-01-22 21:55:42,661] : Running test transaction ....
[2021-01-22 21:55:44,678] : Running prepatch script...
....
[2021-01-22 21:58:27,896] : Upgrading software packages ....
[2021-01-22T22:02:10.022] : Setting appliance version to 7.0.1.00200 build 17327517
[2021-01-22 22:02:10,242] : Running patch script.....
[2021-01-22 22:11:34,245] : Starting all services ....
[2021-01-22T22:11:35.022] : Services started.
[2021-01-22T22:11:35.022] : Installation process completed successfullyThat was it. The actual update took about 20 minutes, and although the UI said no reboot was necessary vCenter did reboot during the update. When it was done vCenter was running version 7.0.1.00200.
The vCenter appliance Update “Stage Only” and “Stage and Install” buttons are still greyed-out and unselectable, but right now there are no updates available so that’s how they should be. I’ll have to wait for the next update to see if they’re working again. If the buttons are still broken, at least now I know how to use the command line to install an update.
Hope you find this useful.
