Upgrade vRA 7.1 to 7.2 journey

Challenges are one of the reasons that keeps my interest in IT.  I’ve supported many products in my career.  Every product I’ve supported have one thing in common…..UPGRADES..  ( Typed that in upper case on purpose).

There are couple strategies to upgrades.

  • Blue / green deploys, a recent cloudy term to stand up new, deploy, test and cutover.   Before this was a term, I prefer this method.  If there is an issue, it’s easy to revert to original environment.
  • In-place upgrade of existing environment.  This is NOT my favorite method, but necessary sometimes.

This post shares experiences on a recent upgrade from vRealize Automation 7.1 to 7.2.    These are my notes, they include links to formal documentation, which in my experiences with any product, there are a few tips / tricks between the lines docs don’t cover.  I’ve included my raw notes below.  I hope this helps in your adventure.

I was fortunate enough to have a separate environment to test before doing the real environment.  Since this was an in-place upgrade, going through the experience helped prepare what to expect, as well as to revert back to original environment.

With the magic of cloning Linux appliances, snapshots on IaaS windows machines, and a SQL Server backup.  It helped simulate the ability to go back to original infrastructure.  It’s doesn’t make me as comfortable standing up new, but was better than no backout plan at all.

  • Read the upgrade docs, search forums for others reporting issues
  • Test the upgrade multiple times.  Also, think about if something goes wrong, how to revert to original version.  Do this multiple times until you are sick of it, that means you’ll be prepared
  • Have backups, clones and use snapshots on windows machines
  • Disable Backups before starting.  Contention on machines could cause issues
  • If you have support with VMware, open a proactive case and have them review your environment.
  • Clean-up any un-submitted requests
  • Clean-up any In-Progress requests that are orphaned
  • Give yourself 4 to 8 hours.  Communicate to your end users.  (Under promise, over deliver)
  • Coordinate with your users to test use cases after upgrade
  • Remember external systems like vRealize Business, Log Insight that access vRA.
  • Make sure end users disable automated build requests.

Here are my raw notes including links including  steps I followed.   These were reminders to help the overall process, the order of operations.  I’ve found having these types of notes help to refer to.

Prep work, review documents and blog posts.

Items I did as prep work.

Launch config using the command: java.exe -jar <vRPT Jar file> config

  • Download updated management agent. KB article with updated management agent, Workaround issue on upgrade on IaaS components
    http://kb.vmware.com/kb/2147926

Load-balancing

The network guys will wonder, what?!  Mine did, just tell them the vendor requires the change, double and triple check this step.

Cloning each machine vRA to upgrade

  • Turn off vRA appliances – Clone each machine (good to have backups of original appliances)
  • Turn off IaaS machines, snapshot.
  • Turn on all machines, verify services both IaaS and vRA are healthy (both appliances) I learned by testing if you clone windows, I see extra machines in the VAMI you’ll see “several called clone”
  • https://<vra-url>:5480/#cafe-services (make sure all services are started)
  • Close any opened files in the pgdata directory and remove any files with a .swp suffix. (on primary vRA appliance)
    –/var/vmware/vpostgres/current/pgdata/
  • Backup SQL Server database (Get with DBA to schedule ahead of time)
  • Upload ISO to datastore, mount to primary and secondary appliances if you don’t have internet access for updates.
  • Adjust settings in VAMI on primary and secondary to use cd-rom
  • ***- Run on a remote machine well connected to network, not laptop on wireless***
  • ***Disable vCenter backups on the cluster so snapshots aren’t taken***
  • De-register vRB via vRB appliance (because we haven’t used it yet) – might not apply to you.
  • Run update on primary (Look for text similar listed below when completed)
  • Reboot primary and secondary vRA appliances after upgrade completed
  • Verify both appliances upgraded
  • Deploy updated Management agent on IaaS machines
  • Deploy updated java (version 1.8)
  • Reboot each IaaS machine
  • Verify services on IaaS and appliances are healthy
  • Create upgrade.properties on primary ( Backup file = cp -p upgrade.properties upgrade.properties.password)
  • run ./upgrade (cross your fingers) – I had to run three times and my install FINALLY upgrade all six windows machines.

Raw text after upgrade
Version 7.2.0.381 Build 4660246

Last Check:
Tuesday, 2017 January 31 15:46:07 UTC-5 (Using update CD found on: /dev/sr0)

VA-check: finished

Pre-install: finished

After all appliances are upgraded, ssh to the master appliance and go to /usr/lib/vcac/tools/upgrade. Populate all the required data in upgrade.properties and execute ./upgrade script

Replica nodes are upgraded successfully. Reboot master node to trigger the reboot of replica nodes

Post-install: finished

Update finished successfully.
WARNING: Immediately update any vRealize Automation IaaS nodes after reboot to avoid product version mismatches.

Last Install:
Tuesday, 2017 January 31 16:29:07 UTC-5

VA-check: finished

Pre-install: finished

After all appliances are upgraded, ssh to the master appliance and go to /usr/lib/vcac/tools/upgrade. Populate all the required data in upgrade.properties and execute ./upgrade script

Replica nodes are upgraded successfully. Reboot master node to trigger the reboot of replica nodes

Post-install: finished

Update finished successfully.
WARNING: Immediately update any vRealize Automation IaaS nodes after reboot to avoid product version mismatches.

Hope this helps,

Steve Schofield
#vExpert 2017

@steveschofield
http://vsteve.me

Getting used to web client only world…Using VMware HTML 5 fling….

Looking for some hope?  This post hopefully will make you smile, give you some hope.  Grab your favorite beverage and let’s begin.  I’ve been working in an environment with multiple vCenters.   Many were either 5.5 or 6.0.   We still had access to the famed ‘full C# client’, even though the Flash Client was available, many didn’t use and would continue to use C# client until we were forced to change (me included).

For long-time admins, the full client is like comfort food or that favorite beverage they are used to, don’t make me change. With anything in IT, change is part of the job.

In one evening, we upgraded multiple (5) vCenters to 6.5, putting the C# out to pasture.  On one hand, we were thrilled the upgrades and migrations from windows to appliance worked (couple bumps, but we were able to get past).  for those wondering what bumps, we had to remove / re-add the PSC to Active Directory.

On the other hand, there was a small empty feeling.  I try look at the bright side in any situation. ( I really do although there are others who would disagree).

As part of the 6.5 rollout, there is two clients.

  • The Flash client (full functionality and some stresses to using it!)
  • The new HTML 5 shiny client

The links are accessible within the landing page when navigating to the vCenter by name. I’ll give VMware credit putting the wording (partial functionality) on the landing page.   This blog post isn’t here to debate Flash vs HTML5, that has been settled elsewhere.  Remember, this blog post is about giving hope. 🙂

html5-1

Did I say this blog post was about providing some hope.

Dennis Lu apparently likes taking on big challenges.  He is a frequent contributor and main person for something called HTML 5 fling (more info here)  For those unaware or haven’t checked this in a while, it’s grown up.

As part of our rollout, I deployed a separate HTML 5 fling appliance. The appliance is used on more frequently used vCenters accessed by customers. Plus, you can give the appliance a handy DNS name. We call ours vhtml.example.com (have to get a little “v” in the name)

When I first explored the HTML 5 fling, the appliance required a re-deploy every time. Although the HTML 5 fling was “kewl”, it wasn’t functional enough to use in our environment.

Fast forward, the current release is 3.9 as of this blog post. A few weeks ago, I deployed the 3.3x release appliance.  I’ve used the update feature twice without issues (remember to snapshot before upgrade).  Good Job Dennis and crew!  Handy feature here.

html5-2

To access this functionality, go to https://<ApplianceIPorName>:5490 (note 5490, not 5480 like I type a few times).  Login and click update.

The update will take a few minutes.  I noticed the finalized update status appears to not always notify when done..   I waited a few minutes and refreshed my browser (Chrome is my preferred one).

Here is a screenshot of the update in progress.

html5-3

The reason we deployed the extra appliance was to give ability to have a client end users could access, that gets updated more frequently than the HTML 5 client hosted on the vCenter appliance.

To update the HTML 5 hosted on vCenter, requires an upgrade as far as I know.  Would be handy to do it separately from a vCenter upgrade.  (@VMware hint hint!)

We generally try to limit upgrading vCenter to once or twice a year.  Using an external appliance, we get new features faster, safer with less hassle and risk upgrading vCenters.

I hope you enjoyed this slice of hope, there is part of me that misses the C# as my every day tool, we have a couple of vCenters it still works on, although there is little need to access them regularly.

Thanks Dennis and team for providing this option.  It’s made the transition a little less painful.  The disclaimer is use at your own risk, test in a non-production environment first.

PS – The appliance appears it needs internet access, so you’ll have to check with the security group or whoever manages the firewalls to download updates. I’m not sure if there is a way to do offline updates to an existing appliance, probably a reload is required.

Enjoy,

Steve Schofield
#vExpert 2017

@steveschofield
http://vsteve.me

What is this section for?  It’s a separate way to share ideas to pass along that I thought of while typing my blog post.  If you know some of the answers, I’m on twitter at @steveschofield

  • Love to have the appliance automatically redirect port 80 to 443.  We have to type in https:// (maybe a browser issue now HTTPS is more common)
  • Ability to externalize web client on multiple machines, load-balance vs. being a single point of failure + the authentication window that appears on a Platform service controller
  • Update HTML 5 client / Flash web client separately from vCenter appliance
  • Single Appliance access multiple, separate vCenters hosted in separate SSO domains.