After vCenter 6.5 U1g newly cloned VMs report “HA not enabled”

Our environment is:

  • vCenter 6.5 U1g (8024368) running on a Windows Server 2008 R2 Standard VM (we’re moving to an appliance once we go 6.7)
  • vSphere 5.5 7504623, 8 node cluster

Since upgrading to the latest vCenter every new VM we clone into our production cluster reports “The virtual machine failed to become vSphere HA Protected and HA may not attempt to restart it after a failure”

After contacting VMware Support they informed me that this is a known issue with the versions of vCenter and vSphere we’re using and to upgrade to vSphere 6.0 or 6.5.

Since vSphere 5.5 is EOL in September no patches are coming to resolve this.

It also turns out the issue is a GUI issue only. The VMs displaying this error are still protected via HA and will recover fine if a host goes down.

A workaround for the issue is to disable/enable HA. Doing so will clear the error.

How to replace the boot volume on a RaspberryPi

Just ran into what I think is called the “rainbow screen of death” where a fully functioning RaspberryPi running Raspbian stopped working on me after a reboot.

I know my MicroSD card is good quality and I wasn’t mucking with the boot file system before the reboot so it baffled me why things would just stop working.

I ran then MicroSD card through SpinRite and that didn’t solve the problem. If I connected the microSD card to another PC all the files were there.

Based on my research I found this happens from time to time where something goes wrong with the /boot volume. I could have just re-loaded and re-setup my RPi (it would have taken less time at this point) but I decided I wanted to try and fix it with out losing all of my data.

This is what worked for me:

  1. Image a fresh spare microSD card with the same distro of Raspbian you’re using on the non-functional microSD
  2. Connect the non-functional microSD to the same PC you have your freshly imaged microSD connected
  3. In Windows there is going to be one partition (disk) under My Computer you can actually view, that’s the boot partition
  4. On the non-functioning microSD copy ‘cmdline.txt’ and ‘config.txt’ to some place safe on your computer
  5. Copy the entire contents of the boot partition from the new microSD to the bad one overwriting all files when prompted
  6. Copy the backups of ‘cmdline.txt’ and ‘config.txt’ on to the bad microSD

If you’ve never run ‘rpi-update’ then you’re done. You should be able to put the microSD into your RPi and it will boot again.

If you have run ‘rpi-update’ you have a bit more work to do.

  1. Go to https://github.com/raspberrypi/firmware/releases and download the release that matches the date of the last time you ran rpi-update
    Note: If you don’t know when this is you might be able to boot the RPi now, login locally and dig through your ‘history’ or look at the date/time stamps on the files in /boot
  2. Extract the downloaded file
  3. Copy the contents of firmware-#.#######\boot into \boot on your bad microSD overwriting all files when prompted

Now you’re done. You should be able to put that bad microSD back into a RPi and have it boot

Once your RPi is backup and running I highly recommend manually running a ‘rpi-update’

References

How to remove the NetApp Host Utilities hotfix check

Sick of the hotfix requirements for NetApp Host Utilities? Me to.

On a test/dev server I have I’ve been experimenting with Veeam, iSCSI, ReFS and Server 2016. I wanted to run the NetApp Host Utilities against my Server 2016 box and was told I was missing Q3197954:

Problem is this 2016 server is fully patched and if I go manually download Q3197954 (which appears to be a previous rollup) I can’t install it on my fully patch server.

On a hunch I figured I might be able to crack open the MSI and find a way around the hotfix check and I was right. Here is how you do it:

  1. Download and install Orca MSI Editor (Alternate Download)
  2. Download the NetApp Host Utilities package from https://now.netapp.com/ (v7.1 as of this writing)
  3. Launch Orca and open the MSI file with it
  4. Select ‘InstallUISequence’ on the left and then click the ‘Sequence’ column on the top right to sort by lowest value first
  5. Locate ‘CheckMSHotfixes’, right click on it and choose ‘Drop row’
  6. Click ‘Save’
  7. Close Orca
  8. Copy the edited MSI to your server and run the installation as per normal

Digging around the MSI a bit I found references to “IGNORE_HOTFIX_CHECK” listed. I suspect it’s some kind of flag or environment variable that could be set to accomplish the same thing but I was unable to find any documentation on how and a few guesses around using environment variables didn’t pan out so I decided to stick with the above solution.

A note of caution. This is likely 100% unsupported by NetApp and if they found out you did this to get the Host Utilities on your server you might run into support issues.

Importing a OVF exported from vCloud into VMware Workstation fails

We’re backing out of a vCloud provider and trying to drag our VMs back into our local vSphere cluster.

I’ve used ovftool to export our VMs from vCloud into OVF templates. I then import the OVF into VMware Workstation 14 and from there drag/drop the VM into our vSphere cluster. There is likely a way to get the ovftool to export in a format that will work directly with vSphere but since this is working I’m just going with it.

This process worked fine on all of our VMs until I got to a group of them that had access to an extra network in vCloud. When trying to import these VMs into VMware Workstation I get the following error:

The source contains more than one network. This target supports at most one network.

I cracked open the ovf file in a text editor and found this near the very top:

In here you can see two networks, “myorg-it-pa-protected” and “server-net” on Line 3 and 6

The networking configuration doesn’t really matter to me since it doesn’t match with our vSphere deployment all I want to do is get these VMs imported. I’ll edit their networking afterwards.

I ended up deleting “myorg-it-pa-protected” by taking out lines 6-8 and lines 29-45. I then save/closed the OVF file and ran it through a hashing app to get the files SHA256 value.

I then opened the .mf file that sits in the same directory as the OVF file and updated the SHA256 entry for the OVF file. I was then able to import my VMs into VMware Workstation.

On Mac/Linux you can use “sha256sum <filename>” to get the SHA256 value of the edited OVF file. On Windows I use tools like HashTab and HashCalc OR if you have the Linux Subsystem installed on Windows 10 you can just use “sha256sum <filename>”.

Patterson EagleSoft v18 crashing during auto-backups

We’re running Patterson EagleSoft v18.10.05 on a Windows 2008 R2 Standard server.

We have auto-backup configured to run when ever the database starts up and to keep 14 backups:

We have a scheduled task on the server that runs “D:\EagleSoft\Shared Files\techaid.exe -stop” at 3:00am and then “D:\EagleSoft\Shared Files\techaid.exe -start” at 3:15am to trigger a nightly automatic backup while the database isn’t in use.

We just noticed that backups haven’t been occurring since 2017-12-31. Funny how they stopped 2018-01-01.

Then around 2018-02-12 the EagleSoft database would stop as planned at 3:00am and then crash/fail to start at 3:15am with no backup being generated. We’d then have to manually start EagleSoft’s database the next day so users could access it.

We found files like these in the EagleSoft auto backup directory:

  • DotNetZip-5tusrxsj.tmp
  • DotNetZip-fybwnf0z.tmp
  • DotNetZip-hete0py2.tmp
  • DotNetZip-zjlqhwj0.tmp

After contacting Patterson for support they informed us this is a known issue with EagleSoft v18 and the fix was to upgrade to v19.

I dug into the backup zip files and found they contain only three files:

  • Eaglesoft.Server.Configuration.data
  • PattersonPM.db
  • PattersonPM.log

All of which are located in “<EAGLESOFT INSTALL DIR>\Data”.

Some further digging showed there were additional LOG and DB files in the Data DIR that aren’t being backed up by auto backup.

Upgrading right now isn’t exactly an option for us so I wrote a PowerShell script to replace EagleSofts built-in auto backup. We run full backups of the server with Veeam so this script is just a little extra insurance for the database specifically.

Note: This script requires PowerShell v5.0 or newer. As of this writing v5.1 appears to be the newest and can be installed on Windows Server 2008 R2 or newer.

You can save this as ‘eagleSoftBackups.ps1‘ (or anything really) on your EagleSoft server and then use ‘Scheduled Tasks’ to schedule it to run nightly or how ever often you want.