How to Fix Hyper-V Backup Errors Quickly

When backing up Hyper-V VMs, VSS errors (Volume Shadow Copy Service) are not uncommon. The error potential for VM backups is much higher than for physical servers because many resources are shared and live backup requires the (error-free) participation of all VSS aware services inside each VM. Any resource shortage of service configuration issue inside a VM or on the host can easily break the entire live backup process, and is beyond the control of the backup software being used.

VSS integrates with BackupChain and Hyper-V to provide a live backup of a VM. A signal is sent to the VM, too, to prepare for live backup, which in turn signals all VSS-aware services inside the VM. As you can see, you can quickly end up with over 100 applications becoming involved in a VM backup phase, especially when multiple VMs are backed up simultaneously.  See Live Hyper-V Backup using Hyper-V VSS Writer: How Backup Works and 18 Hyper-V and Hyper-V Backup Pitfalls You Need to Know

The best strategy when you see VM backups fail is to check the logs in the Windows Event Viewer, as shown here: //backupchain.com/hyper-v-backup/Troubleshooting-First-Steps.html, especially the Hyper-V logs shown at the bottom of the article as well. Look for VolSnap, Disk, and VSS entries in Application and System, around the time when BackupChain reports the first failure. Subsequent errors are likely to be consequential.

A typical scenario is also that Hyper-V Integration Services are outdated, there’s a lack of space on the host or inside the VM, or the VM vetoed. The latter case requires the VM’s Event Viewer to be inspected.

How to Fix Hyper-V Backup Errors

First a couple of recommendations for Hyper-V backups:

• Use Hyper-V Backup tasks only, not universal
• Use sequential backups. Simutaneous VM backups are only meant to provide multiple VM consistency and cause a lot of system stress, especially for VSS, and thereby increase the likelihood of a backup issue. On a well equipped system you won’t have issues with simultaneous backups; hard drive and CPU speeds (and sometimes network speeds and free disk space) are the important factors.
• Use the following optimization on all your servers (please run this also for all drives in each server):
  vssadmin resize shadowstorage /for=c: /on=c: /maxsize=unbounded
• Use VSSUIRUN.EXE and disable scheduled shadow creation on all drives

In a nutshell, Hyper-V backups work like this:

1. BackupChain works with Hyper-V to prepare the disk as well as the VM for live backup
2. A signal is passed from Hyper-V to the VM via Hyper-V integration services (hence all integration services inside each VM must be up-to-date)
3. Hyper-V decides whether live backup or saved state backup is appropriate
4. One or more disk snapshots are created and supported by VSS
5. BackupChain can finally do its job and releases VSS when done

Note that during each of the steps above, something can go wrong and since all VMs share the same resources, it can get very complex to identify an error. Also error details aren’t provided to BackupChain most of the time as they potentially invoice dozens of different sources.

Errors could originate from:

• VSS on host
• VolSnap
• Disk driver
• Hyper-V
• Hyper-V host interaction with Hyper-V integration services inside the VM
• VSS inside the VM
• VSS aware services inside the VM, such as Exchange, SQL, etc.
• VolSnap and Disk drivers inside the VM

Now, to drill down to find the actual cause you’ll need to inspect various logs from about the time BackupChain started the backup to the time when it reported it, see BackupChain’s Log tab for exact timings.
Note that BackupChain will also add entries into the Event Viewer logs; however, these are placed there for automated monitoring software and do not contain any error details.

Each time you run into a VSS-related backup error, check

• Event Viewer logs on host: Application + System for any VSS, VolSnap, Disk, or other related issue
• Hyper-V logs in Applications and Services Logs -> Microsoft -> Windows -> Hyper-V XXXXXXX. There are several nodes in the Event Viewer and each needs to be looked into
• The above two steps are shown here: //backupchain.com/en/troubleshooting-volume-shadow-copy-errors-first-steps/
  and may be automated using our VssDiag tool: //backupchain.com/en/vssdiag/

Usually the error is to be found in one or more of the above logs.
However, sometimes the VM vetoes being backed up live. In that case you may find a hint in the Hyper-V host logs above but the real cause is likely logged in the VM’s Event Viewer in full detail. Hence, the next step is to log into the VM and open its Event Viewer. Again you would check the Application and System logs for VSS related entries and other error entries from VSS aware services, such as SQL Server, Exchange, etc.

Sometimes the error is due to lack of free disk space or some Exchange configuration issue. There are probably over a thousand different ways Hyper-V backups can fail and it usually boils down to resource and configuration issues.

Please to contact our helpline with questions as needed.