Live Hyper-V Backup using Hyper-V VSS Writer: How Backup Works

Live backup of Hyper-V virtual machines may be done with or without the use of the Microsoft Hyper-V VSS Writer, which is part of the Hyper-V Role in Windows Server 2008 and later.

The key integration element powering live backups in Windows is VSS, also called Volume Shadow Copy Service, which allows services and applications in Windows, such as Hyper-V, to integrate with Windows and the backup infrastructure that the backup software application sets up.

Backup software applications, such as BackupChain, a backup software for Windows Server, integrate with the Hyper-V VSS Writer to provide live virtual machine backups in Hyper-V.

General Requirements for Virtual Machine Backups

While there are many different ways to run backups on Microsoft Windows, the general requirement for a good, crash-consistent as well as application-consistent backup of a virtual machine in Hyper-V is to use VSS (Volume Shadow Copy Service) along with specialized backup software optimized for virtual machine backups. See application consistency vs. crash-consistency for more info on this subject.

VSS allows services, such as Microsoft SQL Server, Exchange Server, Oracle, and Hyper-V, to provide means for live backup of their respective data. For example, when a backup is initiated, Microsoft Exchange receives a signal and prepares its data structures for backup. By doing so, the data files that Exchange manages are now guaranteed by Exchange itself to be consistent; hence, it’s no longer a responsibility of the backup software to ensure consistency—it’s completely managed by the data owner. While Microsoft has developed quite a pile of rubbish in the past and got away with it, such as Windows 8, the evolution of VSS is quite an achievement. Yet, there is a lot of work ahead for backup software developers as well as Microsoft to improve VSS design in the future.

The interplay of all installed services with VSS is important to ensure the entire machine is in a consistent state before backup commences; however, this also complicates the IT administrator’s job quite a bit, too, especially when backups fail. Because backups depend on VSS and VSS may depend on dozens of services to work properly, any one of those services (SQL Server, etc.) may bring down the entire system. Backing up virtual machine hosts is, hence, several times more complex than backing up a physical machine.

Backing up virtual machines running in Hyper-V in a live state is even more complex because now you have perhaps a dozen of operating systems with each a dozen of services installed. The IT admin needs to attend to literally hundreds of services which could each potentially break and veto a live backup process. Prevention is therefore the most important aspect to keep the Hyper-V IT infrastructure going and all backups running smoothly.

Hyper-V Integration Services and How they Affect Hyper-V Backups

Each time the host server runs a Windows Update it’s critical to also update each VM’s Hyper-V Integration Services. This is of paramount importance because without a proper communication channel between host and guest operating system, you can’t have an application-consistent guest VM backup. In addition, integration services include drivers and other components that are important for regular VM operations at runtime.

Unfortunately these services do not update themselves. On Windows Server 2012 you can see the Hyper-V Integration Services version number in the Hyper-V Manager without opening and logging into each VM separately. By the way, we also have a PowerShell script on our website to help you automate checking the version numbers on Windows Server 2012 hosts. On VMware the so-called VMware Tools can be configured to update automatically when they are outdated; however, Microsoft is still behind on this issue and it takes some manual labor for admins to ensure the versions are all compatible.

Hard Disk Space Requirements

Free hard disk space is also very important for live backups to run well. Keep at least 15% space free or at least 10GB. To be precise, on the host you should have at least 10GB + 2x the write volume expected for the entire duration of a backup. For example, if you’re backing up a 1 TB VHDX and expect to see 5% change, that’s 51GB, so keep at least about 100GB free as a bare minimum to provide VSS with enough buffering space.

NTFS and Basic Disk Layouts versus Dynamic Disks and Non-NTFS Formats

At this time the Microsoft Hyper-V VSS Writer requires virtual machine disks to be formatted using basic disk layouts (GPT is fine on newer versions of HyperV) and NTFS. If you have non-NTFS or dynamic disks inside the VM, the VM will go into a Saved State when backup is initiated. This has been addressed in Windows Server 2012 R2 but is still an issue on previous versions of Windows Server.

VSS Configuration Requirements for Live Backups

Another Microsoft blunder is the VSS configuration itself. In fact, VSS is one the most complex modules in Windows and configuring it properly isn’t easy. One of the default values that should be changed right after installation is the VSS storage area maximum limit. You may change that value using VSSUIRUN.EXE or VSSADMIN from the command line. The command is:

vssadmin resize shadowstorage /for=C: /on=C: /MaxSize=unbounded

Using VSSUIRUN you can do all that using a nice GUI that is part of Windows Server.

Note that the above command on some Windows versions won’t allow unbounded values. If you run into this issue you need to specify a large number, such as 100GB.

Also please note that live backups using VSS don’t reserve space; it’s just an upper limit to buffer write operations during backup.

Where did Microsoft mess up? It’s the default setting of 300MB, which is far too low, especially on a Windows Server used for Hyper-V. The paging file inside the VM alone might consume 300MB in just seconds.

Also note that the above command allows VSS storage space to be allocated on another drive. For example if c: is full you could use drive d: to provide VSS storage area for drive C. That works on the host but the Hyper-V VSS Writer doesn’t like it when you configure a VM like that. Live backups of VMs fail as of Windows Server 2012 R2 if the VM contains VSS configurations where the storage area is assigned to a different drive.

Free, Contiguous RAM Requirement

A certain amount of free RAM is required to run VSS during the backup initiation phase, where most of the RAM I/O is taking place. It may not be immediately obvious that there is significant risk for RAM fragmentation and that it may break Hyper-V if RAM becomes too fragmented, even though the total amount of free RAM may be sufficient. For example, if a VM is pulled into Saved State during backup, for example because it’s an unsupported Linux variant where Hyper-V doesn’t allow live backups, the VM may not be able to come back online due to RAM fragmentation. Once the VM went into Saved State, Windows may have taken over that chunk of memory for VSS purposes. Now, when the VM is supposed to come back online the RAM is no longer available in one large piece and hence the VM cannot be started.

Dynamic memory allocation of virtual machine memory may prevent or actually make RAM fragmentation more likely to occur, depending on the actual server and VM configuration.

How much free and contiguous RAM the server needs is difficult to tell ahead of time. You’ll need to experiment and provide as much free RAM as you can afford. The absolute minimum is probably upwards of 3 GB.

Microsoft Limits PC Version of Hyper-V: There’s No Hyper-V VSS Writer in Windows 8 or Windows 10

Backups taken without the use of Hyper-V’s VSS Writer, which doesn’t ship with Windows 8, are still crash-consistent, but not application-consistent. See application consistent backups vs. crash consistent backups.

Because the Hyper-V Integration Service passes on the VSS signal into the VM, it requires the Hyper-V VSS Writer to begin the notification process; hence, this phase is entirely missing when backing up Windows 8 virtual machines in Hyper-V.

Why is that an issue? Some data will be lost, as backing up in a crash-consistent state rather than an application-consistent state is similar to a power failure; however, NTFS, Exchange, SQL Server and other modern database systems are all transaction based (MySQL is sometimes not) and can rollback their changes. While application consistency is the premium way to do live backups, crash-consistency is better than not having any backup at all. If data loss is to occur, it should be limited to the current transaction at hand, provided transaction based services are being used. A FAT-based file system, however, is quite likely to suffer from corruption issues when power fails, and that’s a good reason not to use FAT inside VMs.

Live Backup of Hyper-V VMs Using BackupChain and Hyper-V VSS Writer

BackupChain Hyper V backup integrates with Hyper-V VSS Writer and requests each VM separately to prepare for backup, when the default sequential VM backup method is being used.

The advantage of using sequential VM backups in Hyper-V is that it dramatically reduces the load on the host system and also reduces the risk for errors that may occur when too many services are asked to prepare for live backup simultaneously. There are, however, times when you must use simultaneous VM backups, at least for a group of VMs:

Simultaneous VM backups are also offered in BackupChain and it’s a feature for the purpose of taking multiple consistent virtual machine backups. For example, if you have an application server hosted in one VM and two database VMs hosted in their respective VMs, you may need to have a consistent backup across 3 VMs. For those scenarios you would select the N virtual machines belonging together and use the simultaneous backup options in BackupChain.

BackupChain Virtual Backup Features

As a backup software for Windows Server, BackupChain goes far above the common feature set of a typical Windows 10 backup tool and offers server cloud backup, Windows Server backup, version backup, and virtual machine backup.
BackupChain’s background service handles data backup, file backup over FTP, and deduplication using a multithreaded backup engine that also allows backup of a virtual machine live without interruption. BackupChain is also more than just a backup program; it also includes an FTP server and DriveMaker, which you can use to map an FTP drive to a drive letter, and there by set up your own online backup system.
BackupChain may also be used as a backup software for Microsoft Exchange, a backup for SQL Server, or a backup software for NAS servers. Its granular restore and granular backup features are unique in the industry and offer speedy recovery in various restore scenarios.
For virtual machine farms, BackupChain supports various hypervisors, so you can take a backup for VirtualBox, a backup for cluster shared volume VMs, or a backup for VMware Workstation VMs using the same software and interface.
Browse through our collection of free software for Windows and download BackupChain today! Chances are you will join the other 35,000 users worldwide who know that BackupChain is indeed the best backup software for their environment.