It’s 2018, and backing up Hyper-V hosts is solved, right? I certainly expected it to be when I took on the task of finding a simple and reliable way to back up three hosts colocated in a large datacenter. This article summarizes my journey and the findings resulting from it.
It became immediately obvious that Windows Server 2016 does not come with any useful backup functionality for Hyper-V. Looking for commercial products, I found the following three candidates looking promising:
- Altaro Backup 7.5
- Nakivo Backup & Replication 7.3
- Veeam Backup & Replication 9.5 Update 2
There are other products out there, of course, but looking at their websites none of them seemed enticing enough for me to include them in a trial.
My expectations for a Hyper-V backup product are moderate enough, I believe. I do not want to become a full-time backup administrator. I have precious little time as it is. The products we use for our IT need to “just work”. At least, that is the theory.
First of all, I need a backup product to be easy to install and use. The UI should ideally be self-documenting so it can be used without studying manuals, FAQs and KB articles.
Secondly, a backup software needs to do its job, reliably, every single day. Backups are fail-safes. It cannot have the fail-safes fail.
This is more of a nice-to-have than a hard requirement, but it would sure be nice to have a product that does its job swiftly while efficiently deduplicating the backup storage. Also, a light agent and a small installation footprint are always a plus.
I intended to store the backups on an SMB file share which the company that is hosting the servers is offering at low cost. I had used that type of SMB share for manual backup jobs for several years.
Some products offer to host the backups on a dedicated backup server. That was never an option for me. Our environment is relatively small, and standing up and managing another physical server just to host backups was out of the question. I was looking for a simple, elegant solution, not additional complexity.
Once you think about it, this seems to be the obvious choice, at least for Hyper-V hosts located in a datacenter with high-speed internet connectivity (like ours). Storing (encrypted) backups in Amazon S3 or Azure Storage solves multiple problems at once: the backups are located in a physically different site, vastly improving disaster recoverability while at the same time reducing the dependency on one hosting provider. With backups stored in the cloud moving VMs between datacenters becomes so much easier.
The reason why I did not choose this option is as sad as it is simple: none of the products support cloud storage as primary backup targets. This may (and probably will) change in future versions, but today not even Microsoft’s Azure Backup Server has the capability to send backups directly to Azure. As the docs state:
In the current architecture of Azure Backup Server, the Azure Backup vault holds the second copy of the data while the local storage holds the first (and mandatory) backup copy.
Following are the notes from my trial installations of the three products mentioned earlier, two of which I used for several weeks each before writing this.
This was the first product I tried, simply because Veeam is the largest and most well-known brand out there. During my initial installation attempt, I was a little shocked about the huge 2 GB download. When I was finally presented with the product’s console, I could not find any way to add an SMB share as a backup target. As it turned out in my second attempt several weeks later, you have to locate a tiny little chevron-style button placed near the bottom of the screen:
Once you manage to find and click that, a dialog pops up that lets you enable the “backup infrastructure” tab. Why that is not enabled by default is beyond me. Veeam could probably double their sales by that simple change.
As it stands I failed to locate the magic button during my first attempt, so I uninstalled the product, which is a tedious procedure because Veeam litters the Programs and Features window with a good half dozen entries.
Only after I had tried Altaro and Nakivo and failed did I look at Veeam again. During this second run, I located and skimmed a manual which, if printed out, would dwarf any city’s phone book. My newfound knowledge enabled me to find the magic button and configure the product optimally for our needs.
Veeam Backup & Replication is certainly powerful, but unnecessarily complex. Related settings are spread across different wizards, dialogs and other parts of the UI. However, once configured, it seems to be a reliable workhorse.
Cloud storage as the primary backup target is not supported as of version 9.5. I found a note on a blog that support for object-based storage is to come with version 10, so, hopefully, that will give us direct cloud backup.
For optimal backup consistency with a diverse set of Windows and Linux VMs, I recommend configuring Veeam so that application-consistency is attempted first, crash-consistency second.
Application-consistent backups require Veeam to trigger VSS in the VM, so admin rights in the guest are mandatory for this to work. Sometimes VSS gets confused and fails, or you do not have admin credentials for all VMs. That is when you need the crash-consistent variant as your second option, which basically backs up a VM as if it had suffered a sudden power loss. Unfortunately, this two-phased approach is not enabled by default. To configure it, you have to dig pretty deep:
Hyper-V crash-consistency needs to be specifically enabled in a different dialog. While you are there make sure to switch on changed block tracking (CBT):
With 273 MB is Altaro is a nice small download. Installation is similarly quick. The UI is so intuitive that a product manual is not required. Altaro employs the sensible backup strategy of trying application-consistent first, crash-consistent second, by default.
All in all, I was really happy with Altaro until it started failing every single time with the error The backup location contains too many bad data blocks. Error code DEDUP_064:
This happened with two different SMB shares, in the first case after two to three weeks, in the second case after only a few days. I suspect it is caused by an incompatibility with the Linux-based SMB shares our provider offers. Altaro’s support was not able to find a solution. It would be nice if they offered a tool or script to test a backup target for compatibility (which they do not). In any case, Veeam seems to be able to work with the SMB shares just fine.
With regards to cloud backup, Altaro can send secondary copies to Azure, but that capability is reserved for the most expensive Unlimited Plus edition.
With just 192 MB Nakivo is an even smaller download than Altaro. That, unfortunately, is the most positive thing I have to say about it.
The installer leaves critical tasks to the admin which do not even seem to be documented:
- WS-Man needs to be enabled over the network and firewall rules need to be added
- The installer creates a self-signed certificate that browsers reject
Once you are past the mandatory manual WS-Man configuration and try to access the product’s console in IE (IE being preinstalled on Windows Server) you are greeted with a recommendation not to use IE.
When I tried to add the SMB backup target for which I had a username, a password, and a UNC path, specifying the username turned out to be more of a challenge than I had anticipated:
- Just the username did not work
- “.\username” did not work either
- “user\user” finally did the trick
Again, that does not seem to be documented. Altaro and Veeam did not have any issues authenticating to the share.
Connecting to a second share did not work at all, not even with the “user\user” hack. I verified the credentials were OK by mapping a drive on the command line, which was no problem, of course.
As for the UI, I found Nakivo’s console to be slow and badly designed.
Previous versions of Nakivo required SMB1, which the current 7.3 finally got rid of. Now SMB2 is used instead.
Altaro is a nice product, easy to use, yet very capable. Give it a try if you plan to store your backups on a Windows machine. Veeam is the workhorse whose power comes with quite a bit of complexity. Nakivo cannot be recommended at this point.
Direct backup to the cloud is a feature I would like to see in the future. Today, cloud storage can only be used as a secondary backup target.