DPM 2016 Modern Backup Storage and Deduplication performance issue

Modern Backup Storage and Deduplication

One of the great features of Hyper-V in combination with a virtual DPM server was the ability to use deduplication. On the Hyper-V host you have a volume which is enabled for dedup where you place your .vhdx files on that serves as backup storage for the virtual DPM server. This way you can safe a lot of disk space.

Since DPM 2016 you can use Modern Backup Storage (MBS). With MBS you can use Storage Spaces to create a volume with shares that you present to DPM. Now I hear you thinking hey, we can enable Dedup on that volume and let DPM backup data to that volume and it get’s deduped so we can use a physical machine to… Unfortunately not, because MBS requires ReFS as file system on the disk, Dedup is ruled out because it is not yet supported and not available on ReFS volumes.

So we still need a physical Hyper-V host with an NTFS volume for the .vhdx files with dedup enabled. In the VM we create the storage space with a virtual disk and a ReFS volume and you are ready to cruise with your DPM server.

The setup

So the situation above also discribes the setup a bit. In short we have a physical host that has storage for the virtual DPM server. In this case it was a disk enclosure attached to the host with several SATA disks that we added to a storage space. On top of the storage space a mirrored volume was created to place the backup .vhdx files on. The host takes care of the deduplication of the backup data inside the .VHDX files. The vm takes care of the Storage Space and DPM.

The physical host has to take care of some dedub jobs like optimizing, garbage collection and scrubbing. These jobs cost a big amount of IO and a large amount of memory. In this case the physical host did not have a large amount of memory in spare. The DPM VM was take about 10GB of memory were the physical box only had 16GB. So there was 6GB left for the Hyper-V host itself and the Deduplication process.

The problem

The customer demanded a pretty heavy backup schedule were about 400 resources (databases, exchange and file shares) were synced every hour. Because of this there is no gab of low I/O or idle environment for dedub to do his magic and things started to get nasty. Dedub jobs started to get backlogs and were running during backup so backups and dedub were take all I/O and the underling hardware was not able to keep up. Then the memory on the host was at 99%. And the VM was using 10gb and the dedup process 5+ Gig so there is no room for the dedup process to increase in memory to increase in speed.

Because the VM is unaware of the underlying hardware the VM was receiving time outs on the disks (because there was no I/O left on the actual hardware) so it was thinking there are problems with the disks. DPM default behaviour is that it checks for disks error’s and if a disk receives to much error’s and appears unhealthy it takes the disk offline to prevent corruption on the backup disk.


So DPM 2016 with Deduplication and MBS gives you a big efficiency advantage on capacity but it’s very imported to make sure the deduplication jobs don’t run together with the DPM Jobs. If things start to overlap consider adding a couple of SSD’s to the storage space on the host, or disable deduplication jobs and manually schedule the jobs for Deduplication optimization. Because running DPM jobs and Deduplication jobs at the same time on hardware that is not able to keep up wil result in I/O Errors and disks going offline.

Leave a Comment