How to backup Virtual Domain Controllers is a subject and a question that many users have when they Virtualize their Domain Controllers. Is not an easy answer and some rules and requirements need to be set before we backup or restore Domain Controllers (or even Active Directory objects).

With the new Windows Server feature, VM-Generation ID now Virtualize Domain Controllers can reduce most of the issues we found in previous Windows versions(before Windows Server 2012).

The VM-Generation ID is a counter that is kept in the DC VM and managed by system operation and the Hypervisor. When there is a change(like a snapshots, clone, replication or copy virtual disks) in the DC VM, the counter is incremented. With this, the DC VM keep track of any changes that were supposed not to happen and fixed.

If the  VM-Generation ID counter inside the virtual machine as changed and doesn’t match the one in the Hypervisor then the Active Directory will not trust is RID and will not accept any replication or changes from this DC. The DC with the VM-Generation ID counter changed, will then fetch a database from a working domain controller.

This a very cool feature to protect Virtualize Domain Controllers inside of your Active Directory.

This board shows what type of tasks can change the VM-Generation ID.

Scenario VM-Generation – ID Change

• VMware vMotion®/VMware vSphere Storage vMotion / Hyper-V Live Migration – No
• Virtual machine pause/resume – No
• Virtual machine reboot – No
• vSphere host reboot – No
• Delete VM Snapshot – No
• Import virtual machine – Yes
• Cold clone – Yes
• Hot clone – Yes
  Note: Either Microsoft or VMware do not support virtual domain controllers hot cloning. Do not attempt hot cloning under any circumstances.
• New virtual machine from VMware Virtual Disk Development Kit (VMDK) copy – Yes
• Cold snapshot revert (while powered off or while running and not taking a memory snapshot) – Yes
• Hot snapshot revert (while powered on with a memory snapshot) – Yes
• Restore from virtual machine level backup – Yes
• Virtual machine replication (using both host-based and array-level replication) – Yes

You can check your DC VM-Generation ID by running a Powershell command:

Run a Powershell console as “Run as Administrator”

This is the output:

So the VM-Generation ID for this Domain Controller is: 133 33 220 110 3 50 9 240

Best Practices to Virtualize DCs

First I will go trough some rules to Virtualize your Domain Controllers. Even is not the purpose of this article(plan to write a full article about how to properly Virtualize your Domain Controllers) I will just add some of the basic rules.

• Timekeeping sync

This is one of the most important requirements when Virtualize a DC. We can use our Hypervisor as a time source (VMware or Hyper-V), but this is not entirely recommended, particularly if you have any physical AD, so always use an external NTP source (with multiple sources) to Time for your Domain Controllers.

As best practices, never synchronize your DCs with the Hypervisor.

Note: Use the some NTP source for your Hypervisors. Active Directory forest should use the same time sources as the ESXi hosts. Do not synchronize ESXi hosts with a virtualized domain controller. Which is especially useful when Hypervisor is configured for Active Directory integration. Using the same time sources also maintains accurate virtual domain controller time during boot cycles.

• Use Hypervisor High Available for AD VMs
  Use HA (for VMware) and Failover Cluster(for Hyper-V) for Hypervisor where your AD VMs are running in case a Hypervisor failure. Also, create an affinity rule so that your AD VMs run in different Hypervisor.
• Distribute the FSMO Roles
  First DC that you will deploy(if you are creating a new Domain) will always own all the roles. Always follow Microsoft operations master roles placement best practices
  
• Do not perform AD P2V (Physical-to-Virtual )
  I have seen this a lot. Customers and administrators to save time, just do a p2v of their physical domain controllers and hope everything will be working after. Wrong! In the long run we will get issues regarding sync and AD replication. Is easier and faster just to create fresh VM and then promote to AD and then replicate with the physical AD. If needed, move the FMSO roles to the new AD and remove the physical one from the Domain.

You can read here more about best practices how to Virtualize Domain Controllers in Hypervisor:

• For VMware
  
• For Hyper-V

How to Backup Domain Controllers

For this section, we will use NAKIVO Backup & Replication v7.1 and use Integration with Active Directory and the Hypervisor is vSphere ESXi v6.0.5572656.

Note: To use the VM-Generation-ID in VMware you need to run your Virtual Domain Controller in version vCenter v5.0(update 2), and ESXi v5.0(update 2) or newer and Virtual Hardware (vHW) need to be v8 or later. Check VMware KB HERE.

First, we will create a backup job for the DCs.

Just create a normal Backup Job.

Just choose your Domain Controllers to backup.

Select the Backup Repository.

Next options are the options that are very important when we Backup our Virtual Domain Controllers.

• Option 1
  Enable the App-aware (is enabled by Default in Nakivo) feature in the backup job.
  Note: For this option work properly, latest VMware Tools should be installed in the VM Guest OS.
• Option 2
  Enable Screenshot verification. Even this option is not mandatory; you should verify if your Backup is 100% reliable. After a VM backup is completed, NAKIVO Backup & Replication recover the VM with Flash VM Boot, disables networking on the VM to prevent network connections, makes a screenshot of the booted OS, discards the recovered VM, and sends a report with the screenshot via email.
• Option 3
  Set the transporters to manual for all VMs.
• Option 4
  Disable Automatically select the replacement for unavailable transporter. We need to disable this so that we only use the transporter that we set for only run one concurrent task at a time.
• Option 5
  Select the transporter that you set only to run one concurrent at time (check how next).

For the Screenshot verification we need to select the ESXi and the Datastore that Nakivo will use for the power on and test of this VM.

Very Important regarding Option 3, 4 and 5:

As a best Practices, never Backup your 2 (or more) Virtual Domain Controllers at the same time. This will save the integrity and sync of the DC USNs Active Directory database instance.

So in the Backup Transporter that you are using in your Nakivo DCs backup job, make sure you set to run only 1 concurrent task. Or we can create one Backup Job for each Virtual DC in your environment and run one at a time.

To finish the backup job Click Save, or Save & Run to run the Backup immediately.

After we run the Backup, everything finish without any issue. So now we have a Backup of our Virtual Domain Controllers.

After we finish our DCs backup I did some nasty stuff at in the DC-02 at Guest OS level, so that was broken and cannot restart anymore.

When trying to boot the DSC-02 is broken.

But before we restore the DC-02 last do some tasks in the DC-01 so that we make sure that the DC-02 have the last AD updates (while was not reachable).

First I delete one user (To be Deleted 01) and create a second user (To DC02 Break). When DC-02 is restored and sync in the AD it should not have the “To be Deleted 01” user and should have one extra “To DC02 Break”.

Delete user”

Create User:

How to Restore Domain Controllers

Before we restore the broken Domain Controller, some Domain Controllers restore Best Practices.

• If you are restoring a broken DC in a multi DCs environment, then you can do a normal restore and follow then next restore process.
• If you are restoring all DCs in your Domain, then you should first restore the one with all RMSO roles (this information you should have before you lost your DCs).
• If you are restoring the DC that had the FMSO roles and you want to keep that way(meaning all the existing DCs will sync from your restored DC), then you need to do follow Microsoft How to recover authoritative restore .

Now let us create a Restore Job to restore the Domain Controller that is broken.

Just click Recover and select VMs from Backup.

Select the VM that you need to restore (for this test is the DC 02) and choose the restore point (we have 4, but we will select the latest)

Select the ESXi host and the Datastore were to restore the DC.

Next we don’t need to change must in the default options. For this case, I will not Power ON after the restore and will leave the VM name as is. But this is not mandatory for this type of restore.

Next Finish the Restore Job, or Finish & Run to start the Job after we finish.

As we can see, the restore finish (in my home lab the restore did take 25 minutes  to finish) with success without any issues.

Now let us power on our restored Virtual Domain Controller and hope that synchronize with the DC-01.

When the restored Domain Controller is power on, we will see some error in the events. Particularly in the Directory Service.

Since now the VM-Generation ID as change the DC-01 will block the DC-02 (the restored DC) from sending any ID pool and changes to invocation ID for the Active Directory (AD) database (such as the database identifier) until the VM-Generation ID is fix.  With this will ensure only a proper replication between DCs can happen.

These are some of the event issues that we will see:

Information about the VM-Generation ID from this DC.

Then some errors not matching VM-Generation ID.

Then Active Directory will block all new updates from this DC-02 and starts a new connection to replace the AD DB from the DC-01.

Then Active Directory fix the issue we get the information that everything is ok and back to normal state and replication between DCs is working again.

To double check if everything is good and check if restored DC-02 as all updated from the DC-01 (even the ones while it was offline), lets check if the new User Created and the one we delete previously is now in the DC-02.

As we can see in the next image from DC-02, user “To be Deleted 01” does not exists and the user created “To DC02 Break” exists in DC-02.

Just as a final test, I did create one new user in DC-01 and another one in DC-02. Wait a few seconds and then both DCs had the new users create in different DCs.

With this final checking and tests we finish our Virtual Domain Controller and this article.

When starting this article I was planning to talk about restoring AD Objects (like users, computers etc.), but since this article is already to long, I will create an article just with that subject (how to restore AD objects from your VMs backups.

Note: Share this article, if you think it is worth sharing.

©2017 ProVirtualzone. All Rights Reserved