This topic was on my list for a long time but had been a little bit busy and also in AWS training in the last two weeks, plus a lot of issues in work that needs a lot of troubleshooting hours. Now is time to share some of my tests in this How to backup 4M files with Veeam Backup NAS blog post that I did in my home lab.

The plan by testing Veeam Backup NAS is to check how the Veeam works while we are back up a large number of files and folders. This is 4,25 million files, divided trough 10 folders (not many folders here). All data in the 4.25M files is around 7Tb.

My plan is to divide this blog into three parts:

1. How to backup 4M files with Veeam Backup NAS – Backup NAS directly from file share (this blog post).
2. How to backup 4M files with Veeam Backup NAS – Backup NAS from a VSS Snapshot (only available for SMB or Windows File Servers).
3. How to backup 4M files with Veeam Backup NAS – Backup from a Storage Snapshot.

In this blog post, we will backup directly from a file share.

Note: This first blog post about Veeam Backup NAS will be a bit longer than the other two since we also explain how to adding File Share and File Proxies.

Before we go through the backup process, let us go through a quick review of how the Veeam NAS Backup works.

We can backup from the following sources:

• SMB
• NFS
• Linus File Server
• Windows File Server

Veeam NAS Backup components.

Since this is just a simple explanation of how the NAS Backup process works, I will just show two components of the process that start trough file proxies and also the Changed File Tracking that is stored in the Cache Repository.

These are the components that are part of the Veeam NAS Backup.

Image Veeam Source

Veeam NAS Backup is using File Proxies for the backup process. The default (Veeam Server) and other that admins can create to balance the load and add more concurrent tasks to the backup, but at the same time, Cache Repository that stores and maintains all changes in this source file share using Changed File Tracking.

NAS Backup File Proxies.

As we can check in the above image, File Proxies are the main component in the NAS Backup since it is used to communicate and transfer data between the source(Shared folder) and the destination (Backup Repository).

You can use the File Backup Proxy that is installed by default in v10, or you can add another File Backup Proxy as a secondary proxy to use in a particular NAS Backup job.

Changed File Tracking

Image Veeam Source

We can think the Changed File Tracking (CFT) like the VMware Changed Block Tracking (CBT) is a process that tracks all the file backup and files or object changes that are store in the Cache Repository that had to happen between backups. Using this process, the Veeam NAS Backup backup process runs faster (for example, recognizing which are the files that needs to be back up).

As we can see in the above image, there are different changes in the files or folder that had to happen and are restored in the cache repository and is tracked by the CFT.

Finishing the quick Veeam NAS Backup, let us star the NAS backup process and what are the steps.

Environment:

• Veeam Backup Server: Windows 2019 Standard with Veeam Backup & Replication v10 build 4384.
• Second Veeam File Proxy: Windows Core 2019.
• Destination Backup Repository: 2x Windows ReFS Disk 600 Gb each ( provided by StarWind vSAN 8.0 build 13481).
• Source NFS Share folder: Synology 7Tb NSF Share with 4,25M files (only 6 folders).
• File Backup NAS Cache Repository:  Synology 750Gb NFS Share.

NFS Share source files and folder:

In this test environment was created exactly 4.25 Milton files. Files have different sizes (from 1Kb, 1Mb, 1oMb, 1Gb, and 10Gb) all create for this testing backup.

To create the files I used two small scripts (each one create 250 thousandths of files with 1Gb each):

Windows:

Note: Running this script in the NSF Share mounted in a Windows takes more time.

Linux:

Note: Running the script directly on the Storage system console was able to create thousandths of files just in 20/30m (depending on the file size).

To Update files for a second Backup changes.

This changes is just to trigger Changed File Tracking (CFT) and have a second version of the file.

Windows

For Windows this is just a very simple dos batch file to do the changes in the file.

Linux

In Linux I use just a simple command line to update files with a text “Veeam Update”.

Start NAS Backup.

To start the NAS Backup, there is some simple requirements that need to be in place.

The NFS/SMB Share needs to be added to Inventory – Files Shares. If there is no shared folder added when you try to select Backup File Share you get the following error:

How to Add File Share (NFS).

Start by adding a File Share in the Inventory tab.

We have tree type file shares that we can backup File Server (Windows or Linux Server), NFS Share, and SMB share.

In this case, we will use an NFS Share.

Add the path for your NFS and click Advanced.

Here is where you set what type of backup will NAS Backup do. For NFS, only direct backup from a file share and backup from Storage Snapshots.

Note: For Windows File Server and SMB Share, there also a third option to backup using VSS Snapshot.

Select the option of how backup will be processed.

1. What proxies will use.
2. What Cache Repository (default or one repository added).
3. What type of impact we can afford in the Storage using the NFS Share.

1 – Proxies

Select the proxies that you want to use in for this NFS Share (proxies are select in the NFS Share not in the backup job).

Note: Again, it is important to remember that only File Proxy can be used in a File Share Backup.

2 – Cache Repository.

We can use the default Cache Repository that Veeam created in the Install (by default will be C:\Backup\File backup cache), also double check if you have enough space to hold the Backup File Share Cache.

You should use a different Cache Repository (another NFS share or a disk) that could handle fast read/write to speed up your backups.

Note: Only a non-used backup repository can be used for Cache Repository. If the repository is already used in any backup job, it is not available.

For our test, I created a second NFS share with 750Gb and added to the Backup Repository to be used for NAS Backups.

Even we do not need a Cache this big, always take into account the size of your Backup, the number of files, and the retention points. Every time you back up and files changes, all those changes are saved in the Cache Repository.

3 – Backup I/O control.

Here you can control and define the Backup I/O and how fast File Share Proxies can read the NSF file Share.

Always take into account your  NSF Share source system resources. Setting faster I/O can quickly stress your Storage System and use most of its resources for the Backup Process. So always check your NAS System resources. You can do this by doing some backup tests and check if you can increase or need to reduce these settings. By default, the middle is defined.

In the next step, we apply all the configuration and settings that we had select above and clicked Finish.

Now we have our NSF Share added to Veeam NAS BAckup, and we can start and create a File Share backup job.

How to create a File Share Backup Job.

Creating the File Backup job is as simple as the standard VM Backup Job. On the backup option, select the File Share option.

Give it a name and click Next.

Next, click Add, and in Server, you can select your NSF Share(if you have more than one), then you can see all NSF Files and Folder. You can add Folders, Files, or just add the root and all NFS Share will be back up.

For this test, we will add the full NFS Share.

After you add the files\folders or NFS Share, you can click in Excluding section, and you can include (by default is *.* all files included) or exclude files, type of files(like mpeg, avi, mp3, etc.).

By default snapshot files are excluded, after checking you include\exclude files(if you need to), click Next.

If no need to include, exclude files, you can just skip and click Next.

Next, we need to configure the retention points, and also archiving is necessary.

1 – Select the Backup Repository that will store this backup.

2 – Enter the number of retention points that is necessary for this backup. Retention points can be days or months. With this option, backup will save file versions (file changes) for 7 days. But it could also be for 7 months. During that 7 days/7 months, all backups changes will be stored and will not be deleted.

3 – This option is optional and is for archiving. If you need to archive your files for a more extended period, you can enable this option to select the amount of time that files versions should be archived.

4 – When enabling archiving, you need to add a new Backup Repository for the archiving. Veeam NAS Backup will archive files in this repository.

5 – In this option, you have the option to select the type of files that you want to archive. For example, you don’t want to archive all NFS Share files, but only for example doc files, or another type of document files (this is just an example you can include all types of files).

The next step is the secondary Target. If you plan to have a second copy of your backup is here that you configure. In this step, we will not use it, so we will skip this step.

Next is the usual scheduling of your File Backup and click Apply and/or Finish to create the File Backup Job.

Now that we have our NFS Share added to Inventory, and the File Backup job created, we can start testing the 4,25M file NAS Backup.

Did a full backup of 4M files, then I was adding more files for the next run and save in Cache Repository the file changes.

Also, I run a script to had some text inside each file (just randomly did in 100k or 200k files) so that changes are done, and Cache Repository gets the new file version.

NAS Backup Results.

In the first run, there were only 4M files in the NFS Share, in the second run I did changes in 2M files.

In the first and second run, these were the resources that this full backup used.

Veeam Backup Server used resources.

As we can above, the backup did not use much of the resources of the Veeam Backup Server even if it was backing up 4M files. The average is around 50% for CPU and Memory.

Second File Backup Server used resources.

As we can see in the image above, File Proxy usage resources were also low (maximum 20% for CPU and Memory).

Synology NFS Share used resources.

Finally, we can check how the Synology did manage the backups regarding resource usage.

First Backup I/O I used the default (middle I/O) and the last backup (adding the 250k new files) was used the Fast Backup I/O to test how the Storage System would handle, and initially, we see CPU reaching 83% for some minutes, but then drop-down to average values. But I notice only CPU changes, memory used was all the time flat line.

NAS Backup License.

We will finish this first How to backup 4M files with Veeam Backup NAS blog post with the licenses that we need to use for NAS Backup.

For each 250Gb File, Backup needs 1 license. So, in this case, for almost 7Gb, Veeam Backup Server used 27 instances for the full 7Gb NAS Backup.

With the last information about the license, we finish this blog post about How to backup 4M files with How to backup 4M files with Veeam Backup NAS – Backup NAS directly from the file share.

The second blog post about Veeam NAS Backup will be How to backup 4M files with Veeam Backup NAS – Backup NAS from a VSS Snapshot. The second part will try to test all backups (full and incremental) to use Fast Backup I/O to see how Storage, Veeam Server, and File Proxy resource behave.

• More Information about Veeam Backup & Replication v10 HERE.
• Information about Veeam NAS Backup HERE.

Final thoughts:

In this How to backup 4M files with Veeam Backup NAS blog post, we saw that using Veeam Backup & Replication v10  to backup your File Shares or File Servers is very straightforward, and we can backup a big amount of files without any stress in our environments.

Creating the proper environment, like adding a Cache Repository, more File Proxies and using the proper Backup I/O for your File Share, you can achieve good results and fast backups.

All environments are not the same, so change your environment according to your infrastructure and data to backup. Add more File Proxies if you see that File Server backup is taking too long and Veeam Server resources are not being able to manage all that data.

Note: This environment is a test environment and all files are just dummy files, no real data is being backup, so in real production with different types of files, results could be very different. Please use this blog post as a reference only, not as a standard of how your environment will behave when you try to backup millions of different types of files.

I hope this information was useful, and help you to backup and config your File Shares with Veeam Backup & Replication v10.

Note: Share this article if you think it is worth sharing.

©2020 ProVirtualzone. All Rights Reserved