Category: Uncategorized

Accelerator backup slow? Don’t ignore these warnings.

Accelerator is a time and storage-saving feature first introduced in NetBackup 7.5. Using dedicated track log, Accelerator can determine which files have changed when the subsequent backup runs, and only send the changes to the media server. This applies to full backup as well, so imagine having a full backup at the speed of incremental backup.

If there aren’t many changes to the files, you will be laughing all the way back to your server. The backup is really that quick.

But, yes, BUT you need to know how to use it to make the most out of it. I won’t discuss the initial configuration as it can be found here.

What I’m trying to say here is: Once you have completed the initial configuration and kicked off some jobs, please check the job detail to see if there is any Accelerator related warning mentioned below. If there is, fix it, else you won’t get the full performance benefit.

Scenario 1:
Dec 17, 2018 8:21:44 PM – Info bpbkar (pid=8528) not using change journal data for <D:\>: no forced rescan schedule configured
Dec 17, 2018 8:21:44 PM – Info bpbkar (pid=8528) not using change journal data for enumeration for <D:\> but will use it for change detection

Solution: Create a full backup schedule with Accelerator forced rescan option checked.

Scenario 2:
Dec 17, 2018 8:30:10 PM – Info bpbkar32 (pid=1216) not using change journal data for <D:\>: hard link or reparse point change detected
Dec 17, 2018 8:30:10 PM – Info bpbkar32 (pid=1216) not using change journal data for enumeration for <D:\> but will use it for change detection

Solution: The chance is the data being backed up resides in an NTFS deduplicated volume. Refer to https://www.veritas.com/support/en_US/article.000127405 for further info. You can contact Veritas NetBackup support for assistance.

Scenario 3:
Dec 17, 2018 8:38:15 PM – Info bpbkar32 (pid=844) not using change journal data for <D:\>: not configured for use
Dec 17, 2018 8:38:15 PM – Info bpbkar32 (pid=844) not using change journal data for enumeration for <D:\> but will use it for change detection

Solution: For Windows client, it is best to let Accelerator tap into NTFS change journal. It can be enabled by going to NetBackup Java Admin Console > Host Properties > Clients > double click the client > Properties > Windows Client > Client Settings > tick the Use Change Journal option. Click OK to confirm.

MSDP performance issue after upgrading to NetBackup 8.1 / Appliance 3.1

A lot of NetBackup customers would have upgraded to version 8.1 (3.1 if appliance) or later by now. One of the hottest topic in my team at the moment is how some customers encounter MSDP performance issue soon after upgrading to this version.

The stronger fingerprint algorithm in NetBackup 8.1’s MSDP is definitely good in the long run, but for those who recently upgraded from older NetBackup, the algorithm conversion from MD5 to SHA-2 can be a pain point – especially for the first few days post upgrade.

The issue may manifest in different ways: NetBackup Deduplication Engine (spoold) crashing, replication and/or duplication hanging and general slowness for MSDP-related operations.

If you think you are affected, download and install EEB 3935219 attached to technote: https://www.veritas.com/support/en_US/article.100041057

Note that if you already have EEB 3935219 installed, check the revision. The above technote contains the latest revision which is 11. Anything lower than 11 should be uninstalled and replaced by revision 11.

[9 May 2018] Update:

Some users may still notice performance issue even after data conversion has finished. If you are one of them, first, try lowering the amount of memory used by NetBackup deduplication engine (spoold) down from the default value of 75% to 50%.

Second, tune the OS kernel a little bit.

The steps are:

1. Open /msdp/data/dp1/pdvol/etc/puredisk/contentrouter.cfg file and search for MaxCacheSize. If the value is not 50%, set it to 50%. Save the file and restart NetBackup services. In theory you just need to restart spoold, but if you can afford to restart all NetBackup services, you will have a cleaner start.

2. Edit /etc/sysctl.conf file and ensure the 3 parameters below are set to the values given here (if these parameters don’t exist, just create them).

vm.overcommit_ratio = 100
kernel.numa_balancing = 0

Save the file and run sysctl -p to make the change effective.

NetBackup Appliance: update_clients script does not work even though client package has been installed correctly

NetBackup has a nifty script called update_clients to upgrade multiple clients at once. I wrote this technote a couple of years back to explain what it is and how it works.

Before you can use this script, the appropriate NetBackup client packages must be installed on the NetBackup Server (otherwise there is nothing to push).

For example: if you have a NetBackup Appliance server version 3.1, you can download the client packages from here then follow this procedure to copy and install the packages.

Now back to the issue mentioned earlier. To encounter that, you need a rather specific scenario:

  • Your NetBackup Server must be an Appliance,
  • The Appliance must already have NetBackup client packages installed,
  • The Appliance is then upgraded without removing the old client packages.

When you install a new client package in that scenario, the package will be symlinked incorrectly.

nb-appliance:/usr/openv/netbackup/client/Solaris # ls -l
total 12
lrwxrwxrwx 1 root root 101 Dec 7 13:43 Solaris -> /inst/client/.packages/NetBackup_8.1_CLIENTS/NBClients/anb/Clients/usr/openv/netbackup/client/Solaris
drwxr-xr-x 2 root bin 4096 Dec 7 13:02 Solaris10
drwxr-xr-x 2 root bin 4096 Dec 7 13:02 Solaris_x86_10_64

The correct one should be:

nb-appliance:/usr/openv/netbackup/client # ls -l
total 40
drwxr-xr-x 3 root bin 4096 Dec 7 17:11 HP-UX-IA64
drwxr-xr-x 5 root bin 4096 Jul 5 16:38 Linux
drwxr-xr-x 4 root bin 4096 Dec 7 17:11 Linux-s390x
drwxr-xr-x 3 root bin 4096 Dec 7 17:11 NDMP
drwxr-xr-x 3 root bin 4096 Dec 7 17:11 Novell
drwxr-xr-x 3 root bin 4096 Dec 7 17:11 OpenVMS
drwxr-xr-x 3 root bin 4096 Dec 7 17:11 RS6000
lrwxrwxrwx 1 root root 101 Dec 7 18:17 Solaris -> /inst/client/.packages/NetBackup_8.1_CLIENTS/NBClients/anb/Clients/usr/openv/netbackup/client/Solaris
drwxr-xr-x 14 root bin 4096 Dec 7 17:11 Windows-x64
drwxr-xr-x 8 root bin 4096 Dec 7 17:11 Windows-x86

To get the update_clients script to work, you will need to fix the symlink manually.

In my example, I would move out the existing /usr/openv/netbackup/client/Solaris:

mv /usr/openv/netbackup/client/Solaris /usr/openv/netbackup/client/Solaris_old

and create a fresh symlink:

ln -s /inst/client/.packages/NetBackup_8.1_CLIENTS/NBClients/anb/Clients/usr/openv/netbackup/client/Solaris /usr/openv/netbackup/client/Solaris