SharePoint 2016 GRT restore fails with status code 5: the restore failed to recover the requested files. Granular restore log (ncfgre) throws error 0xE0008488:Access is denied.

I encountered this problem recently and with only access to the restore logs, it took me a while to figure out that the root cause is as simple as restore destination pointing to the wrong box!

Longer story:

I have the following evidence on hand:

NCFGRE log from SQL Server

22/06/2018 08:49:55.508 [Debug] NB 51216 ncfrai 158 PID:4960 TID:9240 File ID:352 [No context] 4 [[fsys\spsv3] ] <from Consumer::initializeResource> WNetAddConnection2 returned: 67. Either connection failed or the network path is not accessible (../BEDSContext.cpp:159)
22/06/2018 08:49:55.509 [Debug] NB 51216 ncfrai 158 PID:4960 TID:9240 File ID:352 [No context] 1 [ResourceChildBEDS::_attach()] FS_AttachToDLE() Failed! (0xE0008488:Access is denied.) (../ResourceChild.cpp:674)

Additional environment info

• SharePoint 2016 front-end server and SQL back-end server are both VMWare virtual machines.
• SharePoint farm is backed up using VMWare policy with Enable SharePoint Recovery option checked.
• Now we need to restore some granular items.

Assuming you have experience with NetBackup SharePoint agent before, do you notice anything suspicious?

Well, I didn’t initially.

NetBackup always generates NCFGRE log in the SQL Server for as long as I remember. Until… SharePoint 2016 came along.

To connect to SharePoint 2016, NetBackup doesn’t use custom API anymore, instead it uses a standardized method called SharePoint Server Object Model (SSOM). With that, the granular recovery process has been moved to the SharePoint front-end server. It is no longer on the back-end server.

If you notice, the NCFGRE log in my case was created in the SQL Server, so that should raise a suspicion. Yes, this happens if the restore destination is pointing to the SQL Server.

Just bear this in mind: As far as NetBackup is concerned, SharePoint restore destination must always point to the SharePoint front-end server. This will solve the problem I mentioned at the beginning.

 

 

 

 

 

SharePoint farm backup fails if using TLS 1.2 connection and NetBackup client is version 8.1 or 8.1.1.

Transport Secure Layer (TLS) facilitates secure computer communication over network. TLS 1.2 is the latest available version that is gaining momentum as it is much more secure than its predecessors (TLS 1.1/1.0 and SSL 1.0/2.0/3.0).

Microsoft SharePoint Server supports TLS 1.2 from version 2010 onwards. If you want to enable it, obviously, you also need to configure SQL connection to strictly use TLS 1.2.

From version 8.1 onwards, NetBackup introduces a new feature called Secure Communication. Basically it is now mandatory for all NetBackup hosts to communicate securely via TLS 1.2.

Bear in mind, however, that the above feature does not yet extend to NetBackup’s interaction with SharePoint farm. If you upgrade the NetBackup software on your SharePoint farm to 8.1 or 8.1.1, and the servers happen to use TLS 1.2, content database backup and restore will fail. SQL agent backup does work fine, however.

You will see the following error in SQL Server’s bpbkar log:

03:42:10.599 [1220.796] <2> ov_log::V_GlobalLog: INF - SQL Error Source:
Microsoft OLE DB Provider for SQL Server
03:42:10.599 [1220.796] <2> ov_log::V_GlobalLog: INF - SQL Error Description:
[DBNETLIB][ConnectionOpen (SECCreateCredentials()).]SSL Security error.

 

PS: This log is located in \Veritas\NetBackup\logs\bpbkar\. Create the folder if it does not exist and re-run the backup to get the log.

This issue will be fixed in NetBackup 8.1.2 which should be released soon. Meanwhile, you may want to roll back the NetBackup client software on the SharePoint Servers back to 7.7.3.