After upgrading NetBackup Master Server to version 8.0, you may encounter the following error when logging in to the Java Administration Console:
Could not connect to NetBackup Service Layer. You may not be able to perform the functions in the Administration Console that depends on connectivity to this service. Please ensure the nbsl service is up and running.
There can be various reasons why it happens. I have compiled a list of things to look at based on past experience. Hopefully one of them can resolve your issue.
Checklist:
1. NetBackup EMM database must be running. Verify by running below command:
Linux/UNIX: /usr/openv/db/bin/nbdb_ping
Windows: install_path\Veritas\NetBackup\bin\nbdb_ping
2. Master server’s Client_Name, EMMSERVER and the first entry in Server list must be identical with the host name mentioned in NetBackup’s CA certificate.
Linux/UNIX:
* Get the CA cert detail by running:
/usr/openv/netbackup/bin/nbcertcmd -listcacertdetails
* Compare the hostname with the above parameters stored in /usr/openv/netbackup/bp.conf
Windows:
* Get the CA cert detail by running:
install_path\NetBackup\bin\nbcertcmd -listcacertdetails
* Launch registry editor and go to HKEY_LOCAL_MACHINE \ SOFTWARE \ Veritas \ NetBackup \ CurrentVersion \ Config.
For example:
C:\Program Files\Veritas\NetBackup\bin>nbcertcmd -listcacertdetails Subject Name : /CN=nbatd/OU=root@nbumaster /O=vx Start Date : Jun 30 13:55:39 2015 GMT Expiry Date : Jun 25 15:10:39 2035 GMT SHA1 Fingerprint : C4:81:6C:B6:66:25:C1:DA:82:E3:06:F3:23:26:4F:51:85:3B:B8:71
In this case, make sure Client_Name, EMMSERVER and the first Server entry are all listed as nbumaster.
3. In databases.conf file, NBDB database must be on the first line.
Reference: https://www.veritas.com/support/en_US/article.000126459
Good example:
Linux/UNIX:
# cat /usr/openv/var/global/databases.conf "/usr/openv/db/data/NBDB.db" -n NBDB "/usr/openv/db/data/NBAZDB.db" -n NBAZDB
Windows:
type C:\Program Files\Veritas\NetBackupDB\conf\databases.conf "C:\Program Files\Veritas\NetBackupDB\data\NBDB.db" -n NBDB "C:\Program Files\Veritas\NetBackupDB\data\NBAZDB.db" -n NBAZDB
4. The following processes must be running for NetBackup Java Administration Console authentication:
- nbatd
- nbwmc
- nbsl
You can run this command to verify:
Linux/UNIX: /usr/openv/netbackup/bin/bpps
Windows: install_path\NetBackup\bin\bpps
TIPS: If nbsl is not running, try (re)starting it by following: https://www.veritas.com/support/en_US/article.100033680
If step 1-4 are good and nbatd and nbwmc are not running, I suggest to contact NetBackup Technical Support as it may require more in-depth troubleshooting.
5. If the 3 processes above are running and you are still getting the same error, check whether the Tomcat certificate has expired.
A) First, enable NBCURL logging.
Linux/UNIX: Add below line in /usr/openv/netbackup/bp.conf
ENABLE_NBCURL_VERBOSE = 1
Windows: Add a DWORD (32-bit) key in HKEY_LOCAL_MACHINE \ SOFTWARE \ Veritas \ NetBackup \ CurrentVersion \ Config, called: ENABLE_NBCURL_VERBOSE
Double click the key and put a value of: 1
B) Second, verify the Tomcat certificate.
Linux/UNIX: /usr/openv/netbackup/bin/nbcertcmd -ping
Windows: install_path\NetBackup\bin\nbcertcmd -ping
If the Tomcat certificate has expired, you will see an entry similar to this:
* Server certificate: * subject: CN=nbumaster; OU=TOMCAT@nbumaster; O=vx * start date: 2017-01-31 21:59:12 GMT * expire date: 2018-01-31 23:14:12 GMT * issuer: CN=broker; OU=root@nbumaster; O=vx * SSL certificate verify result: certificate has expired (10), continuing anyway.
If you try below command, it will fail as well:
C:\Program Files\Veritas\NetBackup\bin>nbcertcmd -getcertificate -force
nbcertcmd: The -getCertificate operation failed for server winref.
EXIT STATUS 8506: The certificate has expired.
C) In that case, we need to renew the Tomcat certificate. Below steps are the same as this technote, with additional precautionary measure.
Linux:
- Stop NetBackup services first and make a copy of the following directory:
/usr/openv/var - Start NetBackup services again.
- Run:
export WEBSVC_PASSWORD=web_service_user_password - Run:
/usr/openv/netbackup/bin/admincmd/nbcertconfig -t -user netbackup_web_service_user - And:
/usr/openv/wmc/bin/install/configureCerts
Windows:
-
- Stop NetBackup services first and make a copy of the following directory:
install_path\NetBackup\var\ - Start NetBackup services again.
- Run: set WEBSVC_PASSWORD=web_service_user_password
- And:
install_path\NetBackup\bin\admincmd\nbcertconfig -t -user netbackup_web_service_user - And:
install_path\NetBackup\wmc\bin\install\configureCerts.bat - Note: netbackup_web_service_user is usually nbwebsvc.
- Stop NetBackup services first and make a copy of the following directory:
D) Verify the Tomcat certificate again.
Linux/UNIX: /usr/openv/netbackup/bin/nbcertcmd -ping
Windows: install_path\NetBackup\bin\nbcertcmd -ping
The Tomcat certificate should be good now. For example:
* Server certificate: * subject: CN=nbumaster; OU=TOMCAT@nbumaster; O=vx * start date: 2018-03-06 22:52:27 GMT * expire date: 2019-03-07 00:07:27 GMT * issuer: CN=broker; OU=root@nbumaster; O=vx * SSL certificate verify ok.
E) At this point, you can retrieve/update the rest of the certificates as required by NetBackup.
Linux/UNIX, run:
/usr/openv/netbackup/bin/nbcertcmd -getCaCertificate /usr/openv/netbackup/bin/nbcertcmd -getCertificate -force /usr/openv/netbackup/bin/admincmd/bpnbaz -ProvisionCert
Windows, run:
install_path\NetBackup\bin\nbcertcmd -getCaCertificate install_path\NetBackup\bin\nbcertcmd -getCertificate -force install_path\NetBackup\bin\admincmd\bpnbaz -ProvisionCert
F) Try logging in to the NetBackup Java Administration Console again.
If issue persists, I strongly recommend to contact NetBackup Technical Support.
W 