Monitoring Fails Due to SSL Handshake or Authentication Failures
KB1186 - Nov 03, 2016 - Last reviewed on Jun 27, 2018
Description: What to do if monitoring your systems with BMC TrueSight Capacity Optimization or BMC PATROL/TrueSight Operations Management fails due to SSL handshake.
Additional Keywords: Java, SSL Handshake, Close_notify
Because manufacturers and Java increased the security level of SSL connections, the monitoring of your systems may fail due to a SSL handshake failure or if your certificates do not comply with the java security constraints. The following errors will occur:
EXCEPTION CertificateException : Certificates does not conform to algorithm constraints
Error: Received fatal alert: close_notify
Connection has been shutdown: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
When using Java 1.6
Java 1.6 sends a SSLv2 Hello message during the SSL negotiation (“handshake”). Because this message is not supported by most secured services, the handshake will fail. To solve this issue, upgrade to the latest version of java available.
When using Java 1.8 or higher
Use the testssl.sh utility or any other SSL scanner to test the SSL encryption and verify the server preferences. The command to be run is
If you are testing the connection against:
- an SMI-S provider: Use port 5989
an HTTP API: Use port 443.
In the example below, we tested the SSL encryption of a NetApp Filer configured in 7-mode and with Data ONTAP API v7.3.3 installed:
As you can see in the screenshot above:
- the negotiated protocol is TLSv1
- the server key size is 512 bit
- the signature algorithm is MD5.
Open the java.security file to verify that the server preferences comply with the current java constraints. The java.security file is to be found in:
opt/bmc/BCO/jre/lib/security/(for BMC TrueSight Capacity Optimization)
%JAVA_home%/lib/security(for BMC PATROL/BMC TrueSight Operations Management while using java 1.8)
%JAVA_home%/conf/security(for BMC PATROL/BMC TrueSight Operations Management while using java 9 and higher)
Search for the following lines:
jdk.certpath.disabledAlgorithms=MD2, MD5, RSA keySize < 1024
jdk.tls.disabledAlgorithms=SSLv3, RC4, MD5withRSA, DH keySize < 768
jdk.jar.disabledAlgorithms=MD2, MD5, RSA keySize < 1024
Which means that we will face a handshake failure since our signature algorithm (MD5) is disabled and our server key size is lower than 1024.
If the server preferences do not match the java security constraints, update the java.security file accordingly and save your changes. In our example, we will remove MD5 from the jdk.certpath.disabledAlgorithms and jdk.jar.disabledAlgorithms lines and set the key size to 511.
In this case, we are adapting the java constraints to a specific API. You may have to modify them again if another certificate requires lower properties. You can disable those constraints by commenting them out using
If you are using:
- BMC TrueSight Capacity Optimization, restart the scheduler
- BMC PATROL/TrueSight Operations Management, restart the PATROL agent.
- Failing to Upgrade to Java v1.8.0_45
- Installation or Upgrade Fails Due to Locked Out Jar Files
- Monitoring Studio X Unexpectedly Hangs or Consumes High CPU When Running a JRE Higher Than 8
- Other Sentry KMs Stop Working After EMC KM 3.1.00 Install
- SSH-based monitoring fails due to incompatible Java versions
- java.sql.SQLException: I/O Error: Connection Reset When Monitoring Microsoft SQL Server 2008 R2