8295343: sun/security/pkcs11 tests fail on Linux RHEL 8.6 and newer

8231351: Add notes for PKCS11 tests in the test doc

Reviewed-by: lucy
Backport-of: 6ce0ebb858d3112f136e12d3ad595f805f6871a0

Author: Andrew Lu

doc/testing.html

@@ -5,7 +5,7 @@
   <meta name="generator" content="pandoc" />
   <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
   <title>Testing the JDK</title>
-  <style type="text/css">
+  <style>
       code{white-space: pre-wrap;}
       span.smallcaps{font-variant: small-caps;}
       span.underline{text-decoration: underline;}
@@ -21,7 +21,7 @@
 <header id="title-block-header">
 <h1 class="title">Testing the JDK</h1>
 </header>
-<nav id="TOC">
+<nav id="TOC" role="doc-toc">
 <ul>
 <li><a href="#using-the-run-test-framework">Using the run-test framework</a><ul>
 <li><a href="#configuration">Configuration</a></li>
@@ -40,6 +40,7 @@ <h1 class="title">Testing the JDK</h1>
 <li><a href="#notes-for-specific-tests">Notes for Specific Tests</a><ul>
 <li><a href="#docker-tests">Docker Tests</a></li>
 <li><a href="#client-ui-tests">Client UI Tests</a></li>
+<li><a href="#pkcs11-tests">PKCS11 Tests</a></li>
 </ul></li>
 </ul>
 </nav>
@@ -58,7 +59,7 @@ <h3 id="configuration">Configuration</h3>
 <p>To be able to run JTReg tests, <code>configure</code> needs to know where to find the JTReg test framework. If it is not picked up automatically by configure, use the <code>--with-jtreg=&lt;path to jtreg home&gt;</code> option to point to the JTReg framework. Note that this option should point to the JTReg home, i.e. the top directory, containing <code>lib/jtreg.jar</code> etc. (An alternative is to set the <code>JT_HOME</code> environment variable to point to the JTReg home before running <code>configure</code>.)</p>
 <h2 id="test-selection">Test selection</h2>
 <p>All functionality is available using the run-test make target. In this use case, the test or tests to be executed is controlled using the <code>TEST</code> variable. To speed up subsequent test runs with no source code changes, run-test-only can be used instead, which do not depend on the source and test image build.</p>
-<p>For some common top-level tests, direct make targets have been generated. This includes all JTReg test groups, the hotspot gtest, and custom tests (if present). This means that <code>make run-test-tier1</code> is equivalent to <code>make run-test TEST=&quot;tier1&quot;</code>, but the latter is more tab-completion friendly. For more complex test runs, the <code>run-test TEST=&quot;x&quot;</code> solution needs to be used.</p>
+<p>For some common top-level tests, direct make targets have been generated. This includes all JTReg test groups, the hotspot gtest, and custom tests (if present). This means that <code>make run-test-tier1</code> is equivalent to <code>make run-test TEST="tier1"</code>, but the latter is more tab-completion friendly. For more complex test runs, the <code>run-test TEST="x"</code> solution needs to be used.</p>
 <p>The test specifications given in <code>TEST</code> is parsed into fully qualified test descriptors, which clearly and unambigously show which tests will be run. As an example, <code>:tier1</code> will expand to <code>jtreg:$(TOPDIR)/test/hotspot/jtreg:tier1 jtreg:$(TOPDIR)/test/jdk:tier1 jtreg:$(TOPDIR)/test/langtools:tier1 jtreg:$(TOPDIR)/test/nashorn:tier1 jtreg:$(TOPDIR)/test/jaxp:tier1</code>. You can always submit a list of fully qualified test descriptors in the <code>TEST</code> variable if you want to shortcut the parser.</p>
 <h3 id="common-test-groups">Common Test Groups</h3>
 <p>Ideally, all tests are run for every change but this may not be practical due to the limited testing resources, the scope of the change, etc.</p>
@@ -101,8 +102,8 @@ <h2 id="test-results-and-summary">Test results and summary</h2>
 <p>Additional work data is stored in <code>build/$BUILD/test-support/$TEST_ID</code>. For some frameworks, this directory might contain information that is useful in determining the cause of a failed test.</p>
 <h2 id="test-suite-control">Test suite control</h2>
 <p>It is possible to control various aspects of the test suites using make control variables.</p>
-<p>These variables use a keyword=value approach to allow multiple values to be set. So, for instance, <code>JTREG=&quot;JOBS=1;TIMEOUT=8&quot;</code> will set the JTReg concurrency level to 1 and the timeout factor to 8. This is equivalent to setting <code>JTREG_JOBS=1 JTREG_TIMEOUT=8</code>, but using the keyword format means that the <code>JTREG</code> variable is parsed and verified for correctness, so <code>JTREG=&quot;TMIEOUT=8&quot;</code> would give an error, while <code>JTREG_TMIEOUT=8</code> would just pass unnoticed.</p>
-<p>To separate multiple keyword=value pairs, use <code>;</code> (semicolon). Since the shell normally eats <code>;</code>, the recommended usage is to write the assignment inside qoutes, e.g. <code>JTREG=&quot;...;...&quot;</code>. This will also make sure spaces are preserved, as in <code>JTREG=&quot;VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug&quot;</code>.</p>
+<p>These variables use a keyword=value approach to allow multiple values to be set. So, for instance, <code>JTREG="JOBS=1;TIMEOUT=8"</code> will set the JTReg concurrency level to 1 and the timeout factor to 8. This is equivalent to setting <code>JTREG_JOBS=1 JTREG_TIMEOUT=8</code>, but using the keyword format means that the <code>JTREG</code> variable is parsed and verified for correctness, so <code>JTREG="TMIEOUT=8"</code> would give an error, while <code>JTREG_TMIEOUT=8</code> would just pass unnoticed.</p>
+<p>To separate multiple keyword=value pairs, use <code>;</code> (semicolon). Since the shell normally eats <code>;</code>, the recommended usage is to write the assignment inside qoutes, e.g. <code>JTREG="...;..."</code>. This will also make sure spaces are preserved, as in <code>JTREG="VM_OPTIONS=-XshowSettings -Xlog:gc+ref=debug"</code>.</p>
 <p>(Other ways are possible, e.g. using backslash: <code>JTREG=JOBS=1\;TIMEOUT=8</code>. Also, as a special technique, the string <code>%20</code> will be replaced with space for certain options, e.g. <code>JTREG=VM_OPTIONS=-XshowSettings%20-Xlog:gc+ref=debug</code>. This can be useful if you have layers of scripts and have trouble getting proper quoting of command line arguments through.)</p>
 <p>As far as possible, the names of the keywords have been standardized between test suites.</p>
 <h3 id="general-keywords-test_opts">General keywords (TEST_OPTS)</h3>
@@ -152,7 +153,7 @@ <h4 id="run_problem_lists">RUN_PROBLEM_LISTS</h4>
 <p>Set to <code>true</code> or <code>false</code>. If <code>true</code>, JTReg will use <code>-match:</code> option, otherwise <code>-exclude:</code> will be used. Default is <code>false</code>.</p>
 <h4 id="options">OPTIONS</h4>
 <p>Additional options to the JTReg test framework.</p>
-<p>Use <code>JTREG=&quot;OPTIONS=--help all&quot;</code> to see all available JTReg options.</p>
+<p>Use <code>JTREG="OPTIONS=--help all"</code> to see all available JTReg options.</p>
 <h4 id="java_options-1">JAVA_OPTIONS</h4>
 <p>Additional Java options to JTReg (<code>-javaoption</code>).</p>
 <h4 id="vm_options-1">VM_OPTIONS</h4>
@@ -169,7 +170,7 @@ <h4 id="repeat">REPEAT</h4>
 <p>Default is 1. Set to -1 to repeat indefinitely. This can be especially useful combined with <code>OPTIONS=--gtest_break_on_failure</code> to reproduce an intermittent problem.</p>
 <h4 id="options-1">OPTIONS</h4>
 <p>Additional options to the Gtest test framework.</p>
-<p>Use <code>GTEST=&quot;OPTIONS=--help&quot;</code> to see all available Gtest options.</p>
+<p>Use <code>GTEST="OPTIONS=--help"</code> to see all available Gtest options.</p>
 <h4 id="aot_modules-2">AOT_MODULES</h4>
 <p>Generate AOT modules before testing for the specified module, or set of modules. If multiple modules are specified, they should be separated by space (or, to help avoid quoting issues, the special value <code>%20</code>).</p>
 <h2 id="notes-for-specific-tests">Notes for Specific Tests</h2>
@@ -178,6 +179,11 @@ <h3 id="docker-tests">Docker Tests</h3>
 <pre><code>$ make run-test TEST=&quot;jtreg:test/hotspot/jtreg/containers/docker&quot;</code></pre>
 <p>To run these tests correctly, additional parameters for the correct docker image are required on Ubuntu 18.04 by using <code>JAVA_OPTIONS</code>.</p>
 <pre><code>$ make run-test TEST=&quot;jtreg:test/hotspot/jtreg/containers/docker&quot; JTREG=&quot;JAVA_OPTIONS=-Djdk.test.docker.image.name=ubuntu -Djdk.test.docker.image.version=latest&quot;</code></pre>
+<h3 id="pkcs11-tests">PKCS11 Tests</h3>
+<p>It is highly recommended to use the latest NSS version when running PKCS11 tests. Improper NSS version may lead to unexpected failures which are hard to diagnose. For example, sun/security/pkcs11/Secmod/AddTrustedCert.java may fail on Ubuntu 18.04 with the default NSS version in the system. To run these tests correctly, the system property <<code>jdk.test.lib.artifacts.&lt;NAME&gt;</code> is required on Ubuntu 18.04 to specify the alternative NSS lib directories.The<code>&lt;NAME&gt;</code> component should be replaced with the name element of the appropriate <code>@Artifact</code> class. (See<code>test/jdk/sun/security/pkcs11/PKCS11Test.java</code>)</p>
+<p>For example:</p>
+<pre><code>$ make test TEST=&quot;jtreg:sun/security/pkcs11/Secmod/AddTrustedCert.java&quot; JTREG=&quot;JAVA_OPTIONS=-Djdk.test.lib.artifacts.nsslib-linux_aarch64=/path/to/NSS-libs&quot</code></pre>
+<p>For more notes about the PKCS11 tests, please refer to test/jdk/sun/security/pkcs11/README.</p>
 <h3 id="client-ui-tests">Client UI Tests</h3>
 <p>Some Client UI tests use key sequences which may be reserved by the operating system. Usually that causes the test failure. So it is highly recommended to disable system key shortcuts prior testing. The steps to access and disable system key shortcuts for various platforms are provided below.</p>
 <h4 id="macos">MacOS</h4>

doc/testing.md

@@ -370,6 +370,22 @@ required on Ubuntu 18.04 by using `JAVA_OPTIONS`.
 
     $ make run-test TEST="jtreg:test/hotspot/jtreg/containers/docker" JTREG="JAVA_OPTIONS=-Djdk.test.docker.image.name=ubuntu -Djdk.test.docker.image.version=latest"
 
+### PKCS11 Tests
+
+It is highly recommended to use the latest NSS version when running PKCS11 tests.
+Improper NSS version may lead to unexpected failures which are hard to diagnose.
+For example, sun/security/pkcs11/Secmod/AddTrustedCert.java may fail on Ubuntu
+18.04 with the default NSS version in the system.
+To run these tests correctly, the system property `jdk.test.lib.artifacts.<NAME>` is required on
+Ubuntu 18.04 to specify the alternative NSS lib directory. The `<NAME>`
+component should be replaced with the name element of the appropriate
+`@Artifact` class. (See `test/jdk/sun/security/pkcs11/PKCS11Test.java`)
+For example:
+
+    $ make test TEST="jtreg:sun/security/pkcs11/Secmod/AddTrustedCert.java" JTREG="JAVA_OPTIONS=-Djdk.test.lib.artifacts.nsslib-linux_aarch64=/path/to/NSS-libs"
+
+For more notes about the PKCS11 tests, please refer to test/jdk/sun/security/pkcs11/README.
+
 ### Client UI Tests
 
 Some Client UI tests use key sequences which may be reserved by the operating

test/jdk/ProblemList.txt

@@ -634,12 +634,7 @@ sun/security/provider/PolicyFile/GrantAllPermToExtWhenNoPolicy.java 8039280 gene
 sun/security/provider/PolicyParser/ExtDirsChange.java           8039280 generic-all
 sun/security/provider/PolicyParser/PrincipalExpansionError.java 8039280 generic-all
 
-sun/security/tools/keytool/NssTest.java                         8295343,8204203 linux-all,windows-all
-sun/security/pkcs11/Signature/TestRSAKeyLength.java             8295343 linux-all
-sun/security/pkcs11/rsa/TestSignatures.java                     8295343 linux-all
-sun/security/pkcs11/rsa/TestKeyPairGenerator.java               8295343 linux-all
-sun/security/pkcs11/rsa/TestKeyFactory.java                     8295343 linux-all
-sun/security/pkcs11/KeyStore/Basic.java                         8295343 linux-all
+sun/security/tools/keytool/NssTest.java                         8204203 linux-all,windows-all
 
 ############################################################################
 

test/jdk/sun/security/pkcs11/PKCS11Test.java

@@ -48,7 +48,6 @@
 import java.security.spec.ECParameterSpec;
 import java.util.ArrayList;
 import java.util.Arrays;
-import java.util.HashMap;
 import java.util.Iterator;
 import java.util.List;
 import java.util.Map;
@@ -384,30 +383,19 @@ private static Path getNSSLibPath() throws Exception {
 
     static Path getNSSLibPath(String library) throws Exception {
         String osid = getOsId();
-        String[] nssLibDirs = getNssLibPaths(osid);
-        if (nssLibDirs == null) {
-            System.out.println("Warning: unsupported OS: " + osid
+        String nssLibDir = fetchNssLib(osid);
+        if (nssLibDir == null) {
+            throw new SkippedException("Warning: unsupported OS: " + osid
                     + ", please initialize NSS library location, skipping test");
-            return null;
-        }
-        if (nssLibDirs.length == 0) {
-            System.out.println("Warning: NSS not supported on this platform, skipping test");
-            return null;
         }
 
-        Path nssLibPath = null;
-        for (String dir : nssLibDirs) {
-            Path libPath = Paths.get(dir).resolve(System.mapLibraryName(library));
-            if (Files.exists(libPath)) {
-                nssLibPath = libPath;
-                break;
-            }
-        }
-        if (nssLibPath == null) {
-            System.out.println("Warning: can't find NSS library on this machine, skipping test");
-            return null;
+        String libraryName = System.mapLibraryName(library);
+        Path libPath = Paths.get(nssLibDir).resolve(libraryName);
+        if (!Files.exists(libPath)) {
+            throw new SkippedException("NSS library \"" + libraryName + "\" was not found in " + nssLibDir);
         }
-        return nssLibPath;
+
+        return libPath;
     }
 
     private static String getOsId() {
@@ -731,77 +719,6 @@ private static ECParameterSpec getECParameterSpec(Provider p, String name)
         return parameters.getParameterSpec(ECParameterSpec.class);
     }
 
-    // Location of the NSS libraries on each supported platform
-    private static Map<String, String[]> getOsMap() {
-        if (osMap != null) {
-            return osMap;
-        }
-
-        osMap = new HashMap<>();
-        osMap.put("SunOS-sparc-32", new String[] { "/usr/lib/mps/" });
-        osMap.put("SunOS-sparcv9-64", new String[] { "/usr/lib/mps/64/" });
-        osMap.put("SunOS-x86-32", new String[] { "/usr/lib/mps/" });
-        osMap.put("SunOS-amd64-64", new String[] { "/usr/lib/mps/64/" });
-        osMap.put("Linux-i386-32", new String[]{
-                "/usr/lib/i386-linux-gnu/",
-                "/usr/lib32/",
-                "/usr/lib/"});
-        osMap.put("Linux-amd64-64", new String[]{
-                "/usr/lib/x86_64-linux-gnu/",
-                "/usr/lib/x86_64-linux-gnu/nss/",
-                "/usr/lib64/"});
-        osMap.put("Linux-ppc64-64", new String[]{"/usr/lib64/"});
-        osMap.put("Linux-ppc64le-64", new String[]{"/usr/lib64/"});
-        osMap.put("Linux-s390x-64", new String[]{"/usr/lib64/"});
-        osMap.put("Windows-x86-32", new String[]{});
-        osMap.put("Windows-amd64-64", new String[]{});
-        osMap.put("MacOSX-x86_64-64", new String[]{});
-        osMap.put("Linux-arm-32", new String[]{
-                "/usr/lib/arm-linux-gnueabi/nss/",
-                "/usr/lib/arm-linux-gnueabihf/nss/"});
-        // Exclude linux-aarch64 at the moment until the following bug is fixed:
-        // 8296631: NSS tests failing on OL9 linux-aarch64 hosts
-//        osMap.put("Linux-aarch64-64", new String[] {
-//                "/usr/lib/aarch64-linux-gnu/",
-//                "/usr/lib/aarch64-linux-gnu/nss/",
-//                "/usr/lib64/" });
-        return osMap;
-    }
-
-    private static String[] getNssLibPaths(String osId) {
-        String[] preferablePaths = getPreferableNssLibPaths(osId);
-        if (preferablePaths.length != 0) {
-            return preferablePaths;
-        } else {
-            return getOsMap().get(osId);
-        }
-    }
-
-    private static String[] getPreferableNssLibPaths(String osId) {
-        List<String> nssLibPaths = new ArrayList<>();
-
-        String customNssLibPaths = System.getProperty("test.nss.lib.paths");
-        if (customNssLibPaths == null) {
-            // If custom local NSS lib path is not provided,
-            // try to download NSS libs from artifactory
-            String path = fetchNssLib(osId);
-            if (path != null) {
-                nssLibPaths.add(path);
-            }
-        } else {
-            String[] paths = customNssLibPaths.split(",");
-            for (String path : paths) {
-                if (!path.endsWith(File.separator)) {
-                    nssLibPaths.add(path + File.separator);
-                } else {
-                    nssLibPaths.add(path);
-                }
-            }
-        }
-
-        return nssLibPaths.toArray(new String[0]);
-    }
-
     private final static char[] hexDigits = "0123456789abcdef".toCharArray();
 
     private static final String distro = distro();

test/jdk/sun/security/pkcs11/README

@@ -4,14 +4,15 @@ perform as a result of bugs or features in NSS or other pkcs11 libraries.
 - How to get NSS libraries?
 The libraries come from the following sources.
 
-1. Specified by system property test.nss.lib.paths
-System property test.nss.lib.paths can specify a set of absolute paths to
-the local NSS library directories. The paths are separated by comma.
+1. Specified by system property jdk.test.lib.artifacts.<NAME>
+The system property, jdk.test.lib.artifacts.<NAME>, can specify an absolute path
+to the local NSS library directory. The <NAME> component should be replaced with
+the name element of the appropriate @Artifact class.
+(See `test/jdk/sun/security/pkcs11/PKCS11Test.java`)
 
 2. Pre-built NSS libraries from artifactory server
-If the value of system property test.nss.lib.paths is not set, the tests will try
-to download pre-built NSS libraries from artifactory server. Currently, the
-tests only looks for libraries for Windows and MacOSX platforms on artifactory.
+If the value of system property jdk.test.lib.artifacts.<NAME> is not set, the
+tests will try to download pre-built NSS libraries from artifactory server.
 Please note that JIB jar MUST be present in classpath when downloading the
 libraries.
 

test/jdk/sun/security/tools/keytool/NssTest.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2018, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2018, 2023 Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * This code is free software; you can redistribute it and/or modify it
@@ -56,8 +56,9 @@ private static void copyFiles() throws IOException {
 
         Path dbPath = srcPath.getParent().getParent()
                 .resolve("pkcs11").resolve("nss").resolve("db");
-        Files.copy(dbPath.resolve("cert8.db"), Paths.get("cert8.db"));
-        Files.copy(dbPath.resolve("key3.db"), Paths.get("key3.db"));
-        Files.copy(dbPath.resolve("secmod.db"), Paths.get("secmod.db"));
+        Path destDir = Path.of( "tmpdb");
+        Files.createDirectory(destDir);
+        Files.copy(dbPath.resolve("cert9.db"), destDir.resolve("cert9.db"));
+        Files.copy(dbPath.resolve("key4.db"), destDir.resolve("key4.db"));
     }
 }

test/jdk/sun/security/tools/keytool/p11-nss.txt

@@ -6,7 +6,7 @@ slot = 2
 
 library = ${nss.lib}
 
-nssArgs = "configdir='.' certPrefix='' keyPrefix='' secmod='secmod.db'"
+nssArgs = "configdir='sql:./tmpdb' certPrefix='' keyPrefix='' secmod='secmod.db'"
 
 #forceLogin = true