[WFCORE-5712] Update elytron-tool scripts to make use of jboss-modules #4879

fjuma · 2021-11-26T16:53:57Z

https://issues.redhat.com/browse/WFCORE-5712
https://issues.redhat.com/browse/WFCORE-5720
https://issues.redhat.com/browse/WFCORE-5730

Depends on #4878 (I've included those commits in this PR to also un-ignore ElytronToolScriptTestCase)

wildfly-ci · 2021-11-26T16:58:03Z

Core - Full Integration Build 11108 outcome was UNKNOWN using a merge of 27b0e97
Summary: Canceled (Error while applying patch; cannot find commit 1790120 in the https://github.com/wildfly/wildfly-core.git repository, possible reason: refs/pull/4879/merge branch was updated and the commit selected for the ... Build time: 00:02:38

fjuma · 2021-11-26T17:00:01Z

@jamezp When you are back next week, would it be possible to get a review on the updates to the elytron-tool.[sh|bat|ps1] scripts? Thanks.

github-actions · 2021-11-26T17:00:35Z

Dependency Tree Analyzer Output:

New Dependencies:

commons-cli:commons-cli:jar:1.4:compile
org.apache.commons:commons-lang3:jar:3.11:compile
org.wildfly.core:wildfly-elytron-tool-wrapper:jar:18.0.0.Beta6-SNAPSHOT:compile

CC @wildfly/prod

wildfly-ci · 2021-11-26T19:30:41Z

Core - Full Integration Build 11109 outcome was FAILURE using a merge of 8103730
Summary: Tests failed: 1 (1 new), passed: 4078, ignored: 44 Build time: 02:32:15

Failed tests

org.jboss.as.test.layers.LayersTestCase.testServlet: java.lang.Exception: Some unreferenced modules are un-expected [org.apache.commons.cli, org.apache.commons.lang3, org.wildfly.security.elytron-tool]
	at org.jboss.as.test.layers.LayersTest.test(LayersTest.java:109)
	at org.jboss.as.test.layers.LayersTestCase.testServlet(LayersTestCase.java:146)

fjuma · 2021-11-26T20:38:24Z

I can't reproduce the ElytronToolScriptTestCase failures locally on Linux. Need to look closer at what's causing this to fail on CI.

wildfly-ci · 2021-11-26T21:04:04Z

Core - Full Integration Build 11113 outcome was FAILURE using a merge of 137925a
Summary: Tests failed: 1, passed: 4079, ignored: 43 Build time: 02:32:31

Failed tests

org.jboss.as.test.layers.LayersTestCase.testServlet: java.lang.Exception: Some unreferenced modules are un-expected [org.apache.commons.cli, org.apache.commons.lang3, org.wildfly.security.elytron-tool]
	at org.jboss.as.test.layers.LayersTest.test(LayersTest.java:109)
	at org.jboss.as.test.layers.LayersTestCase.testServlet(LayersTestCase.java:146)

wildfly-ci · 2021-11-26T21:32:44Z

Core - Full Integration Build 11114 outcome was FAILURE using a merge of 137925a
Summary: Tests failed: 1, passed: 4077, ignored: 45 Build time: 02:13:07

Failed tests

org.jboss.as.test.layers.LayersTestCase.testServlet: java.lang.Exception: Some unreferenced modules are un-expected [org.apache.commons.cli, org.apache.commons.lang3, org.wildfly.security.elytron-tool]
	at org.jboss.as.test.layers.LayersTest.test(LayersTest.java:109)
	at org.jboss.as.test.layers.LayersTestCase.testServlet(LayersTestCase.java:146)

wildfly-ci · 2021-11-26T22:25:14Z

Core - Full Integration Build 11115 outcome was FAILURE using a merge of 137925a
Summary: Tests failed: 1, passed: 4079, ignored: 43 Build time: 02:11:47

Failed tests

org.jboss.as.test.layers.LayersTestCase.testServlet: java.lang.Exception: Some unreferenced modules are un-expected [org.apache.commons.cli, org.apache.commons.lang3, org.wildfly.security.elytron-tool]
	at org.jboss.as.test.layers.LayersTest.test(LayersTest.java:109)
	at org.jboss.as.test.layers.LayersTestCase.testServlet(LayersTestCase.java:146)

gaol · 2021-11-29T08:07:09Z

I am not sure about the ElytronToolScriptTestCase failures, but the failure of LayersTestCase.testServlet() in full integration should be fixed via wildfly/wildfly#14970.

jamezp · 2021-11-29T16:42:33Z

@fjuma Is there a reason we remove then re-add the module? I assume it's for backwards compatibility, but will that work if the wrapper library is not provided?

fjuma · 2021-11-29T16:50:08Z

@jamezp ELYTRON_TOOL_ADDONS allows a user to optionally specify their own JAR that contains a custom credential store implementation. Before we updated the script to make use of jboss-modules, if a user defined ELYTRON_TOOL_ADDONS, it was just added to the classpath by the elytron-tool script.

After updating the script to make use of jboss-modules, this classpath approach could no longer work. So instead, if a user defines ELYTRON_TOOL_ADDONS, then the script will just create a new module that contains this JAR using the module add command. For the module add command to succeed, the module cannot already exist. That's why the script first removes the custom module if it happened to exist and then adds it.

core-feature-pack/common/src/main/resources/content/bin/elytron-tool.bat

wildfly-ci · 2021-11-29T17:22:26Z

Core - Full Integration Build 11122 outcome was UNKNOWN using a merge of 0cfad96
Summary: Canceled (Error while applying patch; cannot find commit 62ac754 in the https://github.com/wildfly/wildfly-core.git repository, possible reason: refs/pull/4879/merge branch was updated and the commit selected for the ... Build time: 00:00:16

fjuma · 2021-11-29T17:48:14Z

@jamezp Thanks for the feedback! I've just merged a commit from @Ashpan that should address your comments on the .bat script.

wildfly-ci · 2021-11-29T18:16:57Z

Core - Full Integration Build 11236 outcome was UNKNOWN using a merge of ccfd744
Summary: Canceled (Error while applying patch; cannot find commit 62ac754 in the https://github.com/wildfly/wildfly-core.git repository, possible reason: refs/pull/4879/merge branch was updated and the commit selected for the ... Build time: 00:00:17

wildfly-ci · 2021-11-29T18:43:56Z

Core - Full Integration Build 11125 outcome was UNKNOWN using a merge of ccfd744
Summary: Canceled (Error while applying patch; cannot find commit 1f7c4d5 in the https://github.com/wildfly/wildfly-core.git repository, possible reason: refs/pull/4879/merge branch was updated and the commit selected for the ... Build time: 00:00:09

wildfly-ci · 2021-11-29T18:44:07Z

Core - Full Integration Build 11238 outcome was UNKNOWN using a merge of ccfd744
Summary: Canceled (Error while applying patch; cannot find commit 1f7c4d5 in the https://github.com/wildfly/wildfly-core.git repository, possible reason: refs/pull/4879/merge branch was updated and the commit selected for the ... Build time: 00:00:12

jamezp

CI is still waiting to run, but the change itself looks okay to me. I did run the script tests on a Windows VM and Linux. They passed on both platforms.

wildfly-ci · 2021-11-29T23:21:04Z

Core - Full Integration Build 11127 outcome was FAILURE using a merge of ccfd744
Summary: Tests failed: 1, passed: 4092, ignored: 41 Build time: 02:41:07

Failed tests

org.jboss.as.test.layers.LayersTestCase.testServlet: java.lang.Exception: Some unreferenced modules are un-expected [org.apache.commons.cli, org.apache.commons.lang3, org.wildfly.security.elytron-tool]
	at org.jboss.as.test.layers.LayersTest.test(LayersTest.java:109)
	at org.jboss.as.test.layers.LayersTestCase.testServlet(LayersTestCase.java:146)

jamezp · 2021-11-29T23:45:01Z

That galleon failure needs to be addressed. There is likely an issue on Windows too with the PowerShell script test. This can be fixed with a commit like e6dd928.

fjuma · 2021-11-30T14:12:42Z

Thanks very much @jamezp! I've included your commit in this PR.

The galleon failure should be addressed by wildfly/wildfly#14970.

wildfly-ci · 2021-11-30T17:24:56Z

Core - Full Integration Build 11130 outcome was FAILURE using a merge of fb45e1a
Summary: Tests failed: 1, passed: 4074, ignored: 43 Build time: 02:35:51

Failed tests

org.jboss.as.test.layers.LayersTestCase.testServlet: java.lang.Exception: Some unreferenced modules are un-expected [org.apache.commons.cli, org.apache.commons.lang3, org.wildfly.security.elytron-tool]
	at org.jboss.as.test.layers.LayersTest.test(LayersTest.java:109)
	at org.jboss.as.test.layers.LayersTestCase.testServlet(LayersTestCase.java:146)

wildfly-ci · 2021-11-30T18:16:59Z

Core - Full Integration Build 11131 outcome was FAILURE using a merge of fb45e1a
Summary: Tests failed: 1, passed: 4092, ignored: 41 Build time: 02:27:15

Failed tests

org.jboss.as.test.layers.LayersTestCase.testServlet: java.lang.Exception: Some unreferenced modules are un-expected [org.apache.commons.cli, org.apache.commons.lang3, org.wildfly.security.elytron-tool]
	at org.jboss.as.test.layers.LayersTest.test(LayersTest.java:109)
	at org.jboss.as.test.layers.LayersTestCase.testServlet(LayersTestCase.java:146)

wildfly-ci · 2021-11-30T18:59:57Z

Core - Full Integration Build 11244 outcome was FAILURE using a merge of fb45e1a
Summary: Tests passed: 7058, ignored: 128; exit code 137 (Step: Build & test full (Maven)) (new) Build time: 03:10:13

yersan · 2021-12-01T10:49:48Z

@fjuma I did not realize it was going to conflict with another one merged. Could you rebase and resolve the conflict? thanks

… modules

fjuma · 2021-12-01T14:45:51Z

@yersan I've rebased and resolved the conflict. Thanks!

yersan · 2021-12-01T15:45:54Z

testsuite/layers/src/test/java/org/jboss/as/test/layers/LayersTestCase.java

@@ -49,6 +49,10 @@
        "org.wildfly.event.logger",
        // wildfly-elytron-http-stateful-basic
        "org.wildfly.security.http.sfbasic"
+        // wildfly-elytron-tool
+        "org.apache.commons.cli",


Hello @fjuma , missed comma "," after "org.wildfly.security.http.sfbasic"

Sorry, fixed.

…n-tool.[sh|bat|ps1] should be used instead

…tCase

scripts

…on.conf.ps1 https://issues.redhat.com/browse/WFCORE-5712 Signed-off-by: James R. Perkins <jperkins@redhat.com>

yersan · 2021-12-01T18:09:43Z

The RemotingLegacySubsystemTestCase.testSubsystemWithConnectorPropertyChange failing on "Linux JDK 11" Job is an intermittent error that it is unrelated to this PR, see XNIO-385

wildfly-ci · 2021-12-01T20:27:43Z

Core - Full Integration Build 11248 outcome was FAILURE using a merge of b9e9d27
Summary: Tests failed: 1 (1 new), passed: 7124, ignored: 128 Build time: 03:14:04

Failed tests

org.wildfly.test.scripts.AppClientScriptTestCase: java.io.UncheckedIOException: java.nio.file.FileSystemException: /store/work/tc-work/37b47ae8b9c60325/full/testsuite/scripts/target/wildfly/docs/schema/wildfly-elytron_1_0.xsd -> /store/work/tc-work/37b47ae8b9c60325/full/testsuite/scripts/target/wildfly (core)/docs/schema/wildfly-elytron_1_0.xsd: No space left on device
	at org.wildfly.test.common.ServerConfigurator.copy(ServerConfigurator.java:60)
	at org.wildfly.test.common.ServerConfigurator.configure(ServerConfigurator.java:47)
	at org.wildfly.test.scripts.ScriptTestCase.configureEnvironment(ScriptTestCase.java:74)
Caused by: java.nio.file.FileSystemException: /store/work/tc-work/37b47ae8b9c60325/full/testsuite/scripts/target/wildfly/docs/schema/wildfly-elytron_1_0.xsd -> /store/work/tc-work/37b47ae8b9c60325/full/testsuite/scripts/target/wildfly (core)/docs/schema/wildfly-elytron_1_0.xsd: No space left on device
	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
	at sun.nio.fs.UnixCopyFile.copyFile(UnixCopyFile.java:253)
	at sun.nio.fs.UnixCopyFile.copy(UnixCopyFile.java:581)
	at sun.nio.fs.UnixFileSystemProvider.copy(UnixFileSystemProvider.java:253)
	at java.nio.file.Files.copy(Files.java:1274)
	at org.wildfly.test.common.ServerConfigurator$1.visitFile(ServerConfigurator.java:76)
	at org.wildfly.test.common.ServerConfigurator$1.visitFile(ServerConfigurator.java:65)
	at java.nio.file.Files.walkFileTree(Files.java:2670)
	at java.nio.file.Files.walkFileTree(Files.java:2742)
	at org.wildfly.test.common.ServerConfigurator.copyDirectory(ServerConfigurator.java:65)
	at org.wildfly.test.common.ServerConfigurator.copy(ServerConfigurator.java:57)
	... 21 more

yersan · 2021-12-02T09:05:50Z

Holding this PR until see how it integrates with wildfly/wildfly#14988, at least at product branches there is some kind of relationship between them. We can follow the integration here:

https://ci.wildfly.org/viewLog.html?buildTypeId=WF_WildFlyCoreIntegrationExperiments&buildId=285378

fjuma force-pushed the WFCORE-5712 branch from cf3455c to 27b0e97 Compare November 26, 2021 16:55

github-actions bot added the deps-changed Dependencies have been checked, and there are changes highlighted in a comment label Nov 26, 2021

fjuma mentioned this pull request Nov 26, 2021

[WFLY-15721] Remove org.apache.commons.cli and org.apache.commons.lang3 modules and update wildfly-elytron-tool.jar wildfly/wildfly#14967

Merged

fjuma force-pushed the WFCORE-5712 branch 2 times, most recently from 01d225e to 8103730 Compare November 26, 2021 19:16

fjuma force-pushed the WFCORE-5712 branch from 8103730 to 137925a Compare November 26, 2021 20:07

gaol mentioned this pull request Nov 28, 2021

[WFCORE-5708] Add a new "org.wildfly.security.http.sfbasic" module. #4866

Merged

fjuma force-pushed the WFCORE-5712 branch 2 times, most recently from 050e055 to e6174a1 Compare November 29, 2021 15:48

jamezp requested changes Nov 29, 2021

View reviewed changes

jamezp approved these changes Nov 29, 2021

View reviewed changes

jamezp approved these changes Nov 30, 2021

View reviewed changes

yersan added the ready-for-merge This PR is ready to be merged and fulfills all requirements label Dec 1, 2021

[WFCORE-5720] Add org.apache.commons.cli and org.apache.commons.lang3…

3b14c3d

… modules

fjuma force-pushed the WFCORE-5712 branch from fb45e1a to 912ed53 Compare December 1, 2021 14:45

yersan reviewed Dec 1, 2021

View reviewed changes

fjuma and others added 9 commits December 1, 2021 12:10

[WFCORE-5712] Add org.wildfly.security.elytron-tool module

91dc9a1

[WFCORE-5712] Update elytron-tool.sh to make use of jboss-modules

d2f3892

[WFCORE-5712] Update elytron-tool.[bat|ps1] to make use of jboss-modules

5c712af

[WFCORE-5712] Update wildfly-elytron-tool.jar to indicate that elytro…

fcb3765

…n-tool.[sh|bat|ps1] should be used instead

[WFCORE-5730] Un-ignore org.wildfly.scripts.test.ElytronToolScriptTes…

65bfca7

…tCase

[WFCORE-5712] Fix ElytronToolScriptTestCase

bb6479b

[WFCORE-5712] Fixed failure with spaces in directory path for Windows

77f7fc4

scripts

[WFCORE-5712] Fix the powershell test by adding JAVA_OPTS to the comm…

48b406e

…on.conf.ps1 https://issues.redhat.com/browse/WFCORE-5712 Signed-off-by: James R. Perkins <jperkins@redhat.com>

[WFCORE-5712] Fix JAVA_OPTS in elytron-tool.ps1

b9e9d27

fjuma force-pushed the WFCORE-5712 branch from 912ed53 to b9e9d27 Compare December 1, 2021 17:10

yersan added the hold Do not merge this PR label Dec 2, 2021

yersan removed the hold Do not merge this PR label Dec 2, 2021

yersan merged commit cfb4d11 into wildfly:main Dec 2, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WFCORE-5712] Update elytron-tool scripts to make use of jboss-modules #4879

[WFCORE-5712] Update elytron-tool scripts to make use of jboss-modules #4879

fjuma commented Nov 26, 2021 •

edited

Loading

wildfly-ci commented Nov 26, 2021

fjuma commented Nov 26, 2021 •

edited

Loading

github-actions bot commented Nov 26, 2021 •

edited

Loading

wildfly-ci commented Nov 26, 2021

fjuma commented Nov 26, 2021

wildfly-ci commented Nov 26, 2021

wildfly-ci commented Nov 26, 2021

wildfly-ci commented Nov 26, 2021

gaol commented Nov 29, 2021

jamezp commented Nov 29, 2021

fjuma commented Nov 29, 2021

wildfly-ci commented Nov 29, 2021

fjuma commented Nov 29, 2021

wildfly-ci commented Nov 29, 2021

wildfly-ci commented Nov 29, 2021

wildfly-ci commented Nov 29, 2021

jamezp left a comment

wildfly-ci commented Nov 29, 2021

jamezp commented Nov 29, 2021

fjuma commented Nov 30, 2021

wildfly-ci commented Nov 30, 2021

wildfly-ci commented Nov 30, 2021

wildfly-ci commented Nov 30, 2021

yersan commented Dec 1, 2021

fjuma commented Dec 1, 2021

yersan Dec 1, 2021

fjuma Dec 1, 2021

yersan commented Dec 1, 2021

wildfly-ci commented Dec 1, 2021

yersan commented Dec 2, 2021 •

edited

Loading

[WFCORE-5712] Update elytron-tool scripts to make use of jboss-modules #4879

[WFCORE-5712] Update elytron-tool scripts to make use of jboss-modules #4879

Conversation

fjuma commented Nov 26, 2021 • edited Loading

wildfly-ci commented Nov 26, 2021

fjuma commented Nov 26, 2021 • edited Loading

github-actions bot commented Nov 26, 2021 • edited Loading

wildfly-ci commented Nov 26, 2021

Failed tests

fjuma commented Nov 26, 2021

wildfly-ci commented Nov 26, 2021

Failed tests

wildfly-ci commented Nov 26, 2021

Failed tests

wildfly-ci commented Nov 26, 2021

Failed tests

gaol commented Nov 29, 2021

jamezp commented Nov 29, 2021

fjuma commented Nov 29, 2021

wildfly-ci commented Nov 29, 2021

fjuma commented Nov 29, 2021

wildfly-ci commented Nov 29, 2021

wildfly-ci commented Nov 29, 2021

wildfly-ci commented Nov 29, 2021

jamezp left a comment

Choose a reason for hiding this comment

wildfly-ci commented Nov 29, 2021

Failed tests

jamezp commented Nov 29, 2021

fjuma commented Nov 30, 2021

wildfly-ci commented Nov 30, 2021

Failed tests

wildfly-ci commented Nov 30, 2021

Failed tests

wildfly-ci commented Nov 30, 2021

yersan commented Dec 1, 2021

fjuma commented Dec 1, 2021

yersan Dec 1, 2021

Choose a reason for hiding this comment

fjuma Dec 1, 2021

Choose a reason for hiding this comment

yersan commented Dec 1, 2021

wildfly-ci commented Dec 1, 2021

Failed tests

yersan commented Dec 2, 2021 • edited Loading

fjuma commented Nov 26, 2021 •

edited

Loading

fjuma commented Nov 26, 2021 •

edited

Loading

github-actions bot commented Nov 26, 2021 •

edited

Loading

yersan commented Dec 2, 2021 •

edited

Loading