Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jakarta Security 3.0 #462

Merged
merged 5 commits into from Jun 14, 2022
Merged

Jakarta Security 3.0 #462

merged 5 commits into from Jun 14, 2022

Conversation

arjantijms
Copy link
Contributor

@arjantijms arjantijms commented Mar 1, 2022

Specification PR template

When creating a specification project release review, create PRs with the content defined as follows.

Include the following in the PR:

Note: If any item does not apply, check it and mark N/A below it.

Signed-off-by: Arjan Tijms arjan.tijms@gmail.com

Signed-off-by: Arjan Tijms <arjan.tijms@gmail.com>
@netlify
Copy link

netlify bot commented Mar 1, 2022

Deploy Preview for jakartaee-specifications ready!

Name Link
🔨 Latest commit 571a7c6
🔍 Latest deploy log https://app.netlify.com/sites/jakartaee-specifications/deploys/626ebc1a78ceb10008370f11
😎 Deploy Preview https://deploy-preview-462--jakartaee-specifications.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

Signed-off-by: Arjan Tijms <arjan.tijms@gmail.com>
@ivargrimstad ivargrimstad added the release review Use this label on PRs that are filed for release review label Mar 2, 2022
Signed-off-by: Arjan Tijms <arjan.tijms@gmail.com>
@jeanouii
Copy link
Contributor

jeanouii commented Apr 5, 2022

It looks good to me.
Thanks Arjan and sorry for the delay

@jeanouii jeanouii added the approved The ballot was approved by the Specification Committee label Apr 5, 2022
@jeanouii jeanouii self-requested a review April 5, 2022 12:45
@jeanouii jeanouii removed the approved The ballot was approved by the Specification Committee label Apr 5, 2022
@ivargrimstad
Copy link
Member

ivargrimstad commented Apr 19, 2022

Mentor's Spec Review Checklist

  1. Spec PR
  1. _index.md
  1. javadocs
  • Footer contains Eclipse copyright and link to license
  • ESFL license is included, usually as doc-files/speclicense.html
  • no META-INF directory in PR
  • javadocs-jar artifact matches apidocs (optional for this release)
  1. Spec PDF
  • Correct spec title
  • Version number of the form x.y, not x.y.z
  • Correct Eclipse copyright line
  • No DRAFT or SNAPSHOT
  • Correct Logo
  1. Spec HTML
  • Same as PDF
  1. TCK zip file
  • README file (optional for this release)
  • EFTL license file, preferably named LICENSE.md
  • User's Guide (or equivalent documentation)
  • How to test the Compatible Implementation(s) listed in _index.md above with the TCK (may be in UG)
  1. TCK User's Guide (or equivalent documentation)
  • Software requirements listed
  • Installation and configuration described
  • How to run tests
  • Where to file challenges
  1. Compatibility certification request
  • Request follows template
  • SHA-256 fingerprint matches staged TCK zip file
  • Request issue has certification label.
  1. TCK results summary
  • Page is hosted by Compatible Implementation project
  • Includes all information from certification request
  • Summary includes number of tests passed, failed, errors
  • SHA-256 fingerprint matches staged TCK zip file on cert request
  1. If a Release Review is required, the specification project team contacts the EMO to initiate the review by sending an email to emo@eclipse.org.
    (A Release Review is not required if the current release is a Service Release based on a previously successful Major or Minor
    release as indicated by a release record on the project's Releases page, e.g., the Jakarta Servlet releases page.)

  2. Update Jakarta EE API jar

  • Update the Jakarta EE API jar by submitting a PR to the jakartaee-api project that updates the version number of your API jar file.

@dblevins
Copy link
Contributor

@arjantijms Can you provide some insight on where we are with this overall? Looks like we're still missing staged binaries and a TCK run, so I suspect you all might still be working on the spec itself. Is there an ETA on when this spec is ready to go final?

@dblevins
Copy link
Contributor

From the Security dev list, it looks like there's still work to be done to add the needed coverage for the features introduced in Security 3.0 https://www.eclipse.org/lists/es-dev/msg00162.html

@arjantijms
Copy link
Contributor Author

arjantijms commented Apr 20, 2022

@dblevins

so I suspect you all might still be working on the spec itself.

Nope, that's not correct. The spec has been fully ready since about a month. The binaries should really be there. I see them here:

https://jakarta.oss.sonatype.org/content/repositories/staging/jakarta/security/enterprise/jakarta.security.enterprise-api/3.0.0/

From the Security dev list, it looks like there's still work to be done to add the needed coverage for the features introduced in Security 3.0

That's not correct. The new Security 3.0 features are covered, at least, to a reasonable degree. For something as large as OpenID Connect there's always more that can be tested.

The new tests are:

@dblevins
Copy link
Contributor

@arjantijms Thanks. I read your email to Security dev a bit closer, it looks like you were talking about tests still lacking in the Authorization TCK, not the Security TCK.

As the Security spec and TCK are stable, is there any blocker or ETA to checking off more items in the checklist such as staged API jars, staged TCK binary and compatible implementation TCK results?

@arjantijms
Copy link
Contributor Author

As the Security spec and TCK are stable, is there any blocker or ETA to checking off more items in the checklist such as staged API jars, staged TCK binary and compatible implementation TCK results?

The item for the staged API jars can surely be checked-off.

@arjantijms
Copy link
Contributor Author

CCR: jakartaee/security#239

@arjantijms
Copy link
Contributor Author

@arjantijms
Copy link
Contributor Author

@arjantijms
Copy link
Contributor Author

Release review tracking issue: https://gitlab.eclipse.org/eclipsefdn/emo-team/emo/-/issues/279

@starksm64
Copy link
Contributor

@arjantijms You mentioned the userguide might need work, but I have gone through the guide and candidate TCK zip and it meets all of the requirements.

@arjantijms
Copy link
Contributor Author

@starksm64 if it does that, then it's fine. I haven't looked at it yet, but does it also contain the parts about the new TCK? (the junit / arquillian bits)

@arjantijms
Copy link
Contributor Author

@starksm64 additionally, does it correctly say how many tests need to pass?

This is the current output from a TCK run:

********************************************************************************
Completed running 116 tests.
Number of Tests Failed      = 0
Number of Tests with Errors = 0
********************************************************************************
[INFO]      [exec] [javatest.batch] Number of Tests Passed      = 84
[INFO]      [exec] [javatest.batch] Number of Tests Failed      = 0
[INFO]      [exec] [javatest.batch] Number of Tests with Errors = 0
[INFO]      [exec] [javatest.batch] ********************************************************************************
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/autoapplysession/Client.java#testAutoApplySession
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/basic/Client.java#testBasicHAMHasCorrectQualifier
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/basic/Client.java#testBasicHAMValidateRequest
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/basic/Client.java#testBasicHAMValidateRequest_wrongPassword
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/customform/base/Client.java#testCustomFormHAMHasCorrectQualifier
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/customform/base/Client.java#testCustomFormHAMValidateRequest
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/customform/base/Client.java#testCustomFormLoginToContinueErrorPage
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/customform/base/Client.java#testLoginToContinueuseRedirectToLogin
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/customform/expression/Client.java#testLoginToContinueuseForwardToLoginExpression
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/form/Client.java#testFormHAMHasCorrectQualifier
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/form/Client.java#testFormHAMValidateRequest
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/form/Client.java#testLoginToContinueerrorPage
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/form/Client.java#testLoginToContinueLoginPage
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/rememberme/test1/Client.java#testRememberMecookieHttpOnly
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/rememberme/test1/Client.java#testRememberMecookieMaxAgeSeconds
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/rememberme/test1/Client.java#testRememberMeCookieNameandisRememberMeExpression
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/rememberme/test2/Client.java#testRememberMecookieHttpOnlyFalse
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/rememberme/test2/Client.java#testRememberMecookieSecureOnly
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/rememberme/test3/Client.java#testRememberMecookieHttpOnlyExpression
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/rememberme/test3/Client.java#testRememberMecookieMaxAgeSecondsExpression
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/rememberme/test3/Client.java#testRememberMecookieSecureOnlyExpression
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/rememberme/test3/Client.java#testRememberMeisRememberMe
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/sam/obtainbean/Client.java#testSAMObtainBean
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/workflow/cleansubject/Client.java#testHAMCleanSubject
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/workflow/secureresponse/Client.java#testHAMSecureResponse
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/workflow/validaterequest/Client.java#testCallValidateRequestBeforeService
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/workflow/validaterequestduringauthen/Client.java#testCallValidateRequestDuringAuthenticate
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/ham/workflow/validaterequestwithfilter/Client.java#testCallValidateRequestBeforeFilter
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/basic/Client.java#testIdentityStoreInstall
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/customhandler/Client.java#testIdentityStore_customHandler
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/database/basic/Client.java#testAnnotationDBIDStore_Basic
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/database/hashalgorithm/Client.java#testAnnotationDBIDStore_HashAlgorithmDefault
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/database/hashalgorithmparam/Client.java#testAnnotationDBIDStore_HashAlgorithmParam
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/database/invalidcallerquery/Client.java#testAnnotationDBIDStore_Invalidcallerquery
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/database/invaliddatasource/Client.java#testAnnotationDBIDStore_Invaliddatasource
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/database/invalidgroupsquery/Client.java#testAnnotationDBIDStore_Invalidgroupsquery
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/database/invalidhashalgorithmparam/Client.java#testAnnotationDBIDStore_InvalidHashAlgorithmParam
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/database/invalidpriorityuseforexpr/Client.java#testAnnotationDBIDStore_invalidpriorityuseforexpr
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/database/multi/Client.java#testAnnotationDBIDStore_multi_withLdap
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/database/notvalidated/Client.java#testAnnotationDBIDStore_notValidated
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/database/priorityuseforexpr/Client.java#testAnnotationDBIDStore_priorityuseforexpr
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/database/priorityuseforexprbean/Client.java#testAnnotationDBIDStore_priorityuseforexprbean
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/database/useforgroup/Client.java#testAnnotationDBIDStore_useforgroup
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/database/useforvalidation/Client.java#testAnnotationDBIDStore_useforvalidation
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/idstorepermission/Client.java#testIdentityStore_customHandlerWithoutIDStorePermission
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/basic/Client.java#testAnnotationLdapIDStore_Basic
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/binddn/Client.java#testAnnotationLdapIDStore_Binddn
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/groupmemberof/Client.java#testAnnotationLdapIDStore_groupMemberOf
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/groupmemberofnotexist/Client.java#testAnnotationLdapIDStore_groupMemberOfNotExist
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/invalidbinddn/Client.java#testAnnotationLdapIDStore_invalidBinddn
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/invalidbinddnpassword/Client.java#testAnnotationLdapIDStore_invalidBinddnPassword
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/invalidcallerbasedn/Client.java#testAnnotationLdapIDStore_invalidCallerBasedn
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/invalidcallernameattr/Client.java#testAnnotationLdapIDStore_invalidCallerNameAttribute
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/invalidcallersearchbase/Client.java#testAnnotationLdapIDStore_invalidCallerSearchBase
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/invalidcallersearchfilter/Client.java#testAnnotationLdapIDStore_invalidCallerSearchFilter
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/invalidgroupmemberattr/Client.java#testAnnotationLdapIDStore_invalidGroupMemberAttribute
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/invalidgroupnameattr/Client.java#testAnnotationLdapIDStore_invalidGroupNameAttribute
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/invalidgroupsearchbase/Client.java#testAnnotationLdapIDStore_invalidGroupSearchBase
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/invalidgroupsearchfilter/Client.java#testAnnotationLdapIDStore_invalidGroupSearchFilter
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/invalidsearchscopeexpr/Client.java#testAnnotationLdapIDStore_invalidsearchScopeExpression
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/invalidurl/Client.java#testAnnotationLdapIDStore_invalidURL
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/notvalidated/Client.java#testAnnotationLdapIDStore_NotValidated
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/priorityuseforexpr/Client.java#testIdentityStore_ldap_priorityuseforexpr
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/searchscopebothonelevel/Client.java#testAnnotationLdapIDStore_searchScopeBothOneLevel
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/searchscopebothsubtree/Client.java#testAnnotationLdapIDStore_searchScopeBothSubTree
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/searchscopecalleronelevelgroupsubtree/Client.java#testAnnotationLdapIDStore_searchScopeCallerOneLevelGroupSubTree
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/searchscopecallersubtreegrouponelevel/Client.java#testAnnotationLdapIDStore_searchScopeCallerSubTreeGroupOneLevel
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/searchscopeexpr/Client.java#testAnnotationLdapIDStore_searchScopeExpression
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/useforgroup/Client.java#testIdentityStore_ldap_useforgroup
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/ldap/useforvalidation/Client.java#testIdentityStore_ldap_useforvalidation
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/multi/Client.java#testIdentityStoreValidate_multiIDStore
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/multi/Client.java#testIdentityStoreValidate_multiIDStore_INVALID
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/multi/Client.java#testIdentityStoreValidate_multiIDStore_INVALIDWithNOTVALIDATED
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/multi/Client.java#testIdentityStoreValidate_multiIDStore_NOTVALIDATED
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/multiauthz/Client.java#testIdentityStore_getGroups_multiGroupStore_highPriority_valid
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/multiauthz/Client.java#testIdentityStore_getGroups_multiGroupStore_lowerPriority_valid
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/useforgroup/Client.java#testIdentityStore_validationType_useforgroup
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/idstore/useforvalidation/Client.java#testIdentityStore_validationType_useforvalidation
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/securitycontext/authenticate/Client.java#testSecurityContextAuthenticate
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/securitycontext/authenticate/Client.java#testSecurityContextAuthenticate_wrongCredential
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/securitycontext/callerdata/Client.java#testSecurityContextHasAccessToWebResource
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/securitycontext/callerdata/Client.java#testSecurityContextIsCallerInRole
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/securitycontext/ejb/Client.java#testSecurityContextAvailableInEJB
[INFO]      [exec] [javatest.batch] PASSED........com/sun/ts/tests/securityapi/securitycontext/getprincipalsbytype/Client.java#testSecurityContextGetPrincipalsByType
[INFO]      [exec] [javatest.batch] 
[INFO]      [exec] [javatest.batch] Apr 27, 2022, 8:51:13 PM Finished executing all tests, wait for cleanup...
[INFO]      [exec] [javatest.batch] Apr 27, 2022, 8:51:13 PM Harness done with cleanup from test run.
[INFO]      [exec] [javatest.batch] Total time = 194s
[INFO]      [exec] [javatest.batch] Setup time = 0s
[INFO]      [exec] [javatest.batch] Cleanup time = 0s
[INFO]      [exec] [javatest.batch] Test results: passed: 84
[INFO]      [exec] [javatest.batch] Results written to /home/jenkins/agent/workspace/2_security-run-tck-against-staged-build/jakarta-security-tck-3.0.0/old-tck/run/target/security-tck/securitywork/security.
[INFO]      [exec] [javatest.batch] Report written to /home/jenkins/agent/workspace/2_security-run-tck-against-staged-build/jakarta-security-tck-3.0.0/old-tck/run/target/security-tck/securityreport/security
[INFO]      [exec] 
[INFO]      [exec] BUILD SUCCESSFUL
[INFO]      [exec] Total time: 3 minutes 16 seconds
[INFO]      [exec] Picked up JAVA_TOOL_OPTIONS: -XX:+IgnoreUnrecognizedVMOptions -XX:+UnlockExperimentalVMOptions 
[INFO]      [exec] Waiting for the domain to stop .
[INFO]      [exec] Command stop-domain executed successfully.
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-failsafe-plugin:3.0.0-M5:integration-test (default) @ glassfish-external-tck-security ---
[INFO] No tests to run.
[INFO] 
[INFO] --- maven-surefire-report-plugin:3.0.0-M6:failsafe-report-only (default) @ glassfish-external-tck-security ---
[INFO] 
[INFO] --- maven-failsafe-plugin:3.0.0-M5:verify (default) @ glassfish-external-tck-security ---
[INFO] 
[INFO] --- maven-install-plugin:2.4:install (default-install) @ glassfish-external-tck-security ---
[INFO] Installing /home/jenkins/agent/workspace/2_security-run-tck-against-staged-build/jakarta-security-tck-3.0.0/old-tck/run/pom.xml to /home/jenkins/.m2/repository/org/eclipse/ee4j/security/tck/glassfish-external-tck-security/3.0.0/glassfish-external-tck-security-3.0.0.pom
[INFO] 
[INFO] --- maven-surefire-report-plugin:3.0.0-M6:failsafe-report-only (default-cli) @ glassfish-external-tck-security ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Jakarta Security TCK - main 3.0.0:
[INFO] 
[INFO] Jakarta Security TCK - main ........................ SUCCESS [ 56.659 s]
[INFO] common ............................................. SUCCESS [ 10.930 s]
[INFO] app-securitycontext ................................ SUCCESS [ 25.370 s]
[INFO] app-securitycontext-auth ........................... SUCCESS [ 17.201 s]
[INFO] app-securitycontext-customprincipal ................ SUCCESS [ 17.386 s]
[INFO] app-mem ............................................ SUCCESS [ 17.361 s]
[INFO] app-db ............................................. SUCCESS [ 21.171 s]
[INFO] app-ldap ........................................... SUCCESS [ 19.079 s]
[INFO] app-ldap2 .......................................... SUCCESS [ 18.821 s]
[INFO] app-ldap3 .......................................... SUCCESS [ 19.964 s]
[INFO] app-custom ......................................... SUCCESS [ 19.935 s]
[INFO] app-multiple-store ................................. SUCCESS [ 20.000 s]
[INFO] app-multiple-store-backup .......................... SUCCESS [ 19.371 s]
[INFO] app-openid ......................................... SUCCESS [ 32.713 s]
[INFO] app-openid2 ........................................ SUCCESS [ 42.490 s]
[INFO] app-openid3 ........................................ SUCCESS [ 37.370 s]
[INFO] app-mem-basic ...................................... SUCCESS [ 20.056 s]
[INFO] app-mem-basic-decorate ............................. SUCCESS [ 19.857 s]
[INFO] app-mem-form ....................................... SUCCESS [ 22.115 s]
[INFO] app-mem-customform ................................. SUCCESS [ 22.619 s]
[INFO] app-custom-session ................................. SUCCESS [ 18.902 s]
[INFO] app-custom-rememberme .............................. SUCCESS [ 23.540 s]
[INFO] app-custom-identity-store-handler .................. SUCCESS [ 21.127 s]
[INFO] app-jaxrs .......................................... SUCCESS [ 19.510 s]
[INFO] Old Jakarta Security TCK - build ................... SUCCESS [02:22 min]
[INFO] Old Jakarta Security TCK - main .................... SUCCESS [  1.265 s]
[INFO] Old Jakarta Security TCK - run ..................... SUCCESS [05:19 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  17:10 min
[INFO] Finished at: 2022-04-27T20:51:17Z
[INFO] ------------------------------------------------------------------------
SHA256_API=81c0d29cc28b57ee6c254c5609b8f7c15c2dcdc299174fac4e1bad123225e6d5
SHA256_IMPL=e755077c3a27c03c44c86222a27f76c6d10ff147f88fde0b2d4cdde070fe84b0
SHA256_TCK=c5235d3ef2a5ecdde7bfdfc7d2d808f7a84502c1b9096027ea47528eb7499536
TCK_download=https://download.eclipse.org/es/jakartaee10/staged/eftl/jakarta-security-tck-3.0.0.zip
OS2=Debian GNU/Linux 10
OS3=10.12
OS4=PRETTY_NAME="Debian GNU/Linux 10 (buster)" NAME="Debian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"
JDK_VERSION=Picked up JAVA_TOOL_OPTIONS: -XX:+IgnoreUnrecognizedVMOptions -XX:+UnlockExperimentalVMOptions openjdk version "17.0.1" 2021-10-19 OpenJDK Runtime Environment (build 17.0.1+12-39) OpenJDK 64-Bit Server VM (build 17.0.1+12-39, mixed mode, sharing)

I guess this bit is what we should ask for?

********************************************************************************
Completed running 116 tests.
Number of Tests Failed      = 0
Number of Tests with Errors = 0
********************************************************************************
[INFO]      [exec] [javatest.batch] Number of Tests Passed      = 84
[INFO]      [exec] [javatest.batch] Number of Tests Failed      = 0
[INFO]      [exec] [javatest.batch] Number of Tests with Errors = 0

So 116 new tests, and 84 old tests, right?

@starksm64
Copy link
Contributor

The guides don't generally say how many tests need to pass as it can vary based on optional features and environment.

@jeanouii
Copy link
Contributor

Hey @arjantijms
Few comments ...

Downloaded the TCK from the link at the top of this PR and ran the shasum command.

$ shasum -a 256 ~/Downloads/jakarta-security-tck-3.0.0.zip
c5235d3ef2a5ecdde7bfdfc7d2d808f7a84502c1b9096027ea47528eb7499536  /Users/jlmonteiro/Downloads/jakarta-security-tck-3.0.0.zip

Both the CSR jakartaee/security#239 and the result page https://eclipse-ee4j.github.io/soteria/certifications/jakarta-security/3.0/TCK-Results do reference another TCK.

I noticed there is a TCK 3.0.1 with JDK11 support in the index file, but did not find it to download for additional checking.

In your comment to Scott, you dropped the log of the TCK run which on the other hand seems to be the same as I have download.

SHA256_API=81c0d29cc28b57ee6c254c5609b8f7c15c2dcdc299174fac4e1bad123225e6d5
SHA256_IMPL=e755077c3a27c03c44c86222a27f76c6d10ff147f88fde0b2d4cdde070fe84b0
SHA256_TCK=c5235d3ef2a5ecdde7bfdfc7d2d808f7a84502c1b9096027ea47528eb7499536
TCK_download=https://download.eclipse.org/es/jakartaee10/staged/eftl/jakarta-security-tck-3.0.0.zip

Can you double check this please?

security/3.0/apidocs/doc-files/speclicense.html Outdated Show resolved Hide resolved
security/3.0/_index.md Outdated Show resolved Hide resolved
@arjantijms
Copy link
Contributor Author

I noticed there is a TCK 3.0.1 with JDK11 support in the index file, but did not find it to download for additional checking.

That is probably a mistake. Could be a build of the old TCK that's still lingering around?

Signed-off-by: Arjan Tijms <arjan.tijms@gmail.com>
Signed-off-by: Arjan Tijms <arjan.tijms@gmail.com>
@arjantijms
Copy link
Contributor Author

Downloading the TCK manually and checking SHA on the terminal:

shasum -a 256 ~/Downloads/jakarta-security-tck-3.0.0.zip
c5235d3ef2a5ecdde7bfdfc7d2d808f7a84502c1b9096027ea47528eb7499536

@jeanouii
Copy link
Contributor

jeanouii commented May 2, 2022

I think this is all looking good now.
I'll finish the review and create the ballot.

Thanks a lot @arjantijms

@jeanouii jeanouii added ballot Delivered to the Specification Committee for ballot and removed release review Use this label on PRs that are filed for release review labels May 3, 2022
@ivargrimstad ivargrimstad added the release review Use this label on PRs that are filed for release review label May 3, 2022
@Pandrex247
Copy link
Contributor

Am I being blind or does the staged TCK not have a user guide?
It's got a README with a couple of sentences on how to run the tests in Docker, but that's not really a user guide - it doesn't cover how to file challenges for example.

@arjantijms
Copy link
Contributor Author

I'll have to double check the actual archive that got uploaded, but from the assembly they should be there:

@Pandrex247
Copy link
Contributor

The tck directory doesn't exist on the 3.0.0-RELEASE tag (which is what I assume the 3.0.0 artefact aligns to?), which is presumably why it's not included in the staged TCK. Although having said that it doesn't exist on the 2.0 branches or tags either so perhaps there's some build shenanigans going on? It only seems to exist on the master branch.

@Pandrex247
Copy link
Contributor

Checked the staged zip again, the user guide PDF I expect to be there under docs (where it is in all other TCKs) isn't; the only things in there are the exclude list and assertions. A search for any PDFs within the entire archive also turns up nothing.

Is it in a different format and located somewhere else within the zip?

@arjantijms
Copy link
Contributor Author

Hmmm, then we have to think about what to do with the ballot. If it would be okay to just update the zip with the docs that were technically already there (in the repo, but have magically not been included in the zip), or that we have to restart the ballot.

I also have to check what exactly happened. At one point I had a local zip file with the docs in it, but it's clearly not there now.

@jeanouii
Copy link
Contributor

@arjantijms Ballot is now canceled.
Can you please re-roll the TCK and then update the SHA everywhere, including updating the CCR.
You will have to run the compatible implementation against the new TCK ZIP.

Lemme know if you need something

@arjantijms
Copy link
Contributor Author

@jeanouii I'll re-roll the TCK, thanks!

@jeanouii
Copy link
Contributor

@arjantijms When do you think you can update the TCK and following PRs?

@arjantijms
Copy link
Contributor Author

I'd like to get it done before the weekend, or at the very latest this Saturday.

@jeanouii
Copy link
Contributor

Sorry @arjantijms . Don't want to be a pain but clock is running and we need some days to create the ballot and finalize the release. Do you think you can still do it or should we find a plan B?

@arjantijms
Copy link
Contributor Author

@jeanouii @Pandrex247 The updated TCK is staged here: https://download.eclipse.org/ee4j/jakartaee-tck/jakartaee10/staged/eftl/jakarta-security-tck-3.0.0.zip

I checked that it contained the user guide this time.

The SHA is now '696776046dfeaed74266a5d1c4dac7fea5437b6f51743b7fe10962dde755ff8f' and updated here: jakartaee/security#239 and here https://eclipse-ee4j.github.io/soteria/certifications/jakarta-security/3.0/TCK-Results

@jeanouii jeanouii added approved The ballot was approved by the Specification Committee and removed ballot Delivered to the Specification Committee for ballot release review Use this label on PRs that are filed for release review labels Jun 14, 2022
@jeanouii
Copy link
Contributor

Ballot successful
Proceeding with remaining steps

@jeanouii
Copy link
Contributor

jeanouii commented Jun 14, 2022

On ballot completion, the specification committee mentor:

  • adds this final checklist to the main PR.
  • adds the approved label to the PRs, and sends out the Ballot Summary per this template to the public Jakarta EE Specification Committee email list
  • calculates the staged EFTL TCK signature and promotes it to the committee download area
    using the https://ci.eclipse.org/jakartaee-spec-committee/job/promote-release/ job. Manually editing the jenkins Build Information will help identify the build (ie. Mail 2.0 or CDI 3.0).
  • merges the specification (and apidocs) PRs, ensuring the "date:" field in the _index.md file has an appropriate value to allow publishing.
  • updates the specification page with the ballot results. This is normally done via a separate PR that should be reviewed, approved, and merged.
  • notifies the EMO of the ballot results by email to emo@eclipse-foundation.org. Just forward the ballot summary note sent earlier to the public Spec Committee email list.
  • creates an issue in the specification project that includes the following checklist for the specification project team: Finalize the Jakarta Security 3.0 Release security#252

@jeanouii jeanouii merged commit 8b39606 into jakartaee:master Jun 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved The ballot was approved by the Specification Committee
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants