workflows: Simplify testing

[jku]

• Don't try to handle sslib main test within the matrix
• Put it in a separate workflow
• Include the new workflow in CI but not in CD
• Bonus: Make cache-dependency-path more complete

Fixes #2604

---

[coveralls]

Pull Request Test Coverage Report for Build 8701688096

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 97.592%

---

Totals
Change from base Build 8700988091:	0.0%
Covered Lines:	1466
Relevant Lines:	1491

---

💛 - Coveralls

[jku]

@lukpueh what do you think?

• should fix the dependency issue
• workflow now does not need a Github PHD to modify
• PRs still get a "some checks were not successful" which is a little annoying for external contributors but I think it's ok: we do want know if things break with sslib main...

[jku]

Hmm, the check names now changed somehow -- it's different from what the expected check names are?

Maybe:

• "CI" workflow only had one job previously: GitHub decides to not include "CI" in the name
• but now there are two jobs in "CI" so Github includes the name?

Github tries to be helpful and manages to be annoying.

[lukpueh]

Hmm, the names now changed somehow -- it's different from what the expected test names are?

Yes, we just have to uncheck old expected checks and check new ones.

Maybe:

• "CI" workflow only had one job previously: GitHub decides to not include "CI" in the name
• but now there are two jobs in "CI" so Github includes the name?

Could be. It also doesn't include the toxenv py in the name anymore.

[jku]

It also doesn't include the toxenv py in the name anymore.

Oh I missed that!

[lukpueh]

Looks great!

Do I understand correctly that we don't need to set continue-on-error, because default is false and so a failure in tests, which is included in cd.yml and needed by build, blocks build, but a failure in sslib-main, which is is not included or needed, does not?

[lukpueh]

Hmm, the names now changed somehow -- it's different from what the expected test names are?

Yes, we just have to uncheck old expected checks and check new ones.

IIUC, right now, I can only un-require the old checks, but the settings UI won't show me the new ones until the PR is merged. I suggest we merge as is, bypassing the branch protection, and then I migrate the settings.

[jku]

Do I understand correctly that we don't need to set continue-on-error, because default is false and so a failure in tests, which is included in cd.yml and needed by build, blocks build, but a failure in sslib-main, which is is not included or needed, does not?

Yes that's my understanding.

For CI workflows:

• CI workflow will fail if _test_sslib_main.yml fails
• but that's ok since the "required statuses" feature uses specific status checks and not the workflow state

[jku]

merged, I'll modify the required checks now...