Add test-reports workflow for develop and main with badges

[nate-credoai]

Describe your changes

test-reports.yml workflow will run on pushes to develop and main. It can also be triggered for other branches but the badges will not be updated. Badges are updated by pushing a generated badge to S3 that is referenced in the README.md. The README.md references the badges for the main branch but the URL to retrieve the badges for develop can be discovered easily.

The permissions to write to the S3 bucket are limited to this workflow running on the develop and main branch.

Examples of the badges generated for the last run on this branch before lockdown in AWS:

Issue ticket number and link

This was a request in Slack.

Known outstanding issues that are not fully accounted for

Checklist before requesting a review

• I have performed a self-review of my code
• I have built basic tests for new functionality (particularly new evaluators)
• If new libraries have been added, I have checked that readthedocs API documentation is constructed correctly
• Will this be part of a major product update? If yes, please write one phrase about this update.

Extra-mile Checklist

• I have thought expansively about edge cases and written tests for them

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
credoai
__init__.py	3	0	100%
credoai/artifacts
__init__.py	7	0	100%
credoai/artifacts/data
__init__.py	0	0	100%
base_data.py	105	12	89%	55, 153, 156, 171, 178, 185, 189, 193, 197, 209, 212, 219
comparison_data.py	63	13	79%	53, 60, 71, 76, 81, 90, 96, 100, 105, 114, 147, 153, 156
tabular_data.py	40	6	85%	52, 73, 77, 96, 98, 105
credoai/artifacts/model
__init__.py	0	0	100%
base_model.py	36	2	94%	52, 84
classification_model.py	20	0	100%
comparison_model.py	11	0	100%
regression_model.py	11	4	64%	43–45, 48
credoai/evaluators
__init__.py	15	0	100%
data_fairness.py	147	12	92%	83–90, 205, 260–261, 287, 311, 334–340, 356
data_profiler.py	34	2	94%	57, 60
deepchecks.py	40	3	92%	113–122
equity.py	153	31	80%	78, 181–184, 204, 230–257, 281–296, 307–309, 358–359
evaluator.py	70	8	89%	50, 58, 61, 80, 106, 126, 174, 181
fairness.py	145	12	92%	117, 238, 246–251, 310–319, 321, 333–336
feature_drift.py	59	1	98%	66
identity_verification.py	112	2	98%	144–145
model_profiler.py	74	12	84%	128–131, 145–148, 165, 182–183, 192–193, 231
performance.py	117	14	88%	110, 137–143, 230–239, 241, 258–261
privacy.py	118	4	97%	410, 447–449
ranking_fairness.py	134	14	90%	136–137, 157, 178, 184–185, 382–404, 409–439
security.py	96	1	99%	297
shap.py	87	14	84%	119, 127–128, 138–144, 170–171, 253–254, 284–292
survival_fairness.py	67	50	25%	29–33, 36–48, 53–64, 67–78, 81–99, 102, 105, 108
credoai/evaluators/utils
__init__.py	3	0	100%
fairlearn.py	18	2	89%	46, 59
utils.py	8	1	88%	9
validation.py	80	28	65%	14, 34–35, 37–39, 46, 67–74, 80–86, 89, 95–98, 105, 108, 111, 114–115, 119–121
credoai/governance
__init__.py	1	0	100%
credoai/lens
__init__.py	2	0	100%
lens.py	189	12	94%	173–174, 210–215, 272, 314, 338, 420, 435, 439, 451
pipeline_creator.py	60	12	80%	20–21, 37, 79–91
utils.py	39	28	28%	20–27, 49–52, 71–82, 99, 106–109, 128–135
credoai/modules
__init__.py	3	0	100%
constants_deepchecks.py	2	0	100%
constants_metrics.py	17	0	100%
constants_threshold_metrics.py	3	0	100%
metric_utils.py	24	18	25%	15–30, 34–55
metrics.py	60	7	88%	62, 66, 69–70, 73, 83, 120
metrics_credoai.py	115	44	62%	31–40, 45–47, 70–98, 114–117, 144, 168–169, 232–234, 310–316, 352–353, 423–424
stats.py	39	28	28%	11–14, 17–22, 25–27, 30–35, 38–52, 55–60
stats_utils.py	5	3	40%	5–8
credoai/utils
__init__.py	5	0	100%
common.py	102	40	61%	55, 68–69, 75, 84–91, 96–104, 120–126, 131, 136–141, 152–159, 186
constants.py	2	0	100%
dataset_utils.py	61	35	43%	23, 26–31, 50, 54–55, 88–119
logging.py	55	13	76%	10–11, 14, 19–20, 23, 27, 44, 58–62
model_utils.py	30	11	63%	14–19, 28–29, 34–39
version_check.py	11	1	91%	16
TOTAL	2698	500	81%

[esherman-credo]

The badges don't seem to render for me on the front page of the branch:

Is that simply because the branch has been removed from the workflow file?

[nate-credoai]

The badges don't seem to render for me on the front page of the branch: Is that simply because the branch has been removed from the workflow file?

Yes. I updated the README.md to reference the main branch as that is what it would be when we merge to main. The goal of this is to show the status of the workflow that is run against the stable/release branch which is what main is right now. I could entertain the idea that develop is more important to show at this time as it represents the next release. Unfortunately, markdown doesn't have variable substitution that would allow us to make it dynamic per branch. As I write that, I wonder if I can do something on the badge hosting side to server based on CORS...

[esherman-credo]

Sounds good, just wanted to confirm

[esherman-credo]

Lgtm