Expand validation to annotations

[jchunkins]

This is the first step in implementing issue operator-framework/operator-registry#568

I started with the annotations that I knew were problematic (e.g. using wrong case olm.skiprange when the annotation should be olm.skipRange) and also searched for other case sensitive annotations I could find. I also expanded the validation types to include OperatorGroups as well since it can also use case sensitive annotation names. Note that OperatorGroups are not directly hooked up to the command line execution of operator-verify (similar to how PackageManifestValidator is not accessible from the command line either).

I intend to open up new PRs in operator-registry once this is merged here so we can pick up the changes for CSV validation.

• Add validation function for well known case sensitive annotation names
• Use new validation function in CSV and OperatorGroups
• Add minimal OperatorGroup validator calling new function
• Add / update unit tests and test data
• update usage readme

Signed-off-by: John Hunkins jhunkins@us.ibm.com

[dinhxuanvu]

Lookin good. Just a few comments.

[dinhxuanvu]

/lgtm

[awgreene]

Great work @jchunkins, left some non-blocking nits.

[awgreene]

/lgtm

[njhale]

Ah, this looks interesting! Although it does seem fairly limited; i.e. in order to catch similar mistakes (e.g. olm.OperatorGroups, olm.operator-group, etc), we need to add more custom logic.

Have you considered something more generalized? maybe this could use a string distance algorithm (see agnivade/levenschtein) to catch non-exact matches and suggest the closest correct key?

[jchunkins]

@njhale I appreciate the idea and can see how that can make this more useful. However, considering a crawl/walk/run approach, maybe we start off with this implementation before committing to additional complexity.

My primary concern would be introducing false positives when you start handling more complex scenarios. A secondary concern would be the computation time cost involved during validation (which admittedly might not be that large). Right now we know that we have real world examples of folks making simple mistakes with case sensitivity being the root problem.

[benluddy]

Thanks for your contribution! I don't have a problem with a specific case-sensitivity rule like this one.

It might eventually be useful to validate that any annotation key beginning with "olm." (?) can be found on an annotation allowlist, which would cover the other mistakes Nick brought up. That would take a bit of extra effort to offer a "did you mean" like the one your patch provides, though.