Added repo validation based on project-blueprint

[maoo]

See finos/open-developer-platform#31 for requirements.

This PR introduces a new command called check-project-repos that generates a full report (finos-repo-validation.json) including each GitHub repository across all our orgs - add file
2. The script runs the list of validations below (including text generated in the report)
3. A repository can define a .finos-blueprint.json file to ignore one or more criteria (in case it doesn't apply), see below.

The output can be easily integrated with a GitHub Action, in order to output the results, for example creating a GitHub issue for each repository that has at least one issue flagged.

Validations

• has-admin - One or more admin collaborators were found in this GitHub repository.. FINOS Governance doesn't allow GitHub users to have Admin rights on repositories, therefore it must be removed.
• has-user - One or more user collaborators were found in this GitHub repository. FINOS Governance only allows GitHub users to be added via Teams. Please remove it, therefore it must be removed.
• disabled-issues - This GitHub repository does not have GitHub Issues enabled; make sure that there is a documented way to submit questions, feature requests and other communications to the project team.
• no-teams - This GitHub repository does not grant permissions to any FINOS Team, although it should be configured to grant access to the program and project specific teams defined in https://github.com/orgs/finos/teams. Please email help@finos.org and coordinate changes to the repository access permissions.
• no-issue-templates - This GitHub repository does not use issue templates; please check the issue template blueprints.
• no-contributing - CONTRIBUTING.md file is missing; check the CONTRIBUTING.md template."
• no-code-conduct - CODE_OF_CONDUCT.md file is missing; check the CODE_OF_CONDUCT.md template.
• notice-nok - NOTICE file is incomplete; check line 4 of the NOTICE template.
• no-notice - NOTICE file is missing; check the NOTICE template.
• no-readme - README.md file is missing; check the README.md template.
  no-description - This GitHub repository does not have a general description defined; the Edit button is seen when on the repositories main page, which is the Code tab.
• is-archivable - This repository belongs to project {{project-name}} which is archived, therefore the GitHub repository is expected to be archived too. @finos-staff will get in touch with the project lead to sort it out.
• readme-nok - README.md file is incomplete; check the README.md template and make sure that ## Contributing and ## License sections exist.
• no-badge - README.md file is missing the FINOS badge; check the README.md template and make sure that it embeds one of SVG FINOS badges.
• wrong-badge - Our internal records state that this project is in {{project-state}} state, whereas README.md states {{readme-state}}; make sure that README.md embeds the right FINOS badge."
  :repo-not-on-file "We don't have this repository on file. We will fix this issue on our side as soon as possible and keep you posted.
• no-whitesource - WhiteSource configuration was not found; make sure that dependencies are scanned against security vulnerabilities. Read more on the WhiteSource Wiki page.

.finos-blueprint.json example

{
  "ignore" : [
    "no-issue-templates",
    "readme-nok"
  ]
}

Testing check-project-repos

git checkout blueprint-check
lein run -- check-project-repos -m ../metadata

Here's a useful command to transform the output in CSV

cat finos-repo-validation.json| jq -r '.[] | [.org, .["repo-name"], .validations["has-admin"], .validations["has-user"], .validations["no-teams"], .validations["no-issues"], .validations["no-issue-templates"], .validations["no-contributing"], .validations["no-code-conduct"], .validations["notice-nok"], .validations["no-notice"], .validations["no-readme"], .validations["no-description"], .validations["is-archivable"], .validations["readme-nok"], .validations["no-badge"], .validations["wrong-badge"], .validations["repo-not-on-file"], .validations["no-whitesource"]] | @csv'

Sample output is available on https://gist.github.com/maoo/ec1bfbfa6e6d9b43fbe09b1f10337e5e

Syncing CLAs with GitHub membership

Based on the GitHub access control documented on our Wiki page, Project Collaboration on Github, we want to promote affiliation with github.com/finos org as much as possible.

For this reason we've built the invite-clas-to-finos-org command, which will pull all GitHub ids covered by a FINOS CLA and subtract existing members and pending invitations, then send them an invitation to join the org. The run is scheduled nightly, along with the other FINOS metadata batch processes.

IMPORTANT! The script doesn't trigger the invitation yet, as we want to first announce it to the community. /CC @mcleo-d

Testing invite-clas-to-finos-org

This command will send out invitations to github.com/orgs/finos/people to all metadata registered github ids that are covered by a CLA with FINOS.

git checkout blueprint-check
lein run -- invite-clas-to-finos-org -m ../metadata

[maoo]

Just added Syncing CLAs with GitHub membership to the issue description (/CC @mindthegab )

I've also added a no-teams validation, so that we'll be able to track repos who don't have the proper team configuration.

[tschady]

since this function is a nop, I would not yet include it.

[maoo]

It does print out the results, instead of sending actual emails, which is why I'd like to keep it. This way we can run the full test against the master definition of FINOS metadata.

Note that this feature is expected to be rolled out next week.

[tschady]

the matcher later on is case sensitive, so # CONTRIBUTING would fail.

[maoo]

I see that readme-content is defined as (str/lower-case readme), so we should be fine.

[mcleo-d]

@maoo and @tschady

I'm in awe of your collaboration. It's a great thread of work to follow 👍

James.

[mcleo-d]

Code review dome with @maoo 👍