[Epic] Improve cross-project contributor experience

[Andersson007]

Problem statement

The Ansible Ecosystem is made of over 20 projects our engineering organization is the biggest contributor to.
Despite all those projects being part of the same ecosystem their contribution practices and experience vary widely.

Objective

To provide a well crafted, comprehensive and consistent Contributor Experience across the majority of the projects to enable contributors to easily be able to land and expand in our ecosystem.

I walked through all the projects. Below is what I found out.

Spotted patterns and thoughts

• Purpose and wide use applicability: Some projects are created and contributed to only by red hatters. It can indicate that they are probably used only by red hatters (for convenience) or maybe play some auxiliary very narrow-case role like receptor, ansible-sign, compat library and some others.
• Licenses: We can see a wide range of licenses used: Apache 2.0, GPLv3, GPLv2, MIT, LGPLv2, BSD. I’m not a lawyer but it looks like they are GPLv3 compatible. I think it’s not an obstacle to contributing, just an observation.
• Repo level docs:
  • README: Quality of README varies significantly. In some cases, it’s almost empty. Often, it’s outdated. There’s no common structure or at least a minimal set of required sections. For example, it could contain the following sections: Mission statement (it welcomes visitors to contribute), Communication (sync, async channels), Contributing, Versioning model used, etc. We could provide templates.
  • Issue: it’s often not clear how to open it.
  • CONTRIBUTING.md: in some cases contradicts with what is written in a doc site or is irrelevant. The fragmentation issue is also relevant here.
  • CoC: is often not mentioned or it’s only present like a badge which is not very visible, IMO. Even if not many reads it carefully, I think it’s worth mentioning it explicitly to emphasize that it does work in the project.
• Communication:
  • Chat channels: often no info or there’s only IRC
  • Async channels: almost no Forum, no info, or only ML/GH Discussions
• Doc sites:
  • Some projects don’t have a docsite or its pages are in the WIP state not filled with content.
  • The contributor guide is often missed or very poor. Doc and other non-code forms of contributions are not mentioned at all.
  • The dev guide is often missed as well as testing and maintainer guides. Project architectures are not explained at all. No quick-start guides that lead newbies through the whole process.
  • Maybe a recommended docsite structure should be developed and offered?
• Versioning, releasing, and changelog:
  • No release plan at all in most of the projects. No version support timeline.
  • In most cases, it’s not clear which versioning model (e.g. SemVer) is used and how it works
  • Changelogs are often a mess
  • One project even doesn’t use tags
• Community involvement:
  • Community maintainers/committers: are present only in collections (not saying it’s bad, just a fact).
  • Community contributors: absent entirely in ⅓-½ of the projects (even no issue reporters); in many others are present only as issue reporters.
• Testing frameworks (which is a barrier for cross-project participation if there are different ones used): in our Python projects it’s Pytest everywhere, so should be fine.
• Governance: no governance info (except some collections). Which is expected as the projects are maintained and decisions are made by red hatters.

Possible solutions to some common issues

1. Should we come up with a common README template with a minimal required set of sections and submit PRs across selected projects (will be most of them)? I think any recommendations will not work and we should raise the PRs ourselves.
2. Develop common docsite section templates and submitting PRs? Example: a section template for “Communication”. Maybe even a common docsite structure for projects?
3. Make sure roles of subprojects of bigger projects are explained in those bigger projects and in many cases, I think, independent contributors have just no idea they exist and what they are for.
4. We should submit issues in selected/all the repos to remind folks about the Forum (with a link to the group request guide) and Matrix.
5. Develop a common approach and submit PRs to fix Versioning, releasing, and changelog. They IMO don’t have to be the same in all the projects but it should be at least clear how it works.

I think we should try to come up with a more or less global solution, not trying to tackle small bits incrementally right after they come to mind. I think we could use the project_template repo to polish stuff like, say, the README template or docsite file/section structure trying to engage the broader community in the process.
Also we should do (or at least initiate in PR form) most of the work themselves -and resp first come up with solutions that can be done by us- as it’s practically hard to make maintainers do anything even if they agree.

Any inputs would be much appreciated. I could start coming up with implementations ASAP.

Sub-tasks:

• [Task] Cross-projects contributor exp: Create two lists of tasks #419
  • Related google doc
• [Task] Cross-project contributor experience: create README template #420
• [Task] Cross-project contributor exp: Define comprehensive, ready-to-use docsite template in project_template #422
• [Task] Cross-project contributor exp: Auto-publishing of the project_template repo docs content on readthedocs #423
• [Task] Cross-project contributor exp: do research how to make the projects use the docsite template and README #526
• [Task] Cross-project contributor exp: ask the community #538

Related discussions:

• Ecosystem page content forum discussion

[Andersson007]

cc @cybette @Spredzy @GregSutcliffe @oraNod @samccann @anweshadas @leogallego please share your thoughts:) If no thoughts, feel free to ignore:)

[samccann]

@Andersson007 we have the start of recommendations in the Project Onboarding toolkit (including a recommended docs structure), and the github project template that matches it.

[samccann]

On the things we'll have difficulty with - we can recommend projects follow the github project template and docs structure, but we don't have the authority to ensure it happens so to speak. We lack an overarching governance model that could help enforce these items.

[Andersson007]

@samccann thanks for the links i'll take a look

On the things we'll have difficulty with - we can recommend projects follow the github project template and docs structure, but we don't have the authority to ensure it happens so to speak. We lack an overarching governance model that could help enforce these items.

Even if there's the gov model working, practically it'd be hard to force folks do things anyway imo. Speaking more radically, maybe we even shouldn't force them. Like that semver vs calver vs number-of-digits-in-calver thing - as long as a used versioning convention is documented and it's easy to find, is it really an issue preventing people from contributing? But it's another topic..

There are thing we can do ourselves now.

Let's take one example: one of the spotted issues is the lack of communication guides in the projects.
To solve this, we could come up with a template that will contain some cross-project channels (like Bullhorn and social Matrix channel, etc.) and leave space for project-specific ones once or fill up them two if we know).
Then we could add this as project communications guides. Once we have a README template, this should contain the Communication section containing a link to a project-specific communication guide. In this case every project will have a similarly structured part of README about communication guide and similarly structured communication guide on its docsite. I'm sure nobody will be against if we submit the PRs themselves. A real example is c.postgres communication guide we now use in several collections with minimal modifications (they don't have docsites, so we put it just in their READMEs).
Another example: it looks like the projects mostly follow SemVer but there's no explicit info about it. So there could be a section in the README template called Versioning convention.
This are just examples what we can do ourselves now.

So my proposal for now is:

1. I'll take a look at the repos you shared
2. I submit PRs to improve/change stuff if needed
3. I'll try to see what else can be done everywhere re-iterating through steps 1 and 2 if needed
4. Then i can submit PRs in all the projects to fix things

How about the plan?

[oraNod]

@Andersson007 Just reading through this and my thoughts are similar to what you and @samccann have already said. In here there are things we can do something about and other things that are purely up the respective project.

It would probably help to break items into categories of what we can control and take action there, meaning we can submit PRs for things like the Communication sections as you said. I'm willing to help with that side of things.

For things we can't control I think we could at least outline recommendations and notify teams, asking them to adopt or apply where possible.