Automate reporting for BROKEN libraries

[SableRaf]

Right now, broken contributions are only visible in contributions.yaml through the status: BROKEN flag.

If a contribution go missing, we have to wait for someone to report that they don't appear in the contribution manager (like here for example)

It would be helpful to automate the reporting process when a library is first marked as BROKEN.

For example, we could open an issue automatically when a library is reported as BROKEN for the first time.

On a related note, we could add optional fields in contributions.yaml and submission issue template for the repository URL and GitHub username(s) of the author(s). This would make it easier to tag maintainers when their contribution goes down.

Would love to hear thoughts on this. Happy to help if we decide to move forward!

[mingness]

Great idea @SableRaf , it would really help keep an eye on the state of the contributions.
In addition to this, I'd want to check the code that pings the urls, to make sure that it's giving the url a chance. But what you are suggesting, the surfacing of the "broken" urls, will help in identifying the cases when the ping fails.
I think a great first step would be to create issues when a library is "broken" just to surface to ourselves that this happened. only when we are completely happy that the code is fully robust, let's notify maintainers.

with regards to the optional field for the github username in the issue template, which is then added to the contributions.yaml, that sounds good to me.

[SableRaf]

Thanks @mingness!

I made a PR to update the issue templates, but then realized that since we generate contributions.yaml from the properties files, the properties files should be the source of truth for these new fields as well. Right?

We could add the following to release.properties in the processing-library-template repo:

# (Optional) A URL to the source repository for your library (e.g., https://github.com/yourUser/yourLibrary).
repository_url=

# (Optional) Your GitHub username. Useful for linking contributions.
github_username=

# (Optional) The SPDX identifier for your library's license (e.g., LGPL-2.1-or-later).
license_spdx=

Then could add optional properties to add_new_contribution_to_yaml.py something like so (excuse my Copilot, I'm not too familiar with Python):

# Define a mapping of properties to YAML fields
optional_fields = {
    'repository_url': 'repositoryUrl',
    'github_username': 'githubUsername',
    'license_spdx': 'licenseSpdx',
}

# Add optional fields if present
for prop_key, yaml_key in optional_fields.items():
    value = props.get(prop_key)
    if value:
        props[yaml_key] = value
        del props[prop_key]

Let me know if I'm on the right track!

[mingness]

I agree that we want the library template to collect this information. I already added an issue to the processing-library-template regarding the licensing, processing/processing-library-template#87

I think it's good if we get this information from multiple pathways. For instance, not everyone is going to port their library over to the new template, or use our template to create a new library. In that case they may not know about the extra fields we want to populate, and we can collect this information when they submit their new library. But yes, if the library properties has this information, that is the source, and that is best. What do you think? Do you think the repetitive collecting of information will be annoying?

Wrt the file add_new_contribution_to_yaml.py, nothing needs to be added.
line 16 creates the dictionary from the json, which contains the new fields. The extra processing is only needed for specific fields. line 48 then takes all the fields in props and adds them to contributions, including our new fields.

The script that parses the properties file and creates the json also shouldn't need to be changed, since we tell Pydantic to accept extra fields

[SableRaf]

Thanks @mingness for your feedback!

I think it's good if we get this information from multiple pathways. For instance, not everyone is going to port their library over to the new template, or use our template to create a new library. In that case they may not know about the extra fields we want to populate, and we can collect this information when they submit their new library. But yes, if the library properties has this information, that is the source, and that is best. What do you think?

Makes sense. We would need to make it clear to the user in both the form and the release.properties which source takes precedence in case the two values don't match.

Wrt the file add_new_contribution_to_yaml.py, nothing needs to be added.
The script that parses the properties file and creates the json also shouldn't need to be changed

Noted. Thanks for clarifying!

I already added an issue to the processing-library-template regarding the licensing, processing/processing-library-template#87

The issue only talks about the license. Should I make a separate issue for adding the repo URL and author's GitHub handle?

[mingness]

Makes sense. I think, the priority should be properties file, and then issue field?

Feel free to revise the existing new issue to include other information?