Automation/process support needed for PHEPs

[jtniehof]

I'm opening this issue to capture technical and process implementation needed to support the PHEP process (#22). The technical details of implementation should be able to change without having to revisit the PHEP so it seems worth separating that implementation discussion out. Opening here because it's likely to interact with #23, but a fair bit of implementation probably will live in heliophysicsPy/heliophysicsPy.github.io.

TODO list (is anything missing?)

• Index of PHEPs on the main website
• Auto-builder of PHEPs, maybe to put them on the website?
• Minting of DOIs as PHEPs are Final. Needs to be tied to the file not just the repo, but should also connect to the repo/commit ID somehow. Do we just upload the file itself? MD, PDF, both? (Note that there may be supplemental files beyond the MD itself.)
• Handling of DOI updates for "superseded", this can be punted a bit to the future but don't want to paint ourselves into a corner now....
• Auto-checking of formatting, spelling, proofreading...
• Set up the phep-editors group and permissions
• Merge vs. rebase (should not do squash commit because history should be maintained)

[jtniehof]

@sapols , I suggest "assigning" this to you although I hope you'll be able to get some community input for the spadework.

[sapols]

@jtniehof agreed, assigned to myself. And thanks for making this separate issue for these discussions. I agree it makes more sense to discuss here.

Only thing I can think to add to the TODO list would be automated proofreading/spellchecking. Kinda gold plating at this point, but it's something I'd like to have.

[rebeccaringuette]

You could use Zenodo to create the PHEP DOIs. The file can be uploaded to Zenodo, the link to the version of the file(s) in that commit can be included both in the description and in the metadata fields (e.g. Alternate Identifier, URL; or Related Works, DOI for the correct version of the repository, which will need to be generated via the Zenodo-GitHub link). The version/commit ID of the file(s) can also be included in the metadata and in the description, which provides an easy way to handle superceded items.

Item to add: Instructions are needed for how to perform the correct actions and in the right order (e.g. the 'not squash commit' note). Maybe even a workflow in GitHub developed for this to decrease human errors.

[jtniehof]

Walkthrough of the Zenodo/DOI process, to put some meat on Rebecca's comments:

1. Select "new upload" (plus sign in top bar), or https://zenodo.org/uploads/new
2. Pick "no" under "Do you already have a DOI"--at this point, a DOI can be reserved without doing the upload.
3. Resource Type should probably be "Publication / Standard"
4. Title --PHEP # plus the actual title? Just the title?
5. Creator -- probably the author(s)
6. Description -- should set some standards for this
7. License -- CC0 (there is no "public domain" option explicitly)
8. Contributors -- ? Editors?
9. There's the possibility to add a set of dates, for milestones like "Created" and "Accepted"--does it matter?
10. Version: should be the revision number
11. Related works to point to the repo: I suggest this is "Is variant form of", Scheme should be "URL", Identifier is the github identifier of the tag, so it's going to be something like https://github.com/heliophysicsPy/standards/releases/tag/phep-xxxx --I haven't done much tagging on GH without making releases, it may make sense to make a release as each PHEP is approved. Alternatively, or in addition, could have "Is variant form of" pointing straight to the commit itself and the Markdown file, e.g. https://github.com/heliophysicsPy/standards/blob/4f8a6c0dd90e9fa94e3c49107e5c4574c80941ad/README.md (but that will not include any auxiliary files, e.g. images). I'm thinking both
12. Related works for revisions of a single PHEP: Use "Is new version of", scheme DOI, and the Zenodo DOI of the previous revision. On the previous revision, add a new related work of "Is previous version of" and put in the new Zenodo DOI (this field should be updatable after the DOI is minted). Type is again Publication/Standard.
13. Related works for PHEPs that replace: Use "Obsoletes", scheme DOI, and the Zenodo DOI of any PHEPs that are replaced; similarly update their records to "Is obsoleted by" and the new PHEP's Zenodo DOI. Type is again Publication/Standard.
14. Repository URL: this is in the "software" category but I think we should still use it and point to https://github.com/heliophysicsPy/standards/
15. Programming language: Markdown
16. Repository status: Active
17. Add it to the PyHC community, which I forget how to do--I think it's done after the DOI is minted. EDIT: Nope, there's a "select a community" button at the top, above the files

All that can (and probably should) be saved as draft before files are uploaded.

I'm not sure if it makes more sense to just upload the PDFs, or also include the source (markdown + other).

I think it probably makes most sense for this to be an editor's job, but the author could in principle do it.

Finally, I can write all this up--I suggest we have a RELEASE.md (or a better name) in the standards repo that documents this process.

[rebeccaringuette]

Great outline, Jon. For #9, the created date can be the date when the first draft is completed and the accepted date can be the date when the final vote to accept the standard is completed. Note that the accepted date can be added at a later time in a revised version of the PHEP on Zenodo. Files can be uploaded at any time in the creation process. Note that a DOI is created per version and that a 'net DOI' can be used to refer to the entire set, defaulting to the latest version. If you do this process from GitHub via releases, it looks a bit different. I think either way will work, but maybe we should prefer the way you outlined due to the greater flexibility (e.g. can point to a specific PHEP file instead of a whole repository).

…

On Mon, Feb 12, 2024 at 3:00 PM Jon Niehof ***@***.***> wrote: Walkthrough of the Zenodo/DOI process, to put some meat on Rebecca's comments: 1. Select "new upload" (plus sign in top bar), or https://zenodo.org/uploads/new 2. Pick "no" under "Do you already have a DOI"--at this point, a DOI can be reserved without doing the upload. 3. Resource Type should probably be "Publication / Standard" 4. Title --PHEP # plus the actual title? Just the title? 5. Creator -- probably the author(s) 6. Description -- should set some standards for this 7. License -- CC0 (there is no "public domain" option explicitly) 8. Contributors -- ? Editors? 9. There's the possibility to add a set of dates, for milestones like "Created" and "Accepted"--does it matter? 10. Version: should be the revision number 11. Related works to point to the repo: I suggest this is "Is variant form of", Scheme should be "URL", Identifier is the github identifier *of the tag*, so it's going to be something like https://github.com/heliophysicsPy/standards/releases/tag/phep-xxxx --I haven't done much tagging on GH without making releases, it may make sense to make a release as each PHEP is approved. Alternatively, or in addition, could have "Is variant form of" pointing straight to the commit itself and the Markdown file, e.g. https://github.com/heliophysicsPy/standards/blob/4f8a6c0dd90e9fa94e3c49107e5c4574c80941ad/README.md (but that will not include any auxiliary files, e.g. images). I'm thinking both 12. Related works for revisions of a single PHEP: Use "Is new version of", scheme DOI, and the Zenodo DOI of the previous revision. On the previous revision, add a new related work of "Is previous version of" and put in the new Zenodo DOI (this field should be updatable after the DOI is minted). Type is again Publication/Standard. 13. Related works for PHEPs that replace: Use "Obsoletes", scheme DOI, and the Zenodo DOI of any PHEPs that are replaced; similarly update their records to "Is obsoleted by" and the new PHEP's Zenodo DOI. Type is again Publication/Standard. 14. Repository URL: this is in the "software" category but I think we should still use it and point to https://github.com/heliophysicsPy/standards/ 15. Programming language: Markdown 16. Repository status: Active 17. Add it to the PyHC community, which I forget how to do--I think it's done after the DOI is minted All that can (and probably should) be saved as draft before files are uploaded. I'm not sure if it makes more sense to just upload the PDFs, or also include the source (markdown + other). I think it probably makes most sense for this to be an editor's job, but the author could in principle do it. Finally, I can write all this up--I suggest we have a RELEASE.md (or a better name) in the standards repo that documents this process. — Reply to this email directly, view it on GitHub <#24 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALX7QXQRMZ2ZLPVLOY75CCLYTJYGTAVCNFSM6AAAAAA67WAPR2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZZGQ3DMOJXGI> . You are receiving this because you commented.Message ID: ***@***.***>

[jtniehof]

Ah, thanks for mentioning the per-version ("version") and overall ("concept") DOIs, we definitely should use that for revisions of a PHEP but not, I think, for "replacement" PHEPs. Details here and I'll do some updates.

Do we want the main cite of a PHEP to be "version" or "concept" DOI? I can see advantages either way, and would require some tweaking to PHEP-1 text.

I don't think the automatic GitHub release integration is what we want here, since it's usually tied to the entire state of the repo.

[rebeccaringuette]

Agreed, the automatic GitHub pathway is too broad for the PHEPs. Well, the 'concept' DOI redirects to the most recent version of the item by default, even though it is created from the beginning, so the ideas are intertwined a bit. But, it is generally best practice to cite the most recent version of an item *unless* the desire is to point to a specific version. For example, if someone wanted to cite a specific version of the item screenshotted below, they could choose the DOI for the specific version they wish to cite instead of the general one. More info here <https://zenodo.org/help/versioning>. What are others' thoughts on this? [image: image.png]

…

On Tue, Feb 13, 2024 at 9:47 AM Jon Niehof ***@***.***> wrote: Ah, thanks for mentioning the per-version ("version") and overall ("concept") DOIs, we definitely should use that for revisions of a PHEP but not, I think, for "replacement" PHEPs. Details here <https://help.zenodo.org/faq/#versioning> and I'll do some updates. Do we want the main cite of a PHEP to be "version" or "concept" DOI? I can see advantages either way, and would require some tweaking to PHEP-1 text. I don't think the automatic GitHub release integration is what we want here, since it's usually tied to the entire state of the repo. — Reply to this email directly, view it on GitHub <#24 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALX7QXUWEP4TFADWV5WH3M3YTN4HDAVCNFSM6AAAAAA67WAPR2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBRGY3TSMZUG4> . You are receiving this because you commented.Message ID: ***@***.***>

[jtniehof]

Sorry, by "main cite" I mean "the cite that we include in the copyright statement of the PHEP".

[jtniehof]

With my limited Jekyll knowledge, I'm thinking the best way to get these onto the main website is going to be something like:

• Set up a data table and index page much like for the projects. This is going to involve putting some metadata in twice--once in the PHEP itself in the standards repo, and once in the YAML table in the website repo
• Either incorporate standards as a git submodule in the main website repo, so that the markdown is rendered directly, or have the index page link to the standards blob and Zenodo DOI, as is done for the current standards

[rebeccaringuette]

I imagine the YAML table could be generated from the PHEP in the standards repo.
I prefer the index page link approach.

[jtniehof]

I'm getting to be more in favor of not using the Zenodo GH integration after tripping over it with the latest SpacePy release--if it doesn't like your metadata after release you're sort of stuck. That doesn't preclude some other automation, of course.

Would our .zenodo.json cause problems? Is Zenodo integration already set up for this repo?

[rebeccaringuette]

Zenodo likely uses the DataCite DOI API to create the DOIs, and there is nothing preventing us from using the API ourselves. It does require an account, which may be an HDRL question (given the funding chain). However, there would need to be some form of protection layer to avoid skyrocketing fees for generating DOIs (which are NOT free, btw, but only about $10 each I think). The method to obtain a DOI doesn't seem to be integral to the PHEP process itself, though. We can chat about this separate from the vote.

[jtniehof]

The method to obtain a DOI doesn't seem to be integral to the PHEP process itself, though. We can chat about this separate from the vote.

Agreed--I figured there would be an API option later, for now I'm just going to document a procedure and we can find automation opportunities as we go.