Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add citation file #695

Merged
merged 12 commits into from Mar 8, 2023
Merged

Add citation file #695

merged 12 commits into from Mar 8, 2023

Conversation

glatterf42
Copy link
Member

@glatterf42 glatterf42 commented Mar 2, 2023

Add a citation file of format CITATION.cff to address issue #691.

How to review

To review, please

  • Check that the information in the file is corrected
  • Check that it is formatted correctly (i.e. behaves as is intended)

PR checklist

  • Continuous integration checks all ✅
  • Add or expand tests; coverage checks both No change in behaviour.
  • Add, expand, or update documentation.
  • Update release notes.

@codecov
Copy link

codecov bot commented Mar 2, 2023

Codecov Report

Merging #695 (b54d6d0) into main (139c832) will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##            main    #695   +/-   ##
=====================================
  Coverage   94.9%   94.9%           
=====================================
  Files         42      42           
  Lines       3400    3400           
=====================================
  Hits        3229    3229           
  Misses       171     171           

@glatterf42 glatterf42 marked this pull request as ready for review March 2, 2023 12:43
Copy link

@phackstock phackstock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good to me, a few small comments below in line.

CITATION.cff Outdated Show resolved Hide resolved
CITATION.cff Outdated Show resolved Hide resolved
CITATION.cff Outdated Show resolved Hide resolved
CITATION.cff Outdated Show resolved Hide resolved
CITATION.cff Outdated Show resolved Hide resolved
Copy link
Member

@danielhuppmann danielhuppmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, going in the right direction!

CITATION.cff Outdated Show resolved Hide resolved
CITATION.cff Show resolved Hide resolved
CITATION.cff Show resolved Hide resolved
CITATION.cff Outdated Show resolved Hide resolved
CITATION.cff Outdated Show resolved Hide resolved
@glatterf42
Copy link
Member Author

Do we need to update https://docs.messageix.org/en/stable/notice.html or the corresponding NOTICE.rst in the main repo? The information there is still correct, we might still want to add a section about using our new file for citation. I don't yet know how to do that, but I could read up on it. However, if you want it done asap and have worked with cff files before, you might want to take on that part.

@glatterf42 glatterf42 self-assigned this Mar 3, 2023
@glatterf42 glatterf42 linked an issue Mar 3, 2023 that may be closed by this pull request
@danielhuppmann
Copy link
Member

Good idea @glatterf42! I suggest to add a reference/link to the citation file as an item in the list of section 2.

Copy link

@phackstock phackstock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks all good to me.
One small final side note, maybe you could write somewhere that this file needs to be updated with every release as it contains a direct reference to the version number.

Copy link
Member

@danielhuppmann danielhuppmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having the version hard-coded in the citation file means that this has to be changed explicitly as part of the release-PR. In pyam, there is a specific Release Procedure - does this exist for message_ix as well?

Please add an reminder to change the citation.cff to the release-procedure document - or better to not state the version explicitly in this file at all.

cc @khaeru

CITATION.cff Outdated Show resolved Hide resolved
CITATION.cff Outdated Show resolved Hide resolved
@glatterf42
Copy link
Member Author

I think it might be useful to include some version information. Future versions might not be able to reproduce results, so project scientists might want to pinpoint specific versions as the ones that worked for them. On that regard, I've found a Github Action to automatically update the citation file, so we should be able to include this information without needing to worry about manual updates. The only downside is that I haven't really worked with Github Actions yet, so I'll have to spend some time understanding how they work, but then I should be able to adapt this workflow to our needs.
The question would then be: do we want a whole version only or also single commits as they are more accurate?

@danielhuppmann
Copy link
Member

But the version-information is already included in your workflow/script simply by doing message_ix.version (including the commit hash if you're in a non-release state) - I'm very skeptical of duplicating this information.

@glatterf42
Copy link
Member Author

On that note, the pyam citation file also contains a version keyword, referencing version 1.0.
I think this is again a matter of what the use case is: I thought we wanted to include this citation file so that people can simply cite MESSAGEix correctly from the main repo e.g. by extracting a Bibtex entry. This entry will only include version information if we put it in the citation file, if I understand correctly.

@danielhuppmann
Copy link
Member

On that note, the pyam citation file also contains a version keyword, referencing version 1.0.

True, but the pyam-citation refers to the manuscript in Open Research Europe, which is specifically linked to v1.0 of the package. But this could obviously lead to some confusion there, too...

Change version to valid type

Reintroduce commit parameter

Add workflow to automatically update both
@glatterf42
Copy link
Member Author

So: in part, I just wanted to see if I could do it. Even if we don't end up using what I cooked up here, I have learned a lot about Github Actions in the process :)
Despite your explanation above, I still think we should include the version information in the citation file. Since we are going to provide one either way, I think this will make citing MESSAGEix much more comfortable. However, in order to be accurate, we then also need to include the commit that the citation is referring to. A current example is that pandas 2.0.0 will abandon some features that are still in use in version 3.6.0 of MESSAGEix, but have been fixes by a more recent commit. Thus, citing version 3.6.0 of MESSAGEix will lead to people being unable to reproduce results because they won't have the fixes included that every recent clone of the main repo has.
Updating the version and commit manually would be tedious, which is why I have written a Github workflow to automatically update them. With this workflow, every new release would trigger another commit that updates the citation file to include the newest version. Each push to the main branch would update the latest commit.
As @phackstock pointed out, this would lead to people not having the latest version of the citation file locally available even if they just pushed to the main branch without running git pull. So, if they were to use the local contents of the file, they might produce an incorrect citation. However, I don't know how frequently this would actually happen. If I understand correctly, the main purpose of CITATION.cff is to provide a citation option directly in the main Github repo. Anyone wanting to cite MESSAGEix might therefore visit our main repo and create e.g. a Bibtex entry with the up-to-date information from there. Furthermore, I think git pull is a quicker alternative. If people want to cite the code locally (and don't already have a script using message_ix.version), they'd either have to write such a script or simply run git pull. And if they already have such a script, the information in there will still be correct.

All in all, I think providing the version and commit information is comfortable, accurate, and not much work with the addition of the workflow.

cc @khaeru

 * Remove workflow to automatically update those
@glatterf42
Copy link
Member Author

Together with @khaeru, I just noticed that the automatically created Bibtex entry does not contain a lot of information. For starters, it's just one entry, so we can choose for it to be the software entry or the paper entry. Even if we choose the software entry, the only versioning information would be month and year of the "date-released" entry. So I also don't think there's a point to keep them automatically up-to-date for this feature. Maybe some of the other tools could make use of the data, but for now, I removed them from the citation file.
Since I still think the version and commit should be mentioned when citing software, I will adapt the corresponding docs section with how to do that.

Copy link
Member

@danielhuppmann danielhuppmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thank you!

@glatterf42 glatterf42 merged commit 3018e93 into iiasa:main Mar 8, 2023
@glatterf42 glatterf42 deleted the add_citation_file branch March 8, 2023 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a citation file
3 participants