Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Author, year citations #1090

Open
amichuda opened this issue Nov 10, 2020 · 34 comments
Open

Author, year citations #1090

amichuda opened this issue Nov 10, 2020 · 34 comments
Labels
enhancement New feature or request

Comments

@amichuda
Copy link

Currently, jupyter-book uses sphinxcontrib-bibtex for creating citation keys in text using the cite directive.

However, for academic citations (at least for my field) the convention is to use an author, year citation key.

There's an issue open about this on sphinxcontrib-bibtex: mcmtroffaes/sphinxcontrib-bibtex#201

I think it comes down to just creating a custom citation class using pybtex.

I think this is something in my wheelhouse to do a PR for if it's something you all think has value.

Would this be a matter of just adding it to some jupyter-book conf.py? And then registering it somehow?

@amichuda amichuda added the enhancement New feature or request label Nov 10, 2020
@welcome
Copy link

welcome bot commented Nov 10, 2020

Thanks for opening your first issue here! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.

If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).

Welcome to the EBP community! 🎉

@choldgraf
Copy link
Collaborator

I am not sure exactly how to do this, but it would be great to see it done! If you can figure out how to get sphinxcontrib-bibtex to display more custom outputs, then we can write up some docs as a pattern for users that wish to control this. (perhaps we could also upstream some improvements for certain documentation styles to sphinxcontrib-bibtex?)

@stefanuddenberg
Copy link

Seconded; it would be nice to have APA style citations in particular for my own work.

@jstac
Copy link
Contributor

jstac commented Nov 10, 2020

+1, we use author (year) in economics. Thanks @amichuda, would be very happy to have this.

@choldgraf
Copy link
Collaborator

Y'all should +1 (👍) the top comment in this issue so that it shows up on the issue tracker ;-)

@stefanuddenberg
Copy link

I was able to integrate the code for this package into pybtex, but there are still two issues I need to resolve: (1) removing the square brackets from the in-line citation, and (2) removing the citation 'name' from the entry in the bibliography. Pictures of odd formatting below.

PDFXEdit_GkBwHlFIu8
PDFXEdit_69O3oqdtQh

@amichuda
Copy link
Author

I was able to integrate the code for this package into pybtex, but there are still two issues I need to resolve: (1) removing the square brackets from the in-line citation, and (2) removing the citation 'name' from the entry in the bibliography. Pictures of odd formatting below.

That's great! I've been playing around with inheriting from the alpha class and I've run into the same issue. Is there a way I can take a look at the code?

My main question, though, right now is where to throw in the code... From reading the code, I take it that it has to be integrated somehow into default_config.yml? Or is there something in commands/ that can append the lines into the final config?

Sorry for the dumb question. Based on the issue above, you're just supposed to include the overriden classes into conf.py, so what would be the best way to do that?

@stefanuddenberg
Copy link

@amichuda There is an associated PyPI package for that repo I linked earlier. You can try installing that, but it didn't work for me when I tried it some months ago. Instead, I took all of the relevant files from that repo and put them into the corresponding folders in the pybtex folder for my conda environment. So formatting/apa.py went into pybtex/formatting/apa.py, and so on. I think that's all I had to do, and then it knows about the apa style when I call for the book to be built. Still having trouble figuring out the bracket and extra citation key issue though...

@stefanuddenberg
Copy link

stefanuddenberg commented Nov 11, 2020

pip install pybtex-apa-style works for me now. My bracket issue remains, however...

@amichuda
Copy link
Author

My bracket issue remains, however...

Checking the source of sphinxcontrib-bibtex, I think it's hard-coded. See issue I opened up here. Assuming I read the code right, it doesn't seem like something that pybtex could overcome right?

mcmtroffaes/sphinxcontrib-bibtex#203

@stefanuddenberg
Copy link

stefanuddenberg commented Nov 11, 2020

I stumbled across the same piece of code as well; I think you are right. Not sure whether there's an alternative to making a separate fork of sphinxcontrib-bibtex at this point.

@choldgraf
Copy link
Collaborator

Perhaps we could make some upstream contributions to sphinxcontrib-bibtex?

@stefanuddenberg
Copy link

Would be happy to help with that; putting my wish-list for publication-ready articles on here for posterity's sake, as I imagine they're all related to sphinxcontrib-bibtex's particular implementation (I understand that these would require separate issues to be raised):

  • bracketless references (this will allow multiple citations to be present in the same line for free)
  • bibliography without redundant in-line citation key before each reference
  • custom title for bibliography (e.g., "References")
  • parenthetical citations (e.g., "AuthorName (YYYY) found that...")
  • custom line spacing of the document (e.g., single spacing, double spacing)
  • custom page alignment for headings (e.g., left, center, right)

@choldgraf
Copy link
Collaborator

Nice - thanks for that wishlist, do you think you could make a new issue out of it? I think it's valuable to track general lists of improvements we want to see to citations support.

In general my approach is that we should upstream everything as much as possible. If we hit an impasse with sphinxcontrib-bibtex then we can discuss creating a new implementation as a last resort

@jobovy
Copy link

jobovy commented Dec 15, 2020

In case it helps, author-year citations of various forms are supported in a very hacky way by my sphinx-astrorefs Sphinx extension:

https://github.com/jobovy/sphinx-astrorefs

It's really way too hacky to incorporate here or in sphinxcontrib.bibtex, but the pybtex author-year and sorting style that's defined might be useful longer term.

@amichuda
Copy link
Author

In case it helps, author-year citations of various forms are supported in a very hacky way by my sphinx-astrorefs Sphinx extension:

Uhhhh that actually seems kind of perfect...

How did you get rid of square brackets when dealing with inline citations?

It's really way too hacky to incorporate here or in sphinxcontrib.bibtex, but the pybtex author-year and sorting style that's defined might be useful longer term.

Can you explain what's hacky about it. From a cursory glance at the description, it seems like a custom style using pybtex no?

The author of sphinxcontrib.bibtex is starting to do the work of incorporating changes that would sort of get it where your extension already is.

mcmtroffaes/sphinxcontrib-bibtex#213

@jobovy
Copy link

jobovy commented Dec 15, 2020

It's hacky, because it takes the HTML and LaTeX outputs and processes them to replace the square brackets with parentheses or nothing. It seems to work for now, but such a hack is probably not very long time stable wrt changes to the way Sphinx generates the HTML/LaTeX and it may also not work for all Sphinx themes. I think the proper way would be to create new Sphinx roles for :citep:, :citet: and :citealt roles, but that's currently beyond my pay grade 😄

I saw the mcmtroffaes/sphinxcontrib-bibtex#213 PR, but that doesn't allow for the full set of different citation commands that I need.

@amichuda
Copy link
Author

I saw the mcmtroffaes/sphinxcontrib-bibtex#213 PR, but that doesn't allow for the full set of different citation commands that I need.

Agreed, but that PR would be able to help at least with the issue of processing the files again, correct?

I like your idea of defining new roles, I wonder how difficult that would be to implement? Would that require PRs in Sphinx, itself?

@jobovy
Copy link

jobovy commented Dec 15, 2020

I saw the mcmtroffaes/sphinxcontrib-bibtex#213 PR, but that doesn't allow for the full set of different citation commands that I need.

Agreed, but that PR would be able to help at least with the issue of processing the files again, correct?

I like your idea of defining new roles, I wonder how difficult that would be to implement? Would that require PRs in Sphinx, itself?

I'm not sure. I think you can define new roles outside of Sphinx, but the :cite: role has a bit of a special place in how it's handled in some cases. For example, I develop many of my Sphinx as jupyter notebooks and then the way you cite in notebooks (with <cite data-cite=...) has to be converted to RST using a notebook converter. So you'd probably have to edit the notebook converter itself (I'm not sure how new roles are translated...).

@amichuda
Copy link
Author

Some really cool news from sphinxcontrib-bibtex:

mcmtroffaes/sphinxcontrib-bibtex#203

@choldgraf
Copy link
Collaborator

amazing!

@joh
Copy link

joh commented Feb 24, 2022

I would also love to have this option available in jupyter-book. As far as I can understand, custom formatting is now supported by sphinxcontrib-bibtex. However, according to the documentation, custom formatting involves adding some custom python code to conf.py. Is there a way to do this in jupyter-book?

@jbusecke
Copy link
Contributor

jbusecke commented Mar 7, 2022

Hi everyone, I am trying to homogenize the citation/bibliography style for this project(OceanGlidersCommunity/Oxygen_SOP#88). Ideally we would like the Bibliography to show the same format as the citations (example here).

Example of text citation:
image

Correponding Bibliography entry:
image

I guess what I am most unclear about is how I could implement this custom sphinxcontrib-bibtex code in the _config.yml syntax if that is even possible.

If this is not possible, could we export and modify the conf.py file and then use that in the book build? Sorry if these are pretty basic questions, I only have a little experience with jupyter book so far.

If someone here has some concrete tips on how to implement this, I would very much appreciate it. I think this is broadly relevant for many folks in e.g. the earth science community. If there is a solution, I'd be happy to work on adding that to the docs if needed.

Thanks!

@observingClouds
Copy link

Hi everyone,

while searching for a possibility to get the typical academic citation style (author, year) in a Jupyter book I found this issue. You have done so much great work here and it helped me to come up with a solution ( at least for what I wanted ):

  • citations with round brackets ( author et al. ( year) and ( author et al. , year ) )
  • citation keys of the format author et al. ( year) in the bibliography

For this Jupyter book this has been achieved with this PR.

I want to mention that I'm also very new to this environment and did not know how to bring all the mentioned resources together in the _config.yml as @jbusecke mentioned as well.

I decided to create an extension folder _ext, where I included both the plugin for round brackets and for custom reference styles. The latter I adapted to make use of the pydtex-apa-style package.

These extensions can then be loaded within _config.yml:

sphinx:
   local_extensions:
     apastyle: _ext/
     bracket_citation_style: _ext/

and referenced:

bibtex_reference_style    : author_year_round
bibtex_default_style        : myapastyle

It's important that the files in the extension folder have a setup function because this is called when loading the extension.

As a result I end up with:


and
image or image
depending on the usage of cite:t or cite:p

For all the details please check out the PR

I hope this is useful!

@jbusecke
Copy link
Contributor

Thank you so much for outlining these steps @observingClouds! I will try to implement this soon (cc @soerenthomsen ) and report back.

@jbusecke
Copy link
Contributor

Hi @observingClouds, I am running into a problem with this, and get ModuleNotFoundError: No module named 'formatting'.

Where is the formatting package coming from? I was not able to see that in your requirements.txt, but I might be missing something.

Again, thank you so much for sharing your code, this is less than trivial for me.

@observingClouds
Copy link

Hi @jbusecke, the formatting module comes from the pybtex-apa-style-package. It is confusing, I agree.

@soerenthomsen
Copy link
Contributor

Thanks @observingClouds also from my side!

@jbusecke
Copy link
Contributor

Huh, weird I did install pybtex-apa-style in my environment. But let me wipe and reinstall to confirm.

@jbusecke
Copy link
Contributor

That solved it. I think an old pip version might be to blame here. Sorry for the confusion! Works like a charm now!

@jbusecke
Copy link
Contributor

Ok so I tried this out locally and it worked fine, but in our actual deployment relevant PR it doesnt. Here is the current preview compared to what I got locally:
image

I have tried switching to a mamba based env setup (which is what I used locally), but somehow this still doesnt work. I wonder if I am still missing a dependency?

I am getting some warnings which I didnt see during the local build, but these do not seem related on the problem at hand. Sorry for all the spam, thankful for any ideas how to debug this.

@choldgraf
Copy link
Collaborator

Hmmm, I wonder if some of these issues would also be resolved by this issue:

Feel free to add thoughts there too!

@jbusecke
Copy link
Contributor

Thanks for the pointer @choldgraf.
I generally am still to inexperienced to contribute something useful to that issue, but from a quick read that seems very helpful to adapt more general 'sphinx' stuff from within jupyterbook.

I think the most useful action here would to add instructions for this quite common use case to the jupyterbook docs? @observingClouds s PR has all the ingredients (if you actually commit them, not like me yesterday 😭).

@Brazuca228
Copy link

Currently, jupyter-book uses sphinxcontrib-bibtex for creating citation keys in text using the cite directive.

However, for academic citations (at least for my field) the convention is to use an author, year citation key.

There's an issue open about this on sphinxcontrib-bibtex: mcmtroffaes/sphinxcontrib-bibtex#201

I think it comes down to just creating a custom citation class using pybtex.

I think this is something in my wheelhouse to do a PR for if it's something you all think has value.

Would this be a matter of just adding it to some jupyter-book conf.py? And then registering it somehow?.github/ISSUE_TEMPLATE/config.yml

delfanbaum added a commit to oreillymedia/jupyter-book-to-htmlbook that referenced this issue Jan 4, 2023
This is a first pass at handling jupyter book-style bibliographies.
There are a number of compromises made but we'll get to that.

As a commit:

* This adds tests around both end-of-book bibliographies as
  well as end-of-chapter and end-of-chapter-part-file bibliographies, as
  well as code to pass those tests.

* There was a small refactor in the
  `chapter_processing`-type functions to yank out the code that ultimately
  went into `get_main_section()` to avoid duplication. A chapter-file
  bibliography gets added as its own section, so we needed a way to
  capture that and doing it in one function as opposed to in both the
  chapter and chapter part processing functions made more sense.

* HTMLBook bibliographies are appendixes, so that data-type is added
  when the bib is a single file.

* The actual handling of the bibliography happens in the
  `process_citations` function, which rips out the <dt> pointers and leaves
  only the <dd> citations. Handling the actual citations to these
  bibliographical references will come in a later commit.

Some notes and caveats:

* There is not a good way to reformat the citations in this script. The
  way to do it would be to modify the Bibtex formatting, but that feels
  outside the scope of our current work. This means that bibliographies
  will have to be worked on after intake; this feels like an OK
  compromise for the time being.

  Should we go down that path, there is some discussion [here](link),
  and related [here](other_link).

* Since we don't really have a good way to modify the bibliographical
  entries themselves, we likewise don't have a great way to organize
  combined bibliographies from multiple chapter sub-files. Right now
  we're doing a sort of "dumb" combine, appending one to the other
  (i.e., we're not sorting them in any way). While we could do some list
  sorting, I'm not confident it would be good enough or that the data
  will be standard enough to really do that in a reliable way, so for
  now this compromise feels not _great_ but worthwhile.

* As mentioned, handling for the citations themselves will come in a
  subsequent commit.

Sadly, we're starting to get to the brittler points wherein I'm
beginning to think that we ought to instead be writing some kind of a
backend against Jupyter Book/Sphinx like we do for asciidoc, but we're
in this far, so we'll just need to keep on top of Jupyter Book updates
and be careful about targeted version(s). This commit tries to honor
making the script "just work" while compromising enough that hopefully
what work done here is relatively durable (i.e., not making too much
assumptions about the formatting of things outside of the document
structure itself, which _in theory_ is going to be relatively stable
going forward.

link: jupyter-book/jupyter-book#1090
other_link: mcmtroffaes/sphinxcontrib-bibtex#203
delfanbaum added a commit to oreillymedia/jupyter-book-to-htmlbook that referenced this issue Jan 5, 2023
This is a first pass at handling jupyter book-style bibliographies.
There are a number of compromises made but we'll get to that.

As a commit:

* This adds tests around both end-of-book bibliographies as
  well as end-of-chapter and end-of-chapter-part-file bibliographies, as
  well as code to pass those tests.

* There was a small refactor in the
  `chapter_processing`-type functions to yank out the code that ultimately
  went into `get_main_section()` to avoid duplication. A chapter-file
  bibliography gets added as its own section, so we needed a way to
  capture that and doing it in one function as opposed to in both the
  chapter and chapter part processing functions made more sense.

* HTMLBook bibliographies are appendixes, so that data-type is added
  when the bib is a single file.

* The actual handling of the bibliography happens in the
  `process_citations` function, which rips out the <dt> pointers and leaves
  only the <dd> citations. Handling the actual citations to these
  bibliographical references will come in a later commit.

Some notes and caveats:

* There is not a good way to reformat the citations in this script. The
  way to do it would be to modify the Bibtex formatting, but that feels
  outside the scope of our current work. This means that bibliographies
  will have to be worked on after intake; this feels like an OK
  compromise for the time being.

  Should we go down that path, there is some discussion [here](link),
  and related [here](other_link).

* Since we don't really have a good way to modify the bibliographical
  entries themselves, we likewise don't have a great way to organize
  combined bibliographies from multiple chapter sub-files. Right now
  we're doing a sort of "dumb" combine, appending one to the other
  (i.e., we're not sorting them in any way). While we could do some list
  sorting, I'm not confident it would be good enough or that the data
  will be standard enough to really do that in a reliable way, so for
  now this compromise feels not _great_ but worthwhile.

* As mentioned, handling for the citations themselves will come in a
  subsequent commit.

Sadly, we're starting to get to the brittler points wherein I'm
beginning to think that we ought to instead be writing some kind of a
backend against Jupyter Book/Sphinx like we do for asciidoc, but we're
in this far, so we'll just need to keep on top of Jupyter Book updates
and be careful about targeted version(s). This commit tries to honor
making the script "just work" while compromising enough that hopefully
what work done here is relatively durable (i.e., not making too much
assumptions about the formatting of things outside of the document
structure itself, which _in theory_ is going to be relatively stable
going forward.

link: jupyter-book/jupyter-book#1090
other_link: mcmtroffaes/sphinxcontrib-bibtex#203

Update: check for existance of lis after processing
delfanbaum added a commit to oreillymedia/jupyter-book-to-htmlbook that referenced this issue Jan 5, 2023
This is a first pass at handling jupyter book-style bibliographies.
There are a number of compromises made but we'll get to that.

As a commit:

* This adds tests around both end-of-book bibliographies as
  well as end-of-chapter and end-of-chapter-part-file bibliographies, as
  well as code to pass those tests.

* There was a small refactor in the
  `chapter_processing`-type functions to yank out the code that ultimately
  went into `get_main_section()` to avoid duplication. A chapter-file
  bibliography gets added as its own section, so we needed a way to
  capture that and doing it in one function as opposed to in both the
  chapter and chapter part processing functions made more sense.

* HTMLBook bibliographies are appendixes, so that data-type is added
  when the bib is a single file.

* The actual handling of the bibliography happens in the
  `process_citations` function, which rips out the <dt> pointers and leaves
  only the <dd> citations. Handling the actual citations to these
  bibliographical references will come in a later commit.

Some notes and caveats:

* There is not a good way to reformat the citations in this script. The
  way to do it would be to modify the Bibtex formatting, but that feels
  outside the scope of our current work. This means that bibliographies
  will have to be worked on after intake; this feels like an OK
  compromise for the time being.

  Should we go down that path, there is some discussion [here](link),
  and related [here](other_link).

* Since we don't really have a good way to modify the bibliographical
  entries themselves, we likewise don't have a great way to organize
  combined bibliographies from multiple chapter sub-files. Right now
  we're doing a sort of "dumb" combine, appending one to the other
  (i.e., we're not sorting them in any way). While we could do some list
  sorting, I'm not confident it would be good enough or that the data
  will be standard enough to really do that in a reliable way, so for
  now this compromise feels not _great_ but worthwhile.

* As mentioned, handling for the citations themselves will come in a
  subsequent commit.

Sadly, we're starting to get to the brittler points wherein I'm
beginning to think that we ought to instead be writing some kind of a
backend against Jupyter Book/Sphinx like we do for asciidoc, but we're
in this far, so we'll just need to keep on top of Jupyter Book updates
and be careful about targeted version(s). This commit tries to honor
making the script "just work" while compromising enough that hopefully
what work done here is relatively durable (i.e., not making too much
assumptions about the formatting of things outside of the document
structure itself, which _in theory_ is going to be relatively stable
going forward.

link: jupyter-book/jupyter-book#1090
other_link: mcmtroffaes/sphinxcontrib-bibtex#203

Update: check for existance of lis after processing
delfanbaum added a commit to oreillymedia/jupyter-book-to-htmlbook that referenced this issue Jan 5, 2023
This is a first pass at handling jupyter book-style bibliographies.
There are a number of compromises made but we'll get to that.

As a commit:

* This adds tests around both end-of-book bibliographies as
  well as end-of-chapter and end-of-chapter-part-file bibliographies, as
  well as code to pass those tests.

* There was a small refactor in the
  `chapter_processing`-type functions to yank out the code that ultimately
  went into `get_main_section()` to avoid duplication. A chapter-file
  bibliography gets added as its own section, so we needed a way to
  capture that and doing it in one function as opposed to in both the
  chapter and chapter part processing functions made more sense.

* HTMLBook bibliographies are appendixes, so that data-type is added
  when the bib is a single file.

* The actual handling of the bibliography happens in the
  `process_citations` function, which rips out the <dt> pointers and leaves
  only the <dd> citations. Handling the actual citations to these
  bibliographical references will come in a later commit.

Some notes and caveats:

* There is not a good way to reformat the citations in this script. The
  way to do it would be to modify the Bibtex formatting, but that feels
  outside the scope of our current work. This means that bibliographies
  will have to be worked on after intake; this feels like an OK
  compromise for the time being.

  Should we go down that path, there is some discussion [here](link),
  and related [here](other_link).

* Since we don't really have a good way to modify the bibliographical
  entries themselves, we likewise don't have a great way to organize
  combined bibliographies from multiple chapter sub-files. Right now
  we're doing a sort of "dumb" combine, appending one to the other
  (i.e., we're not sorting them in any way). While we could do some list
  sorting, I'm not confident it would be good enough or that the data
  will be standard enough to really do that in a reliable way, so for
  now this compromise feels not _great_ but worthwhile.

* As mentioned, handling for the citations themselves will come in a
  subsequent commit.

Sadly, we're starting to get to the brittler points wherein I'm
beginning to think that we ought to instead be writing some kind of a
backend against Jupyter Book/Sphinx like we do for asciidoc, but we're
in this far, so we'll just need to keep on top of Jupyter Book updates
and be careful about targeted version(s). This commit tries to honor
making the script "just work" while compromising enough that hopefully
what work done here is relatively durable (i.e., not making too much
assumptions about the formatting of things outside of the document
structure itself, which _in theory_ is going to be relatively stable
going forward.

link: jupyter-book/jupyter-book#1090
other_link: mcmtroffaes/sphinxcontrib-bibtex#203

Update: check for existance of lis after processing

Update: typo fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

10 participants