Bump `DocumenterCitations` to `v1.0.0` #2605

lgoettgens · 2023-07-28T14:49:44Z

(Follow-up to #2598 being broken somehow)

DocumenterCitations (the package that we use for references in the documentation) has received a major update including some breaking changes and a lot of new fancy stuff.
I will first try to get it to work exactly like it does currently with the old version and then try to nice things up.

Preview at https://docs.oscar-system.org/previews/PR2605/references/

lgoettgens · 2023-07-28T14:57:38Z

This now furthermore includes the following changes:

Custom style: Use the labels from the .bib file as cite labels.
Show all references, even if they are not cited anywhere.
Slightly adapted some .bib entries that behaved weirdly.

Known problems:

When there is no year in a bib entry, there are additional () printed. See Do not print non-existing years JuliaDocs/DocumenterCitations.jl#30
author fields without names behave weirdly (see e.g. OEIS and Stacks). If someone has an idea, please let me know. Showing first names of all authors instead of initials should work, but wouldn't look as nice.

If somebody finds more broken bib entries in the bibliography overview, please lmk.

A preview should be available at https://docs.oscar-system.org/previews/PR2605/references/ once CI finishes.

Pinging @fingolfin as he motivated the above changes.

codecov · 2023-08-30T18:25:50Z

Codecov Report

Merging #2605 (1fd6681) into master (f8049ce) will decrease coverage by 0.16%.
Report is 15 commits behind head on master.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2605      +/-   ##
==========================================
- Coverage   80.39%   80.24%   -0.16%     
==========================================
  Files         456      462       +6     
  Lines       65235    65443     +208     
==========================================
+ Hits        52447    52514      +67     
- Misses      12788    12929     +141

see 36 files with indirect coverage changes

fingolfin · 2023-09-19T08:45:38Z

Restarted CI to see if 1.0.1/1.0.2 fix the year issue.

docs/src/references.md

fingolfin · 2023-09-19T11:55:37Z

Documenter 1.0 has been released -- with the change in this PR, nothing blocks the CI from using Documenter 1.0, and so it does, and so things fail.

I think for now we should modify docs/Project.toml to restrict Documenter to 0.27 (?), and sort things out here. Then we can worry about switching to Documenter 1.0 in a separate PR?

lgoettgens · 2023-09-19T12:14:35Z

Thanks @benlorenz! This check and duplication can be removed again once we change the Documenter compat to 1.0.0.

lgoettgens · 2023-09-19T12:49:31Z

Restarted CI to see if 1.0.1/1.0.2 fix the year issue.

The year issue is fixed.

OEIS and Stacks are still broken.

Furthermore, @Article without journal are printed a bit weirdly (e.g. BES19). This currently happens for arXiv preprints.

fingolfin · 2023-09-19T13:58:18Z

Yeah there is a reason why preprints were marked as @Misc -- I also do this in "real" .bib files I use for writing books. What's the reason to change those to @Article, by the way?

docs/oscar_references.bib

lgoettgens · 2023-09-19T14:10:28Z

Yeah there is a reason why preprints were marked as @Misc -- I also do this in "real" .bib files I use for writing books. What's the reason to change those to @Article, by the way?

If I remember correctly, @Misc was formatter even weirder. I'll check again and provide a screenshot here

fingolfin · 2023-09-19T21:02:54Z

Is there a chance we can get @Misc fixed either by a fix to DocumenterCitations, or some custom formatter or wahtever here?

lkastner · 2023-09-28T09:23:01Z

What is the status of this @lgoettgens ? I need the changes by @benlorenz that make the documenter helpers work with newer versions. Should I move this to a separate PR?

lgoettgens · 2023-09-28T09:25:32Z

What is the status of this @lgoettgens ? I need the changes by @benlorenz that make the documenter helpers work with newer versions. Should I move this to a separate PR?

If you need it urgently, it would be best to move it to a separate PR. I will be on vacation the next week and will only find time to work on this afterwards.

fingolfin · 2023-10-07T21:31:29Z

Just as a note relevant to the @misc discussion above: the DocumenterCitations documentation says this about preprints:

For documents that are available only as an arXiv eprint, the best result is obtained with a BibTeX entry using the @article class, with, e.g., arXiv:2003.10132 in the Journal field, and, e.g., 10.48550/ARXIV.2003.10132 in the Doi field (but no Eprint field)

fingolfin · 2023-10-07T21:32:36Z

Also archiveprefix = {arXiv}, is not necessary as this is the default. Probably a good idea to read all of https://juliadocs.org/DocumenterCitations.jl/v1.2/syntax/#Preprint-support

lgoettgens · 2023-10-07T21:58:11Z

Just as a note relevant to the @misc discussion above: the DocumenterCitations documentation says this about preprints:

For documents that are available only as an arXiv eprint, the best result is obtained with a BibTeX entry using the @article class, with, e.g., arXiv:2003.10132 in the Journal field, and, e.g., 10.48550/ARXIV.2003.10132 in the Doi field (but no Eprint field)

Thanks! I will look into this in the upcoming week.

even if they are never cited anywhere

as advised by DocumenterCitations docs. Some escapes didn't work as expected before.

- Add some missing DOIs I found while researching other stuff - Format HAL preprints - Remove weirdly printed `\url` - Change `pdf =` to `url =` to get it included in docs - Fix typo in title of `DES93` - Replace dead url of `OM78`

fingolfin · 2023-10-11T05:41:08Z

docs/oscar_references.bib

@@ -25,14 +26,15 @@ @Article{ABS97
 }

 @Article{AG10,
-  author        = {Arzhantsev, Ivan V. and Ga\v{i}fullin, Sergei A.},
+  author        = {Arzhantsev, Ivan V. and Gaĭfullin, Sergey A.},


This change seems unrelated?

Some umlaut/accent escaped were parsed/printed badly. As DocumenterCitations.jl now supports Unicode, my easiest solution was to change all escapes to Unicode symbols. I cannot remember (and didn't check) which exactly had to be changed but just redid all of them.
To ease the review, all of these changes are in well-named commits.

My concern is that this makes it even harder to extract things from the .bib file to use in LaTeX. Maybe not for those using biber, but there are still people using bibtex...

It's not a major problem for me, but it seems a bit gratuitous. Anyway, in the end, I can live with it, but I wonder whether @thofma or @fieker can...

Each bib system has its own quirks, so one needs to adapt the bib file for each.

The machinery available in julia cannot parse generic Latex, but only applies some limited regex replacement. If we decide against Unicode, now and for each future reference that first needs a new Unicode symbol, we need to PR a change to the DocumenterCitations regex list and wait for a release there.

No objection

docs/oscar_references.bib

lgoettgens · 2023-10-11T15:14:16Z

Unfortunately, non-person authors ("corporate names") are currently not supported by BibInternal.jl and DocumenterCitations.jl (waiting for Humans-of-Julia/BibInternal.jl#21 to get released, and see JuliaDocs/DocumenterCitations.jl#44)

The last commit here monkey-patches some functions from DocumenterCitations.jl to use different printing for OEIS and Stacks (that does not initialize first and middle names). This should only be a temporary fix until JuliaDocs/DocumenterCitations.jl#44 is available.

lgoettgens · 2023-10-11T15:18:13Z

I also added some guidelines on how DocumenterCitations.jl expects the .bib file to look like to the dev docs.
Maybe one day I might find the time to write some bibtool test for that.

fingolfin · 2023-10-11T17:32:56Z

docs/oscar_references.bib

@@ -1789,14 +1728,16 @@ @Article{Tay87
  volume        = {3},
  pages         = {76--85},
  year          = {1987},
-  url           = {https://arxiv.org/abs/2201.09155}
+  eprint        = {2201.09155},
+  primaryclass  = {math.GR}


As far as I can tell, the primaryclass is only printed for a very bib entries, like this one; for most other arxiv ones it is not. Presumably it is only printed for those using the eprint field? I guess this is an artifact of the "hack" used to abuse @article for use in eprints. I wonder why they are not simply implement @Misc suitably?

If we want to have them visible for arxiv only papers as well, it needs to go into the journal field. I can adapt that tomorrow

It is not important from my POV, I was just at first confused why it appeared for some papers but not others.

so, don't waste your time on this :-)

As 1.3 will probably break some internals we currently rely on.

fingolfin

I am mostly happy with this now, thank you very much for the effort you put into this, @lgoettgens !

One minor question: the sorting is a bit weird. It seems to be based on the key, but case insensitive? Because it puts ZS98 before Zie95 ? If it easily possible to make the sorting case sensitive would be great. If it takes a lot of effort, forget it or put it on the TODO stack somewhere far from the top.

lgoettgens · 2023-10-16T10:04:33Z

One minor question: the sorting is a bit weird. It seems to be based on the key, but case _in_sensitive? Because it puts ZS98 before Zie95 ? If it easily possible to make the sorting case sensitive would be great. If it takes a lot of effort, forget it or put it on the TODO stack somewhere far from the top.

Nothing I could manage to get working in roughly 20 minutes -> maybe investigate some time way later.
What I can say: it is a bibtool setting. DocumenterCitations.jl keeps the order from the .bib file.

lgoettgens mentioned this pull request Jul 28, 2023

Bump DocumenterCitations to v1.0.0 #2598

Closed

This comment was marked as outdated.

Sign in to view

lgoettgens force-pushed the lg/citations branch from 9ae4a22 to b11fc3a Compare July 28, 2023 14:52

This comment was marked as outdated.

Sign in to view

lgoettgens added the documentation Improvements or additions to documentation label Aug 30, 2023

fingolfin closed this Sep 19, 2023

fingolfin reopened this Sep 19, 2023

fingolfin reviewed Sep 19, 2023

View reviewed changes

docs/src/references.md Show resolved Hide resolved

fingolfin reviewed Sep 19, 2023

View reviewed changes

docs/oscar_references.bib Outdated Show resolved Hide resolved

fingolfin reviewed Sep 19, 2023

View reviewed changes

docs/oscar_references.bib Show resolved Hide resolved

benlorenz mentioned this pull request Sep 28, 2023

support both documenter versions in doctest helper code #2866

Merged

Bump DocumenterCitations

f92452f

lgoettgens force-pushed the lg/citations branch 2 times, most recently from 2c05eba to 970c1b4 Compare October 10, 2023 16:02

lgoettgens added 2 commits October 10, 2023 18:03

Fix two citations

69c612a

Change to custom style

e43b370

lgoettgens added 6 commits October 10, 2023 18:03

Show all .bib entries

7b4f8ee

even if they are never cited anywhere

Bound Documenter version

37cc018

Bump DocumenterCitations to have fixes available

92a4fec

Make all arxiv things consistent

c982d43

Remove duplicate doi and url entries

81c4d3b

Replace escapes by unicode

5e410f1

as advised by DocumenterCitations docs. Some escapes didn't work as expected before.

lgoettgens force-pushed the lg/citations branch from 970c1b4 to 429e611 Compare October 10, 2023 17:49

lgoettgens added 2 commits October 10, 2023 19:50

More .bib changes

429e611

- Add some missing DOIs I found while researching other stuff - Format HAL preprints - Remove weirdly printed `\url` - Change `pdf =` to `url =` to get it included in docs - Fix typo in title of `DES93` - Replace dead url of `OM78`

Add guidelines to dev docs

540ba0d

fingolfin reviewed Oct 11, 2023

View reviewed changes

docs/oscar_references.bib Outdated Show resolved Hide resolved

lgoettgens added 2 commits October 11, 2023 11:47

Remove harmful extra braces from bib file

8628d6e

Monkey-patch DocumenterCitations to work with corporate names

04f6578

lgoettgens force-pushed the lg/citations branch from e2b7ec6 to 04f6578 Compare October 11, 2023 15:13

lgoettgens marked this pull request as ready for review October 11, 2023 15:35

fingolfin reviewed Oct 11, 2023

View reviewed changes

Update DocumenterCitations compat

1fd6681

As 1.3 will probably break some internals we currently rely on.

lgoettgens requested a review from fingolfin October 13, 2023 17:05

fingolfin approved these changes Oct 15, 2023

View reviewed changes

lgoettgens mentioned this pull request Oct 16, 2023

Rename .bib item for ATLAS #2918

Merged

fingolfin merged commit b02fd35 into master Oct 16, 2023
12 of 16 checks passed

fingolfin deleted the lg/citations branch October 16, 2023 06:56

lgoettgens mentioned this pull request Oct 16, 2023

Add some more tests to the bibtool CI job #2919

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bump `DocumenterCitations` to `v1.0.0` #2605

Bump `DocumenterCitations` to `v1.0.0` #2605

lgoettgens commented Jul 28, 2023 •

edited by fingolfin

This comment was marked as outdated.

This comment was marked as outdated.

lgoettgens commented Jul 28, 2023

codecov bot commented Aug 30, 2023 •

edited

fingolfin commented Sep 19, 2023

fingolfin commented Sep 19, 2023

lgoettgens commented Sep 19, 2023

lgoettgens commented Sep 19, 2023 •

edited

fingolfin commented Sep 19, 2023

lgoettgens commented Sep 19, 2023

fingolfin commented Sep 19, 2023

lkastner commented Sep 28, 2023

lgoettgens commented Sep 28, 2023

fingolfin commented Oct 7, 2023

fingolfin commented Oct 7, 2023

lgoettgens commented Oct 7, 2023

fingolfin Oct 11, 2023

lgoettgens Oct 11, 2023

fingolfin Oct 11, 2023

lgoettgens Oct 11, 2023

thofma Oct 12, 2023

lgoettgens commented Oct 11, 2023

lgoettgens commented Oct 11, 2023 •

edited

fingolfin Oct 11, 2023

lgoettgens Oct 11, 2023

fingolfin Oct 11, 2023

lgoettgens Oct 12, 2023

fingolfin left a comment

lgoettgens commented Oct 16, 2023

Bump DocumenterCitations to v1.0.0 #2605

Bump DocumenterCitations to v1.0.0 #2605

Conversation

lgoettgens commented Jul 28, 2023 • edited by fingolfin

This comment was marked as outdated.

This comment was marked as outdated.

lgoettgens commented Jul 28, 2023

codecov bot commented Aug 30, 2023 • edited

Codecov Report

fingolfin commented Sep 19, 2023

fingolfin commented Sep 19, 2023

lgoettgens commented Sep 19, 2023

lgoettgens commented Sep 19, 2023 • edited

fingolfin commented Sep 19, 2023

lgoettgens commented Sep 19, 2023

fingolfin commented Sep 19, 2023

lkastner commented Sep 28, 2023

lgoettgens commented Sep 28, 2023

fingolfin commented Oct 7, 2023

fingolfin commented Oct 7, 2023

lgoettgens commented Oct 7, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lgoettgens commented Oct 11, 2023

lgoettgens commented Oct 11, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fingolfin left a comment

Choose a reason for hiding this comment

lgoettgens commented Oct 16, 2023

Bump `DocumenterCitations` to `v1.0.0` #2605

Bump `DocumenterCitations` to `v1.0.0` #2605

lgoettgens commented Jul 28, 2023 •

edited by fingolfin

codecov bot commented Aug 30, 2023 •

edited

lgoettgens commented Sep 19, 2023 •

edited

lgoettgens commented Oct 11, 2023 •

edited