Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bibliography package improvements on BibTeX syntax #2048

Merged
merged 9 commits into from
Jun 23, 2024

Conversation

Omikhleia
Copy link
Member

@Omikhleia Omikhleia commented Jun 8, 2024

Branch for some bibliography package fixes and improvements

Let's start gently and see where / how it goes.

Closes #2021
Closes #2051
Closes #2050
Closes #1860

@Omikhleia Omikhleia added bug Software bug issue enhancement Software improvement or feature request labels Jun 8, 2024
@Omikhleia Omikhleia self-assigned this Jun 8, 2024
@Omikhleia Omikhleia requested a review from alerque as a code owner June 8, 2024 07:25
@Omikhleia Omikhleia marked this pull request as draft June 8, 2024 07:26
@Omikhleia Omikhleia added the modules:packages Issue relates to core or 3rd party packages label Jun 8, 2024
@Omikhleia Omikhleia force-pushed the bibliography-overhaul branch 2 times, most recently from 404719d to ebbfd5d Compare June 9, 2024 18:58
packages/bibtex/init.lua Outdated Show resolved Hide resolved
packages/bibtex/init.lua Outdated Show resolved Hide resolved
packages/bibtex/init.lua Outdated Show resolved Hide resolved
packages/bibtex/init.lua Outdated Show resolved Hide resolved
packages/bibtex/init.lua Outdated Show resolved Hide resolved
packages/bibtex/init.lua Outdated Show resolved Hide resolved
@Omikhleia Omikhleia changed the title Bibliography package improvements Bibliography package improvements on BibTeX syntax Jun 15, 2024
@Omikhleia Omikhleia force-pushed the bibliography-overhaul branch 2 times, most recently from 433bce2 to 9e4e8fd Compare June 15, 2024 13:19
@Omikhleia
Copy link
Member Author

Omikhleia commented Jun 15, 2024

So this first PR is focused low-hanging fruits, regarding the support of more Bib(La)TeX syntax elements without breaking existing working behavior yet:

  • Ignoring all content between entries (previous implementation mandated using % comments, but this is not correct)
  • Ignoring @preamble elements (specific to TeX-based systems)
  • Supporting @string elements ("abbreviations") and string concatenation with # (BibTeX compatibility)
  • Supporting @xdata elements and xdata fields (Biber/BibLaTeX compatibility) for inheritance
  • Supporting crossref fields with (an initial partial implementation of) the biber/biblatex data inheritance rules
  • Supporting \& as legacy alternative to & for BibTeX compatibility
  • Supporting ~ as convenience for non-breaking space, and \~ to get a tilde (BibTeX compatibility too)
  • XML-escaping (&, <, >) in input, since we are later wrapping them in XML tags (for italics etc.) and processing the formatted output as XML

Later, I'll want to refactor some of the code and reorganize the files differently, in order to tackle properly with issues that will change the output, but this first step should be a decent base for the work to come, and it remains minimally readable (which would become harder after reshuffling some of the logic in different files).

Ready for review.

@Omikhleia Omikhleia marked this pull request as ready for review June 15, 2024 13:32
@alerque alerque self-assigned this Jun 15, 2024
@alerque alerque added this to the v0.15.4 milestone Jun 15, 2024
Copy link
Member

@alerque alerque left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I'm fine with this as is, the comments I made were more because I didn't understand why you did something than because it has to change. You decide.

The commits I pushed are just automatically generated code formatting from stylua with git absorb. The fixup commits should apply to touchup the original commits if you rebase this branch for any reason, or I can too before merging.

I'll be out of pocket for the next week so I can't do much work (maybe a couple of mobile phone based scanning), but if having this merged will facilitate the ongoing work in this direction I can make that happen.

packages/bibtex/init.lua Show resolved Hide resolved
packages/bibtex/init.lua Show resolved Hide resolved
@alerque
Copy link
Member

alerque commented Jun 17, 2024

Just a side note on the Lua style thing.... when you see me push those kind of fixups just know there is absolutely no need to to make an effort to preserve them. If you happen to have other work in a commit or branch and my style fixups are causing rebase conflicts or anything like that just keep the work you have and blow the code formatting fixes away. They are 100% automated and next to zero work to redo at any point, so don't stress if they are causing conflicts. And don't stress about getting the formatting right while writing code either, whatever is easier to write you can write, the auto formatter can take it from any Lua code that runs.

… bibliography

Note that we do not support here expanding "xdata=" contructs inside other
fields i.e. the "granular" per-field inheritance supported by BibLaTeX and
originally Biber, expanding xdata=key-field-index inside other field
values, which requires list fields (e.g. author) to already have been
split and parsed...
Skip `@preamble` as done for `@comment`.
The `@preamble` command is used to provide TeX or LaTeX macros that
are processed by BibTeX program, but it is unspecified what to do
with these when converting to a format that does not use TeX.

See sile-typesetter#2051
Support `@string` blocks in a bibtex file, and the use of the
thus defined strings in other entry fields.
Supports concatenation with the "#" operator.

As part of the syntax clean-up, all content between entries
is now skipped (it doesn't need to start with a percent sign,
everything that is not an entry is a comment.

See sile-typesetter#2051
Add (a part of) the BibLaTeX data inheritance rules for
cross-references.
Use BibLaTeX field names, but still support the legacy
BibTeX file names.
As part of these refactors, allow loading more than one
bibliography file.
Accept `\&` for compatibility with legacy BibTeX, but do not
mandate it to be escaped for compatibility with other engines.
Support unescaped `~` as a non-breaking space for compability with
TeX, this is often found in existing bibliography files.
Support `\~` to render a tilde.
XML-escape the input so it can safely be wrapped in a `<sile>`
construct.

Closes sile-typesetter#2050

Closes sile-typesetter#1860 (replaced by this implementation)
Test `@preamble`, `@xdata`, `@string`, string concatenation with
hash `#`, `xdata` and `crossref` fields.
This test file is already insanely long, so rather than adding
new tests, I preferred hacking a few existing entries:
The result should thus meet the same expectation although the
bibtex file encodes the information slightly differently.
@alerque alerque merged commit 8c5a2a2 into sile-typesetter:master Jun 23, 2024
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Software bug issue enhancement Software improvement or feature request modules:packages Issue relates to core or 3rd party packages
Projects
2 participants