Skip to content

v1.1.0

Latest

Choose a tag to compare

@rsdoiel rsdoiel released this 26 Oct 23:37
· 2 commits to main since this release

v1.1.0 adds a text layer for CrossRef titles and abstracts. Nothing that existed changes.

Added:

  • PlainText(s): a title fragment as plain Unicode text. Tags are removed, entities decoded, subscripts and MathML formulas converted, and the layout whitespace publishers add around markup tidied.
  • (*Message).FullTitle() and (*Message).AlternateTitles(): the title and subtitle as plain text (the subtitle left out when empty or already in the title), and the record's other titles. No language is guessed.
  • HTMLAbstract(s) and (*Message).HTMLAbstract(): a JATS abstract as HTML that Invenio RDM's sanitizer keeps, with MathML as TeX in \( ... \), the leading "Abstract" heading and copyright text dropped, attributes stripped and links limited to http, https, mailto and relative addresses.
  • Normalizer and NewNormalizer(path): the same conversions with an override of the embedded symbol table (symbols.json). A Normalizer never changes after it is built, so it can be shared between goroutines.
  • Warning and its codes (parse-fallback, script-unmapped, math-linear, math-unconverted, multiple-titles): what could not be converted cleanly, returned beside the best text available.
  • A user guide (docs/text_normalization.md), a package overview and runnable examples.

Notes:

  • Whitespace around markup is a heuristic, chosen on real CrossRef records; the guide lists the rules and their known limits.
  • The TeX in an abstract needs MathJax, or another renderer that reads the \( ... \) delimiters, on the page that shows it.
  • The tests use captured CrossRef responses in testdata/ and run offline.