Skip to content

v0.1.0

Latest

Choose a tag to compare

@rsdoiel rsdoiel released this 28 Oct 01:12
  • Breaking: every doi2rdm exit code is now a Caltech Library workspace code (sysexits), so a script can tell what went wrong from the number: 0 the record was written; 1 the DOI was not found (CrossRef and DataCite both answered that they have no record); 2 a bad command line; 65 wrong content (a -diff file that is not JSON, a fetched record that cannot be converted); 66 a file is missing (the options, -diff or symbols file); 69 CrossRef or DataCite could not be reached or failed; 70 an unclassified error; 74 writing the output failed; 75 rate limited, try again later; 78 a configuration file is present but invalid. Earlier versions used 2 for "not found" and 8 for an invalid file. Update any script that reads them, irdm_harvester included, and deploy it together with this release. The default invocation asks DataCite only when CrossRef had no answer for the DOI; a problem with a file or with the record stops the run at once, and an outage at either service is never reported as "not found". The exported ENOENT, ENOEXEC and EAGAIN constants are removed from the Go package (the new classes are in exit.go).
  • Titles from CrossRef are plain text: tags, entities and MathML are converted, layout whitespace is tidied, and the subtitle is appended unless the title already holds it (fixes irdm_harvester#42). The other titles that add something are additional titles, still typed other; a second title that only differs by capitalization is no longer added.
  • Descriptions are HTML that Invenio RDM's sanitizer keeps, from the CrossRef abstract: the leading "Abstract" heading and the copyright line are dropped, structured headings become bold paragraphs, italics stay, and MathML formulas become TeX between \( and \) (fixes irdmtools#36). The page that shows the description needs MathJax for the formulas to appear as formulas; RDM's record pages have it.
  • A new optional symbols: key in the options YAML names a crossrefapi override file for the conversion's symbol table; a relative path is read relative to the options file. A missing file exits 66 and an invalid one 78 before any record is fetched.
  • Conversion warnings go to standard error, one line each, doi2rdm: DOI: field: code: fragment, and never change the JSON on standard output or the exit status. Control characters in a fragment are replaced by spaces so a warning is one line of printable text.
  • The -debug flag is documented: it writes the raw JSON CrossRef or DataCite returned to standard error before it is converted.
  • Every program answers NAME -help topics with a list of its help topics and NAME -help TOPIC with the page for one; -help alone still prints the whole manual. rdmutil has a topic for each of its 38 actions, ep3util for each of its 5, and doi2rdm for its options, options file, text normalization, warnings, exit status and examples; the six single-purpose programs say they have no topics. The topics are cut from each manual, so they cannot drift, and each has a page in the documentation (rdmutil-get_record.1.md and so on, 49 pages). An unknown topic is exit code 2.
  • The rdmutil manual documented get_files twice, left out delete_endpoint and still described query, which was removed earlier; it now documents get_draft_files and delete_endpoint and no longer lists query. A test checks that the manuals of rdmutil and ep3util document exactly the actions the programs run.
  • The manual lists the new text normalization, the warnings and every exit code; a new page, docs/doi2rdm_text_normalization.md, shows before and after records.
  • Uses crossrefapi v1.1.0 and needs Go 1.27.1.
  • Tests: the live tests (make test-live) read the CrossRef and DataCite mailto from the environment variable TEST_MAILTO and skip without it; no address is written in the tests.
  • Known limits, in TODO.md: doi2rdm cannot convert a CrossRef journal issue (its resource type is not mapped), and the crosswalk calls api.ror.org for funders with no timeout or cache.