Skip to content

CRAN v1.4.0

Latest

Choose a tag to compare

@kbenoit kbenoit released this 05 Nov 02:03
6a2520a

spacyr 1.4.0 fixes two bugs that could affect results (input text was altered before reaching spaCy, and an error could leave the spaCy pipeline disabled), makes installation more robust, and adds spacy_installed() and a max_length argument to spacy_initialize().

Install from CRAN with install.packages("spacyr").

Changes that can affect results

  • spacy_parse(), spacy_tokenize(), spacy_extract_entity() and spacy_extract_nounphrases() no longer alter the input text. Previously they deleted all backslashes and converted the literal sequences \n and \t into newlines and tabs, so that for instance C:\Users was tokenized as C, :, Users. Code that relied on literal \n being converted should now convert it in R before calling spacyr.
  • NA documents are now treated as empty documents, rather than as the text "NA".
  • spacy_tokenize(what = "sentence", remove_separators = FALSE) now keeps trailing whitespace regardless of the value of multithread, including a single space after a sentence, so that the sentences join back into the original text.
  • spacy_parse(nounphrase = TRUE) now returns rows in the order of the input documents, as it does without noun phrases, instead of sorted by document name. This fixes a bug where the whitespace column and any additional_attributes were assigned to the wrong tokens when the document names were not in alphabetical order.

New features

  • Added spacy_installed(), which checks without starting Python whether spaCy and, optionally, given language models are installed (#197).
  • Added a max_length argument to spacy_initialize() to allow documents longer than spaCy's default limit of 1,000,000 characters (#218).

Installation

  • spacy_install() now creates its environment with a version of Python supported by the spaCy release being installed, as declared on PyPI, rather than whichever Python is found first; if none is found, it offers to install a compatible one. For an existing environment whose Python is not supported, it stops with an error explaining how to recreate the environment, instead of failing during installation.
  • spacy_initialize() now explains how to install a language model that cannot be found.
  • Fixed spacy_download_langmodel() and spacy_install() not detecting already-installed language models with recent versions of pip (#258).
  • Fixed a bug in spacy_download_langmodel() that prevented installing multiple models in one function call (#249).
  • spacy_install() and spacy_upgrade() no longer fail on Windows when installing into a different environment after Python has been initialised in the session (#258).
  • Updated the installation instructions: language models are installed with spacy_download_langmodel() rather than with pip (#253), and new sections cover changing the Python version, using an existing Python or conda environment (#246), using local models (#180) and the need for git on Windows (#245). The vignette no longer describes arguments that were removed from spacy_initialize().
  • Requires reticulate >= 1.31.

Bug fixes

  • Fixed a bug where an error during spacy_tokenize(), such as a text exceeding spaCy's maximum length, left the spaCy pipeline disabled, so that later calls in the same session failed or silently returned no entities.
  • Fixed a SyntaxWarning about an invalid escape sequence on Python 3.12+ when initialising spacyr (#257, thanks to @thjbdvlt).
  • spacy_parse() no longer warns that lemmatization "may not work properly" for models whose name does not start with "en".
  • spacy_parse() now returns a zero-row result with the usual columns and types when no document contains any tokens (for instance when all documents are NA or empty), instead of failing with dependency = TRUE or returning malformed columns.