openalexVectorComp 0.3.5

openalexVectorComp 0.3.5

Bug Fixes

• With delete_existing = FALSE, embed_corpus() and
  batch_submit_openai() are supposed to skip rows that already have
  embeddings with the same text hash. They never did: the existing
  embeddings were opened with the label folder passed as the schema
  argument of arrow::open_dataset(), which always failed silently. As a
  result every run re-embedded the whole corpus, and batch_submit_openai()
  resubmitted already-embedded rows as new (paid) OpenAI batch jobs. The
  existing label=<label> folder is now read correctly and unchanged rows
  are skipped.
• When a row's text has changed and it is embedded again, its old row is
  now removed, so each id appears exactly once per label=<label> folder
  and distance and score functions no longer see duplicate ids. This applies
  to embed_corpus() with delete_existing = FALSE and to
  batch_collect_openai(). New rows are written to a new batch=<n> folder
  first (embed_corpus() no longer restarts at batch=1, which would have
  overwritten unchanged rows); only then are the old rows removed from older
  shards. Shards and batch= folders left empty are deleted. Duplicates left
  by earlier versions are cleaned up the next time new embeddings are written
  to that label.

openalexVectorComp 0.3.4

Bug Fixes

• The TEI backend's id uses the server's served_model_name before
  model_id.

Continuous integration

• R CMD check also runs on pushes to main and dev; test coverage only
  on pushes to main.

0.3.3 (SPECTER2 + TEI support, PR #9) was merged into main but never got a GitHub release; its changes are part of this release and its archive. See NEWS.md.

v0.3.2

Release for Zenodo archiving.

v0.3.1

Patch release to register with Zenodo.

v0.3.0: Initial Release

What's Changed

• release: v0.2.0 async OpenAI workflow, docs/tutorial refresh, and CI updates by @rkrug in #1

New Contributors

• @rkrug made their first contribution in #1

Full Changelog: https://github.com/rkrug/openalexVectorComp/commits/v0.3.0