ragnar 0.2.0
-
ragnar_store_create()gains a new argument:version, with default2.
Store version 2 adds support for chunk deoverlapping on retrieval and automatic chunk augmentation with headings.
To support these features, the internal schema and ingestion requirements are different.
Seemarkdown_chunk()and new S7 classesMarkdownDocumentandMarkdownDocumentChunks.
Backwards compatibility is maintained with version = 1. (#58, #39, #36) -
ragnar_store_create()now supports Date and POSIXct classes supplied toextra_cols. -
ragnar_store_create()now supports remote MotherDuck Databases specified withmd:<dbname>as
thelocationargument. (#50) -
ragnar_retrieve()and friends gain afilterargument, adding support for efficiently
filtering retrieval results. -
ragnar_retrieve_bm25()gains argumentsb,k, andconjunctive(#56). -
ragnar_retrieve_vss()gains argumentquery_vector, supporting workflows that preprocess the query string before embedding. -
ragnar_retrieve_vss()set of validmethodchoices have been updated to a narrower set to ensure that anHNSWindex scan is used. -
Passing a
tbl(store)toragnar_retrieve()is deprecated. -
New chunker
markdown_chunk()with support for chunk heading context generation,
semantic boundary selection, overlapping chunks, document segmentation, and more. (#56) -
New function
ragnar_chunks_view()for quickly previewing chunks (#42) -
ragnar_register_tool_retrieve()gains optionalnameandtitlearguments
to allow for more descriptive tool registration. These values can also be set
inragnar_store_create()(#43). -
ragnar_read()andread_as_markdown()now accept paths
that begin with~(@topepo, #46, #48). -
Changes to
read_as_markdown()HTML conversion (#40, #51):- New arguments
html_extract_selectorsandhtml_zap_selectorsprovide a flexible way to
exclude some html page elements from being included in the converted markdown. - code blocks now include the language, if available.
- Fixed handling of nested code fences in markdown output.
- New arguments