Skip to content

RayRAG v0.26.0 — reference chunks carry every citation field

Choose a tag to compare

@Junesgone Junesgone released this 04 Oct 00:19
· 1 commit to main since this release

RayRAG v0.26.0

The citation fields a reference chunk owes a client

The API guide prints the shape of a reference chunk in both places it appears. Retrieval:

content, content_ltks, document_id, document_keyword, highlight, id, image_id,
important_keywords, tag_kwd, dataset_id, positions, similarity, term_similarity, vector_similarity

and chat/agent references: id, content, document_id, document_name, document_metadata, dataset_id, image_id, positions, url, similarity, vector_similarity, term_similarity, doc_type.

RayRAG sent its own names instead — chunk_id, doc_id, kb_id, doc_name — and the citation-only
fields (positions, image_id, url, doc_type, content_ltks, important_keywords, tag_kwd) were
simply absent. A client written against RAGFlow had nothing to draw a source with.

One helper now builds that block and every chunk and reference builder merges it:

  • Nothing is invented. A field is the stored value or it is empty — upstream itself sends
    "positions": [] and "image_id": "" when it has no position data, and an empty list is a truthful
    answer while a fabricated coordinate is not. content_ltks is the stored token string when the chunk
    has one, and otherwise tokenised from the content with the same tokenizer the keyword index uses.
  • Positions are parsed from what was stored: an array stays as it is, a JSON string is parsed, a
    comma-separated tuple (12,11,11) is wrapped into upstream's list-of-tuples shape, and malformed
    JSON becomes []
    rather than a guessed position built from half a tuple.
  • RayRAG's own names stay beside upstream's, so every client that already read these responses keeps
    working: on the retrieval path similarity and score are the same number under two names.

What following the contract caught

The /api/v1/retrieval handler lives in server.rs and is a second implementation of the same
response shape as api::search::weighted_search. The first patch reached only the second one, and the
router test failed against the real handler — which is how the duplicate surfaced. The shared helper now
feeds all four builders, which is what stops them drifting apart again.

Measured on the running deployment

POST /api/v1/retrieval   -> code 0, 10 chunks
  id                  "6b16e472-7555-42da-86e2-abdf49e98714_0"
  document_id         "31219a9a-8935-4fe9-a1ab-5418a0829630"
  document_keyword    "embed_test.txt"
  dataset_id          "0bd5976a-6b50-49ba-8a07-07b4952a51cc"
  content_ltks        "RayRAG 是一个用 Rust 实现的知识库系统。 它对标 RAGFlow 的全部功能。 …"
  image_id            ""
  important_keywords  []
  tag_kwd             []
  positions           []
  similarity          0.9640934467315674
RayRAG's own names kept: True; similarity == score: True; panics: 0

2254 tests pass, 0 failures; verify-all.sh passes nine suites, including a router test that indexes
a chunk with stored positions, an image id, a URL, a doc type and keyword lists, then reads every upstream
key back off the response and asserts the values are the stored ones.