Skip to content

Conversation

@harini-venkataraman
Copy link
Contributor

@harini-venkataraman harini-venkataraman commented Mar 6, 2025

What

  • Separated indexing and extraction into distinct APIs in prompt service
  • Refactored the SDK to accommodate the new APIs.
  • Subquestion Retrieval using the Llama Index query engine.
  • Refactored the Retriever service.
  • Skipped VectorDB usage for zero chunk size.
  • Added integration tests for the Indexing and Extraction APIs.

...

Why

Pre requisite for cell type merge
...

How

Refactored the existing code
...

Relevant Docs

Related Issues or PRs

Zipstack/unstract#1172
Zipstack/unstract#1149
...

Dependencies Versions / Env Variables

Notes on Testing

Added integration tests
...

Screenshots

...

Checklist

I have read and understood the Contribution Guidelines.

@CLAassistant
Copy link

CLAassistant commented Mar 12, 2025

CLA assistant check
All committers have signed the CLA.

@harini-venkataraman harini-venkataraman changed the title REFACTOR : Indexing API [FIX] Refactored indexing, extraction and retrieval Mar 12, 2025
@harini-venkataraman harini-venkataraman marked this pull request as ready for review March 12, 2025 14:26
Copy link
Contributor

@chandrasekharan-zipstack chandrasekharan-zipstack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments - my main question is on why we have such helper methods for indexing and an API call to the prompt-service

Copy link
Contributor

@chandrasekharan-zipstack chandrasekharan-zipstack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - left a nit comment

harini-venkataraman and others added 4 commits March 18, 2025 10:24
Copy link
Contributor

@gaya3-zipstack gaya3-zipstack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

Copy link
Contributor

@gaya3-zipstack gaya3-zipstack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good.

Copy link
Contributor

@gaya3-zipstack gaya3-zipstack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine.

@gaya3-zipstack gaya3-zipstack merged commit fc0fe4f into main Mar 19, 2025
1 check passed
@gaya3-zipstack gaya3-zipstack deleted the fix/indexing-refactor branch March 19, 2025 06:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants