Skip to content

Conversation

@chandrasekharan-zipstack
Copy link
Contributor

What

  • Deprecated old function get_vector_store_index_from_storage_context() to index and created a new one index_document() in vector_db.py
  • Moved attributes within new function to simplify logic at caller side
  • Renamed deprecated SimpleNodeParser to the intended SentenceSplitter
  • Bumped SDK to 0.48.0

Why

  • The llama index function VectorStoreIndex.from_documents() expects an attribute transformations which was not set correctly. As a result the user configured values for chunk size and chunk overlap was never set and used

Notes on Testing

  • Attempted indexing locally

Screenshots

  1. With 500 and 50 chunk size / overlap
    image
  2. With 2048 and 256 chunk size / overlap
    image

Checklist

I have read and understood the Contribution Guidelines.

Copy link
Contributor

@gaya3-zipstack gaya3-zipstack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants