Skip to content

Conversation

@tylerhutcherson
Copy link
Collaborator

@tylerhutcherson tylerhutcherson commented Jan 23, 2024

With the shift in the ecosystem to Pydantic 2.x.x, we need to get redisvl to the same level.

Referenced here: https://github.com/RedisVentures/redisvl/issues/86

Fortunately, there is not a ton of complex usage of Pydantic; which makes this easy! Changes include:

  • Shifting the storage classes to use Pydantic to avoid manual (and ugly) field validation. Now, the subclasses inherit from the base and params are all typed as expected.
  • Shifting the vectorizer classes to use Pydantic for the same reasons.
  • Bump and pin pydantic version in the packge requirements to a reasonable range pydantic>=2.0.0,<3
  • use v1 shim for now for safety

@tylerhutcherson tylerhutcherson added the enhancement New feature or request label Jan 23, 2024
@codecov-commenter
Copy link

codecov-commenter commented Jan 23, 2024

Codecov Report

Attention: 27 lines in your changes are missing coverage. Please review.

Comparison is base (24955a2) 76.95% compared to head (183cb68) 77.16%.

❗ Current head 183cb68 differs from pull request most recent head 0569657. Consider uploading reports for the commit 0569657 to get more accurate results

Files Patch % Lines
redisvl/vectorize/text/openai.py 20.00% 8 Missing ⚠️
redisvl/vectorize/text/cohere.py 25.00% 6 Missing ⚠️
redisvl/vectorize/text/vertexai.py 25.00% 6 Missing ⚠️
redisvl/vectorize/text/huggingface.py 58.33% 5 Missing ⚠️
redisvl/storage.py 90.00% 1 Missing ⚠️
redisvl/vectorize/base.py 92.30% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #101      +/-   ##
==========================================
+ Coverage   76.95%   77.16%   +0.21%     
==========================================
  Files          22       22              
  Lines        1380     1384       +4     
==========================================
+ Hits         1062     1068       +6     
+ Misses        318      316       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Spartee
Spartee previously approved these changes Jan 24, 2024
Copy link
Contributor

@Spartee Spartee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Undoubtly will cause some pain with the langchain integration. However, I think as long as there is no inheritance chain that utilizes pydantic v2 with langchain internals we should be fine. the langchain integration should just use langchain.pydantic_v1 for any instantiated classes. As well, any base classes we expect to use in that integration should also utilize langchain.pydantic_v1 if at all possible.

Also, as this is a library we expect to have people use as a dependency within their projects, I would expect to have a wider range of support for pydantic versions even if we have to test multiple versions (we shouldnt, but you never know).

nit: might think about breaking up some of these non-related changes just in case you need to roll this back if it causes major issues at some point down the road.

Up to you, otherwise looks good

CONTRIBUTING.md Outdated

Formatting and linting checks:
```bash
make format
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make check?

requirements.txt Outdated
coloredlogs
pydantic<2.0.0
redis>=5.0.0
pydantic==2.5.3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may think about relaxing this a bit. Pinning to a patch version for this popular of a library is bound to cause some issues. more on this in the top level comment

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do. Good call

@tylerhutcherson tylerhutcherson changed the base branch from dev to main January 26, 2024 17:14
@tylerhutcherson tylerhutcherson dismissed Spartee’s stale review January 26, 2024 17:14

The base branch was changed.

@tylerhutcherson tylerhutcherson merged commit c3e036b into main Jan 26, 2024
@tylerhutcherson tylerhutcherson deleted the pydantic-upgrade branch January 26, 2024 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants