Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add weave to wandbot to track all intermediate steps and calls #76

Merged
merged 4 commits into from
Jun 6, 2024

Conversation

parambharat
Copy link
Contributor

The PR adds weave integration to wandbot to track RAG pipeline calls for query enhancements, retrieval, and response synthesis.

Copy link

socket-security bot commented Jun 5, 2024

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package New capabilities Transitives Size Publisher
pypi/boto3@1.34.119 eval, filesystem, network 0 996 kB aws
pypi/botocore@1.34.119 environment, eval, filesystem, network, shell, unsafe 0 106 MB aws, garnaat
pypi/jmespath@1.0.1 environment, eval, filesystem 0 108 kB aws, jamesls
pypi/s3transfer@0.10.1 environment, filesystem, network 0 780 kB aws

🚮 Removed packages: pypi/appnope@0.1.4, pypi/appnope@0.1.4, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi-bindings@21.2.0, pypi/argon2-cffi@23.1.0, pypi/argon2-cffi@23.1.0, pypi/arrow@1.3.0, pypi/arrow@1.3.0, pypi/async-lru@2.0.4, pypi/async-lru@2.0.4, pypi/babel@2.15.0, pypi/babel@2.15.0, pypi/bleach@6.1.0, pypi/bleach@6.1.0, pypi/comm@0.2.2, pypi/comm@0.2.2, pypi/debugpy@1.8.1, pypi/debugpy@1.8.1, pypi/debugpy@1.8.1, pypi/debugpy@1.8.1, pypi/debugpy@1.8.1, pypi/debugpy@1.8.1, pypi/debugpy@1.8.1, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/faiss-cpu@1.8.0, pypi/fastjsonschema@2.19.1, pypi/fastjsonschema@2.19.1, pypi/flask-cors@4.0.1, pypi/flask-cors@4.0.1, pypi/fqdn@1.5.1, pypi/fqdn@1.5.1, pypi/ipynbname@2024.1.0.0, pypi/ipynbname@2024.1.0.0, pypi/isoduration@20.11.0, pypi/isoduration@20.11.0, pypi/itsdangerous@2.2.0, pypi/itsdangerous@2.2.0, pypi/jedi@0.19.1, pypi/jedi@0.19.1, pypi/json5@0.9.25, pypi/json5@0.9.25, pypi/jsonschema-specifications@2023.12.1, pypi/jsonschema-specifications@2023.12.1, pypi/jsonschema@4.22.0, pypi/jsonschema@4.22.0, pypi/jupyter-client@8.6.2, pypi/jupyter-client@8.6.2, pypi/jupyter-core@5.7.2, pypi/jupyter-core@5.7.2, pypi/jupyter-events@0.10.0, pypi/jupyter-events@0.10.0, pypi/jupyter-lsp@2.2.5, pypi/jupyter-lsp@2.2.5, pypi/jupyter-server-terminals@0.5.3, pypi/jupyter-server-terminals@0.5.3, pypi/jupyter-server@2.14.1, pypi/jupyter-server@2.14.1, pypi/jupyterlab-pygments@0.3.0, pypi/jupyterlab-pygments@0.3.0, pypi/jupyterlab-server@2.27.2, pypi/jupyterlab-server@2.27.2, pypi/jupyterlab@4.3.0a0, pypi/jupyterlab@4.3.0a0, pypi/llvmlite@0.43.0rc1, pypi/llvmlite@0.43.0rc1, pypi/llvmlite@0.43.0rc1, pypi/llvmlite@0.43.0rc1, pypi/llvmlite@0.43.0rc1, pypi/llvmlite@0.43.0rc1, pypi/llvmlite@0.43.0rc1, pypi/llvmlite@0.43.0rc1, pypi/llvmlite@0.43.0rc1, pypi/llvmlite@0.43.0rc1, pypi/llvmlite@0.43.0rc1, pypi/mistune@3.0.2, pypi/mistune@3.0.2, pypi/nbconvert@7.16.4, pypi/nbconvert@7.16.4, pypi/notebook-shim@0.2.4, pypi/notebook-shim@0.2.4, pypi/notebook@7.3.0a0, pypi/notebook@7.3.0a0, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/numba@0.60.0rc1, pypi/pandocfilters@1.5.1, pypi/pandocfilters@1.5.1, pypi/parso@0.8.4, pypi/parso@0.8.4, pypi/pathspec@0.12.1, pypi/pathspec@0.12.1, pypi/pexpect@4.9.0, pypi/pexpect@4.9.0, pypi/pluggy@1.5.0, pypi/pluggy@1.5.0, pypi/prometheus-client@0.20.0, pypi/prometheus-client@0.20.0, pypi/prompt-toolkit@3.0.46, pypi/prompt-toolkit@3.0.46, pypi/ptyprocess@0.7.0, pypi/ptyprocess@0.7.0, pypi/pynndescent@0.5.12, pypi/pynndescent@0.5.12, pypi/pywin32@306, pypi/pywin32@306, pypi/pywin32@306, pypi/pywin32@306, pypi/pywin32@306, pypi/pywin32@306, pypi/pywin32@306, pypi/pywin32@306, pypi/pywin32@306, pypi/pywin32@306, pypi/pywin32@306, pypi/pywin32@306, pypi/pywin32@306, pypi/pywin32@306, pypi/pywinpty@2.0.13, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/pyzmq@26.0.3, pypi/referencing@0.35.1, pypi/referencing@0.35.1, pypi/rfc3339-validator@0.1.4, pypi/rfc3339-validator@0.1.4, pypi/rfc3986-validator@0.1.1, pypi/rfc3986-validator@0.1.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/rpds-py@0.18.1, pypi/send2trash@1.8.3, pypi/send2trash@1.8.3, pypi/terminado@0.18.1, pypi/terminado@0.18.1, pypi/tornado@6.4, pypi/tornado@6.4, pypi/tornado@6.4, pypi/tornado@6.4, pypi/tornado@6.4, pypi/tornado@6.4, pypi/tornado@6.4, pypi/tornado@6.4, pypi/tornado@6.4, pypi/tornado@6.4, pypi/tornado@6.4, pypi/types-python-dateutil@2.9.0.20240316, pypi/types-python-dateutil@2.9.0.20240316, pypi/umap-learn@0.5.6, pypi/umap-learn@0.5.6, pypi/uri-template@1.3.0, pypi/uri-template@1.3.0, pypi/webcolors@1.13, pypi/webcolors@1.13, pypi/zenpy@2.0.49

View full report↗︎

Copy link

socket-security bot commented Jun 5, 2024

👍 Dependency issues cleared. Learn more about Socket for GitHub ↗︎

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

View full report↗︎

@ayulockin
Copy link
Member

Hey @parambharat did you run eval on it?

Do you want me to do it?

@parambharat
Copy link
Contributor Author

@ayulockin , there are no functional changes other than adding weave ops and changing a few langchain imports to work with the new langchain version. I don't think a eval re-run might be necessary here.

if chat_history is None:
chat_history = []

with get_openai_callback() as query_enhancer_cb, Timer() as query_enhancer_tb:
enhanced_query = self.query_enhancer.chain.invoke(
enhanced_query = self.query_enhancer(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the requirement of not doing .chain.invoke a new thing in LECL?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you have added __call__ methods.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haha, yes, this allows the use of a weave.op decorator, making the weave integration easier.

src/wandbot/rag/query_handler.py Show resolved Hide resolved
@ayulockin
Copy link
Member

Hey @parambharat, reviewed and it looks good. I asked a few questions out of curiosity. But going ahead and merging it. Thanks for the weave integration.

@ayulockin ayulockin merged commit 536b44c into main Jun 6, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants