Some updates and fixes for async streaming and telemetry #1100
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* add missing awaits * remove extra coroutine
dtam
commented
Sep 26, 2024
| validate_subschema=True, | ||
| stream=True, | ||
| ) | ||
| # TODO why? how does it happen in the other places we handle streams |
There was a problem hiding this comment.
@nichwch any insights?
There was a problem hiding this comment.
If you're asking about the None check on line 157, it might be a remnant of before we implemented generators all the way down. Before, it was possible for validation to return None of the validators haven't accumulated enough chunks to validate yet. For sync streaming, we've changed this so that the validation logic takes place in a generator and only emits results when enough chunks have been accumulated.
Looking into the async streaming code, it looks like we never changed that to use generators for the validation logic, so validators still emit Nones before they have accumulated enough chunks
|
requires guardrails-ai/interfaces#13 |
zsimjee
approved these changes
Sep 30, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixed an issue with telemetry initialization despite having enable_metrics=true in .rc it was not enabled
Fixed issues with async streaming erroring in several cases
Fixed issues with attributes being added to spans that had be closed already in the case of async streaming