-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add bedrock contract tests. #227
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mxiamxia
reviewed
Jul 26, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in general, the pr looks good to me. the tests covers the new gen ai attributes will be released.
mxiamxia
previously approved these changes
Jul 26, 2024
mxiamxia
reviewed
Aug 15, 2024
thpierce
approved these changes
Aug 16, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The PR is a follow up on bedrock service support PR:
#209
We add contract tests for following Bedrock services that covers all resource attributes we newly support:
GetGuardrail
GetAgent
,GetDataSource
,GetKnowledgeBase
InvokeModel
InvokeAgent
Upgrade
botocore
andboto3
to the latest version1.34.143
so that to support Bedrock services API calls.Upgrade
localstack/localstack
image to the latest version3.5.0
so resolve the SQS API call issue usinglocalstack/localstack:2.0.1
with new version ofboto3
: localstack/localstack#9610Contract test limitation:
The contract tests in current repo is using LocalStackContainer to serve AWS SDK service calls. But it doesn’t has bedrock related service support (This is the full service list it support.). In this case, no matter which bedrock API we call in contract test, the response will always be 4XX.
As a workaround, we inject
inject_200_success
andinject_500_error
directly into the API call to make sure we receive http response with expected status code and attributes._assert_semantic_conventions_span_attributes
function change:In _assert_semantic_conventions_span_attributes function it is checking the input
service
equals torpc.service
, however, we pass the input service with "remote_service", where there is mismatch for example for Bedrock Agent Runtime: we haverpc_service="Bedrock Agent Runtime", remote_service="AWS::Bedrock".
Thus we change to use
rpc_service
if it is provided by:By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.