fix: dynamodb mapper works with images too now #53
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available: N/A
Description of changes: TL;DR this change allows multi-modal agent invocations with the DynamoDB tracer and conversation history.
Fixed the DynamoDB mapper so that it works for binary messages as well (e.g. images). Before we used a trick where we went back and forth to JSON to coerce everything to DynamoDB compatibility--but that didn't work for bytes. So I fixed that and made it more proper now: boto3 wants
Decimalinstead offloat, wrapsbytesinBinary, doesn't understanddatetime. It is not inconceivable there are other types we have to special case at some point but I don't know them yet right now, and what we have now works for the integration test (see next point).Added an integration test showing how to do multi-modal agent invocations.
At some point we should make the GitHub action execute integration tests too, but that requires setting up credentials in the action. That's a bit too much work for me for now, so I'm keeping the integration tests out of the GitHub action. At least we can run them locally (using developer AWS credentials).
Another todo: we should document how to do multi-modal agent invocations in the README, and maybe consider making it easier and directly available on the Agent instance, instead of via conversation history (see test).
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.