Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[]byte would be preferable to *string as the type of Message Bodies #371

Closed
endophage opened this issue Sep 3, 2015 · 3 comments
Closed
Labels
feature-request A feature should be added or improved.

Comments

@endophage
Copy link

A number of the services (SQS is the one I've been dealing with recently), appear to use *string as the type for a message body. Per @jasdel this is apparently to help enforce unicode encoding of the body.

However, with many of these services I and everyone I've discussed them with, put our own encoded (JSON, XML, etc...) data into the body and need to parse it when consuming. Go's encoding package and it's sub-packages consistently use the []byte type as input and output.

This creates the problem that we frequently have to type cast between string and []byte (and vice versa when initially creating the message body), causing additional memory allocations, not overly concerning for things like SQS with 256KB max message size, but not great. More frustratingly though, code ends up sprinkled with what should be unnecessary casts, which are just ugly.

@jasdel
Copy link
Contributor

jasdel commented Sep 3, 2015

Discussion original started in #363 for extra context

@jasdel
Copy link
Contributor

jasdel commented Jan 13, 2016

@endophage thanks again for this request. These fields being *string instead of []byte is a representation of the underling wire service protocols. Since the API service clients are generated from service API models their representation will be very similar to the the model definitions. This is also why pointer types are so commonly implemented in the SDK instead of value for future optional parameter support. In this case the message body parameter is a string within the service's model.

The SDK does have the ability to create customizations, overriding what is specified in the model. In a few specific cases this functionality is used to modify the ingested models specifically based on the SDK's capabilities. This functionality is used sparingly to prevent creating situations where maintainability is hampered, or preventing backwards incompatible breaking changes. There are numerous other types this style of customization could be applied to also. Each one brings the SDK further away from the service API model's definition, and increases the maintenance burden. Increasing the risk inconsistencies in the API which cannot be corrected without a breaking change.

Because of this we would like to hold off making customizations to the service API model like this. Similar types such as Policies and the more extreme CloudFormation Templates would be better and more completely served by concrete types or builders to address their pain points.

@jasdel jasdel added feature-request A feature should be added or improved. and removed review needed labels Jan 13, 2016
@jasdel
Copy link
Contributor

jasdel commented Jan 18, 2016

Thanks again for the feature request. I'm going to close this issue for now since we don't plan on adding this customization to the SDK. Please let us know if you have any addition questions, feedback, or ideas how the SDK can be improved.

@jasdel jasdel closed this as completed Jan 18, 2016
skotambkar added a commit to skotambkar/aws-sdk-go that referenced this issue May 20, 2021
Adds missing sdk error checking when seeking readers. Also adds support for nonseekable io.Reader and support for streamed payloads for unsigned body request.

Fixes aws#371
skotambkar added a commit to skotambkar/aws-sdk-go that referenced this issue May 20, 2021
Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Enhancements
---
* `aws/endpoints`: Expose DNSSuffix for partitions ([aws#369](aws/aws-sdk-go-v2#369))
  * Exposes the underlying partition metadata's DNSSuffix value via the `DNSSuffix` method on the endpoint's `Partition` type. This allows access to the partition's DNS suffix, e.g. "amazon.com".
  * Fixes [aws#347](aws/aws-sdk-go-v2#347)
* `private/protocol`: Add support for parsing fractional timestamp ([aws#367](aws/aws-sdk-go-v2#367))
  * Fixes the SDK's ability to parse fractional unix timestamp values and adds tests.
  * Fixes [aws#365](aws/aws-sdk-go-v2#365)
* `aws/ec2metadata`: Add marketplaceProductCodes to EC2 Instance Identity Document ([aws#374](aws/aws-sdk-go-v2#374))
  * Adds `MarketplaceProductCodes` to the EC2 Instance Metadata's Identity Document. The ec2metadata client will now retrieve these values if they are available.
  * Related to: [aws#2781](aws#2781)
* `aws`: Adds configurations to the default retryer ([aws#375](aws/aws-sdk-go-v2#375))
  * Provides more customization options for retryer by adding a constructor for default Retryer which accepts functional options. Adds NoOpRetryer to support no retry behavior. Exposes members of default retryer.
  * Updates the underlying logic used by the default retryer to calculate jittered delay for retry. Handles int overflow for retry delay.
  * Fixes [aws#370](aws/aws-sdk-go-v2#370)
* `aws` : Refactors request retry behavior path logic ([aws#384](aws/aws-sdk-go-v2#384))
  * Retry utilities now follow a consistent code path. aws.IsErrorRetryable is the primary entry point to determine if a request is retryable.
  * Corrects sdk's behavior by not retrying errors with status code 501. Adds support for retrying the Kinesis API error, LimitExceededException.
  * Fixes [aws#372](aws/aws-sdk-go-v2#372), [aws#145](aws/aws-sdk-go-v2#145)

SDK Bugs
---
* `aws`: Fixes bug in calculating throttled retry delay ([aws#373](aws/aws-sdk-go-v2#373))
  * The `Retry-After` duration specified in the request is now added to the Retry delay for throttled exception. Adds test for retry delays for throttled exceptions. Fixes bug where the throttled retry's math was off.
  * Fixes [aws#45](aws/aws-sdk-go-v2#45)
* `aws` : Adds missing sdk error checking when seeking readers ([aws#379](aws/aws-sdk-go-v2#379))
  * Adds support for nonseekable io.Reader. Adds support for streamed payloads for unsigned body request.
  * Fixes [aws#371](aws/aws-sdk-go-v2#371)
* `service/s3` : Fixes unexpected EOF error by s3manager ([aws#386](aws/aws-sdk-go-v2#386))
  * Fixes bug which threw unexpected EOF error when s3 upload is performed for a file of maximum allowed size
  * Fixes [aws#316](aws/aws-sdk-go-v2#316)
* `private/model` : Fixes generated API Reference docs links being invalid ([387](aws/aws-sdk-go-v2#387))
  * Fixes [aws#327](aws/aws-sdk-go-v2#327)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request A feature should be added or improved.
Projects
None yet
Development

No branches or pull requests

2 participants