[Enhancement]: Reduce size

[pauldraper]

Description

This plugin is 383 MB. (Linux)

That's huge.

Can it be smaller?

Affected Resource(s) and/or Data Source(s)

No response

Potential Terraform Configuration

No response

References

No response

Would you like to implement a fix?

None

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[mattburgess]

I've just run bloaty to try and analyse where the size is coming from. The largest culprit appears to be this:

   FILE SIZE        VM SIZE    
 --------------  -------------- 
  13.5%   104Mi  30.9%   104Mi    [section .gopclntab]

Alas, according to golang/go#36313 there's not much we can do about that directly; that section is needed for certain tools/library calls to work. However, indirectly there might be a way we could decrease the binary size.

   FILE SIZE        VM SIZE    
 --------------  -------------- 
  13.5%   104Mi  30.9%   104Mi    [section .gopclntab]
   4.3%  33.3Mi   3.6%  12.1Mi    github.com/aws/aws-sdk-go-v2/service/ec2
...
   1.8%  13.9Mi   1.2%  4.00Mi    github.com/aws/aws-sdk-go/service/ec2

You'll see there that for the ec2 service there are some parts of the provider using the v1 SDK and others are using the v2 SDK. If we could convert all callers over to the newer SDK we could potentially eliminate 14Mi of binary size. Not massive, in and of itself, but if we repeat that over the various services that are only partially converted over to the newer SDK then we could shave quite some size off. It would also reduce the size of the above mentioned .gopclntab section as the symbols, etc. from that old SDK won't end up in there either.

Unfortunately, that migration work is sizeable & boring (adds no new features in and of itself) so is unlikely to be tackled with any particular fervour I wouldn't have thought. I don't work for or represent Hashicorp though so am only basing that assumption on how I'd prioritize things if I was maintaining a project with issue and PR lists of this project's size. There's also this note from the Contributor Guide:

At present, we are slowly increasing our footprint on SDK v2, but are not actively migrating existing code to use v2. The choice of SDK will be as follows:

For new services, you should use AWS SDK for Go v2. AWS has a migration guide that details the differences between the versions of the SDK.

For existing services, use the version of the SDK that service currently uses. You can determine this by looking at the import section in the service's Go files.

If anyone wants to reproduce my findings or investigate in any more depth I had to have a number of attempts to get the analysis to work:

# Running bloaty on a release binary fails because it's stripped:
$ bloaty -d compileunits -n 0 ~/go/bin/terraform-provider-aws 
bloaty: missing debug info

# Running bloaty on a 'standard' `go install` build fails, presumably due to a bug in bloaty's DWARF parsing?
$ go install
$ bloaty -d compileunits -n 0 ~/go/bin/terraform-provider-aws 
bloaty: premature EOF reading fixed-length DWARF data

# Not compressing the DWARF data got things working
$ go install -ldflags=-compressdwarf=false
$ bloaty -d compileunits -n 0 ~/go/bin/terraform-provider-aws 

   FILE SIZE        VM SIZE    
 --------------  -------------- 
...

[pauldraper]

Thanks for looking at this!

I've looked too, and the builds are indeed using the low-handing fruit of stripping the binary (-s -w).

I don't think there is anything actionable, but I thought I'd see if anyone had ideas.