action_env not passed in to GoCompilePkg actions

[rabbbit]

What version of rules_go are you using?

"https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.34.0/rules_go-v0.34.0.zip",

What version of gazelle are you using?

bazel-gazelle-v0.26.0.tar.gz

What version of Bazel are you using?

> bazel version
Build label: 5.2.0

Does this issue reproduce with the latest releases of all the above?

rules_go is at the latest, this seem to be a go, not bazel issue.

What operating system and processor architecture are you using?

macos, intel.

Any other potentially useful information about your toolchain?

Don't think so.

What did you do?

bazel build -s :mytarget --action_env=GOAMD64=v4

What did you expect to see?

SUBCOMMAND: #  .... [action 'GoCompilePkg src/...',
 configuration: b98097a8192cd37499844c985c17d0baa5f64f76406c1ab83bdd5c41bb00cddb, execution platform: @local_config_platform//:host]                                                                                                                                                     
(cd /private/var/tmp/_bazel_pawel/592aedee41e8f957ae1ee499da1dedeb/execroot/__main__ && \                                                                                                                                                 
exec env - \                                                                                                                                                                                                                             
 CGO_ENABLED=0 \                                                                                                                                                                                                                         
GOARCH=amd64 \                                                                                                                                                                                                                         
 GOOS=linux \                                                                                                                                                                                                                           
 GOPATH='' \      
GOAMD64=v4 \      <<<<<<<<<<<<<<<<<<<<<<<<<<================ HERE                                                                                                                                                                                                                    
GOROOT=bazel-out/darwin-fastbuild-ST-8f6eb6207b9d/bin/external/io_bazel_rules_go/stdlib_ \                                                                                                                                              
GOROOT_FINAL=GOROOT \ 

What did you see instead?

SUBCOMMAND: #  .... [action 'GoCompilePkg src/...', 
configuration: b98097a8192cd37499844c985c17d0baa5f64f76406c1ab83bdd5c41bb00cddb, execution platform: 
@local_config_platform//:host]                                                                                                                                                     (cd 
/private/var/tmp/_bazel_pawel/592aedee41e8f957ae1ee499da1dedeb/execroot/__main__ && \                                                                                                                                                 
exec env - \                                                                                                                                                                                                                              
CGO_ENABLED=0 \                                                                                                                                                                                                                         
GOARCH=amd64 \                                                                                                                                                                                                                          
GOOS=linux \                                                                                                                                                                                                                            
GOPATH='' \                                                                                                                                                                                                                             
GOROOT=bazel-out/darwin-fastbuild-ST-8f6eb6207b9d/bin/external/io_bazel_rules_go/stdlib_ \                                                                                                                                              
GOROOT_FINAL=GOROOT \ 

It seems a bit weird, because I do the flag being passed in to other actions

SUBCOMMAND: # @com_google_protobuf//:api_proto [action 'Generating Descriptor Set proto_library 
@com_google_protobuf//:api_proto', configuration: 
17ab12c7286ec7c0e3ae7d5b7031d9ba6bf19204c617431f8d60e2ee53b7d12a, execution platform: 
@local_config_platform//:host]                                                                                                                                                                                                          
(cd /private/var/tmp/_bazel_pawel/592aedee41e8f957ae1ee499da1dedeb/execroot/__main__ && \                                                                                                                                                 
exec env - \                                                                                                                                                                                                                              
GOAMD64=v7 \  

[fmeum]

Could you try and see what happens if you add a line use_default_shell_env = True here?

I am not yet at a point where I fully understand the implications of adding this line, see bazelbuild/rules_nodejs#1548 for some context.

Could you elaborate on why you need to set an environment variable for the compile action? Could we also expose this functionality in other ways?

[rabbbit]

Thanks for a quick response!

Could you elaborate on why you need to set an environment variable for the compile action? Could we also expose this functionality in other ways?

I don't think the functionality can be exposed in other ways - I'm after GOAMD64 env variable added in Go1.18 - https://github.com/golang/go/wiki/MinimumRequirements#amd64.

Could you try and see what happens if you add a line use_default_shell_env = True here?

I'm at all not familiar with bazel/rules_go, but after some patching, I think it had the reverse effect?

I'm getting the below:

compilepkg: GOROOT not set

After some guessing, it looks like all environment variables are removed? This is what I see by "default".

SUBCOMMAND: # @com_github_samuel_go_thrift//parser:parser [action 'GoCompilePkg external/com_github_samuel_go_thrift/parser/parser.a [for host]', configuration: 3e91b918ec515c683678e3b9094e7f37d15199e29b2032dc369e07f1d4add100, execution platform: @local_config_platform//:host]                                                                                                                                                                                           (cd /private/var/tmp/_bazel_pawel/592aedee41e8f957ae1ee499da1dedeb/execroot/__main__ && \                                                                                                                                                 
exec env - \                                                                                                                                                                                                                              
CGO_ENABLED=1 \                         
    GOARCH=amd64 \                                                                                                  
    GOOS=darwin \                                  
    GOPATH='' \                                                                                                                                                                                                                             GOROOT=external/go_sdk \                                                                                        
    GOROOT_FINAL=GOROOT \                                                                                           
    PATH=/usr/bin:external/local_config_cc:/bin \
  bazel-out/host/bin/external/go_sdk/builder compilepkg .....

This is what I see with the patch:

SUBCOMMAND: # @com_github_samuel_go_thrift//parser:parser [action 'GoCompilePkg external/com_github_samuel_go_thrift/parser/parser.a [for host]', configuration: 
3e91b918ec515c683678e3b9094e7f37d15199e29b2032dc369e07f1d4add100, execution platform: @local_config_platform//:host]                                                                                                                                                                                           (cd /private/var/tmp/_bazel_pawel/592aedee41e8f957ae1ee499da1dedeb/execroot/__main__ && \                                                                                                                                                 
exec env - \                                                                                                                                                                                                                              
PATH=/bin:/usr/bin:/usr/local/bin \                                                                                                                                                                                                   
bazel-out/host/bin/external/go_sdk/builder compilepkg .......

So it looks like my whole environment apart from the PATH (which is actually different) gets wiped. I'm not entirely sure how it is set in the first place though.

This would be somewhat inline with bazelbuild/rules_nodejs#1548 (comment) which suggests if use_default_shell_env is used a lot of other action_envs need to be set. This also seems related to #2057 (comment) - where it surprised @jayconrod too.

[fmeum]

Thanks for the summary, I think use_default_shell_env isn't usable for us here in its current form.

In my opinion, the canonical representation of a microarchitecture level is a platform constraint. I created #3251, could you take a look and say whether this models your use case?

[rabbbit]

Interesting. I didn't even know platforms or constraints existed. This means that for any new build ENV variables you'd need to define new constraints ... which seems painful, but also reasonable - the builds will definitely be more hermetic.

This solves my current use-case (testing). I think "long-term" is also fine - we'd likely want to enable this by default for all builds, but it looks like it should be easy enough to configure (?). @linzhp you know much more about our internal setup - could you take a look too?

[fmeum]

Platform support within Bazel is still being improved, but I'm positive that it owns the future.