Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

py_binary requires a C++ toolchain #8751

Open
ghost opened this issue Jun 28, 2019 · 11 comments
Open

py_binary requires a C++ toolchain #8751

ghost opened this issue Jun 28, 2019 · 11 comments
Labels
not stale Issues or PRs that are inactive but not considered stale P4 This is either out of scope or we don't have bandwidth to review a PR. (No assignee) team-Rules-Python Native rules for Python type: feature request

Comments

@ghost
Copy link

ghost commented Jun 28, 2019 •
edited by ghost

Description of the problem / feature request:

On a Linux machine without GCC installed, I am unable to build py_binary targets out of the box.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

$ touch WORKSPACE
$ cat > BUILD <<EOF
> py_binary(
>     name = "dummy",
>     srcs = ["dummy.py"],
> )
> EOF
$ touch dummy.py
$ bazel build :dummy
...
Auto-Configuration Error: Cannot find gcc or CC; either correct your path or set the CC environment variable

What operating system are you running Bazel on?

CentOS 7.2

What's the output of bazel info release?

release 0.26.0+vmware

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

n/a

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

n/a

Have you found anything relevant by searching the web?

Any other information, logs, or outputs that you want to share?

From #5133, setting BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 leads to a different, more concise error:

$ BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 bazel build :dummy
ERROR: While resolving toolchains for target //:dummy: no matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type
ERROR: Analysis of target '//:dummy' failed; build aborted: no matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type
INFO: Elapsed time: 0.092s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 0 targets configured)

Surprisingly, I am able to work around this by setting CC=/bin/false, confirming that a C/C++ compiler isn't actually required.
@keith
Copy link
Member

keith commented Jun 28, 2019

I believe this was fixed by c73aa26

More conversation in the bazel slack https://bazelbuild.slack.com/archives/CA306CEV6/p1560966350017900

@ghost
Copy link
Author

ghost commented Jun 28, 2019

I believe this was fixed by c73aa26

More conversation in the bazel slack https://bazelbuild.slack.com/archives/CA306CEV6/p1560966350017900

Oh, neat! I'll backport / build Bazel at HEAD and try again.

@ghost
Copy link
Author

ghost commented Jun 28, 2019

I believe this was fixed by c73aa26
More conversation in the bazel slack https://bazelbuild.slack.com/archives/CA306CEV6/p1560966350017900

Oh, neat! I'll backport / build Bazel at HEAD and try again.

I don't want to rule out PEBKAC just yet, but I'm seeing the same behavior after cherry-picking this back to our 0.26.0 branch. I'll double check my work and try building Bazel from master next week.

@ishikhman
Copy link
Contributor

@beasleyr-vmw please update this issue ones you do the test with HEAD, thanks!

@ghost
Copy link
Author

ghost commented Jul 1, 2019 •
edited by ghost

@beasleyr-vmw please update this issue ones you do the test with HEAD, thanks!

Yup, running into the same problem (but with a new reporting bug) on master (HEAD is e6ab3c4):

user@host dummy> bazel build :dummy
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Invocation ID: 7d1dd6a1-a73c-4f70-b762-31a50ab14a70
INFO: Call stack for the definition of repository 'local_config_cc' which is a cc_autoconf (rule definition at .../external/bazel_tools/tools/cpp/cc_configure.bzl:121:15):
 - .../external/bazel_tools/tools/cpp/cc_configure.bzl:164:5
 - /DEFAULT.WORKSPACE.SUFFIX:315:1
ERROR: An error occurred during the fetch of repository 'local_config_cc':
   Traceback (most recent call last):
        File ".../external/bazel_tools/tools/cpp/cc_configure.bzl", line 119
                configure_unix_toolchain(repository_ctx, cpu_value, overriden...)
        File ".../external/bazel_tools/tools/cpp/unix_cc_configure.bzl", line 298, in configure_unix_toolchain
                _find_generic(repository_ctx, "gcc", "CC", overriden...)
        File ".../external/bazel_tools/tools/cpp/unix_cc_configure.bzl", line 269, in _find_generic
                auto_configure_fail(msg)
        File ".../external/bazel_tools/tools/cpp/lib_cc_configure.bzl", line 112, in auto_configure_fail
                fail(("\n%sAuto-Configuration Error:%...)))

Auto-Configuration Error: Cannot find gcc or CC; either correct your path or set the CC environment variable
Internal error thrown during build. Printing stack trace: java.util.UnknownFormatConversionException: Conversion = '.'
user@host dummy> BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 bazel build :dummy                                      
Starting local Bazel server and connecting to it...                                                                                                                                                                                       
INFO: Invocation ID: 69ca4588-705b-473c-9245-9dedb7abc267                                                                                                                                                                                 
ERROR: While resolving toolchains for target //:dummy: no matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type
ERROR: Analysis of target '//:dummy' failed; build aborted: no matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type
INFO: Elapsed time: 3.016s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (10 packages loaded, 24 targets configured)

Scanning bazel query 'deps(:dummy)' output and wondering if there is any possibility this is somehow coming in through the zipper and launcher dependencies. (See https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPyBinaryRule.java#L45-L52 .)

user@host dummy> CC=/bin/false BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 bazel query 'deps(:dummy)'                        
//:dummy                                                  
@bazel_tools//tools/zip:zipper                            
@bazel_tools//tools/zip:zipper/zipper                     
@bazel_tools//third_party/ijar:zipper                     
@bazel_tools//third_party/ijar:zip_main.cc                
@bazel_tools//third_party/ijar:zip                        
@bazel_tools//third_party/ijar:zlib_client                
@bazel_tools//third_party/zlib:zlib                       
@bazel_tools//third_party/zlib:zutil.h                    
@bazel_tools//third_party/zlib:zutil.c                    
@bazel_tools//third_party/zlib:zlib.h                     
@bazel_tools//third_party/zlib:zconf.h                    
@bazel_tools//third_party/zlib:uncompr.c                  
@bazel_tools//third_party/zlib:trees.h                    
@bazel_tools//third_party/zlib:trees.c                    
@bazel_tools//third_party/zlib:inftrees.h                 
@bazel_tools//third_party/zlib:inftrees.c                 
@bazel_tools//third_party/zlib:inflate.h                  
@bazel_tools//third_party/zlib:inflate.c                  
@bazel_tools//third_party/zlib:inffixed.h                 
@bazel_tools//third_party/zlib:inffast.h                  
@bazel_tools//third_party/zlib:inffast.c                  
@bazel_tools//third_party/zlib:infback.c                  
@bazel_tools//third_party/zlib:gzwrite.c                  
@bazel_tools//third_party/zlib:gzread.c                   
@bazel_tools//third_party/zlib:gzlib.c                    
@bazel_tools//third_party/zlib:gzguts.h                   
@bazel_tools//third_party/zlib:gzclose.c                  
@bazel_tools//third_party/zlib:deflate.h                  
@bazel_tools//third_party/zlib:deflate.c                  
@bazel_tools//third_party/zlib:crc32.h                    
@bazel_tools//third_party/zlib:crc32.c                    
@bazel_tools//third_party/zlib:compress.c                 
@bazel_tools//third_party/zlib:adler32.c                  
@bazel_tools//third_party/ijar:zlib_client.h              
@bazel_tools//third_party/ijar:zlib_client.cc             
@bazel_tools//third_party/ijar:zip.h                      
@bazel_tools//third_party/ijar:zip.cc                     
@bazel_tools//third_party/ijar:platform_utils             
@bazel_tools//third_party/ijar:platform_utils.h           
@bazel_tools//third_party/ijar:platform_utils.cc          
@bazel_tools//third_party/ijar:mapped_file_windows.cc     
@bazel_tools//third_party/ijar:mapped_file_unix.cc        
@bazel_tools//third_party/ijar:mapped_file.h              
@bazel_tools//third_party/ijar:common.h                   
@bazel_tools//src:windows                                 
@bazel_tools//tools/python:2to3                           
@bazel_tools//tools/launcher:launcher                     
@bazel_tools//tools/launcher:launcher_windows             
@bazel_tools//tools/launcher:launcher.exe                 
@bazel_tools//src/tools/launcher:launcher                 
@bazel_tools//src/tools/launcher:python_launcher          
@bazel_tools//src/tools/launcher:python_launcher.h        
@bazel_tools//src/tools/launcher:python_launcher.cc       
@bazel_tools//src/tools/launcher:launcher_main.cc         
@bazel_tools//src/tools/launcher:java_launcher            
@bazel_tools//src/tools/launcher:java_launcher.h          
@bazel_tools//src/tools/launcher:java_launcher.cc         
@bazel_tools//src/tools/launcher:bash_launcher            
@bazel_tools//src/tools/launcher:launcher_base            
@bazel_tools//src/tools/launcher/util:data_parser         
@bazel_tools//src/tools/launcher/util:util                
@bazel_tools//src/tools/launcher/util:launcher_util.h     
@bazel_tools//src/tools/launcher/util:launcher_util.cc    
@bazel_tools//src/tools/launcher/util:dummy.cc            
@bazel_tools//src/tools/launcher/util:data_parser.h       
@bazel_tools//src/tools/launcher/util:data_parser.cc      
@bazel_tools//src/tools/launcher:launcher.h               
@bazel_tools//src/tools/launcher:launcher.cc              
@bazel_tools//src/main/cpp/util:filesystem                
@bazel_tools//src/main/native/windows:lib-file            
@bazel_tools//src/main/native/windows:util.h              
@bazel_tools//src/main/native/windows:util.cc             
@bazel_tools//src/main/native/windows:file.h              
@bazel_tools//src/main/native/windows:file.cc             
@bazel_tools//src/main/cpp/util:path_windows.cc           
@bazel_tools//src/main/cpp/util:path_posix.cc             
@bazel_tools//src/main/cpp/util:path_platform.h           
@bazel_tools//src/main/cpp/util:path.h                    
@bazel_tools//src/main/cpp/util:path.cc                   
@bazel_tools//src/main/cpp/util:file_windows.cc           
@bazel_tools//src/main/cpp/util:file_posix.cc             
@bazel_tools//src/main/cpp/util:file_platform.h           
@bazel_tools//src/main/cpp/util:file.h                    
@bazel_tools//src/main/cpp/util:file.cc                   
@bazel_tools//src/main/cpp/util:errors                    
@bazel_tools//src/main/cpp/util:port                      
@bazel_tools//src/main/cpp/util:port.h                    
@bazel_tools//src/main/cpp/util:port.cc                   
@bazel_tools//src/main/cpp/util:logging                   
@bazel_tools//src/main/cpp/util:strings                   
@bazel_tools//src/main/cpp/util:strings.h                 
@bazel_tools//src/main/cpp/util:strings.cc                
@bazel_tools//src/main/cpp/util:logging.h                 
@bazel_tools//src/main/cpp/util:logging.cc                
@bazel_tools//src/main/cpp/util:ijar                      
@bazel_tools//src/main/cpp/util:errors_windows.cc         
@bazel_tools//src/main/cpp/util:errors_posix.cc           
@bazel_tools//src/main/cpp/util:errors.h                  
@bazel_tools//src/main/cpp/util:blaze_exit_code           
@bazel_tools//tools/def_parser:def_parser                 
@bazel_tools//tools/def_parser:no_op.bat                  
@bazel_tools//tools/def_parser:def_parser_windows         
@bazel_tools//tools/def_parser:def_parser.exe             
@bazel_tools//third_party/def_parser:def_parser           
@bazel_tools//tools/cpp:malloc                            
@bazel_tools//third_party/def_parser:def_parser_main.cc   
@bazel_tools//third_party/def_parser:def_parser_lib       
@bazel_tools//third_party/def_parser:def_parser.h         
@bazel_tools//third_party/def_parser:def_parser.cc        
@bazel_tools//src/conditions:remote                       
@bazel_tools//src/main/cpp/util:exit_code.h               
@bazel_tools//src/tools/launcher:dummy.cc                 
@bazel_tools//src/tools/launcher:bash_launcher.h          
@bazel_tools//src/tools/launcher:bash_launcher.cc         
@bazel_tools//src/conditions:windows                      
@bazel_tools//src/conditions:host_windows                 
@bazel_tools//tools/cpp:toolchain                         
//external:cc_toolchain                                   
@local_config_cc//:toolchain                              
@local_config_cc//:local                                  
@local_config_cc//:local_config                           
@local_config_cc//:empty                                  
@bazel_tools//tools/cpp:link_dynamic_library              
@bazel_tools//tools/cpp:link_dynamic_library.sh           
@bazel_tools//tools/cpp:interface_library_builder         
@bazel_tools//tools/cpp:build_interface_so                
@bazel_tools//tools/cpp:grep-includes                     
@bazel_tools//tools/cpp:grep-includes.sh                  
//:dummy.py

@brandjon
Copy link
Member

brandjon commented Jul 8, 2019

It's certainly possible we're still requiring a cpp toolchain through dependencies like the zipper, despite removing the direct implicit dependency from the py_binary rule. I don't think that's something we're likely to eliminate anytime soon. In fact, we may add more things that require cpp compilation, such as a native launcher to replace the stub script.

@brandjon brandjon added P4 This is either out of scope or we don't have bandwidth to review a PR. (No assignee) type: feature request and removed type: bug untriaged labels Jul 8, 2019
@EdSchouten
Copy link
Contributor

Though I have absolutely no problems with py_binary() dragging in C++ toolchains to do all sorts of fancy things, isn't there anything that could be done to improve Bazel's analysis to defer the resolution of the toolchain to the actual point where it's being used? I've noticed that this error currently triggers, even in cases where no actual C++ code ends up being built. For example, when using rules_docker's py3_image().

I am currently working around this by declaring a bogus C++ toolchain using cc_toolchain()+cc_toolchain_config()+toolchain()+register_toolchains().

@ulfjack
Copy link
Contributor

ulfjack commented Feb 17, 2020

Not easily. The py_binary rule has to declare the dependency before it can know whether it's necessary. The built-in rule collects all transitive cc_library rules and then creates a link action (for which it needs a cc toolchain). Technically, that can only happen if the cc_library rules already have a cc toolchain dependency, but there's currently no infrastructure to inherit the toolchain in that case.

One possible option might be to add a tag or attribute to disable the dependency. That would require manual work, though.

Another option might be to find a way to forward the cc toolchain dep from the cc_library. I'm not sure if that's feasible though.

@EdSchouten
Copy link
Contributor

Thanks for clarifying!

@uhthomas uhthomas mentioned this issue Jan 14, 2022
Open
@sgowroji sgowroji added the stale Issues or PRs that are stale (no activity for 30 days) label Feb 15, 2023
@sgowroji
Copy link
Member

Hi there! We're doing a clean up of old issues and will be closing this one. Please reopen if you’d like to discuss anything further. We’ll respond as soon as we have the bandwidth/resources to do so.

@meastham
Copy link

I believe this is still relevant. Would it be possible to use the new optional toolchain support to solve the problem?

@sgowroji sgowroji reopened this Feb 15, 2023
@sgowroji sgowroji added not stale Issues or PRs that are inactive but not considered stale and removed stale Issues or PRs that are stale (no activity for 30 days) labels Feb 15, 2023
@jesses-canva jesses-canva mentioned this issue Mar 27, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
not stale Issues or PRs that are inactive but not considered stale P4 This is either out of scope or we don't have bandwidth to review a PR. (No assignee) team-Rules-Python Native rules for Python type: feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@meastham @keith @EdSchouten @brandjon @ulfjack @ishikhman @sgowroji