New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
First implementation of the Bazel toolchain #8991
Changes from all commits
57b4bcc
c55b677
39a9a11
9e1ea3d
52d8272
3fd5ecd
f9e5a30
6d17dc3
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
from conan.tools.google.toolchain import BazelToolchain | ||
from conan.tools.google.bazeldeps import BazelDeps | ||
from conan.tools.google.bazel import Bazel |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
import os | ||
import json | ||
|
||
from conan.tools import CONAN_TOOLCHAIN_ARGS_FILE | ||
from conans.util.files import load | ||
|
||
class Bazel(object): | ||
def __init__(self, conanfile): | ||
self._conanfile = conanfile | ||
self._get_bazel_project_configuration() | ||
|
||
def configure(self, args=None): | ||
pass | ||
|
||
def build(self, args=None, label=None): | ||
# TODO: Change the directory where bazel builds the project (by default, /var/tmp/_bazel_<username> ) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. About the TODO. Why? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's @memsharded's suggestion. He said that the project should be built in the conan's cache. Shouldn't it? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure that the concept "the directory where bazel builds the project" makes sense, as much as 'the directory of the Bazel project' bazel must be run from somewhere in the source tree under a WORKSPACE file. It executes each build action in a symlink tree created for the action and places results in bazel-out in trees under bazel-out and bazel-bin. After building a particular label or labels, they will appear somewhere under bazel-bin. From there, you might want to copy them out to conan's cache. It's hard to make a particular suggestion without seeing an example of how all the intermediate files should fit together. For example, what would you expect for a configure based project that depends on libfoo where libfoo is built by Bazel. You obviously need to have the glue that says
Somehow we need to be able to tell configure not to used the installed libfoo. It needs to use the file under bazel-bin. At least, that is what I presume. I don't know the conan workflow enough to know. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hi @aiuto, thanks for the feedback! Let me briefly clarify the different flows: For the consumer point of view: a project using bazel to build that wants to use Conan existing packages as dependencies:
For the producer point of view, a Conan package which build system is Bazel.
It would be good if Bazel could be instructed to put the temporary build files in that location, instead of somewhere in There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's look at the consumer view. WORKSPACE
BUILD
and the glue generated by the WORKSPACE
BUILD
libmylib.a: symlink to libmylib.a built as a result of doing the conan build. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The merged generator is pretty similar to your idea. The dependencies are declared in the
Then, the only thing you have to add to the
After that, just update the
Give it a try and let us know what you think :-) |
||
|
||
bazelrc_path = '--bazelrc={}'.format(self._bazelrc_path) if self._bazelrc_path else '' | ||
bazel_config = '--config={}'.format(self._bazel_config) if self._bazel_config else '' | ||
|
||
# arch = self._conanfile.settings.get_safe("arch") | ||
# cpu = { | ||
# "armv8": "arm64", | ||
# "x86_64": "" | ||
# }.get(arch, arch) | ||
# | ||
# command = 'bazel {} build --sandbox_debug --subcommands=pretty_print --cpu={} {} {}'.format( | ||
# bazelrc_path, | ||
# cpu, | ||
# bazel_config, | ||
# label | ||
# ) | ||
|
||
command = 'bazel {} build {} {}'.format( | ||
bazelrc_path, | ||
bazel_config, | ||
label | ||
) | ||
|
||
self._conanfile.run(command) | ||
|
||
def _get_bazel_project_configuration(self): | ||
self._bazel_config = None | ||
self._bazelrc_path = None | ||
|
||
if os.path.exists(CONAN_TOOLCHAIN_ARGS_FILE): | ||
conan_toolchain_args = json.loads(load(CONAN_TOOLCHAIN_ARGS_FILE)) | ||
self._bazel_config = conan_toolchain_args.get("bazel_config", None) | ||
self._bazelrc_path = conan_toolchain_args.get("bazelrc_path", None) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,122 @@ | ||
import textwrap | ||
|
||
from jinja2 import Template | ||
|
||
from conans.util.files import save | ||
|
||
|
||
class BazelDeps(object): | ||
def __init__(self, conanfile): | ||
self._conanfile = conanfile | ||
|
||
def generate(self): | ||
local_repositories = [] | ||
for dependency in self._conanfile.dependencies.transitive_host_requires: | ||
content = self._get_dependency_buildfile_content(dependency) | ||
filename = self._save_dependendy_buildfile(dependency, content) | ||
|
||
local_repository = self._create_new_local_repository(dependency, filename) | ||
local_repositories.append(local_repository) | ||
|
||
content = self._get_main_buildfile_content(local_repositories) | ||
self._save_main_buildfiles(content) | ||
|
||
def _save_dependendy_buildfile(self, dependency, buildfile_content): | ||
filename = 'conandeps/{}/BUILD'.format(dependency.ref.name) | ||
save(filename, buildfile_content) | ||
return filename | ||
|
||
def _get_dependency_buildfile_content(self, dependency): | ||
template = textwrap.dedent(""" | ||
load("@rules_cc//cc:defs.bzl", "cc_import", "cc_library") | ||
|
||
{% for lib in libs %} | ||
cc_import( | ||
name = "{{ lib }}_precompiled", | ||
static_library = "{{ libdir }}/lib{{ lib }}.a" | ||
) | ||
{% endfor %} | ||
|
||
cc_library( | ||
name = "{{ name }}", | ||
{% if headers %} | ||
hdrs = glob([{{ headers }}]), | ||
{% endif %} | ||
{% if includes %} | ||
includes = [{{ includes }}], | ||
{% endif %} | ||
{% if defines %} | ||
defines = [{{ defines }}], | ||
{% endif %} | ||
visibility = ["//visibility:public"] | ||
) | ||
|
||
""") | ||
|
||
dependency.new_cpp_info.aggregate_components() | ||
|
||
if not dependency.new_cpp_info.libs and not dependency.new_cpp_info.includedirs: | ||
return None | ||
|
||
headers = [] | ||
includes = [] | ||
|
||
for path in dependency.new_cpp_info.includedirs: | ||
headers.append('"{}/**"'.format(path)) | ||
includes.append('"{}"'.format(path)) | ||
|
||
headers = ', '.join(headers) | ||
includes = ', '.join(includes) | ||
|
||
defines = ('"{}"'.format(define) for define in dependency.new_cpp_info.defines) | ||
defines = ', '.join(defines) | ||
|
||
context = { | ||
"name": dependency.ref.name, | ||
"libs": dependency.new_cpp_info.libs, | ||
"libdir": dependency.new_cpp_info.libdirs[0], | ||
"headers": headers, | ||
"includes": includes, | ||
"defines": defines | ||
} | ||
|
||
content = Template(template).render(**context) | ||
return content | ||
|
||
def _create_new_local_repository(self, dependency, dependency_buildfile_name): | ||
snippet = textwrap.dedent(""" | ||
native.new_local_repository( | ||
name="{}", | ||
path="{}", | ||
build_file="{}", | ||
) | ||
""").format( | ||
dependency.ref.name, | ||
dependency.package_folder, | ||
dependency_buildfile_name | ||
) | ||
|
||
return snippet | ||
|
||
def _get_main_buildfile_content(self, local_repositories): | ||
template = textwrap.dedent(""" | ||
def load_conan_dependencies(): | ||
{} | ||
""") | ||
|
||
if local_repositories: | ||
function_content = "\n".join(local_repositories) | ||
function_content = ' '.join(line for line in function_content.splitlines(True)) | ||
else: | ||
function_content = ' pass' | ||
|
||
content = template.format(function_content) | ||
|
||
return content | ||
|
||
def _save_main_buildfiles(self, content): | ||
# A BUILD file must exist, even if it's empty, in order for bazel | ||
# to detect it as a bazel package and allow to load the .bzl files | ||
save("conandeps/BUILD", "") | ||
|
||
save("conandeps/dependencies.bzl", content) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
import json | ||
|
||
from conan.tools import CONAN_TOOLCHAIN_ARGS_FILE | ||
from conans.util.files import save | ||
|
||
|
||
class BazelToolchain(object): | ||
|
||
def __init__(self, conanfile): | ||
self._conanfile = conanfile | ||
|
||
def generate(self): | ||
bazel_config = self._conanfile.conf["tools.google.bazel:config"] | ||
bazelrc_path = self._conanfile.conf["tools.google.bazel:bazelrc_path"] | ||
|
||
save(CONAN_TOOLCHAIN_ARGS_FILE, json.dumps({ | ||
"bazel_config": bazel_config, | ||
"bazelrc_path": bazelrc_path | ||
})) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If there is no configure step for bazel you can nicely remove it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As I said to @memsharded in the other PR (I had to close it due to merge complications):
As far as I've seen, the convention when writing the build() method on a project's conanfile.py is:
Wouldn't deleting the configure method break this pattern?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not necessarily, the
MSBuild
helper does not haveconfigure()
step. It can be removed.