[bazel] Add support for building lldb

[keith]

This adds build configuration for building LLDB on macOS and Linux. It uses a default subset of features that should work out of the box with macOS + Ubuntu. It is notably missing python support right now, although some of the scaffolding is there, because of the complexity of linking a python dylib, especially if you plan to distribute the resulting liblldb.so.

Most of this build file is pretty simple, one of the unfortunate patterns I had to use was to split the header and sources cc_library targets to break circular dependencies.

[aaronmondal]

Wow this looks pretty good!

The only concern I have is the -l* in the linkopts. I remember going to fairly great lengths to remove those for other third-party deps like zlib because they made the build susceptible to cache poisoning and had reproducibility issues. The deps used there are fairly complex, so I'm not sure how viable it is to add all of them as third-party deps. However, the bazel central registry seems to have at least some of them. Maybe we could borrow some build files from there.

[keith]

thanks for the review!

[chsigg]

Hi Keith, the bazel buildkite is not happy with this change.

ERROR: Target @llvm-project//lldb:DebugServerCommon is incompatible and cannot be built, but was explicitly requested.
Dependency chain:
    @llvm-project//lldb:DebugServerCommon (4f455a)
    @llvm-project//lldb:DebugServerMacOSX (189b94)   <-- target platform (@local_config_platform//:host) didn't satisfy constraint @platforms//:incompatible

Is this something you can fix forward?

[hokein]

IIUC, the debugserver binary and its dependencies (e.g. DebugServerCommon) are only available on MacOS. If we build these targets explicitly with bazel on Linux, bazel will give the incompatible and cannot be built, but was explicitly requested. error message.

The bazel buildkite setup is like:

1. run bazelisk query //... + @llvm-project//... command to list all available targets;
2. run bazelisk test on each target which is list in step1;

Step1 lists some incompatible targets like DebugServerCommon which causes the issue. I guess the fix is to not emit these incompatible targets in step1 for non-MacOS platforms.

[keith]

Would it be safe to change that CI configuration to only do a bazel test @llvm-project//...? Otherwise I think we'd need to manually filter out incompatible targets, since query //... doesn't do that for us

[chsigg]

I don't know the reason behind bazel query + bazel test. There probably was one at some point, but I can't find where it came from. It always seemed a bit surprising, exactly because it will trip up on incompatible targets and also ignore things like manual tags.

[rupprecht]

I landed bee3377 to make buildkite green.

As for why we do query+test instead of just test: I think this comes from https://github.com/openxla/iree/blob/bb7e536ecd31ea24007aa7202f3a0c41b897e05c/build_tools/bazel/build_test_all.sh

# We use `bazel query //...` piped to `bazel test` rather than the simpler
# `bazel test //...` because the latter excludes targets tagged "manual". The
# "manual" tag allows targets to be excluded from human wildcard builds, but we
# want them built by CI unless they are excluded with tags.

While that may be true for iree, I don't think it's (currently) true for LLVM. i.e., I don't know if we have any tags = ["manual"] stuff that we explicitly want to build in CI.

---

Separate from the buildkite issue: we have some internal rules for building LLDB, and they look a bit different than this, but that's partly because it requires a bunch of other internal stuff. I'll see if there's anything we have that we can contribute.

[keith]

If the goal is to include manual targets, we can instead do:

bazel query "attr(tags, 'manual', //...)"

and test those with test //... + those