Bazel project with Lombok fails to build on Bazel 4.0.0 rc10

[bradsherman]

Description of the problem / feature request:

I have a java project that uses Project Lombok. Currently it is building with bazel v3.7.2 but when I try to upgrade to bazel v4.0.0rc10, the build errors out because it cannot find symbols that are being generated by lombok.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

git clone https://github.com/bradsherman/examples.git
cd examples/java-tutorial
git checkout lombok-bazel
wget https://github.com/bazelbuild/bazel/releases/download/3.7.2/bazel-3.7.2-installer-linux-x86_64.sh
chmod +x bazel-3.7.2-installer-linux-x86_64.sh
./bazel-3.7.2-installer-linux-x86_64.sh --user
~/bin/bazel run //:ProjectRunner
# builds, prints out "Hi!", then "Test"

wget https://releases.bazel.build/4.0.0/rc10/bazel-4.0.0rc10-installer-linux-x86_64.sh
chmod +x bazel-4.0.0rc10-installer-linux-x86_64.sh
./bazel-4.0.0rc10-installer-linux-x86_64.sh --user
~/bin/bazel run //:ProjectRunner
# build error out "cannot find symbol"

What operating system are you running Bazel on?

Ubuntu 18.04

What's the output of bazel info release?

release 4.0.0rc10

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

https://github.com/bradsherman/examples.git
4183fc709c26a00366665e2d60d70521dc0b405d
22236db1d29e75a8165a94f5dc9cb10ff890480c

[cushon]

cc @comius

I can reproduce:

$ bazel run :ProjectRunner 
...
src/main/java/com/example/ProjectRunner.java:8: error: constructor LombokExample in class LombokExample cannot be applied to given types;
        LombokExample l = new LombokExample("Test");
                          ^
  required: no arguments
  found: String
  reason: actual and formal argument lists differ in length
src/main/java/com/example/ProjectRunner.java:9: error: cannot find symbol
        System.out.println(l.getName());
                            ^
  symbol:   method getName()
  location: variable l of type LombokExample

The lombok processor logs the following warning when running as part of the action that generates bazel-bin/src/main/java/com/example/cmdline/liblombok_example-gensrc.jar. The diagnostic doesn't cause that action to fail because lombok logs it as a warning and not an error, and it doesn't show up on the console due to a different bug.

<>: error: You aren't using a compiler supported by lombok, so lombok will not work and has been disabled.
Your processor is: com.google.turbine.processing.TurbineProcessingEnvironment
Lombok supports: OpenJDK javac, ECJ

Since lombok doesn't run, the symbols it generates (like LombokExample.getName) aren't visible later.

This is the result of 10ffddb, because lombok relies on javac internals and doesn't support the non-javac-based turbine implementation.

It's possible to work around this by disabling the 'header compilation' feature by passing --nojava_header_compilation. (The main downside is that some builds will get slower, the output should be identical.)

[bradsherman]

Thank you for the explanation and the workaround. Passing --nojava_header_compilation worked for me!

[illicitonion]

Would it be possible to add an attribute or tag or similar to disable turbine for individual targets, rather than at a whole-build level? It'd be nice to get most of the speed-up, and only fall back to ijar where it's really necessary.

[cushon]

I agree having an attribute to configure this for individual targets might be nice, I'll defer to @comius on the priority of that for Bazel.

I can also think of a few alternatives that work today:

• It's possible to disable --java_header_compilation on particular targets (or in general to set native Bazel flags) using configuration transitions, e.g. for the example in Bazel project with Lombok fails to build on Bazel 4.0.0 rc10 #12837 (comment):

def _java_header_compilation_transition(settings, attr):
    _ignore = (settings, attr)
    return {"//command_line_option:java_header_compilation": "False"}

java_header_compilation_transition = transition(
    implementation = _java_header_compilation_transition,
    inputs = [],
    outputs = ["//command_line_option:java_header_compilation"],
)

def _java_library_without_header_compilation(ctx):
    return [java_common.merge([d[JavaInfo] for d in ctx.attr.dep])]

java_library_without_header_compilation = rule(
    implementation = _java_library_without_header_compilation,
    attrs = {
        "dep": attr.label(
            providers = [JavaInfo],
            mandatory = True,
            cfg = java_header_compilation_transition,
        ),
        "_allowlist_function_transition": attr.label(
            default = "@bazel_tools//tools/allowlists/function_transition_allowlist",
        ),
    },
)

load("//:java_library_without_header_compilation.bzl", "java_library_without_header_compilation")

java_library_without_header_compilation(
    name = "lombok_example",
    dep = ":lombok_example_impl",
    visibility = ["//visibility:public"],
)

java_library(
    name = "lombok_example_impl",
    srcs = ["LombokExample.java"],
    deps = ["@lombok"],
)

• Another option would be to use https://projectlombok.org/features/delombok in a genrule, to preprocess the code using lombok separately from compilation.

[hobofan]

As it took me quite some time to figure out a good workaround (the mentioned ones didn't work) as a Bazel newcomer, here is how I got it working.

Assuming a installed version of 1.18.20 for lombok via maven_install, add the following to your BUILD file:

java_library(
    name = "lombok",
    exports = [
        "@maven//:org_projectlombok_lombok",
    ],
    exported_plugins = [
        ":lombok_plugin"
    ],
)
java_plugin(
    name = "lombok_plugin",
    processor_class = "lombok.launch.AnnotationProcessorHider$AnnotationProcessor",
    deps = [
        ":lombok_jar",
    ],
)
java_import(
    name = "lombok_jar",
    jars = [
      "@maven//:v1/https/repo1.maven.org/maven2/org/projectlombok/lombok/1.18.20/lombok-1.18.20.jar"
    ],
)

:lombok can now be used as a dep for your java_library:

java_library(
    name = "mylib",
    srcs = glob(["src/main/java/**/*.java"]),
    deps = [
        ":lombok",
        # ... more deps
    ],
)

[cushon]

@hobofan that example isn't affected. The issue is that the API generated by lombok for mylib won't be visible to libraries that depend on it. If everything is compiled in one java_library, or if the API generated by lombok isn't used by downstream java_library rules, this should have always worked.

If you have an example like:

java_library(
    name = "a",
    srcs = ["A.java"],
    deps = [
        ":lombok",
    ],
)

java_library(
    name = "b",
    srcs = ["B.java"],
    deps = [
        ":a",
    ],
)

Where B.java refers to the lombok-generated APIs in A.java, that's when the workarounds in #12837 (comment) are necessary.

[hobofan]

Hmm strange. I was only trying to use the methods generated by lombok from inside the same library and this was the only way I was able to get it to work.

I don't know if the underlying cause is the same, but at least from a users perspective it looked the same as the reported issues, so maybe someone else with the same problem as me will stumble upon it and find it helpful 🤷

[cushon]

I was only trying to use the methods generated by lombok from inside the same library and this was the only way I was able to get it to work.

What else did you try that didn't work? Is the report more that java_plugin and exported_plugins were unintuitive?

[hobofan]

What else did you try that didn't work?

IIRC I tried these things in roughly that order:

• Putting "@maven//:org_projectlombok_lombok" into deps of my java_library; Starting point, just like I was used to from Maven
• The --nojava_header_compilation workaround; No visible change, but I also didn't encounter the turbine warning/error mentioned above which made me think something was off
• Putting "@maven//:org_projectlombok_lombok" into plugins of my java_library; Fails with an error as java_plugin is expected and not a jvm_import
• I tried the route via delombok in a genrule; This where I probably spent the most time (trying to also learn the basics of writing a macro or rule). One thing I found a bit surprising is that there isn't a rule like e.g. java_run that makes it easy to invoke an existing .jar file with the ability to set the environment (CLASSPATH, etc.) in a similar way to how you would do for a java_library via deps. I hope that makes sense? Without it, I deemed it to complicated to try and construct the environment myself, and abandoned that route.

And then I finally found that workaround, or rather "verbose correct way". I think bazelbuild/rules_jvm_external#144 is probably the right way to handle it and what I was missing, but all web searches for Bazel + lombok led me here with seemingly fitting error messages.

[manuelnaranjo]

We've created some rules that do delombok: https://github.com/bookingcom/rules_lombok_java_library

And we have a sample project that shows usage as well based on the examples shared on this thread

[nvachhar]

What else did you try that didn't work?

IIRC I tried these things in roughly that order:

• Putting "@maven//:org_projectlombok_lombok" into deps of my java_library; Starting point, just like I was used to from Maven
• The --nojava_header_compilation workaround; No visible change, but I also didn't encounter the turbine warning/error mentioned above which made me think something was off
• Putting "@maven//:org_projectlombok_lombok" into plugins of my java_library; Fails with an error as java_plugin is expected and not a jvm_import
• I tried the route via delombok in a genrule; This where I probably spent the most time (trying to also learn the basics of writing a macro or rule). One thing I found a bit surprising is that there isn't a rule like e.g. java_run that makes it easy to invoke an existing .jar file with the ability to set the environment (CLASSPATH, etc.) in a similar way to how you would do for a java_library via deps. I hope that makes sense? Without it, I deemed it to complicated to try and construct the environment myself, and abandoned that route.

And then I finally found that workaround, or rather "verbose correct way". I think bazelbuild/rules_jvm_external#144 is probably the right way to handle it and what I was missing, but all web searches for Bazel + lombok led me here with seemingly fitting error messages.

I'm also new to Bazel. But I think the original commenter already had Lombok set up as a plugin. So the --nojava_header_compilation workaround is only necessary, as was previously pointed out, if the Lombok generated code is used in a different library/binary than the target using lombok. But for completeness, I think the following is the plugin set up you want:

java_plugin(
    name = "lombok_plugin",
    generates_api = True,
    processor_class = "lombok.launch.AnnotationProcessorHider$AnnotationProcessor",
    deps = [
        "@maven//:v1/https/repo.maven.apache.org/maven2/org/projectlombok/lombok/1.18.22/lombok-1.18.22.jar",
    ],
)

java_library(
    name = "lombok",
    exported_plugins = [
        ":lombok_plugin",
    ],
    visibility = ["//visibility:public"],
    exports = [
        "@maven//:org_projectlombok_lombok",
    ],
)

The 2 differences from what was posted earlier are the generates_api flag and not needing the java_import

[Mivr]

I tried the patch proposed with Bazel 6.0.0 and JVM external rules version 4.5 and got this error:

target '@maven//:v1/https/repo1.maven.org/maven2/org/projectlombok/lombok/1.18.12/lombok-1.18.12.jar' is not visible from target '//:lombok_jar'.

I managed to work around that in the following way:

# some tools file
java_plugin(
    name = "lombok_plugin",
    generates_api = True,
    processor_class = "lombok.launch.AnnotationProcessorHider$AnnotationProcessor",
    deps = ["@maven//:org_projectlombok_lombok"],
)

java_library(
    name = "lombok",
    exported_plugins = [
        ":lombok_plugin",
    ],
    visibility = ["//visibility:public"],
    exports = [
        "@maven//:org_projectlombok_lombok",
    ],
)

# sources build file
java_library(
    name = "java_sourcces_with_lombok_anotations",
    srcs = [
        "ClassWithLombokAnnotations.java",
    ],
    deps = [
        "//:lombok",
    ],
)

[davido]

@cushon

Thanks a lot for the workaround for disabling --java_header_compilation. It works as expected when applied for one of gerrit plugins.

However, I noticed one issue: when creating java_binary from library built with java_library_without_header_compilation, the lombok library is unexpectedly shaded in the final artifact.

This can be of course workaround using virtual lombok-deploy-env binary like this:

java_plugin(
    name = "lombok_plugin",
    generates_api = True,
    processor_class = "lombok.launch.AnnotationProcessorHider$AnnotationProcessor",
    visibility = ["//visibility:public"],
    deps = ["@lombok//jar"],
)

java_library(
    name = "lombok",
    exported_plugins = [":lombok_plugin"],
    visibility = ["//visibility:public"],
    exports = ["@lombok//jar"],
)

java_binary(
    name = "lombok-deploy-env",
    main_class = "Dummy",
    visibility = ["//visibility:public"],
    runtime_deps = [
        ":lombok",
    ],
)

So that now t would work as expected and not leaking the lombok itself into the final artifact:

java_binary(
    name = "github-oauth",
    deploy_env = ["//plugins/github:lombok-deploy-env"],
    main_class = "Dummy",
    runtime_deps = [":github-oauth-lib"],
)

java_library_without_header_compilation(
    name = "github-oauth-lib",
    dep = ":github-oauth-lib-impl",
    visibility = ["//visibility:public"],
)

java_library(
    name = "github-oauth-lib-impl",
    srcs = glob(["src/main/java/**/*.java"]),
    deps = PLUGIN_DEPS_NEVERLINK,
)

Why this complication is happening in the first place, and is there any simpler workarounds?

[cushon]

@davido it looks like your example has a java_binary with a non-neverlink dependency on lombok, so it's expected that the dependency would get linked in to the binary. Is this related to java_library_without_header_compilation, or do you see the same behaviour depending on the underlying java_library?

[cushon]

I'm going to close this out, because it isn't a bug in Bazel. The Lombok annotation processor depends on unsupported compiler internals to work with javac and ecj, and doesn't work with other implementations of the annotation processing API.

Other annotation processors that generate code for value classes, like AutoValue, are fully supported by Bazel. For code using Java 14+, records are also a good option.

For projects that needs to use Lombok with Bazel, I recommend using https://projectlombok.org/features/delombok instead of relying on the annotation processor.

[davido]

Is this related to java_library_without_header_compilation, or do you see the same behavior depending on the underlying java_library?

You are totally right, it's unrelated to java_library_without_header_compilation. I see exactly the same shading behavior also without it.

My impression was, that java_plugin dependencies are excluded from shading into the final artifact. But now I remember, that we had a similar issue for auto-value and friends annotation processors as well, see: [1].

Thanks for mentioning it's a much simpler workaround is to use neverlink = True in lombok's java_library definition:

java_library(
    name = "lombok",
    exported_plugins = [":lombok_plugin"],
    visibility = ["//visibility:public"],
    neverlink = True,
    exports = ["@lombok//jar"],
)

[1] https://gerrit-review.googlesource.com/c/gerrit/+/352391