Access absolute output path

[teh]

Really liking buck2 so far, great work!

I've been trying to implement a proto_library rule in starlark but am stuck on a small thing - when creating the command I need access to the absolute directory of an artifact in order to pass it to protoc, i.e. what do I pass for ??? in the snippet below? If I use e.g. . then protoc just writes into my current git checkout.

cmd = cmd_args(["protoc", "--cpp_out=???"])

I've tried various combinations of

• $(location) => does not get substituted
• declaring an output with dir=True => "conflicts with the following output paths"

The closest thing I can find in the code is in genrule.bzl but also doesn't appear to work (nor would I expect it to work based on the Rust code)

"GEN_DIR": cmd_args("GEN_DIR_DEPRECATED"),  # ctx.relpath(ctx.output_root_dir(), srcs_path)

[dtolnay]

Take a look at how rustdoc works. It also writes to an output directory.

https://github.com/facebookincubator/buck2/blob/485b689d9b124b1c670bb242caa4b2c687428971/prelude/rust/build.bzl#L132

[ndmitchell]

Generally you would declare an output directory, and write it out.

output = ctx.actions.declare_output("protoc", dir=True)
cmd = cmd_args("protoc", "--cpp_out", output.as_output())

You will normally get an error about overlapping paths if you have defined that one output is in foo, and then you say the whole output is in . because the . output will overlap/contain the foo, so is no longer isolated from foo.

The above will given you a relative path, which I guess is probably fine (it will be relative to the root of the repo, which will also be the $PWD for the command). If you really need absolute, then use realpath in a shell script to convert, but I suspect relative is just fine.

If you have to do --cpp_out=something rather than just --cpp_out something, then the approach @dtolnay described is the way to do it.

[teh]

With that approach I get an output directory. I'm using cxx_library_parameterized though which needs srcs, see ctx.attrs.srcs.extend(ccs) in the snippet below.

glob isn't available so I'm missing the link to go from a directory of files to a list of individual source artifacts. Is there anything to support that?

def proto_library_impl(ctx: "context") -> ["provider"]:
    headers = []
    ccs = []

    out = ctx.actions.declare_output("out", dir=True)
    ctx.attrs.headers.append(out)
    cmd = cmd_args(["protoc", "--cpp_out", out.as_output()])

    for s in ctx.attrs.srcs:
        name = paths.replace_extension(s.short_path, "")
        h = ctx.actions.declare_output(out.short_path + "/" + name + ".pb.h") # conflict warning 
        cc = ctx.actions.declare_output(out.short_path + "/" + name + ".pb.cc") # conflict warning
        cmd.hidden([h.as_output(), cc.as_output()])
        ccs.extend([cc])
        headers.extend([h])

    cmd.add(ctx.attrs.srcs)

    # replace sources because cxx_library_parameterized uses ctx.srcs
    ctx.actions.run(cmd, category="protoc", identifier = "compile", no_outputs_cleanup = True)
    ctx.attrs.srcs.clear()
    ctx.attrs.srcs.extend(ccs)

    # ... snip much more code to call cxx_library_parameterized

[teh]

After a bit more of trial and error I landed on the following solution which relies on the fact that rule
outputs are generated in the same directory. This way I can declare non-conflicting
individual outputs and the out directory for protoc.

out = ctx.actions.declare_output('out')
cmd = cmd_args("protoc")
# ... snip
cmd.add("--cpp_out")
cmd.add(cmd_args(out.as_output()).parent())
cmd.add(ctx.attrs.srcs)

I don't know whether this is always true but works for now.