Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unable to compile syn #159

Open
mobileink opened this issue May 26, 2020 · 7 comments
Open

unable to compile syn #159

mobileink opened this issue May 26, 2020 · 7 comments

Comments

@mobileink
Copy link
Contributor

mobileink commented May 26, 2020

I'm new to Rust but not to Bazel. I've spent most of the past two days trying to compile syn. I've learned a lot, but I admit defeat.

MWE using unroll, which depends on syn:

~/tmp/unroll/WORKSPACE:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(name = "io_bazel_rules_rust",
             url = "https://github.com/bazelbuild/rules_rust/archive/57c6f4ec6b25b07958d91c658d0142dc0943e848.tar.gz",
             strip_prefix = "rules_rust-57c6f4ec6b25b07958d91c658d0142dc0943e848",
             sha256 = "8ceeeedcd55f444dce73494ad01f0d20551ff68d91ab655bf98043a672c65300"
)
http_archive(name = "bazel_skylib",
             sha256 = "9a737999532daca978a158f94e77e9af6a6a169709c0cee274f0a4c3359519bd",
             strip_prefix = "bazel-skylib-1.0.0",
             url = "https://github.com/bazelbuild/bazel-skylib/archive/1.0.0.tar.gz",
)
load("@io_bazel_rules_rust//rust:repositories.bzl", "rust_repositories")
rust_repositories(version = "1.43.1")
load("@io_bazel_rules_rust//:workspace.bzl", "bazel_version")
bazel_version(name = "bazel_version")

~/tmp/unroll/cargo/Cargo.toml:

[package]
name = "compile_with_bazel"
version = "0.1.0"
[dependencies]
unroll = "0.1.4"
[lib]
path = "fake_lib.rs"
[raze]
workspace_path = "//cargo"
target = "x86_64-unknown-linux-gnu"
output_buildfile_suffix = "BUILD.bazel"
[raze.crates.proc-macro2.'0.4.30']
gen_buildrs = true
[raze.crates.syn.'0.15.44']
gen_buildrs = true

Run cargo vendor --versioned-dirs && cargo raze from the cargo dir. Then cd cargo/vendor/syn-0.15.44 && cargo build. Builds ok. Then try building with Bazel, from ~/tmp/unroll: $ bazel build //cargo/vendor/syn-0.15.44:syn. Fails with

error[E0277]: the trait bound `proc_macro2::TokenStream: std::convert::From<proc_macro::TokenStream>` is not satisfied
   --> cargo/vendor/syn-0.15.44/src/buffer.rs:109:20
    |
109 |         Self::new2(stream.into())
    |                    ^^^^^^^^^^^^^ the trait `std::convert::From<proc_macro::TokenStream>` is not implemented for `proc_macro2::TokenStream`

Ditto if you try to build unroll. If you remove proc-macro from the syn build targets, it builds ok, but then unroll breaks with a different error.

I've tried a hundred variations on this, compared the compile commands, etc. and nothing works. I'm out of ideas, I hope somebody can help.

Thanks,

Gregg

@kornholi
Copy link
Contributor

kornholi commented May 29, 2020

This smells like gen_buildrs not working correctly with proc-macro2 (could you check what cargo-raze generates?).

I'd try manually configuring the cfg flags:

[raze.crates.proc-macro2.'xxx']
additional_flags = [
    "--cfg=use_proc_macro",
    "--cfg=wrap_proc_macro",
]

@mobileink
Copy link
Contributor Author

mobileink commented May 29, 2020

@kornholi
Copy link
Contributor

kornholi commented May 30, 2020

I just tried your example and it builds fine after my suggested change:

$ bazel build //cargo/vendor/syn-0.15.44:syn
INFO: Invocation ID: 3c00aff8-7e90-4676-9324-f498474e64d4
INFO: Analyzed target //cargo/vendor/syn-0.15.44:syn (4 packages loaded, 95 targets configured).
INFO: Found 1 target...
Target //cargo/vendor/syn-0.15.44:syn up-to-date:
  bazel-out/k8-fastbuild-ST-5e74b77704d3a70b08875590eb0f067cbb9a6e09f41f090f307cf0d79d4b2461/bin/cargo/vendor/syn-0.15.44/libsyn-158068785.rlib
INFO: Elapsed time: 9.685s, Critical Path: 9.27s
INFO: 3 processes: 3 linux-sandbox.
INFO: Build completed successfully, 4 total actions

It did fail with the same error beforehand. We don't really ever use gen_buildrs but I don't think it can propagate compiler options due to Bazel's architecture.

  1. Perhaps you forgot to run cargo raze after changing Cargo.toml?
  2. Are you running latest/recent cargo-raze? Your comment about the editions makes me suspicious as edition related issues were fixed a long time ago.

those flags must be passed as
"--cfg foo" (no = sign), as I found by examining the output of cargo build.

--cfg foo and --cfg=foo are equivalent in the eyes of rustc

@UebelAndre
Copy link
Collaborator

UebelAndre commented Aug 9, 2020

Why is this necessary? It seems like there are a few issues about being unable to compile without needing to add some flags to specific dependencies. Is there an issue outlining why that is?

Maybe @acmcarther might know?

@GregBowyer
Copy link
Contributor

GregBowyer commented Nov 4, 2020

Sorry I just noticed this, the issue with needed flags typically comes from build.rs files.

If you look at proc_macro2 you will find a build.rs like so https://github.com/alexcrichton/proc-macro2/blob/master/build.rs

This file is run by cargo to configure the build, run generators, do autoconf style workloads. Even if you were to run the build.rs (gen_buildrs = true) there is (to my knowledge) no good way of getting those flags into the bazel rules for the rust target (and even if there was, its kinda a way to kiss goodbye to being hermetic).

This ship sailed in cargo and is known to rust and us (https://internals.rust-lang.org/t/build-script-capabilities/8635).

Initially AFAIK the idea was to avoid C style #ifdef-o-rama, e.g. just how do you detect in a sane way you are x86 64bit solaris with say a none typically libc? However what I think happened was we wound up reimplementing it in many crates in subtle and different ways.

There is an RFC to work towards addressing this gap for the whole does my environment look like rustc using llvm on windows type deal (https://github.com/rust-lang/rfcs/blob/master/text/2523-cfg-path-version.md)

With that I would guess that 80% of build.rs work would vanish as people ould start using the following over feature flags and build.rs env sniffing.

#[cfg(accessible(::std::u128))]
impl From<u128> for MyFancyCryptoThing {}

For the record I thing the other 10% of build.rs work is typically generating source codes which is reasonably handled with gen_buildrs = True, 8% is native compilation (pkg-config, cc etc) and 2% is a hedge of mine for anything just .. well ... weird.

For a reference of all the things that bite people during a build #41

I have a crazy side project to infer the bulk of these flags using symbolic execution but I doubt I will complete it any time soon.

@dayfine
Copy link

dayfine commented Jul 3, 2022

Any update on this? I might have hit this when trying to use https://crates.io/crates/strum_macros which depends on syn.

ERROR: /private/var/tmp/_bazel_difan/f78d9d7b4392e86afba7ecb9acf5e2e3/external/raze__strum_macros__0_24_2/BUILD.bazel:34:16: Compiling Rust proc-macro strum_macros v0.24.2 (19 files) failed: (Exit 1): process_wrapper failed: error executing command
  (cd /private/var/tmp/_bazel_difan/f78d9d7b4392e86afba7ecb9acf5e2e3/sandbox/darwin-sandbox/219/execroot/xlab && \
  exec env - \
    APPLE_SDK_PLATFORM=MacOSX \
    APPLE_SDK_VERSION_OVERRIDE=12.3 \
    CARGO_CFG_TARGET_ARCH=x86_64 \
    CARGO_CFG_TARGET_OS=darwin \
    CARGO_CRATE_NAME=strum_macros \
    CARGO_MANIFEST_DIR='${pwd}/external/raze__strum_macros__0_24_2' \
    CARGO_PKG_AUTHORS='' \
    CARGO_PKG_DESCRIPTION='' \
    CARGO_PKG_HOMEPAGE='' \
    CARGO_PKG_NAME=strum_macros \
    CARGO_PKG_VERSION=0.24.2 \
    CARGO_PKG_VERSION_MAJOR=0 \
    CARGO_PKG_VERSION_MINOR=24 \
    CARGO_PKG_VERSION_PATCH=2 \
    CARGO_PKG_VERSION_PRE='' \
    SYSROOT=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/rust_darwin_x86_64/toolchain_for_x86_64-apple-darwin_impl \
    XCODE_VERSION_OVERRIDE=13.4.1.13F100 \
    ZERO_AR_DATE=1 \
  bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/rules_rust/util/process_wrapper/process_wrapper --arg-file bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__proc_macro2__1_0_40/proc_macro2_build_script.linksearchpaths --arg-file bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__quote__1_0_20/quote_build_script.linksearchpaths --arg-file bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__syn__1_0_98/syn_build_script.linksearchpaths --arg-file bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__rustversion__1_0_7/rustversion_build_script.linksearchpaths --subst 'pwd=${pwd}' -- bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/rust_darwin_x86_64/toolchain_for_x86_64-apple-darwin_impl/bin/rustc external/raze__strum_macros__0_24_2/src/lib.rs '--crate-name=strum_macros' '--crate-type=proc-macro' '--error-format=human' '--codegen=metadata=-225036374' '--codegen=extra-filename=-225036374' '--out-dir=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__strum_macros__0_24_2' '--codegen=opt-level=3' '--codegen=debuginfo=0' '--remap-path-prefix=${pwd}=.' '--emit=dep-info,link' '--color=always' '--target=x86_64-apple-darwin' -L bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/rust_darwin_x86_64/toolchain_for_x86_64-apple-darwin_impl/lib/rustlib/x86_64-apple-darwin/lib '--cap-lints=allow' '--edition=2018' '--codegen=linker=external/local_config_cc/cc_wrapper.sh' --codegen 'link-args=-lc++ -fobjc-link-runtime -headerpad_max_install_names -lSystem -lresolv -lSystem -lresolv -lSystem -lresolv -lSystem -lresolv -no-canonical-prefixes -target x86_64-apple-macosx -mmacosx-version-min=12.3 -lc++ -target x86_64-apple-macosx' '--extern=heck=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__heck__0_4_0/libheck-3309503150.rlib' '--extern=proc_macro2=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__proc_macro2__1_0_40/libproc_macro2-2982902350.rlib' '--extern=quote=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__quote__1_0_20/libquote-1640270559.rlib' '--extern=syn=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__syn__1_0_98/libsyn-487446987.rlib' '--extern=rustversion=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__rustversion__1_0_7/librustversion-1507692243.dylib' '-Ldependency=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__heck__0_4_0' '-Ldependency=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__unicode_ident__1_0_1' '-Ldependency=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__proc_macro2__1_0_40' '-Ldependency=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__quote__1_0_20' '-Ldependency=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__syn__1_0_98' '-Ldependency=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__rustversion__1_0_7' --extern proc_macro)
# Configuration: 096ff4561e7e561745a0b7194853903b6ad0770355c59136f4208699a5df52ed
# Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox
error[E0432]: unresolved import `syn::parse`
 --> external/raze__strum_macros__0_24_2/src/helpers/case_style.rs:6:5
  |
6 |     parse::{Parse, ParseStream},
  |     ^^^^^ could not find `parse` in `syn`

@bsilver8192
Copy link
Contributor

bsilver8192 commented Sep 16, 2022

Except for recent breakage fixed by #524, this works for me now (I know because I first noticed that problem with syn breaking strum_macros).

I think the work since this issue was filed around features fixed it, and @dayfine 's recent comment is due to the recent breakage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants