Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fails to build on Nix #184

Open
knarkzel opened this issue Mar 20, 2023 · 19 comments
Open

Fails to build on Nix #184

knarkzel opened this issue Mar 20, 2023 · 19 comments
Labels
enhancement New feature or request

Comments

@knarkzel
Copy link

I have everything setup except for the esp-idf which this crate should handle. However, when attempting to build, I get following:

esp32-hello-world via (nix-shell-env) ❯ cargo build
    Updating crates.io index
   Compiling esp-idf-sys v0.32.1
error: failed to run custom build command for `esp-idf-sys v0.32.1`

Caused by:
  process didn't exit successfully: `/home/odd/source/nix/temp/esp32-hello-world/target/debug/build/esp-idf-sys-abdf284afcdfa8df/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-env-changed=ESP_IDF_TOOLS_INSTALL_DIR
  cargo:rerun-if-env-changed=ESP_IDF_SDKCONFIG
  cargo:rerun-if-env-changed=ESP_IDF_SDKCONFIG_DEFAULTS
  cargo:rerun-if-env-changed=MCU
  cargo:rerun-if-env-changed=ESP_IDF_SYS_ROOT_CRATE
  cargo:rerun-if-env-changed=ESP_IDF_VERSION
  cargo:rerun-if-env-changed=ESP_IDF_REPOSITORY
  cargo:rerun-if-env-changed=ESP_IDF_CMAKE_GENERATOR
  cargo:rerun-if-env-changed=IDF_PATH
  cargo:rerun-if-env-changed=EXTRA-COMPONENTS
  cargo:rerun-if-env-changed=ESP_IDF_COMPONENTS
  PATH=/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4/tools:$PATH

  --- stderr
  Build configuration: BuildConfig {
      esp_idf_tools_install_dir: None,
      esp_idf_sdkconfig: None,
      esp_idf_sdkconfig_defaults: None,
      mcu: None,
      native: NativeConfig {
          esp_idf_version: Some(
              Branch(
                  "release/v4.4",
              ),
          ),
          esp_idf_repository: None,
          esp_idf_cmake_generator: None,
          idf_path: None,
          extra_components: [],
          esp_idf_components: None,
      },
      esp_idf_sys_root_crate: None,
  }
  Using managed esp-idf repository: RemoteSdk { repo_url: None, git_ref: Branch("release/v4.4") }
  fatal: No names found, cannot describe anything.
  fatal: No names found, cannot describe anything.
  WARNING: Git describe was unsuccessful: b''
  ERROR: This script was called from a virtual environment, can not create a virtual environment again
  Error: Could not install esp-idf

  Caused by:
      command '"python3" "/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4/tools/idf_tools.py" "--idf-path" "/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4" "--non-interactive" "install-python-env"' exited with non-zero status code 1

All the code is located here.

@N3xed
Copy link
Collaborator

N3xed commented Mar 20, 2023

  ERROR: This script was called from a virtual environment, can not create a virtual environment again
  Error: Could not install esp-idf

  Caused by:
      command '"python3" "/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4/tools/idf_tools.py" "--idf-path" "/home/odd/source/nix/temp/esp32-hello-world/.embuild/espressif/esp-idf/release-v4.4" "--non-interactive" "install-python-env"' exited with non-zero status code 1

Make sure that python3 doesn't come from a virtual env (see Determine if Python is running inside virtualenv for more info).
The esp-idf-sys will try to install the needed tools which includes creating a python virtualenv.

@knarkzel
Copy link
Author

knarkzel commented Oct 13, 2023

Ok, I've changed the shell now:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    esp-dev = {
      url = "github:thiskappaisgrey/nixpkgs-esp-dev-rust";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = {
    self,
    nixpkgs,
    esp-dev,
  }: let
    pkgs = import nixpkgs {
      system = "x86_64-linux";
      overlays = [ esp-dev.overlay ];
    };
  in {
    devShells.x86_64-linux.default = pkgs.mkShell {
      buildInputs = with pkgs; [
        gcc-riscv32-esp32c3-elf-bin
        openocd-esp32-bin

        # Tools required to use ESP-IDF.
        git
        wget
        gnumake

        flex
        bison
        gperf
        pkg-config

        cmake
        ninja

        ncurses5

        python3
        python3Packages.pip
        python3Packages.virtualenv
      ];

      shellHook = ''
        export ESP_IDF_VERSION="v4.4"
        export LD_LIBRARY_PATH="${pkgs.lib.makeLibraryPath [ pkgs.libxml2 pkgs.zlib pkgs.stdenv.cc.cc.lib ]}"
        export LIBCLANG_PATH=${pkgs.libclang.lib}/lib
      '';
    };
  };
}

Now I get this:

  [939/941] Linking C executable libespidf.elf
  [940/941] Generating binary image from built executable
  esptool.py v3.2-dev
  Merged 1 ELF section
  Generated /home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/build/libespidf.bin
  [941/941] cd /home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/build/esp-idf/esptool_py && /home/odd/source/nix/hello-esp32c3/.embuild/espressif/python_env/idf4.4_py3.10_env/bin/python /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/partition_table/check_sizes.py --offset 0x8000 partition --type app /home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/build/partition_table/partition-table.bin /home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/build/libespidf.bin
  libespidf.bin binary size 0x21db0 bytes. Smallest app partition is 0x100000 bytes. 0xde250 bytes (87%) free.
  cargo:root=/home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out
  cargo:rerun-if-changed=/home/odd/.cargo/registry/src/index.crates.io-6f17d22bba15001f/esp-idf-sys-0.33.2/src/include/esp-idf/bindings.h
  cargo:rustc-env=EMBUILD_GENERATED_BINDINGS_FILE=/home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/bindings.rs

  --- stderr
  Build configuration: BuildConfig {
      esp_idf_tools_install_dir: None,
      esp_idf_sdkconfig: None,
      esp_idf_sdkconfig_defaults: None,
      mcu: None,
      native: NativeConfig {
          esp_idf_version: Some(
              Tag(
                  "v4.4",
              ),
          ),
          esp_idf_repository: None,
          esp_idf_cmake_generator: None,
          idf_path: None,
          extra_components: [],
          esp_idf_components: None,
      },
      esp_idf_sys_root_crate: None,
  }
  Using managed esp-idf repository: RemoteSdk { repo_url: None, git_ref: Tag("v4.4") }
  Using esp-idf v4.4.0 at '/home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4'
  WARNING: directory for tool riscv32-esp-elf version esp-2021r2-8.4.0 is present, but tool was not found
  WARNING: destination path already exists, removing
  WARNING: directory for tool cmake version 3.20.3 is present, but tool was not found
  WARNING: destination path already exists, removing
  WARNING: directory for tool ninja version 1.10.2 is present, but tool was not found
  WARNING: destination path already exists, removing
  WARNING: directory for tool riscv32-esp-elf version esp-12.2.0_20230208 is present, but tool was not found
  WARNING: destination path already exists, removing
  /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/tools/check_python_dependencies.py:12: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources
  CMake Warning:
    Manually-specified variables were not used by the project:

      CMAKE_TOOLCHAIN_FILE


  Built components: esp_ringbuf, efuse, esp_ipc, driver, esp_pm, mbedtls, bootloader, esptool_py, partition_table, app_update, bootloader_support, spi_flash, nvs_flash, pthread, esp_gdbstub, espcoredump, esp_phy, esp_system, esp_rom, hal, vfs, esp_eth, tcpip_adapter, esp_netif, esp_event, wpa_supplicant, esp_wifi, ieee802154, console, openthread, lwip, log, heap, soc, esp_hw_support, riscv, esp32c3, esp_common, esp_timer, freertos, newlib, cxx, app_trace, asio, bt, cbor, unity, cmock, coap, nghttp, esp-tls, esp_adc_cal, esp_hid, tcp_transport, esp_http_client, esp_http_server, esp_https_ota, esp_https_server, esp_lcd, protobuf-c, protocomm, mdns, esp_local_ctrl, sdmmc, esp_serial_slave_link, esp_websocket_client, expat, wear_levelling, fatfs, freemodbus, idf_test, jsmn, json, libsodium, mqtt, openssl, spiffs, usb, tinyusb, wifi_provisioning
  /nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev/include/limits.h:145:5: error: function-like macro '__GLIBC_USE' is not defined
  /nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev/include/limits.h:184:5: error: function-like macro '__GLIBC_USE' is not defined
  /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/freertos/port/riscv/include/freertos/portmacro.h:439:20: warning: section does not match previous declaration [-Wsection]
  /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/freertos/port/riscv/include/freertos/portmacro.h:271:20: note: previous attribute is here
  /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/esp_hw_support/include/soc/esp_spiram.h:1:2: warning: esp_spiram.h has been replaced by esp32/spiram.h, please include esp32/spiram.h instead [-W#warnings]
  clang diag: /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/freertos/port/riscv/include/freertos/portmacro.h:439:20: warning: section does not match previous declaration [-Wsection]
  clang diag: /home/odd/source/nix/hello-esp32c3/.embuild/espressif/esp-idf/v4.4/components/esp_hw_support/include/soc/esp_spiram.h:1:2: warning: esp_spiram.h has been replaced by esp32/spiram.h, please include esp32/spiram.h instead [-W#warnings]
  Error: failed to generate bindings in file '/home/odd/source/nix/hello-esp32c3/target/riscv32imc-esp-espidf/debug/build/esp-idf-sys-3444d1415f0a5f40/out/bindings.rs'

  Caused by:
      clang diagnosed error: /nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev/include/limits.h:145:5: error: function-like macro '__GLIBC_USE' is not defined
      /nix/store/y8wfrgk7br5rfz4221lfb9v8w3n0cnyd-glibc-2.37-8-dev/include/limits.h:184:5: error: function-like macro '__GLIBC_USE' is not defined

@yuri91
Copy link

yuri91 commented Oct 27, 2023

@knarkzel I am trying to make esp-idf for rust work on nix, same as you.

I took a step back and side-stepped the issue of the idf source not being a git repo, by unsetting IDF_PATH and letting esp-idf-sys download it at runtime (in a nix develop session).

Then, I stumbled on this __GLIBC_USE issue.

I think it has to do with nix wrapping clang and adding include paths to it. But esp-idf does not want to pull in host system headers, which are being mixed up with headers from the cross compiler.

So I removed clang from the shell inputs (but kept libclang), and it worked!

It would still be nice to be able to use the esp toolchain from the nix store though.

@shymega
Copy link

shymega commented Nov 24, 2023

This is something I'm trying to get working too.

I've bumped a Flake I forked with the ESP-IDF, but encountering a Git repo error.

I think this will take some work.....

@ChocolateLoverRaj
Copy link

Does anyone have a working flake? I'm also getting the __GLIBC_USE issue.

@auroraanna
Copy link

@ChocolateLoverRaj i do with the dependencies of the flake above (actually without openocd-esp32-bin, flex, bison and gperf) but i put them in an FHS env with nix because the build process tried to execute it's own non-nix executables.

@Vollbrecht
Copy link
Collaborator

Its a two part process:

The first part is us installing essential stuff like the python virt_env + cmake + ninja + either riscv or xtensa gcc compiler needed for the specific ESP_IDF version, That could be more or less statically provided under the assumption that it follows the same layout ESP_IDF expacts.

The second part and the bigger problem is that ESP_IDF also uses pip and a bunch of python dependancys e.g kconfig wrapper and esp-idf-component manager written in python, that depends on other stuff it pulls in via pip. Keep in mind that upstream has this deeply coupled inside there system so just providing something similar is not enough, it will just fail on certain test that are run inline when the cmake buildsystem is executed.

So a complete solution would need to tackle the second part, the first part is more trivial.

@ChocolateLoverRaj
Copy link

@auroraanna can u provide your flake.nix? This is what I'm using atm: https://github.com/ChocolateLoverRaj/rust-esp32c3-examples/blob/main/flake.nix.

@auroraanna
Copy link

auroraanna commented Jun 3, 2024

@ChocolateLoverRaj

{
  inputs = {
    nixpkgs.url = "nixpkgs/nixos-unstable";
  };

  outputs = {
    self,
    nixpkgs
  }: let
    pkgs = import nixpkgs {
      system = "x86_64-linux";
    };
    fhs = pkgs.buildFHSUserEnv {
      name = "fhs-shell";
      targetPkgs = pkgs: with pkgs; [
        gcc

        pkg-config
        libclang.lib
        gnumake
        cmake
        ninja

        git
        wget

        rustup
        cargo-generate

        espflash
        python3
        python3Packages.pip
        python3Packages.virtualenv
      ];
    };
  in {
    devShells.${pkgs.system}.default = fhs.env;
  };
}

@auroraanna
Copy link

oh noes, just tried to delete all the build directories and see if it still works but somehow there's no compiler

@auroraanna
Copy link

updated the flake now and it works. it seems that with gcc-riscv32-esp32c3-elf-bin there's no compiler in the path and that openocd-esp32-bin isn't needed at all.

@ChocolateLoverRaj
Copy link

I tried your flake.nix but this is the error I get:

[rajas@nixos:~/Documents/rust-esp32c3-examples/VL53L0X]$ cargo r
   Compiling std v0.0.0 (/home/rajas/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std)
   Compiling esp-idf-hal v0.43.1
   Compiling esp-idf-svc v0.48.1
   Compiling esp32c3-vl53l0x v0.1.0 (/home/rajas/Documents/rust-esp32c3-examples/VL53L0X)
error[E0425]: cannot find function `dirfd` in crate `libc`
   --> /home/rajas/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys/pal/unix/fs.rs:898:37
    |
898 |             let fd = unsafe { libc::dirfd(self.0) };
    |                                     ^^^^^ not found in `libc`

For more information about this error, try `rustc --explain E0425`.
error: could not compile `std` (lib) due to 1 previous error

@auroraanna
Copy link

why is your toolchain for x86_64

@ChocolateLoverRaj
Copy link

why is your toolchain for x86_64

Maybe because I have rustup installed with home-manager.

@ar3s3ru
Copy link

ar3s3ru commented Jun 5, 2024

Hi all, hitting a similar issue, can't install using the toolchain provided with my Nix devenv.

It seems to be due to the fact that the folder where I have esp-idf installed (and compiled already) is not a Git repository. esp-idf is set up using the following overlay: https://github.com/mirrexagon/nixpkgs-esp-dev/tree/master

Code can be found here mostly under ./flake.nix and ./firmware: https://github.com/openmoto-org/kontroller/tree/3afc4c3d3f1719c5b877a401ff1d380883fd1d6b

The error log:

cargo build --target riscv32imc-esp-espidf
   Compiling esp-idf-sys v0.34.1
   Compiling firmware v0.1.0 (/Users/ar3s3ru/Code/openmoto-org/kontroller/firmware)
warning: function `main` is never used
 --> build.rs:4:4
  |
4 | fn main() {
  |    ^^^^
  |
  = note: `#[warn(dead_code)]` on by default

The following warnings were emitted during compilation:

warning: esp-idf-sys@0.34.1: (esp-idf-sys) Removed env vars that might affect the ESP IDF C build: `CC=clang;CXX=clang++`

error: failed to run custom build command for `esp-idf-sys v0.34.1`

Caused by:
  process didn't exit successfully: `/Users/ar3s3ru/Code/openmoto-org/kontroller/firmware/target/debug/build/esp-idf-sys-ba1c29dbb5b2bdf7/build-script-build` (exit status: 1)
  --- stdout
  cargo:warning=(esp-idf-sys) Removed env vars that might affect the ESP IDF C build: `CC=clang;CXX=clang++`
  cargo:rerun-if-env-changed=ESP_IDF_TOOLS_INSTALL_DIR
  cargo:rerun-if-env-changed=ESP_IDF_SDKCONFIG
  cargo:rerun-if-env-changed=ESP_IDF_SDKCONFIG_DEFAULTS
  cargo:rerun-if-env-changed=MCU
  cargo:rerun-if-env-changed=ESP_IDF_SYS_ROOT_CRATE
  cargo:rerun-if-env-changed=ESP_IDF_VERSION
  cargo:rerun-if-env-changed=ESP_IDF_REPOSITORY
  cargo:rerun-if-env-changed=ESP_IDF_CMAKE_GENERATOR
  cargo:rerun-if-env-changed=IDF_PATH
  cargo:rerun-if-env-changed=EXTRA-COMPONENTS
  cargo:rerun-if-env-changed=ESP_IDF_COMPONENTS
  cargo:rerun-if-env-changed=ESP_IDF_COMPONENT_MANAGER

  --- stderr
  Build configuration: BuildConfig {
      esp_idf_tools_install_dir: None,
      esp_idf_sdkconfig: None,
      esp_idf_sdkconfig_defaults: Some(
          [
              "/Users/ar3s3ru/Code/openmoto-org/kontroller/firmware/sdkconfig.defaults",
          ],
      ),
      mcu: Some(
          "esp32c3",
      ),
      native: NativeConfig {
          esp_idf_version: None,
          esp_idf_repository: None,
          esp_idf_cmake_generator: None,
          idf_path: Some(
              "/nix/store/j7nh3a5rz8br2qligxl4xfav68rsn26q-esp-idf-v5.1.3",
          ),
          extra_components: [],
          esp_idf_components: None,
          esp_idf_component_manager: None,
      },
      esp_idf_sys_root_crate: None,
  }
  fatal: not a git repository (or any parent up to mount point /)
  Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
  fatal: not a git repository (or any parent up to mount point /)
  Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
  Error: '/nix/store/j7nh3a5rz8br2qligxl4xfav68rsn26q-esp-idf-v5.1.3' is not a git respository

  Caused by:
      0: command 'cd "/nix/store/j7nh3a5rz8br2qligxl4xfav68rsn26q-esp-idf-v5.1.3" && LANGUAGE="" LC_ALL="C.UTF-8" "git" "rev-parse" "--show-toplevel"' exited with non-zero status code 128
      1: fatal: not a git repository (or any parent up to mount point /)
         Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
warning: build failed, waiting for other jobs to finish...

@ar3s3ru
Copy link

ar3s3ru commented Jun 8, 2024

I managed to make it work, here is a write-up of what was necessary: nix-community/fenix#58 (comment)

Hope it helps!

@ChocolateLoverRaj
Copy link

@ar3s3ru is there any reason to use your Flake rather than mine?

@ChocolateLoverRaj
Copy link

ChocolateLoverRaj commented Jun 8, 2024

why is your toolchain for x86_64

I removed rustup from my home-manager config, rm -rf ~/.rustup, and then used your Flake. And it worked. I just had to add ldproxy to it. This is what works:

{
  inputs = {
    nixpkgs.url = "nixpkgs/nixos-unstable";
  };

  outputs = {
    self,
    nixpkgs
  }: let
    pkgs = import nixpkgs {
      system = "x86_64-linux";
    };
    fhs = pkgs.buildFHSUserEnv {
      name = "fhs-shell";
      targetPkgs = pkgs: with pkgs; [
        gcc

        pkg-config
        libclang.lib
        gnumake
        cmake
        ninja

        git
        wget

        rustup
        cargo-generate

        espflash
        python3
        python3Packages.pip
        python3Packages.virtualenv
        ldproxy
      ];
    };
  in {
    devShells.${pkgs.system}.default = fhs.env;
  };
}

it's nice how your solution doesn't need to set any environmental variables so I'll be using it instead of my previous flake.

Another nice thing is that now I removed rustup and clang from my home-manager and just ran rust-rover . inside nix-develop. Now Rust Rover detects the toolchain I need and there's no NixOS issues with it.

@knarkzel
Copy link
Author

Awesome stuff, seems to be solved for now then? Basically get an environment able to build esp-idf right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Todo
Development

No branches or pull requests

8 participants