Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Another attempt to package qsv written in rust #24081

Open
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

peterjc
Copy link
Contributor

@peterjc peterjc commented Sep 26, 2023

Checklist

  • Title of this PR is meaningful: e.g. "Adding my_nifty_package", not "updated meta.yaml".
  • License file is packaged (see here for an example).
  • Source is from official source.
  • Package does not vendor other packages. (If a package uses the source of another package, they should be separate packages or the licenses of all packages need to be packaged).
  • If static libraries are linked in, the license of the static library is packaged.
  • Package does not ship static libraries. If static libraries are needed, follow CFEP-18.
  • Build number is 0.
  • A tarball (url) rather than a repo (e.g. git_url) is used in your recipe (see here for more details).
  • GitHub users listed in the maintainer section have posted a comment confirming they are willing to be listed there.
  • When in trouble, please check our knowledge base documentation before pinging a team.

Continues earlier efforts including #16513

@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/qsv) and found it was in an excellent condition.

This was referenced Sep 26, 2023
@peterjc
Copy link
Contributor Author

peterjc commented Sep 26, 2023

Not a good start, but should be an easy one to fix:

RuntimeError: SHA256 mismatch: '12fa3b54ff805ef9b6e72b29d8355b5b4085a8cda4042a1be790ddd1dca7945b' != 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'

peterjc and others added 6 commits September 26, 2023 21:28
This fix this:

error: Using `cargo install` to install the binaries for the package in current working directory is no longer supported, use `cargo install --path .` instead. Use `cargo build` if you want to simply build the package.
* qsv 18.0 recipe

* second try

patterned after ripgrep recipe
https://github.com/conda-forge/ripgrep-feedstock/tree/master/recipe

* we don't have pcre2 feature

* remove unneeded strip and add missing_dso_whitelist

* added libm.so.6 to whitelist

* run qsv test without parameters

* remove --locked from cargo install

let cargo figure out platform-specific dependencies instead of using cargo.lock file.

Also restored --help parm to qsv test.  The linux and macos errors were caused by something else (regex not being parsed properly by regex crate in docopt crate), which may be caused by cargo --locked parm above.

* Try with pcre2 reqt

* Try with 0.18.2

which now uses docopt v.1.1.0, bringing back all docopt regex features

* with pcre2

* Try for qsv 0.21.0

* tweaks to compile on macos

* try macOS specific build options

* Explicitly setup rust target for macOS

* install rust-std for darwin/osx

* no rustup

* turn on backtrace so we can debug problem

* Another MacOS build attempt w/ qsv 0.24.1

and per @peterjc suggestion

* Update sha256 checksum

* Specify features

* try again with 0.30.1

* do rustup update

as MSRV is 1.58.1

* retry installing rust 1.58.1 via meta.yaml

* try again with 0.31.0

* try specifying rust version

* try with conda branch archive with MSRV set to 1.58

* need python 3.8 for `py` command

* Add python 3.8 as a run requirement per lint

* Don't specify python version

* need python in build stage

* update cargo install command

- remove python,lua
- add luau,fetch,polars,geocode,to
- add --locked option

* first try for 0.114.0

- update source and hash
- remove python from build and run
@peterjc
Copy link
Contributor Author

peterjc commented Sep 26, 2023

New error on linux:

error: failed to compile `qsv v0.115.0 (/home/conda/staged-recipes/build_artifacts/qsv_1695760445157/work)`, intermediate artifacts can be found at `/home/conda/staged-recipes/build_artifacts/qsv_1695760445157/work/target`.
To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

Caused by:
  package `qsv v0.115.0 (/home/conda/staged-recipes/build_artifacts/qsv_1695760445157/work)` cannot be built because it requires rustc 1.72.1 or newer, while the currently active rustc version is 1.72.0
Traceback (most recent call last):
  File "/home/conda/staged-recipes-copy/.ci_support/build_all.py", line 228, in <module>
    build_all(os.path.join(root_dir, "recipes"), args.arch)
  File "/home/conda/staged-recipes-copy/.ci_support/build_all.py", line 118, in build_all
    build_folders(recipes_dir, folders, arch, channel_urls)
  File "/home/conda/staged-recipes-copy/.ci_support/build_all.py", line 174, in build_folders
    conda_build.api.build([recipe], config=get_config(arch, channel_urls))
  File "/opt/conda/lib/python3.10/site-packages/conda_build/api.py", line 180, in build
    return build_tree(
  File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 3078, in build_tree
    packages_from_this = build(metadata, stats,
  File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 2198, in build
    utils.check_call_env(cmd, env=env, rewrite_stdout_env=rewrite_env,
  File "/opt/conda/lib/python3.10/site-packages/conda_build/utils.py", line 451, in check_call_env
    return _func_defaulting_env_to_os_environ("call", *popenargs, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/conda_build/utils.py", line 427, in _func_defaulting_env_to_os_environ
    raise subprocess.CalledProcessError(proc.returncode, _args)
subprocess.CalledProcessError: Command '['/bin/bash', '-o', 'errexit', '/home/conda/staged-recipes/build_artifacts/qsv_1695760445157/work/conda_build.sh']' returned non-zero exit status 101.

Will revert to 0.114.0 which does accept rust 1.72.0

@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/qsv) and found some lint.

Here's what I've got...

For recipes/qsv:

  • requirements: host: rust >= 1.72.0 should not contain a space between relational operator and the version, i.e. rust >=1.72.0

@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/qsv) and found it was in an excellent condition.

@peterjc
Copy link
Contributor Author

peterjc commented Sep 26, 2023

Looks like an (indirect) dependency on C++.

2023-09-26T20:57:51.0271156Z    Compiling arrow-string v34.0.0
2023-09-26T20:57:57.6095689Z    Compiling mlua-sys v0.3.2
2023-09-26T20:57:57.9031888Z error: failed to run custom build command for `libduckdb-sys v0.7.1`
2023-09-26T20:57:57.9032938Z 
2023-09-26T20:57:57.9034192Z Caused by:
2023-09-26T20:57:57.9056307Z   process didn't exit successfully: `/home/conda/staged-recipes/build_artifacts/qsv_1695761611793/work/target/release/build/libduckdb-sys-181fff28972623cd/build-script-build` (exit status: 1)
2023-09-26T20:57:57.9059692Z   --- stdout
2023-09-26T20:57:57.9061045Z   cargo:rerun-if-changed=duckdb/duckdb.hpp
2023-09-26T20:57:57.9062053Z   cargo:rerun-if-changed=duckdb/duckdb.cpp
2023-09-26T20:57:57.9062511Z   TARGET = Some("x86_64-unknown-linux-gnu")
2023-09-26T20:57:57.9062845Z   OPT_LEVEL = Some("3")
2023-09-26T20:57:57.9063242Z   HOST = Some("x86_64-unknown-linux-gnu")
2023-09-26T20:57:57.9063697Z   cargo:rerun-if-env-changed=CXX_x86_64-unknown-linux-gnu
2023-09-26T20:57:57.9064119Z   CXX_x86_64-unknown-linux-gnu = None
2023-09-26T20:57:57.9064564Z   cargo:rerun-if-env-changed=CXX_x86_64_unknown_linux_gnu
2023-09-26T20:57:57.9064969Z   CXX_x86_64_unknown_linux_gnu = None
2023-09-26T20:57:57.9065384Z   cargo:rerun-if-env-changed=HOST_CXX
2023-09-26T20:57:57.9065704Z   HOST_CXX = None
2023-09-26T20:57:57.9066058Z   cargo:rerun-if-env-changed=CXX
2023-09-26T20:57:57.9066368Z   CXX = None
2023-09-26T20:57:57.9066757Z   cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
2023-09-26T20:57:57.9067114Z   CRATE_CC_NO_DEFAULTS = None
2023-09-26T20:57:57.9067402Z   DEBUG = Some("false")
2023-09-26T20:57:57.9067722Z   CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
2023-09-26T20:57:57.9068175Z   cargo:rerun-if-env-changed=CXXFLAGS_x86_64-unknown-linux-gnu
2023-09-26T20:57:57.9068912Z   CXXFLAGS_x86_64-unknown-linux-gnu = None
2023-09-26T20:57:57.9076659Z   cargo:rerun-if-env-changed=CXXFLAGS_x86_64_unknown_linux_gnu
2023-09-26T20:57:57.9077340Z   CXXFLAGS_x86_64_unknown_linux_gnu = None
2023-09-26T20:57:57.9077769Z   cargo:rerun-if-env-changed=HOST_CXXFLAGS
2023-09-26T20:57:57.9078084Z   HOST_CXXFLAGS = None
2023-09-26T20:57:57.9078595Z   cargo:rerun-if-env-changed=CXXFLAGS
2023-09-26T20:57:57.9079546Z   CXXFLAGS = None
2023-09-26T20:57:57.9080771Z   cargo:rerun-if-env-changed=CXX_x86_64-unknown-linux-gnu
2023-09-26T20:57:57.9081369Z   CXX_x86_64-unknown-linux-gnu = None
2023-09-26T20:57:57.9081838Z   cargo:rerun-if-env-changed=CXX_x86_64_unknown_linux_gnu
2023-09-26T20:57:57.9084974Z   CXX_x86_64_unknown_linux_gnu = None
2023-09-26T20:57:57.9153399Z   cargo:rerun-if-env-changed=HOST_CXX
2023-09-26T20:57:57.9155858Z   HOST_CXX = None
2023-09-26T20:57:57.9206211Z   cargo:rerun-if-env-changed=CXX
2023-09-26T20:57:57.9207088Z   CXX = None
2023-09-26T20:57:57.9207572Z   cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
2023-09-26T20:57:57.9208499Z   CRATE_CC_NO_DEFAULTS = None
2023-09-26T20:57:57.9208814Z   CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
2023-09-26T20:57:57.9209301Z   cargo:rerun-if-env-changed=CXXFLAGS_x86_64-unknown-linux-gnu
2023-09-26T20:57:57.9209757Z   CXXFLAGS_x86_64-unknown-linux-gnu = None
2023-09-26T20:57:57.9210250Z   cargo:rerun-if-env-changed=CXXFLAGS_x86_64_unknown_linux_gnu
2023-09-26T20:57:57.9210625Z   CXXFLAGS_x86_64_unknown_linux_gnu = None
2023-09-26T20:57:57.9211013Z   cargo:rerun-if-env-changed=HOST_CXXFLAGS
2023-09-26T20:57:57.9211436Z   HOST_CXXFLAGS = None
2023-09-26T20:57:57.9211804Z   cargo:rerun-if-env-changed=CXXFLAGS
2023-09-26T20:57:57.9212122Z   CXXFLAGS = None
2023-09-26T20:57:57.9212526Z   cargo:rerun-if-env-changed=CXX_x86_64-unknown-linux-gnu
2023-09-26T20:57:57.9212949Z   CXX_x86_64-unknown-linux-gnu = None
2023-09-26T20:57:57.9213367Z   cargo:rerun-if-env-changed=CXX_x86_64_unknown_linux_gnu
2023-09-26T20:57:57.9213721Z   CXX_x86_64_unknown_linux_gnu = None
2023-09-26T20:57:57.9214113Z   cargo:rerun-if-env-changed=HOST_CXX
2023-09-26T20:57:57.9214412Z   HOST_CXX = None
2023-09-26T20:57:57.9214771Z   cargo:rerun-if-env-changed=CXX
2023-09-26T20:57:57.9215055Z   CXX = None
2023-09-26T20:57:57.9215442Z   cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
2023-09-26T20:57:57.9215769Z   CRATE_CC_NO_DEFAULTS = None
2023-09-26T20:57:57.9216099Z   CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
2023-09-26T20:57:57.9216535Z   cargo:rerun-if-env-changed=CXXFLAGS_x86_64-unknown-linux-gnu
2023-09-26T20:57:57.9216969Z   CXXFLAGS_x86_64-unknown-linux-gnu = None
2023-09-26T20:57:57.9217400Z   cargo:rerun-if-env-changed=CXXFLAGS_x86_64_unknown_linux_gnu
2023-09-26T20:57:57.9217787Z   CXXFLAGS_x86_64_unknown_linux_gnu = None
2023-09-26T20:57:57.9218194Z   cargo:rerun-if-env-changed=HOST_CXXFLAGS
2023-09-26T20:57:57.9218499Z   HOST_CXXFLAGS = None
2023-09-26T20:57:57.9218872Z   cargo:rerun-if-env-changed=CXXFLAGS
2023-09-26T20:57:57.9219166Z   CXXFLAGS = None
2023-09-26T20:57:57.9219575Z   cargo:rerun-if-env-changed=CXX_x86_64-unknown-linux-gnu
2023-09-26T20:57:57.9219983Z   CXX_x86_64-unknown-linux-gnu = None
2023-09-26T20:57:57.9220555Z   cargo:rerun-if-env-changed=CXX_x86_64_unknown_linux_gnu
2023-09-26T20:57:57.9220903Z   CXX_x86_64_unknown_linux_gnu = None
2023-09-26T20:57:57.9221289Z   cargo:rerun-if-env-changed=HOST_CXX
2023-09-26T20:57:57.9221603Z   HOST_CXX = None
2023-09-26T20:57:57.9221947Z   cargo:rerun-if-env-changed=CXX
2023-09-26T20:57:57.9222248Z   CXX = None
2023-09-26T20:57:57.9222621Z   cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
2023-09-26T20:57:57.9222963Z   CRATE_CC_NO_DEFAULTS = None
2023-09-26T20:57:57.9223269Z   CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
2023-09-26T20:57:57.9223725Z   cargo:rerun-if-env-changed=CXXFLAGS_x86_64-unknown-linux-gnu
2023-09-26T20:57:57.9224141Z   CXXFLAGS_x86_64-unknown-linux-gnu = None
2023-09-26T20:57:57.9224595Z   cargo:rerun-if-env-changed=CXXFLAGS_x86_64_unknown_linux_gnu
2023-09-26T20:57:57.9224939Z   CXXFLAGS_x86_64_unknown_linux_gnu = None
2023-09-26T20:57:57.9225348Z   cargo:rerun-if-env-changed=HOST_CXXFLAGS
2023-09-26T20:57:57.9225664Z   HOST_CXXFLAGS = None
2023-09-26T20:57:57.9226024Z   cargo:rerun-if-env-changed=CXXFLAGS
2023-09-26T20:57:57.9226335Z   CXXFLAGS = None
2023-09-26T20:57:57.9226727Z   cargo:rerun-if-env-changed=CXX_x86_64-unknown-linux-gnu
2023-09-26T20:57:57.9227350Z   CXX_x86_64-unknown-linux-gnu = None
2023-09-26T20:57:57.9227794Z   cargo:rerun-if-env-changed=CXX_x86_64_unknown_linux_gnu
2023-09-26T20:57:57.9228145Z   CXX_x86_64_unknown_linux_gnu = None
2023-09-26T20:57:57.9228522Z   cargo:rerun-if-env-changed=HOST_CXX
2023-09-26T20:57:57.9228829Z   HOST_CXX = None
2023-09-26T20:57:57.9229186Z   cargo:rerun-if-env-changed=CXX
2023-09-26T20:57:57.9229485Z   CXX = None
2023-09-26T20:57:57.9229859Z   cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
2023-09-26T20:57:57.9230196Z   CRATE_CC_NO_DEFAULTS = None
2023-09-26T20:57:57.9230519Z   CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
2023-09-26T20:57:57.9230956Z   cargo:rerun-if-env-changed=CXXFLAGS_x86_64-unknown-linux-gnu
2023-09-26T20:57:57.9231512Z   CXXFLAGS_x86_64-unknown-linux-gnu = None
2023-09-26T20:57:57.9231953Z   cargo:rerun-if-env-changed=CXXFLAGS_x86_64_unknown_linux_gnu
2023-09-26T20:57:57.9232316Z   CXXFLAGS_x86_64_unknown_linux_gnu = None
2023-09-26T20:57:57.9232702Z   cargo:rerun-if-env-changed=HOST_CXXFLAGS
2023-09-26T20:57:57.9233028Z   HOST_CXXFLAGS = None
2023-09-26T20:57:57.9233392Z   cargo:rerun-if-env-changed=CXXFLAGS
2023-09-26T20:57:57.9233703Z   CXXFLAGS = None
2023-09-26T20:57:57.9234529Z   running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-o" "/home/conda/staged-recipes/build_artifacts/qsv_1695761611793/work/target/x86_64-unknown-linux-gnu/release/build/libduckdb-sys-88188750096947e7/out/duckdb/duckdb.o" "-c" "duckdb/duckdb.cpp"
2023-09-26T20:57:57.9234969Z 
2023-09-26T20:57:57.9235279Z   --- stderr
2023-09-26T20:57:57.9235461Z 
2023-09-26T20:57:57.9235619Z 
2023-09-26T20:57:57.9235917Z   error occurred: Failed to find tool. Is `c++` installed?
2023-09-26T20:57:57.9236137Z 
2023-09-26T20:57:57.9236292Z 
2023-09-26T20:57:57.9236613Z warning: build failed, waiting for other jobs to finish...
2023-09-26T20:58:44.0243763Z error: failed to compile `qsv v0.114.0 (/home/conda/staged-recipes/build_artifacts/qsv_1695761611793/work)`, intermediate artifacts can be found at `/home/conda/staged-recipes/build_artifacts/qsv_1695761611793/work/target`.
2023-09-26T20:58:44.0245901Z To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.
2023-09-26T20:58:44.6195954Z Traceback (most recent call last):
2023-09-26T20:58:44.6197216Z   File "/home/conda/staged-recipes-copy/.ci_support/build_all.py", line 228, in <module>
2023-09-26T20:58:44.6205349Z     build_all(os.path.join(root_dir, "recipes"), args.arch)
2023-09-26T20:58:44.6206123Z   File "/home/conda/staged-recipes-copy/.ci_support/build_all.py", line 118, in build_all
2023-09-26T20:58:44.6206478Z     build_folders(recipes_dir, folders, arch, channel_urls)
2023-09-26T20:58:44.6206962Z   File "/home/conda/staged-recipes-copy/.ci_support/build_all.py", line 174, in build_folders
2023-09-26T20:58:44.6207306Z     conda_build.api.build([recipe], config=get_config(arch, channel_urls))
2023-09-26T20:58:44.6207771Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/api.py", line 180, in build
2023-09-26T20:58:44.6213553Z     return build_tree(
2023-09-26T20:58:44.6214260Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 3078, in build_tree
2023-09-26T20:58:44.6234100Z     packages_from_this = build(metadata, stats,
2023-09-26T20:58:44.6234965Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 2198, in build
2023-09-26T20:58:44.6240547Z     utils.check_call_env(cmd, env=env, rewrite_stdout_env=rewrite_env,
2023-09-26T20:58:44.6241406Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/utils.py", line 451, in check_call_env
2023-09-26T20:58:44.6255157Z     return _func_defaulting_env_to_os_environ("call", *popenargs, **kwargs)
2023-09-26T20:58:44.6257921Z   File "/opt/conda/lib/python3.10/site-packages/conda_build/utils.py", line 427, in _func_defaulting_env_to_os_environ
2023-09-26T20:58:44.6258398Z     raise subprocess.CalledProcessError(proc.returncode, _args)
2023-09-26T20:58:44.6259162Z subprocess.CalledProcessError: Command '['/bin/bash', '-o', 'errexit', '/home/conda/staged-recipes/build_artifacts/qsv_1695761611793/work/conda_build.sh']' returned non-zero exit status 101.
2023-09-26T20:58:52.7172928Z ##[error]Bash exited with code '1'.
2023-09-26T20:58:52.7250731Z ##[section]Finishing: Run docker build

@jqnatividad
Copy link

Hi @peterjc , you may want to remove the to feature as it loops in duckdb.

Another feature you may consider removing is the polars feature which powers the sqlp and joinp commands, but I'd try removing just to first as the sqlp and joinp commands are quite powerful and addresses join's issue of redundant column names which you reported in jqnatividad/qsv#84.

@peterjc
Copy link
Contributor Author

peterjc commented Sep 30, 2023

Another C++ dependency, error: failed to run custom build command for `mlua-sys v0.3.2 on Linux - something different but also mlua on macOS. Windows failing in hashbrown dependency.

@peterjc
Copy link
Contributor Author

peterjc commented Sep 30, 2023

Looks like it builds on Linux now, then:

Packaging qsv
Packaging qsv-0.114.0-he8a937b_0
The install/build script(s) for qsv deleted the following files (from dependencies) from the prefix:
['x86_64-conda-linux-gnu/lib/libgfortran.so.5.0.0', 'x86_64-conda-linux-gnu/lib/libgfortran.so', 'x86_64-conda-linux-gnu/lib/libstdc++.so.6.0.30', 'x86_64-conda-linux-gnu/lib/libgfortran.so.5']
This will cause the post-link checks to mis-report. Please try not to delete and files (DSOs in particular) from the prefix
number of files: 23618
Unknown format
Traceback (most recent call last):
  File "/home/conda/staged-recipes-copy/.ci_support/build_all.py", line 228, in <module>
    build_all(os.path.join(root_dir, "recipes"), args.arch)
  File "/home/conda/staged-recipes-copy/.ci_support/build_all.py", line 118, in build_all
    build_folders(recipes_dir, folders, arch, channel_urls)
  File "/home/conda/staged-recipes-copy/.ci_support/build_all.py", line 174, in build_folders
    conda_build.api.build([recipe], config=get_config(arch, channel_urls))
  File "/opt/conda/lib/python3.10/site-packages/conda_build/api.py", line 180, in build
    return build_tree(
  File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 3078, in build_tree
    packages_from_this = build(metadata, stats,
  File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 2358, in build
    newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)
  File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 1659, in bundle_conda
    files = post_process_files(metadata, initial_files)
  File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 1505, in post_process_files
    post_build(m, new_files, build_python=python)
  File "/opt/conda/lib/python3.10/site-packages/conda_build/post.py", line 1319, in post_build
    check_overlinking(m, files, host_prefix)
  File "/opt/conda/lib/python3.10/site-packages/conda_build/post.py", line 1225, in check_overlinking
    return check_overlinking_impl(m.get_value('package/name'),
  File "/opt/conda/lib/python3.10/site-packages/conda_build/post.py", line 1135, in check_overlinking_impl
    all_needed_dsos, needed_dsos_for_file = _collect_needed_dsos(sysroots_files, files, run_prefix,
  File "/opt/conda/lib/python3.10/site-packages/conda_build/post.py", line 742, in _collect_needed_dsos
    needed = get_linkages_memoized(path, resolve_filenames=True, recurse=False,
  File "/opt/conda/lib/python3.10/site-packages/conda_build/os_utils/liefldd.py", line 975, in __call__
    value = self.func(*args, **kw)
  File "/opt/conda/lib/python3.10/site-packages/conda_build/os_utils/liefldd.py", line 1003, in get_linkages_memoized
    return get_linkages(filename, resolve_filenames=resolve_filenames,
  File "/opt/conda/lib/python3.10/site-packages/conda_build/os_utils/liefldd.py", line 510, in get_linkages
    result_lief = inspect_linkages_lief(filename, resolve_filenames=resolve_filenames, recurse=recurse,
  File "/opt/conda/lib/python3.10/site-packages/conda_build/os_utils/liefldd.py", line 406, in inspect_linkages_lief
    if binary.format == lief.EXE_FORMATS.ELF:
AttributeError: 'NoneType' object has no attribute 'format'
##[error]Bash exited with code '1'.

Failed to link on macOS:

error: linking with `x86_64-apple-darwin13.4.0-clang` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/rustlib/x86_64-apple-darwin/bin:/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/_build_env/bin:/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/.cargo/bin:/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/bin:/Users/runner/Miniforge3/condabin:/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/_build_env/bin:/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/bin:/Users/runner/Miniforge3/bin:/Users/runner/Miniforge3/condabin:.:/usr/local/bin:/usr/local/sbin:/Users/runner/bin:/Library/Frameworks/Python.framework/Versions/Current/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/usr/bin:/bin:/usr/sbin:/sbin:/Users/runner/hostedtoolcache/stack/2.11.1/x64" VSLANG="1033" ZERO_AR_DATE="1" "x86_64-apple-darwin13.4.0-clang" "-arch" "x86_64" "-m64" "/tmp/rustcN7tSlm/symbols.o" "/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/work/target/x86_64-apple-darwin/release/deps/qsv-b94db35a4ad848e3.qsv.42ecf7e7e30678cc-cgu.0.rcgu.o" "-L" "/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/work/target/x86_64-apple-darwin/release/deps" "-L" "/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/work/target/release/deps" "-L" "/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/work/target/x86_64-apple-darwin/release/build/bzip2-sys-55bac3dbe65e7e0f/out/lib" "-L" "/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/work/target/x86_64-apple-darwin/release/build/zstd-sys-b75f49f86003bfc5/out" "-L" "/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/work/target/x86_64-apple-darwin/release/build/decimal-2ba866da295245f8/out" "-L" "/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/work/target/x86_64-apple-darwin/release/build/ring-9fe491cf2ed67b3a/out" "-L" "/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/work/target/x86_64-apple-darwin/release/build/psm-08b7dddc9e61d5ce/out" "-L" "/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/rustlib/x86_64-apple-darwin/lib" "/tmp/rustcN7tSlm/libpsm-2233985725a43c76.rlib" "/tmp/rustcN7tSlm/libzstd_sys-58968c0ca72b2055.rlib" "/tmp/rustcN7tSlm/libbzip2_sys-479d9d266d2acf15.rlib" "/tmp/rustcN7tSlm/libring-83aaded3d6ed0bd3.rlib" "/tmp/rustcN7tSlm/libdecimal-2c2ee1cdb1a9c7d6.rlib" "/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-950dfa4358313a7f.rlib" "-framework" "IOKit" "-framework" "CoreFoundation" "-framework" "Security" "-framework" "CoreFoundation" "-liconv" "-lSystem" "-lc" "-lm" "-L" "/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/rustlib/x86_64-apple-darwin/lib" "-o" "/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/work/target/x86_64-apple-darwin/release/deps/qsv-b94db35a4ad848e3" "-Wl,-dead_strip" "-nodefaultlibs" "-Wl,-rpath,@loader_path"
  = note: Undefined symbols for architecture x86_64:
            "_kCFURLVolumeAvailableCapacityForImportantUsageKey", referenced from:
                _$LT$sysinfo..apple..system..System$u20$as$u20$sysinfo..traits..SystemExt$GT$::new_with_specifics::hd20d3a350e202b2b in qsv-b94db35a4ad848e3.qsv.42ecf7e7e30678cc-cgu.0.rcgu.o
          ld: symbol(s) not found for architecture x86_64
          clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
          

error: could not compile `qsv` (bin "qsv") due to previous error
error: failed to compile `qsv v0.114.0 (/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/work)`, intermediate artifacts can be found at `/Users/runner/Miniforge3/conda-bld/qsv_1696099429170/work/target`.

And on Windows failing to compile dependency hashbrown:

   Compiling hashbrown v0.14.1
error[E0554]: `#![feature]` may not be used on the stable release channel
  --> C:\bld\qsv_1696099538573\_h_env\.cargo\registry\src\index.crates.io-6f17d22bba15001f\hashbrown-0.14.1\src\lib.rs:15:5
   |
15 | /     feature(
16 | |         test,
17 | |         core_intrinsics,
18 | |         dropck_eyepatch,
...  |
24 | |         strict_provenance
25 | |     )
   | |_____^

error[E0554]: `#![feature]` may not be used on the stable release channel
  --> C:\bld\qsv_1696099538573\_h_env\.cargo\registry\src\index.crates.io-6f17d22bba15001f\hashbrown-0.14.1\src\lib.rs:16:9
   |
16 |         test,
   |         ^^^^

error[E0554]: `#![feature]` may not be used on the stable release channel
  --> C:\bld\qsv_1696099538573\_h_env\.cargo\registry\src\index.crates.io-6f17d22bba15001f\hashbrown-0.14.1\src\lib.rs:17:9
   |
17 |         core_intrinsics,
   |         ^^^^^^^^^^^^^^^

error[E0554]: `#![feature]` may not be used on the stable release channel
  --> C:\bld\qsv_1696099538573\_h_env\.cargo\registry\src\index.crates.io-6f17d22bba15001f\hashbrown-0.14.1\src\lib.rs:20:9
   |
20 |         extend_one,
   |         ^^^^^^^^^^

error[E0554]: `#![feature]` may not be used on the stable release channel
  --> C:\bld\qsv_1696099538573\_h_env\.cargo\registry\src\index.crates.io-6f17d22bba15001f\hashbrown-0.14.1\src\lib.rs:21:9
   |
21 |         allocator_api,
   |         ^^^^^^^^^^^^^


For more information about this error, try `rustc --explain E0554`.
error: could not compile `hashbrown` (lib) due to 7 previous errors

@peterjc
Copy link
Contributor Author

peterjc commented Sep 30, 2023

The linux failure looks like very much like conda/conda-build#4787

@peterjc
Copy link
Contributor Author

peterjc commented Sep 30, 2023

The Windows failure looks to be something rust specific - https://doc.rust-lang.org/error_codes/E0554.html - maybe @jqnatividad can interpret that?:

`#![feature]` may not be used on the stable release channel

cargo-bundle-licenses --format yaml --output THIRDPARTY.yml

:: build
cargo install --root "%LIBRARY_PREFIX%" --path . --all-features || goto :error

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@peterjc , as I understand it, bld.bat is used to build for Windows.

I'd change this line to:
cargo install --root "$PREFIX" --path . --locked --features feature_capable,apply,generate,foreach,fetch,geocode

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the --all-features option here explains why hashbrown is being built with nightly, as we do have a nightly feature in qsv Cargo.toml.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, I overlooked the Windows options were in a separate file.

Also changing the root path as per Joel's suggestion.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants