Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix - bake build tools into the scrypto-builder docker image #1703

Merged
merged 5 commits into from
Feb 1, 2024
Merged

Fix - bake build tools into the scrypto-builder docker image #1703

merged 5 commits into from
Feb 1, 2024

Conversation

lrubasze
Copy link
Contributor

@lrubasze lrubasze commented Jan 31, 2024 •
edited

Summary

This PR makes sure that scrypto-builder docker image includes the development tools such as cmake, clang, llvm.

Details

It addresses the issue reported by one of the Scrypto developers

error occurred: Command "clang" 
Updating git repository `https://github.com/radixdlt/radixdlt-scrypto`
    Updating crates.io index
    Updating git repository `https://github.com/radixdlt/const-sha1`
    Updating git repository `https://github.com/dtolnay/syn.git`
    Updating git repository `https://github.com/radixdlt/wasm-instrument`
    Updating git repository `https://github.com/radixdlt/wasmi.git`
    Updating git submodule `https://github.com/robbepop/wasm_kernel.git`
    Updating git submodule `https://github.com/WebAssembly/testsuite.git`
    Updating git repository `https://github.com/bluss/indexmap`
 Downloading crates ...
  Downloaded bech32 v0.9.1
  Downloaded crypto-common v0.1.6
  Downloaded cpufeatures v0.2.11
  Downloaded opaque-debug v0.3.0
  Downloaded strum v0.24.1
  Downloaded zeroize v1.3.0
  Downloaded bitflags v1.3.2
  Downloaded generic-array v0.14.7
  Downloaded rand_core v0.5.1
  Downloaded digest v0.10.7
  Downloaded num-integer v0.1.45
  Downloaded sha2 v0.9.9
  Downloaded either v1.9.0
  Downloaded byteorder v1.5.0
  Downloaded strum_macros v0.24.3
  Downloaded zeroize_derive v1.4.2
  Downloaded ppv-lite86 v0.2.17
  Downloaded subtle v2.5.0
  Downloaded num_cpus v1.16.0
  Downloaded block-buffer v0.9.0
  Downloaded paste v1.0.14
  Downloaded keccak v0.1.5
  Downloaded signature v1.6.4
  ...
  Compiling sha3 v0.10.8
   Compiling utils v1.1.0 (https://github.com/radixdlt/radixdlt-scrypto?tag=v1.1.0#4f2918a8)
   Compiling ed25519-dalek v1.0.1
   Compiling sbor v1.1.0 (https://github.com/radixdlt/radixdlt-scrypto?tag=v1.1.0#4f2918a8)
   Compiling secp256k1 v0.24.3
   Compiling strum v0.24.1
   Compiling bnum v0.7.0
   Compiling blake2 v0.10.6
error: failed to run custom build command for `secp256k1-sys v0.6.1`
Caused by:
  process didn't exit successfully: `/src/target/release/build/secp256k1-sys-f1679135aeb015cf/build-script-build` (exit status: 1)
  --- stdout
  TARGET = Some("wasm32-unknown-unknown")
  OPT_LEVEL = Some("z")
  HOST = Some("x86_64-unknown-linux-gnu")
  cargo:rerun-if-env-changed=CC_wasm32-unknown-unknown
  CC_wasm32-unknown-unknown = None
  cargo:rerun-if-env-changed=CC_wasm32_unknown_unknown
  CC_wasm32_unknown_unknown = None
  cargo:rerun-if-env-changed=TARGET_CC
  TARGET_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("false")
  cargo:rerun-if-env-changed=CFLAGS_wasm32-unknown-unknown
  CFLAGS_wasm32-unknown-unknown = None
  cargo:rerun-if-env-changed=CFLAGS_wasm32_unknown_unknown
  CFLAGS_wasm32_unknown_unknown = None
  cargo:rerun-if-env-changed=TARGET_CFLAGS
  TARGET_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  cargo:rerun-if-env-changed=CC_wasm32-unknown-unknown
  CC_wasm32-unknown-unknown = None
  cargo:rerun-if-env-changed=CC_wasm32_unknown_unknown
  CC_wasm32_unknown_unknown = None
  cargo:rerun-if-env-changed=TARGET_CC
  TARGET_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  cargo:rerun-if-env-changed=CFLAGS_wasm32-unknown-unknown
  CFLAGS_wasm32-unknown-unknown = None
  cargo:rerun-if-env-changed=CFLAGS_wasm32_unknown_unknown
  CFLAGS_wasm32_unknown_unknown = None
  cargo:rerun-if-env-changed=TARGET_CFLAGS
  TARGET_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  running: "clang" "-Oz" "-ffunction-sections" "-fdata-sections" "-fPIC" "--target=wasm32-unknown-unknown" "-I" "depend/secp256k1/" "-I" "depend/secp256k1/include" "-
I" "depend/secp256k1/src" "-I" "wasm/wasm-sysroot" "-Wall" "-Wextra" "-DSECP256K1_API=" "-DENABLE_MODULE_ECDH=1" "-DENABLE_MODULE_SCHNORRSIG=1" "-DENABLE_MODULE_EXTRAKEYS=1" "-DUSE_NUM_NONE=1" "-DUSE_FIELD_INV_BUILTIN=1" "-DUSE_SCALAR_INV_BUILTIN=1" "-DECMULT_GEN_PREC_BITS=4" "-DECMULT_WINDOW_SIZE=15" "-DUSE_EXTERNAL_DEFAULT_CALLBACKS=1" "-DENABLE_MODULE_RECOVERY=1" "-o" "/src/target/wasm32-unknown-unknown/release/build/secp256k1-sys-c41ad8ceab3178f9/out/wasm/wasm.o" "-c" "wasm/wasm.c"
  exit status: 127

  --- stderr
  error occurred: Command "clang" "-Oz" "-ffunction-sections" "-fdata-sections" "-fPIC" "--target=wasm32-unknown-unknown" "-I" "depend/secp256k1/" "-I" "depend/secp256k1/include" "-I" "depend/secp256k1/src" "-I" "wasm/wasm-sysroot" "-Wall" "-Wextra" "-DSECP256K1_API=" "-DENABLE_MODULE_ECDH=1" "-DENABLE_MODULE_SCHNORRSIG=1" "-DENABLE_MODULE_EXTRAKEYS=1" "-DUSE_NUM_NONE=1" "-DUSE_FIELD_INV_BUILTIN=1" "-DUSE_SCALAR_INV_BUILTIN=1" "-DECMULT_GEN_PREC_BITS=4" "-DECMULT_WINDOW_SIZE=15" "-DUSE_EXTERNAL_DEFAULT_CALLBACKS=1" "-DENABLE_MODULE_RECOVERY=1" "-o" "/src/target/wasm32-unknown-unknown/release/build/secp256k1-sys-c41ad8ceab3178f9/out/wasm/wasm.o" "-c" "wasm/wasm.c" with args "clang" did not execute successfully (status code exit status: 127).


warning: build failed, waiting for other jobs to finish...
Error: BuildError(CargoFailure(ExitStatus(unix_wait_status(25856))))

~ took 5m49s

Testing

Added tests to build some scrypto blueprint with scrypto-builder docker image

Update Recommendations

For dApp Developers

Pull latest scrypto-builder docker image

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 31, 2024 •
edited

Benchmark for 8679356

Click to view benchmark
Test Base PR %
costing::bench_prepare_wasm 65.2±0.46ms 65.2±0.25ms 0.00%
costing::decode_sbor 12.9±0.15µs 13.3±0.01µs +3.10%
costing::decode_sbor_bytes 38.0±0.02µs 35.5±0.07µs -6.58%
costing::deserialize_wasm 1236.2±1.51µs 1238.9±1.79µs +0.22%
costing::instantiate_flash_loan 3.9±0.48ms 3.9±0.44ms 0.00%
costing::instantiate_radiswap 5.2±0.06ms 5.2±0.08ms 0.00%
costing::spin_loop 23.1±0.01ms 22.1±0.01ms -4.33%
costing::validate_sbor_payload 24.8±0.02µs 24.3±0.07µs -2.02%
costing::validate_sbor_payload_bytes 354.8±0.32ns 337.3±1.93ns -4.93%
costing::validate_secp256k1 80.5±0.07µs 80.4±0.05µs -0.12%
costing::validate_wasm 34.9±0.04ms 34.7±0.04ms -0.57%
decimal::add/0 7.2±0.00ns 7.2±0.00ns 0.00%
decimal::add/rust-native 9.4±0.05ns 9.3±0.03ns -1.06%
decimal::add/wasmer 135.1±0.09ns 133.4±0.04ns -1.26%
decimal::add/wasmer-call-native 518.6±0.43ns 516.3±0.32ns -0.44%
decimal::add/wasmi 441.1±2.92ns 449.4±6.46ns +1.88%
decimal::add/wasmi-call-native 3.3±0.01µs 3.4±0.02µs +3.03%
decimal::div/0 165.0±0.14ns 164.9±0.04ns -0.06%
decimal::from_string/0 151.7±0.14ns 151.9±0.12ns +0.13%
decimal::mul/0 127.9±0.07ns 127.9±0.08ns 0.00%
decimal::mul/rust-native 133.7±0.07ns 132.0±0.12ns -1.27%
decimal::mul/wasmer 1745.5±0.37ns 1729.9±0.40ns -0.89%
decimal::mul/wasmer-call-native 656.2±0.40ns 649.7±0.12ns -0.99%
decimal::mul/wasmi 26.6±0.05µs 26.4±0.03µs -0.75%
decimal::mul/wasmi-call-native 3.5±0.02µs 3.5±0.01µs 0.00%
decimal::pow/0 622.6±0.50ns 622.5±0.13ns -0.02%
decimal::pow/rust-native 609.0±0.25ns 606.2±0.39ns -0.46%
decimal::pow/wasmer 7.6±0.00µs 7.5±0.00µs -1.32%
decimal::pow/wasmer-call-native 1079.9±0.58ns 1075.9±0.12ns -0.37%
decimal::pow/wasmi 126.5±0.06µs 126.5±0.12µs 0.00%
decimal::pow/wasmi-call-native 3.4±0.01µs 3.4±0.01µs 0.00%
decimal::root/0 9.4±0.01µs 9.2±0.00µs -2.13%
decimal::sub/0 7.2±0.00ns 7.2±0.00ns 0.00%
decimal::to_string/0 497.5±0.43ns 495.9±0.18ns -0.32%
precise_decimal::add/0 8.0±0.00ns 8.0±0.00ns 0.00%
precise_decimal::add/rust-native 10.4±0.01ns 10.4±0.01ns 0.00%
precise_decimal::add/wasmer 144.1±0.05ns 142.8±0.06ns -0.90%
precise_decimal::add/wasmer-call-native 555.8±0.54ns 534.8±0.09ns -3.78%
precise_decimal::add/wasmi 529.2±0.21ns 527.0±0.09ns -0.42%
precise_decimal::add/wasmi-call-native 3.5±0.01µs 3.6±0.02µs +2.86%
precise_decimal::div/0 262.8±0.09ns 262.8±0.09ns 0.00%
precise_decimal::from_string/0 196.7±0.04ns 196.7±0.08ns 0.00%
precise_decimal::mul/0 281.2±0.14ns 281.3±0.33ns +0.04%
precise_decimal::mul/rust-native 262.7±0.11ns 260.4±0.12ns -0.88%
precise_decimal::mul/wasmer 4.0±0.00µs 4.0±0.00µs 0.00%
precise_decimal::mul/wasmer-call-native 821.9±1.02ns 817.5±0.17ns -0.54%
precise_decimal::mul/wasmi 74.0±0.03µs 73.8±0.05µs -0.27%
precise_decimal::mul/wasmi-call-native 3.8±0.01µs 3.9±0.02µs +2.63%
precise_decimal::pow/0 1568.8±0.25ns 1565.8±0.55ns -0.19%
precise_decimal::pow/rust-native 1270.4±0.25ns 1276.9±0.31ns +0.51%
precise_decimal::pow/wasmer 18.9±0.02µs 19.1±0.01µs +1.06%
precise_decimal::pow/wasmer-call-native 2.0±0.00µs 2.0±0.00µs 0.00%
precise_decimal::pow/wasmi 356.1±0.18µs 356.8±0.12µs +0.20%
precise_decimal::pow/wasmi-call-native 7.6±0.03µs 7.6±0.03µs 0.00%
precise_decimal::root/0 61.0±0.02µs 60.6±0.01µs -0.66%
precise_decimal::sub/0 8.3±0.00ns 8.3±0.01ns 0.00%
precise_decimal::to_string/0 745.1±0.27ns 746.0±0.17ns +0.12%
schema::validate_payload 332.9±6.49µs 332.0±0.22µs -0.27%
transaction::radiswap 5.2±0.05ms 5.3±0.05ms +1.92%
transaction::transfer 1690.7±6.19µs 1702.4±3.21µs +0.69%
transaction_processing::prepare 2.4±0.00ms 2.4±0.01ms 0.00%
transaction_processing::prepare_and_decompile 6.1±0.01ms 6.2±0.01ms +1.64%
transaction_processing::prepare_and_decompile_and_recompile 23.3±0.16ms 23.2±0.20ms -0.43%
transaction_validation::validate_manifest 43.9±0.04µs 43.8±0.03µs -0.23%
transaction_validation::verify_bls_2KB 1027.1±14.07µs 1005.9±3.49µs -2.06%
transaction_validation::verify_bls_32B 1009.1±12.33µs 1004.5±2.41µs -0.46%
transaction_validation::verify_ecdsa 78.0±0.09µs 78.0±0.06µs 0.00%
transaction_validation::verify_ed25519 51.5±0.04µs 51.5±0.04µs 0.00%

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 31, 2024 •
edited

Docker tags
docker.io/radixdlt/private-scrypto-builder:8679356096

@lrubasze lrubasze changed the title Fix builder image Fix - bake build tools into the builder image Jan 31, 2024
@lrubasze lrubasze changed the title Fix - bake build tools into the builder image Fix - bake build tools into the scrypto-builder docker image Jan 31, 2024
lrubasze
lrubasze commented Feb 1, 2024
View reviewed changes
.github/workflows/ci-scrypto-builder.yml
@@ -6,6 +6,7 @@ on:
- develop
- main
- release\/*
pull_request:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we really want to build docker image upon every PR commit.
Added this just to check if changes in this PR work as expected.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's leave it as-is. If this causes cost related issues, we will revisit it.

@lrubasze lrubasze force-pushed the fix/builder-image branch 2 times, most recently from ebd9020 to fb581db Compare February 1, 2024 08:38
iamyulong
iamyulong approved these changes Feb 1, 2024
View reviewed changes
Copy link
Member

@iamyulong iamyulong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

.github/workflows/ci-scrypto-builder.yml
@@ -6,6 +6,7 @@ on:
- develop
- main
- release\/*
pull_request:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's leave it as-is. If this causes cost related issues, we will revisit it.

@lrubasze lrubasze changed the base branch from develop to release/anemone February 1, 2024 09:46
@iamyulong iamyulong merged commit cbad73f into release/anemone Feb 1, 2024
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@iamyulong iamyulong iamyulong approved these changes

Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@lrubasze @iamyulong