Skip to content
This repository has been archived by the owner on Oct 5, 2022. It is now read-only.

How to increase the speed? #1

Open
jio-gl opened this issue Feb 24, 2021 · 15 comments
Open

How to increase the speed? #1

jio-gl opened this issue Feb 24, 2021 · 15 comments
Labels
question Further information is requested

Comments

@jio-gl
Copy link

jio-gl commented Feb 24, 2021 •
edited
Loading

Is true is one order of magnitude slower than the Rust. Any ideas on how to match the Rust impl?

Thanks,
J.
@nazar-pc
Copy link
Member

It was according to my tests.

I don't think you'll be able to match Rust. You may try newer Rust compiler, the one that was used here is pretty old and I was very new to Rust back then.
There is also SIMD in WASM at least in some browsers, so that may or may not be useful to try as well.

@nazar-pc nazar-pc added the question Further information is requested label Feb 24, 2021
@jio-gl
Copy link
Author

jio-gl commented Feb 24, 2021

thanks.

@jio-gl
Copy link
Author

jio-gl commented Feb 24, 2021

I have an issue compiling with GMP. I am trying Ubuntu, I can try in MacOS too...

libgmp.a is not built yet, compiling it
./build.sh: line 29: pushd: too many arguments

Here:

   pushd $CACHE_DIR
        curl https://gmplib.org/download/gmp/$GMP_RELEASE.tar.bz2 -O
        tar xf $GMP_RELEASE.tar.bz2
        pushd $GMP_RELEASE
            CC_FOR_BUILD=$(which gcc) emconfigure ./configure --build i386-linux-gnu --host none --disable-assembly --disable-shared --prefix=$(pwd)/build
            patch < ../../src/gmp/config.h.patch
            make -j $(getconf _NPROCESSORS_ONLN)
            cp .libs/libgmp.a $CACHE_DIR/lib/
        popd
        rm -rf $GMP_RELEASE*
    popd

Thanks.

@nazar-pc
Copy link
Member

I think the path to your project folder has whitespaces or special characters, dumb script wasn't designed to handle them 🙃
Try compiling from within directory that doesn't have them.

@jio-gl
Copy link
Author

jio-gl commented Feb 24, 2021

I am compiling on Ubuntu now, will see... Thanks a lot.

@jio-gl
Copy link
Author

jio-gl commented Feb 24, 2021

Could not link with GMP on WASM:

          >>> defined as (i32, i32, i32, i32, i32, i32, i32) -> void in /media/psf/Home/lab/Soliton/vdf-subspace/target/wasm32-unknown-emscripten/release/deps/libclassgroup-c93cd65f7ace5636.rlib(classgroup-c93cd65f7ace5636.classgroup.ci31togf-cgu.4.rcgu.o)
          >>> defined as (i32, i32, i32, i32, i32, i32, i32) -> i32 in .cache/lib/libgmp.a(export.o)
          
          wasm-ld: error: function signature mismatch: __gmpz_tdiv_q_ui
          >>> defined as (i32, i32, i32) -> void in /media/psf/Home/lab/Soliton/vdf-subspace/target/wasm32-unknown-emscripten/release/deps/libclassgroup-c93cd65f7ace5636.rlib(classgroup-c93cd65f7ace5636.classgroup.ci31togf-cgu.1.rcgu.o)
          >>> defined as (i32, i32, i32) -> i32 in .cache/lib/libgmp.a(tdiv_q_ui.o)
          
          wasm-ld: error: function signature mismatch: __gmpz_tdiv_r_ui
          >>> defined as (i32, i32, i32) -> void in /media/psf/Home/lab/Soliton/vdf-subspace/target/wasm32-unknown-emscripten/release/deps/libclassgroup-c93cd65f7ace5636.rlib(classgroup-c93cd65f7ace5636.classgroup.ci31togf-cgu.1.rcgu.o)
          >>> defined as (i32, i32, i32) -> i32 in .cache/lib/libgmp.a(tdiv_r_ui.o)

@nazar-pc
Copy link
Member

Try updating version in https://github.com/subspace/vdf/blob/master/build.sh#L22 to 6.2.1, which seems to be the latest release, probably vdf crate updated its dependencies as well (and don't forget to remove .cache directory so it builds gmp again.

@jio-gl
Copy link
Author

jio-gl commented Feb 25, 2021

There is something with the hardlinks:

copying selected object files to avoid basename conflicts...
libtool: link: ln mpz/abs.o .libs/libgmp.lax/lt1-abs.o || cp mpz/abs.o .libs/libgmp.lax/lt1-abs.o
ln: failed to create hard link '.libs/libgmp.lax/lt1-abs.o' => 'mpz/abs.o': Operation not permitted
libtool: link: ln mpz/add.o .libs/libgmp.lax/lt2-add.o || cp mpz/add.o .libs/libgmp.lax/lt2-add.o
ln: failed to create hard link '.libs/libgmp.lax/lt2-add.o' => 'mpz/add.o': Operation not permitted
libtool: link: ln mpz/add_ui.o .libs/libgmp.lax/lt3-add_ui.o || cp mpz/add_ui.o .libs/libgmp.lax/lt3-add_ui.o
ln: failed to create hard link '.libs/libgmp.lax/lt3-add_ui.o' => 'mpz/add_ui.o': Operation not permitted
libtool: link: ln mpz/clear.o .libs/libgmp.lax/lt4-clear.o || cp mpz/clear.o .libs/libgmp.lax/lt4-clear.o
ln: failed to create hard link '.libs/libgmp.lax/lt4-clear.o' => 'mpz/clear.o': Operation not permitted
libtool: link: ln mpz/clears.o .libs/libgmp.lax/lt5-clears.o || cp mpz/clears.o .libs/libgmp.lax/lt5-clears.o
ln: failed to create hard link '.libs/libgmp.lax/lt5-clears.o' => 'mpz/clears.o': Operation not permitted
libtool: link: ln mpz/cmp.o .libs/libgmp.lax/lt6-cmp.o || cp mpz/cmp.o .libs/libgmp.lax/lt6-cmp.o
...

@jio-gl
Copy link
Author

jio-gl commented Feb 25, 2021

Tried with GMP 6.2.1

@nazar-pc
Copy link
Member

Not sure what it is, you'll have to do some experiments then

@jio-gl
Copy link
Author

jio-gl commented Feb 25, 2021

I think is a VM issue, the Ubuntu is a VM. Will try again in a not shared folder.

@jio-gl
Copy link
Author

jio-gl commented Feb 25, 2021 •
edited
Loading

The WASM linker is too strict, maybe I need to use an older version of the emsdk ?

wasm-ld: error: function signature mismatch: __gmpz_export
          >>> defined as (i32, i32, i32, i32, i32, i32, i32) -> void in /home/parallels/Desktop/vdf-subspace/target/wasm32-unknown-emscripten/release/deps/libclassgroup-c93cd65f7ace5636.rlib(classgroup-c93cd65f7ace5636.classgroup.ci31togf-cgu.4.rcgu.o)
          >>> defined as (i32, i32, i32, i32, i32, i32, i32) -> i32 in .cache/lib/libgmp.a(export.o)
          
          wasm-ld: error: function signature mismatch: __gmpz_tdiv_q_ui
          >>> defined as (i32, i32, i32) -> void in /home/parallels/Desktop/vdf-subspace/target/wasm32-unknown-emscripten/release/deps/libclassgroup-c93cd65f7ace5636.rlib(classgroup-c93cd65f7ace5636.classgroup.ci31togf-cgu.1.rcgu.o)
          >>> defined as (i32, i32, i32) -> i32 in .cache/lib/libgmp.a(tdiv_q_ui.o)
          
          wasm-ld: error: function signature mismatch: __gmpz_tdiv_r_ui
          >>> defined as (i32, i32, i32) -> void in /home/parallels/Desktop/vdf-subspace/target/wasm32-unknown-emscripten/release/deps/libclassgroup-c93cd65f7ace5636.rlib(classgroup-c93cd65f7ace5636.classgroup.ci31togf-cgu.1.rcgu.o)
          >>> defined as (i32, i32, i32) -> i32 in .cache/lib/libgmp.a(tdiv_r_ui.o)

@jio-gl
Copy link
Author

jio-gl commented Apr 26, 2021

Tried disabling a flag due to 3 function mismatch, but I cannot compile subspace-vdf to WASM. Any clues?

           CC_FOR_BUILD=$(which gcc) emconfigure ./configure --build i386-linux-gnu --host none --disable-assembly --disable-shared --prefix=$(pwd)/build CFLAGS='-s ERROR_ON_UNDEFINED_SYM\
BOLS=0' CXXFLAGS='-s ERROR_ON_UNDEFINED_SYMBOLS=0'
            patch < ../../src/gmp/config.h.patch
            make -j $(getconf _NPROCESSORS_ONLN) CFLAGS='-s ERROR_ON_UNDEFINED_SYMBOLS=0' CXXFLAGS='-s ERROR_ON_UNDEFINED_SYMBOLS=0'


  = note: emcc:WARNING: --llvm-lto ignored when using llvm backend
          system_libs:WARNING: main() is in the input files, but "_main" is not in EXPORTED_FUNCTIONS, which means it may be eliminated as dead code. Export it if you want main() to run.
          wasm-ld: error: function signature mismatch: __gmpz_export
          >>> defined as (i32, i32, i32, i32, i32, i32, i32) -> void in /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libclassgroup-d76e008ce2b14c87.rlib(classgroup-d76e008ce2b14c87.classgroup.amxomvt1-cgu.4.rcgu.o)
          >>> defined as (i32, i32, i32, i32, i32, i32, i32) -> i32 in .cache/lib/libgmp.a(export.o)
          
          wasm-ld: error: function signature mismatch: __gmpz_tdiv_q_ui
          >>> defined as (i32, i32, i32) -> void in /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libclassgroup-d76e008ce2b14c87.rlib(classgroup-d76e008ce2b14c87.classgroup.amxomvt1-cgu.1.rcgu.o)
          >>> defined as (i32, i32, i32) -> i32 in .cache/lib/libgmp.a(tdiv_q_ui.o)
          
          wasm-ld: error: function signature mismatch: __gmpz_tdiv_r_ui
          >>> defined as (i32, i32, i32) -> void in /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libclassgroup-d76e008ce2b14c87.rlib(classgroup-d76e008ce2b14c87.classgroup.amxomvt1-cgu.1.rcgu.o)
          >>> defined as (i32, i32, i32) -> i32 in .cache/lib/libgmp.a(tdiv_r_ui.o)
          emcc: error: '/home/parallels/lab/VDF/emsdk/upstream/bin/wasm-ld -o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.wasm -L/home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib -L/home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/self-contained /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.0.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.1.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.2.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.3.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.4.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.5.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.6.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.vdf.569mfwoe-cgu.7.rcgu.o /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/vdf.35gafff193gzq50j.rcgu.o -L/home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps -L/home/parallels/lab/VDF/vdf/target/release/deps -L/home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libvdf-48efadb155ccd181.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libsha2-d8f34aa3ce58a162.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libdigest-05bbe2952105514e.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libopaque_debug-0e4ecea6fa489716.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libfake_simd-8f252f8b25b74a91.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libblock_buffer-49d734868c895388.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libgeneric_array-20013f52121094ec.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libtypenum-11f9fa3a864b9b2f.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libblock_padding-b45aec8ca2ef2d2b.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libbyte_tools-e7622d9515268345.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libbyteorder-2fc6ade1e9b79472.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libclassgroup-d76e008ce2b14c87.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/libnum_traits-2165ba5925761581.rlib /home/parallels/lab/VDF/vdf/target/wasm32-unknown-emscripten/release/deps/liblibc-949c035481ff33b9.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libstd-2d3ec1ab9910bb82.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libpanic_unwind-910e7560fa396a1d.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_demangle-f173843d247e78a2.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libhashbrown-b05e05efe60b4f19.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_std_workspace_alloc-f6ae4ab5a9bc2d08.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libunwind-9f8b5f6efeb94207.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcfg_if-8c4a64bedea9d0fc.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/liblibc-27afec8da4d3cd01.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/liballoc-2f94ab43b6c4843a.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_std_workspace_core-018092b98e265aa8.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcore-49726ec2e30e61f2.rlib /home/parallels/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcompiler_builtins-ccd75a5f965f90ed.rlib .cache/lib/libgmp.a .cache/lib/libgmp.a .cache/lib/libgmp.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libc.a -L.cache/lib --fatal-warnings -L/home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libgl.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libal.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libhtml5.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libc.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libcompiler_rt.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libc++.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libc++abi.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libdlmalloc.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libc_rt_wasm-optz.a /home/parallels/lab/VDF/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libsockets.a -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-cxx-exceptions -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --allow-undefined --strip-debug --export malloc --export free --export generate --export verify --export emscripten_stack_get_end --export emscripten_stack_get_free --export emscripten_stack_init --export stackSave --export stackRestore --export stackAlloc --export __wasm_call_ctors --export fflush --export __errno_location --export __cxa_is_pointer_type --export __cxa_can_catch --export ntohs --export htonl --export htons --export emscripten_main_thread_process_queued_calls --export sysconf --export setThrew --export _get_tzname --export _get_daylight --export _get_timezone --export-table --export __start_em_asm --export __stop_em_asm -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=16777216 --global-base=1024' failed (1)

@nazar-pc
Copy link
Member

Either GMP or something else has changed, but I have no time to dig into it and figure out why unfortunately.

@jio-gl
Copy link
Author

jio-gl commented Apr 27, 2021

Will keep you posted.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nazar-pc @jio-gl