Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fastcomp updates ignored #320

Closed
Beuc opened this issue Aug 19, 2019 · 7 comments · Fixed by #322

Comments

@Beuc
Copy link

commented Aug 19, 2019

I just spent forever tracking a bug only to understand emsdk didn't update itself properly :/

$ ./emsdk install 1.38.33
Installing SDK 'sdk-releases-fastcomp-3b8cff670e9233a6623563add831647e8689a86b-64bit'..
Installing tool 'releases-fastcomp-3b8cff670e9233a6623563add831647e8689a86b-64bit'..
Downloading: /home/me/workdir/emtests/emsdk/zips/3b8cff670e9233a6623563add831647e8689a86b-wasm-binaries.tbz2 from https://storage.googleapis.com/webassembly/emscripten-releases-builds/linux/3b8cff670e9233a6623563add831647e8689a86b/wasm-binaries.tbz2, 214988819 Bytes
Unpacking '/home/me/workdir/emtests/emsdk/zips/3b8cff670e9233a6623563add831647e8689a86b-wasm-binaries.tbz2' to '/home/me/workdir/emtests/emsdk/fastcomp'
Done installing tool 'releases-fastcomp-3b8cff670e9233a6623563add831647e8689a86b-64bit'.
Installing tool 'node-8.9.1-64bit'..
Downloading: /home/me/workdir/emtests/emsdk/zips/node-v8.9.1-linux-x64.tar.xz from https://storage.googleapis.com/webassembly/emscripten-releases-builds/deps/node-v8.9.1-linux-x64.tar.xz, 11387108 Bytes
Unpacking '/home/me/workdir/emtests/emsdk/zips/node-v8.9.1-linux-x64.tar.xz' to '/home/me/workdir/emtests/emsdk/node/8.9.1_64bit'
Done installing tool 'node-8.9.1-64bit'.
Done installing SDK 'sdk-releases-fastcomp-3b8cff670e9233a6623563add831647e8689a86b-64bit'.
$ emcc --version
emcc (Emscripten gcc/clang-like replacement) 1.38.33 (commit d94e9fbc89fce94a7f4135786f551ef7e00bf161)
$ sha1sum fastcomp/emscripten/tools/ports/binaryen.py
4fb43a205745a08ef05703493c2d40a3b8b31a75  fastcomp/emscripten/tools/ports/binaryen.py

$ ./emsdk install 1.38.41
Installing SDK 'sdk-releases-fastcomp-5c6785a63993ae7a4d5362b32b0be9c85138fb96-64bit'..
Installing tool 'releases-fastcomp-5c6785a63993ae7a4d5362b32b0be9c85138fb96-64bit'..
The contents of file 'https://storage.googleapis.com/webassembly/emscripten-releases-builds/linux/5c6785a63993ae7a4d5362b32b0be9c85138fb96/wasm-binaries.tbz2' already exist in destination '/home/me/workdir/emtests/emsdk/fastcomp', skipping.
Done installing tool 'releases-fastcomp-5c6785a63993ae7a4d5362b32b0be9c85138fb96-64bit'.
Installing tool 'node-8.9.1-64bit'..
Downloading: /home/me/workdir/emtests/emsdk/zips/node-v8.9.1-linux-x64.tar.xz from https://storage.googleapis.com/webassembly/emscripten-releases-builds/deps/node-v8.9.1-linux-x64.tar.xz, 11387108 Bytes
Unpacking '/home/me/workdir/emtests/emsdk/zips/node-v8.9.1-linux-x64.tar.xz' to '/home/me/workdir/emtests/emsdk/node/8.9.1_64bit'
Done installing tool 'node-8.9.1-64bit'.
Done installing SDK 'sdk-releases-fastcomp-5c6785a63993ae7a4d5362b32b0be9c85138fb96-64bit'.
$ emcc --version
emcc (Emscripten gcc/clang-like replacement) 1.38.41 (commit d94e9fbc89fce94a7f4135786f551ef7e00bf161)
$ sha1sum fastcomp/emscripten/tools/ports/binaryen.py
4fb43a205745a08ef05703493c2d40a3b8b31a75  fastcomp/emscripten/tools/ports/binaryen.py
# ^ old version

$ rm -rf fastcomp/
$ ./emsdk install 1.38.41
Installing SDK 'sdk-releases-fastcomp-5c6785a63993ae7a4d5362b32b0be9c85138fb96-64bit'..
Installing tool 'releases-fastcomp-5c6785a63993ae7a4d5362b32b0be9c85138fb96-64bit'..
Downloading: /home/me/workdir/emtests/emsdk/zips/5c6785a63993ae7a4d5362b32b0be9c85138fb96-wasm-binaries.tbz2 from https://storage.googleapis.com/webassembly/emscripten-releases-builds/linux/5c6785a63993ae7a4d5362b32b0be9c85138fb96/wasm-binaries.tbz2, 159825127 Bytes
Unpacking '/home/me/workdir/emtests/emsdk/zips/5c6785a63993ae7a4d5362b32b0be9c85138fb96-wasm-binaries.tbz2' to '/home/me/workdir/emtests/emsdk/fastcomp'
Done installing tool 'releases-fastcomp-5c6785a63993ae7a4d5362b32b0be9c85138fb96-64bit'.
Installing tool 'node-8.9.1-64bit'..
Downloading: /home/me/workdir/emtests/emsdk/zips/node-v8.9.1-linux-x64.tar.xz from https://storage.googleapis.com/webassembly/emscripten-releases-builds/deps/node-v8.9.1-linux-x64.tar.xz, 11387108 Bytes
Unpacking '/home/me/workdir/emtests/emsdk/zips/node-v8.9.1-linux-x64.tar.xz' to '/home/me/workdir/emtests/emsdk/node/8.9.1_64bit'
Done installing tool 'node-8.9.1-64bit'.
Done installing SDK 'sdk-releases-fastcomp-5c6785a63993ae7a4d5362b32b0be9c85138fb96-64bit'.
$ emcc --version
emcc (Emscripten gcc/clang-like replacement) 1.38.41 (commit d94e9fbc89fce94a7f4135786f551ef7e00bf161)
$ sha1sum fastcomp/emscripten/tools/ports/binaryen.py
d0ecf8e10c70640f03a0bbd996d6b526cbf1a5b0  fastcomp/emscripten/tools/ports/binaryen.py
# ^ up-to-date version

This heuristic appears faulty (./emsdk):

  # If the archive was already downloaded, and the directory it would be
  # unpacked to has contents, assume it's the same contents and skip.
  if not download_even_if_exists and os.path.exists(download_target) and num_files_in_directory(dest_dir) > 0:
    print("The contents of file '" + zipfile + "' already exist in destination '" + dest_dir + "', skipping.")
    return True
@kripken

This comment has been minimized.

Copy link
Member

commented Aug 19, 2019

Which version of the emsdk is that with? We fixed some bugs with this in the last month or two.

edit: e.g. #316

@Beuc

This comment has been minimized.

Copy link
Author

commented Aug 19, 2019

emcc (Emscripten gcc/clang-like replacement) 1.38.41 (commit d94e9fbc89fce94a7f4135786f551ef7e00bf161)
                                                             ^^^ this one ^_^

also in both cases the archive is already present in zips/, just not extracted (with existing fastcomp/), or re-downloaded and extracted (with rm -rf'd fastcomp/), so that looks like a different issue.

@kripken

This comment has been minimized.

Copy link
Member

commented Aug 19, 2019

That's the emcc commit - what's the emsdk commit?

(emcc is not aware of the emsdk, btw - it's just driven by it)

@Beuc

This comment has been minimized.

Copy link
Author

commented Aug 20, 2019

That /is/ the emsdk commit d94e9fb, I thought you knew.

@kripken

This comment has been minimized.

Copy link
Member

commented Aug 20, 2019

I'm confused... is emcc printing an emsdk commit for you? I don't remember us adding support for that. When I do emcc -v I don't see an emsdk commit.

I believe I have the right fix in #322, please test.

@Beuc

This comment has been minimized.

Copy link
Author

commented Aug 20, 2019

After checking out the "lcean2" branch, I got the bug once more (probably emsdk needed to remove the zips first) and then the bug went away (and now emsdk re-downloads everything everytime).
So it's fixed but it may still bite some users even after an emsdk update.

As for emcc printing the emsdk commit, I believe it's because emcc tentatively runs 'git show' and, since I'm use a emsdk git checkout, it gets a emsdk commit!

@kripken kripken closed this in #322 Aug 20, 2019

kripken added a commit that referenced this issue Aug 20, 2019
Properly clean up downloaded zips (#322)
We got this wrong because we were not using get_download_target, so we were trying to delete the wrong file.

Also add a test.

Fixes #320
@kripken

This comment has been minimized.

Copy link
Member

commented Aug 20, 2019

Ah, yes, existing downloaded files are not automatically removed. Perhaps the emsdk should remove them?

Right, thanks, if we detect git then we show a commit. That's intended for emscripten versions though! :) But I guess it happens to find the emsdk instead, and that actually works well...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.