Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix dup*() syscalls #9396

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

jakogut
Copy link
Contributor

@jakogut jakogut commented Sep 5, 2019

Rework dup syscalls so file descriptors share all flags and also seek position.

Based on #4017

Rebased and modified for sharing flags and position, but not file descriptors or closed status.

Closes #4017

NOTE: The included test passes for me when building and running manually, but fails using the runner.

@jakogut jakogut requested a review from sbc100 September 6, 2019 15:57
@jakogut
Copy link
Contributor Author

jakogut commented Sep 6, 2019

Looks like the test is also failing in CI. Really not sure what's going on there.

@kripken
Copy link
Member

kripken commented Sep 10, 2019

If it works locally but fails remotely, one possibility is using a different version of node. I think our CI uses the default emsdk version.

I see failures in exceptions tests etc. though, so all of those pass for you locally?

@jakogut
Copy link
Contributor Author

jakogut commented Sep 11, 2019

If it works locally but fails remotely, one possibility is using a different version of node. I think our CI uses the default emsdk version.

It works locally when I manually build the test code and run it, but not when using the test runner locally, which is what's weird.

I see failures in exceptions tests etc. though, so all of those pass for you locally?

During development, I only ran test_unistd_dup locally, and that one fails, like I mentioned. Looking at test-upstream-wasm0, it looks like the exception tests are passing?

test_exceptions (test_core.wasm0) ... ok
test_exceptions_2 (test_core.wasm0) ... ok
test_exceptions_3 (test_core.wasm0) ... ok
test_exceptions_alias (test_core.wasm0) ... ok
test_exceptions_convert (test_core.wasm0) ... ok
test_exceptions_custom (test_core.wasm0) ... ok
test_exceptions_destroy_virtual (test_core.wasm0) ... ok
test_exceptions_libcxx (test_core.wasm0) ... ok
test_exceptions_minimal_runtime (test_core.wasm0) ... skipped 'is_wasm_backend : MINIMAL_RUNTIME not yet available in Wasm backend'
test_exceptions_multi (test_core.wasm0) ... ok
test_exceptions_multiple_inherit (test_core.wasm0) ... ok
test_exceptions_multiple_inherit_rethrow (test_core.wasm0) ... ok
test_exceptions_primary (test_core.wasm0) ... ok
test_exceptions_refcount (test_core.wasm0) ... ok
test_exceptions_resume (test_core.wasm0) ... ok
test_exceptions_rethrow (test_core.wasm0) ... ok
test_exceptions_simplify_cfg (test_core.wasm0) ... ok
test_exceptions_std (test_core.wasm0) ... ok
test_exceptions_typed (test_core.wasm0) ... ok
test_exceptions_uncaught (test_core.wasm0) ... ok
test_exceptions_uncaught_2 (test_core.wasm0) ... ok
test_exceptions_virtual_inheritance (test_core.wasm0) ... ok
test_exceptions_white_list (test_core.wasm0) ... ok
test_exceptions_white_list_2 (test_core.wasm0) ... ok

I just ran all the wasm0 tests locally on this branch, and the only failing tests aretest_fs_nodefs_rw and test_unistd_dup. Like I said, I don't know why the latter test is failing, because running it manually works as expected.

└──> emscripten $ >> emcc tests/unistd/dup.c -o dup.js
┌─[14:38:08]─[joseph@JAKWS]
└──> emscripten $ >> node dup.js
DUP
errno: 0
f: 1
f2,f3: 1
1. f3 offset was 0.    Should be 0
2. f  offset set to 1. Should be 1
3. f2 offset set to 2. Should be 2
4. f  offset now is 1. Should be 1
5. f2 offset now is 2. Should be 2
6. f3 offset now is 1. Should be 1 (and not 0)
7. f3 offset set to 3. Should be 3
8. f  offset now is 3. Should be 3 (and not 1)
close(f1): 0
close(f2): 0
close(f3): 0

DUP2
errno: 0
f: 1
f2,f3: 1
close(f1): 0
close(f2): 0
close(f3): -1

Whereas it fails when using the runner.

┌─[✗]─[14:38:48]─[joseph@JAKWS]
└──> emscripten $ >> python2 tests/runner.py wasm0.test_unistd_dup
tests/runner.py:WARNING: use EMTEST_ALL_ENGINES=1 in the env to run against all JS engines, which is slower but provides more coverage
Test suites:
['test_core']
Running test_core: (1 tests)
(checking sanity from test runner)
shared:INFO: (Emscripten: Running sanity checks)
test_unistd_dup (test_core.wasm0) ... (test did not pass in JS engine: ['/usr/bin/node'])
FAIL

======================================================================
FAIL: test_unistd_dup (test_core.wasm0)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/joseph/emscripten/tests/test_core.py", line 111, in decorated
    func(self, js_engines=[NODE_JS])
  File "/home/joseph/emscripten/tests/test_core.py", line 5122, in test_unistd_dup
    self.do_run(src, expected, js_engines=js_engines)
  File "/home/joseph/emscripten/tests/runner.py", line 1140, in do_run
    self.assertContained(expected_output, js_output, check_all=assert_all)
  File "/home/joseph/emscripten/tests/runner.py", line 843, in assertContained
    additional_info
AssertionError: Expected to find 'DUP
errno: 0
f: 1
f2,f3: 1
1. f3 offset was 0.    Should be 0
2. f  offset set to 1. Should be 1
3. f2 offset set to 2. Should be 2
4. f  offset now is 1. Should be 1
5. f2 offset now is 2. Should be 2
6. f3 offset now is 1. Should be 1 (and not 0)
7. f3 offset set to 3. Should be 3
8. f  offset now is 3. Should be 3 (and not 1)
close(f1): 0
close(f2): 0
close(f3): 0

DUP2
errno: 0
f: 1
f2,f3: 1
close(f1): 0
close(f2): 0
close(f3): -1
' in 'DUP
errno: 9
f: 1
f2,f3: 1
1. f3 offset was 0.    Should be 0
2. f  offset set to 1. Should be 1
3. f2 offset set to 2. Should be 2
4. f  offset now is 1. Should be 1
5. f2 offset now is 2. Should be 2
6. f3 offset now is 0. Should be 1 (and not 0)
7. f3 offset set to 3. Should be 3
8. f  offset now is 1. Should be 3 (and not 1)
close(f1): 0
close(f2): 0
close(f3): -1

DUP2
errno: 0
f: 1
f2,f3: 1
close(f1): 0
close(f2): -1
close(f3): -1
', diff:

--- expected
+++ actual
@@ -1,5 +1,5 @@
 DUP
-errno: 0
+errno: 9
 f: 1
 f2,f3: 1
 1. f3 offset was 0.    Should be 0
@@ -7,18 +7,18 @@
 3. f2 offset set to 2. Should be 2
 4. f  offset now is 1. Should be 1
 5. f2 offset now is 2. Should be 2
-6. f3 offset now is 1. Should be 1 (and not 0)
+6. f3 offset now is 0. Should be 1 (and not 0)
 7. f3 offset set to 3. Should be 3
-8. f  offset now is 3. Should be 3 (and not 1)
+8. f  offset now is 1. Should be 3 (and not 1)
 close(f1): 0
 close(f2): 0
-close(f3): 0
+close(f3): -1

 DUP2
 errno: 0
 f: 1
 f2,f3: 1
 close(f1): 0
-close(f2): 0
+close(f2): -1
 close(f3): -1




----------------------------------------------------------------------
Ran 1 test in 1.961s

FAILED (failures=1)

Additionally, test_nodefs_rw appears to work as expected without the closure compiler, but fails with it.

└──> emscripten $ >> emcc -s SYSCALL_DEBUG=1 tests/fs/test_nodefs_rw.c -o test_nodefs_rw.js && node test_nodefs_rw.js
syscall! 5,SYS_open
    (raw: "1157")
    (str: "/working/foobar.txt")
    (raw: "32768")
    (raw: "438")
syscall return: 3
syscall! 145,SYS_readv
    (raw: "3")
    (stream: "/working/foobar.txt")
    (raw: "5245456")
    (raw: "2")
syscall return: 6
syscall! 6,SYS_close
    (raw: "3")
    (stream: "/working/foobar.txt")
syscall return: 0
syscall! 5,SYS_open
    (raw: "1157")
    (str: "/working/foobar.txt")
    (raw: "33345")
    (raw: "438")
syscall return: 3
syscall! 54,SYS_ioctl
    (raw: "3")
    (stream: "/working/foobar.txt")
    (raw: "21523")
error: syscall may have failed with 25 (Not a typewriter)
syscall return: -25
syscall! 146,SYS_writev
    (raw: "3")
    (stream: "/working/foobar.txt")
    (raw: "5245456")
    (raw: "2")
syscall return: 5
syscall! 6,SYS_close
    (raw: "3")
    (stream: "/working/foobar.txt")
syscall return: 0
syscall! 54,SYS_ioctl
    (raw: "1")
    (stream: "/dev/tty")
    (raw: "21523")
syscall return: 0
syscall! 146,SYS_writev
    (raw: "1")
    (stream: "/dev/tty")
    (raw: "5245408")
    (raw: "2")
success
syscall return: 8
┌─[14:44:52]─[joseph@JAKWS]
└──> emscripten $ >> emcc -s SYSCALL_DEBUG=1 --closure 1 tests/fs/test_nodefs_rw.c -o test_nodefs_rw.js && node test_nodefs_rw.js
syscall! 5,SYS_open
    (raw: "1157")
    (str: "/working/foobar.txt")
    (raw: "32768")
    (raw: "438")
syscall return: 3
syscall! 145,SYS_readv
    (raw: "3")
    (stream: "/working/foobar.txt")
    (raw: "5245456")
    (raw: "2")
syscall return: 6
syscall! 6,SYS_close
    (raw: "3")
    (stream: "/working/foobar.txt")
syscall return: 0
syscall! 5,SYS_open
    (raw: "1157")
    (str: "/working/foobar.txt")
    (raw: "33345")
    (raw: "438")
syscall return: 3
syscall! 54,SYS_ioctl
    (raw: "3")
    (stream: "/working/foobar.txt")
    (raw: "21523")
error: syscall may have failed with 25 (Not a typewriter)
syscall return: -25
syscall! 146,SYS_writev
    (raw: "3")
    (stream: "/working/foobar.txt")
    (raw: "5245456")
    (raw: "2")
error: syscall failed with 9 (Bad file number)
syscall return: -9
syscall! 6,SYS_close
    (raw: "3")
    (stream: "/working/foobar.txt")
syscall return: 0
Assertion failed: undefined
Assertion failed: undefined
exception thrown: abort(Assertion failed: undefined) at Error
    at cb (/home/joseph/emscripten/test_nodefs_rw.js:413:11)
    at db (/home/joseph/emscripten/test_nodefs_rw.js:427:11)
    at t (/home/joseph/emscripten/test_nodefs_rw.js:2340:36)
    at assert (/home/joseph/emscripten/test_nodefs_rw.js:138:8)
    at Za (/home/joseph/emscripten/test_nodefs_rw.js:397:3)
    at emscripten_asm_const_iii (/home/joseph/emscripten/test_nodefs_rw.js:1752:16)
    at wasm-function[15]:692
    at wasm-function[16]:3
    at Object.d._main (/home/joseph/emscripten/test_nodefs_rw.js:1925:21)
    at b (/home/joseph/emscripten/test_nodefs_rw.js:2261:21)

@kripken
Copy link
Member

kripken commented Sep 16, 2019

If it works when you build it, but not when the test runner runs it, then there is probably a difference in how it's build. You can build with EMCC_DEBUG=1 in the env to see the full emcc commands the test runner runs. You can also use EM_SAVE_DIR=1 in the env to save the test output temp dir (defaults to /tmp/emscripten_test) and then you can compare that output file to the output when you build it manually.

@jakogut
Copy link
Contributor Author

jakogut commented Nov 26, 2019

If it works when you build it, but not when the test runner runs it, then there is probably a difference in how it's build. You can build with EMCC_DEBUG=1 in the env to see the full emcc commands the test runner runs. You can also use EM_SAVE_DIR=1 in the env to save the test output temp dir (defaults to /tmp/emscripten_test) and then you can compare that output file to the output when you build it manually.

Upon taking a closer look at this, it appears the failure is being caused by testing with NODERAWFS=1.

@jakogut
Copy link
Contributor Author

jakogut commented Nov 27, 2019

@kripken Well that was fun. It turns out that Node doesn't have the same concept of duplicating file descriptors, so I implemented a shared refcount that keeps the stream open until no more fds are referencing it. The test now passes with both FS and NODERAWFS.

@jakogut jakogut force-pushed the fix-dup-syscalls branch 2 times, most recently from fc0e3af to 355cdff Compare November 27, 2019 00:34
@kripken
Copy link
Member

kripken commented Nov 27, 2019

If node doesn't support duplicating fds, then that worries me about how useful this will be, I think?

How does the proposed refcount facility work - are duplicated fds all refering to the original they were duplicated from? Does that have the right behavior (I think they should not pick up changes to the other fd after the duplication?)

@jakogut
Copy link
Contributor Author

jakogut commented Nov 27, 2019

If node doesn't support duplicating fds, then that worries me about how useful this will be, I think?

How does the proposed refcount facility work - are duplicated fds all refering to the original they were duplicated from? Does that have the right behavior (I think they should not pick up changes to the other fd after the duplication?)

@kripken It certainly appears to have the right behavior to me, based on the results of the test running natively, in the browser, and in Node.

dup(2) describes:

After a successful return, the old and new file descriptors may be
used interchangeably.  They refer to the same open file description
(see open(2)) and thus share file offset and file status flags; for
example, if the file offset is modified by using lseek(2) on one of
the file descriptors, the offset is also changed for the other.

This is implemented in the in-memory filesystem by creating a duplicate stream object containing a shared object that stores the stream's position and status flags. Updating either property on one fd will change it on a duplicated fd, which is demonstrated to be correct by running the provided test. Prior to this patch, duplicated streams did not share seek position and status flags.

NODERAWFS is a little different, because opening a file creates a node fd for actual io ops. Closing the node stream will also make duplicated fds unavailable, so upon duplicating an fd, we increment the refcount in the shared object. Closing an fd decrements the refcount, and we only close the stream upon having no more references to it. This also passes the provided test.

The one exception described is for the FD_CLOEXEC flag, which is meaningless in our single-process environment, and this flag can be ignored.

The two file descriptors do not share file descriptor flags (the
close-on-exec flag).  The close-on-exec flag (FD_CLOEXEC; see
fcntl(2)) for the duplicate descriptor is off.

@kripken
Copy link
Member

kripken commented Dec 3, 2019

I see @jakogut, sounds good!

Copy link
Member

@kripken kripken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! Some comments, but basically this all looks good.

Also, if the refcount capability is only for NODERAWFS, it would be good to clarify that, and why it is needed there, in a comment - basically, please explain what you explained in the last comment to me, also in the code.

@jakogut
Copy link
Contributor Author

jakogut commented Dec 3, 2019

By the way, it's been a while since I came across this issue, but if memory serves, FFmpeg uses dup in this way somewhere, and this PR fixes this issue.

@jakogut jakogut requested a review from kripken December 3, 2019 20:36
@jakogut
Copy link
Contributor Author

jakogut commented Dec 3, 2019

Oops, looks like some of the indenting is wrong, sorry for the noise. (I have to fix my text editor)

Copy link
Member

@kripken kripken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Looks good aside from that indentation error, and we need to get tests passing.

About the tests, odd it fails in sdl2_mixer - do you see that locally too? Let me know if you need help debugging that.

if (sharedProperties.indexOf(p) >= 0) {
if (!("shared" in newStream)) {
newStream.shared = {};
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indentation looks wrong here

@kripken
Copy link
Member

kripken commented Dec 5, 2019

(Btw, we recently fixed some sdl2_mixer issue - merging in latest incoming to here may help, but probably a long shot.)

@jakogut
Copy link
Contributor Author

jakogut commented Dec 6, 2019

I wasn't able to reproduce the embuilder sdl2-mixer issue, but I've fixed the whitespace problem, and rebased on incoming.

@jakogut
Copy link
Contributor Author

jakogut commented Dec 6, 2019

Look like test_fs_nodefs_rw is failing as well.

@jakogut
Copy link
Contributor Author

jakogut commented Dec 7, 2019

It looks like this test passes when compiled normally, and fails with the closure compiler.

EDIT: All optimization options pass, including -O1, -O2, -O3, and -Os, the test fails when run with --closure-compiler 1 is specified, and passes again with --closure-compiler 1 -g.

Here's the output of the failing test with SYSCALL_DEBUG=1:

syscall! 5,SYS_open
    (raw: "1157")
    (str: "/working/foobar.txt")
    (raw: "32768")
    (raw: "438")
syscall return: 3
syscall! fd_read
    (stream: "/working/foobar.txt")
syscall return: 0
syscall! fd_close
    (stream: "/working/foobar.txt")
syscall return: 0
syscall! 5,SYS_open
    (raw: "1157")
    (str: "/working/foobar.txt")
    (raw: "33345")
    (raw: "438")
syscall return: 3
syscall! 54,SYS_ioctl
    (raw: "3")
    (stream: "/working/foobar.txt")
    (raw: "21523")
error: syscall may have failed with 59 (Not a typewriter)
syscall return: -59
syscall! fd_write
    (stream: "/working/foobar.txt")
error: syscall failed with 8 (Bad file number)
syscall return: 8
syscall! fd_close
    (stream: "/working/foobar.txt")
syscall return: 0
Assertion failed: undefined
Assertion failed: undefined
exception thrown: RuntimeError: abort(Assertion failed: undefined) at Error
    at t (/home/joseph/emscripten/test_nodefs_rw.js:311:13)
    at assert (/home/joseph/emscripten/test_nodefs_rw.js:131:8)
    at 1274 (/home/joseph/emscripten/test_nodefs_rw.js:370:3)
    at emscripten_asm_const_iii (/home/joseph/emscripten/test_nodefs_rw.js:1538:16)
    at wasm-function[16]:692
    at wasm-function[17]:3
    at d._main (/home/joseph/emscripten/test_nodefs_rw.js:1827:21)
    at b (/home/joseph/emscripten/test_nodefs_rw.js:2171:19)
    at xc (/home/joseph/emscripten/test_nodefs_rw.js:2207:14)
    at wc (/home/joseph/emscripten/test_nodefs_rw.js:2126:9),RuntimeError: abort(Assertion failed: undefined) at Error
    at t (/home/joseph/emscripten/test_nodefs_rw.js:311:13)
    at assert (/home/joseph/emscripten/test_nodefs_rw.js:131:8)
    at 1274 (/home/joseph/emscripten/test_nodefs_rw.js:370:3)
    at emscripten_asm_const_iii (/home/joseph/emscripten/test_nodefs_rw.js:1538:16)
    at wasm-function[16]:692
    at wasm-function[17]:3
    at d._main (/home/joseph/emscripten/test_nodefs_rw.js:1827:21)
    at b (/home/joseph/emscripten/test_nodefs_rw.js:2171:19)
    at xc (/home/joseph/emscripten/test_nodefs_rw.js:2207:14)
    at wc (/home/joseph/emscripten/test_nodefs_rw.js:2126:9)
    at t (/home/joseph/emscripten/test_nodefs_rw.js:327:9)
    at assert (/home/joseph/emscripten/test_nodefs_rw.js:131:8)
    at 1274 (/home/joseph/emscripten/test_nodefs_rw.js:370:3)
    at emscripten_asm_const_iii (/home/joseph/emscripten/test_nodefs_rw.js:1538:16)
    at wasm-function[16]:692
    at wasm-function[17]:3
    at d._main (/home/joseph/emscripten/test_nodefs_rw.js:1827:21)
    at b (/home/joseph/emscripten/test_nodefs_rw.js:2171:19)
    at xc (/home/joseph/emscripten/test_nodefs_rw.js:2207:14)
    at wc (/home/joseph/emscripten/test_nodefs_rw.js:2126:9)

@kripken Any ideas for how to debug this further?

@jakogut jakogut requested a review from kripken December 9, 2019 20:36
@kripken
Copy link
Member

kripken commented Dec 10, 2019

I think the test failures here will be fixed by merging latest incoming to here, as that was an llvm update that we fixed meanwhile.

Copy link
Member

@kripken kripken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks! lgtm aside from the test failures as just mentioned.

@jakogut
Copy link
Contributor Author

jakogut commented Dec 11, 2019

Rebased on incoming.

@jakogut
Copy link
Contributor Author

jakogut commented Dec 11, 2019

Looks like we're still having Freetype failures.

@kripken
Copy link
Member

kripken commented Dec 11, 2019

Very strange... the parent commit you rebased on passed CI, but this doesn't. I don't understand why that is.

One thing to try is to do a merge commit into here (the merge commit will get squashed out anyhow) of latest incoming (might need to wait for the next commit there).

@jakogut
Copy link
Contributor Author

jakogut commented Dec 12, 2019

Still no dice. I checked out incoming at 78071f9, and reproduced the failure locally running wasm0.test_freetype. Does this help?

$ >> python tests/runner.py wasm0.test_freetype
tests/runner.py:WARNING: use EMTEST_ALL_ENGINES=1 in the env to run against all JS engines, which is slower but provides more coverage
Test suites:
['test_core']
Running test_core: (1 tests)
(checking sanity from test runner)
shared:INFO: (Emscripten: Running sanity checks)
test_freetype (test_core.wasm0) ... <building and saving freetype_O0_069aaf9e9c5b51519ef5cf1a62791478 into cache> 
-- configure stdout --

FreeType build system -- automatic system detection

The following settings are used:

  platform                    unix
  compiler                    /home/joseph/emscripten/emcc
  configuration directory     ./builds/unix
  configuration rules         ./builds/unix/unix.mk

If this does not correspond to your system or settings please remove the file
`config.mk' from this directory then read the INSTALL file for help.

Otherwise, simply type `make' again to build the library,
or `make refdoc' to build the API reference (the latter needs python).

Generating modules list in ./objs/ftmodule.h...
* module: truetype  (Windows/Mac font files with extension *.ttf or *.ttc)
* module: type1     (Postscript font files with extension *.pfa or *.pfb)
* module: cff       (OpenType fonts with extension *.otf)
* module: cid       (Postscript CID-keyed fonts, no known extension)
* module: pfr       (PFR/TrueDoc font files with extension *.pfr)
* module: type42    (Type 42 font files with no known extension)
* module: winfnt    (Windows bitmap fonts with extension *.fnt or *.fon)
* module: pcf       (pcf bitmap fonts)
* module: bdf       (bdf bitmap fonts)
* module: sfnt      (helper module for TrueType & OpenType formats)
* module: autofit   (automatic hinting module)
* module: pshinter  (Postscript hinter module)
* module: raster    (monochrome bitmap renderer)
* module: smooth    (anti-aliased bitmap renderer)
* module: smooth    (anti-aliased bitmap renderer for LCDs)
* module: smooth    (anti-aliased bitmap renderer for vertical LCDs)
* module: psaux     (Postscript Type 1 & Type 2 helper module)
* module: psnames   (Postscript & Unicode Glyph name handling)
done.
cd builds/unix; ./configure  '--disable-shared' '--without-zlib'
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for gcc... /home/joseph/emscripten/emcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... TypeError: WebAssembly Instantiation: Import #3 module="wasi_unstable" error: module is not an object or function
RuntimeError: abort(TypeError: WebAssembly Instantiation: Import #3 module="wasi_unstable" error: module is not an object or function) at Error
    at jsStackTrace (/tmp/emscripten_test_wasm0_kmjz40re/building/freetype/builds/unix/conftest:1800:17)
    at stackTrace (/tmp/emscripten_test_wasm0_kmjz40re/building/freetype/builds/unix/conftest:1817:16)
    at abort (/tmp/emscripten_test_wasm0_kmjz40re/building/freetype/builds/unix/conftest:1601:44)
    at /tmp/emscripten_test_wasm0_kmjz40re/building/freetype/builds/unix/conftest:1720:7
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:189:7)
    at Function.Module.runMain (module.js:696:11)
    at startup (bootstrap_node.js:204:16)
    at bootstrap_node.js:625:3

-- end configure stdout --
-- configure stderr --
configure: error: in `/tmp/emscripten_test_wasm0_kmjz40re/building/freetype/builds/unix':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
make: *** [builds/unix/detect.mk:84: setup] Error 1

-- end configure stderr --
ERROR

======================================================================
ERROR: test_freetype (test_core.wasm0)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/joseph/emscripten/tests/runner.py", line 139, in decorated
    return func(self, *args, **kwargs)
  File "/home/joseph/emscripten/tests/test_core.py", line 5946, in test_freetype
    libraries=self.get_freetype_library(),
  File "/home/joseph/emscripten/tests/runner.py", line 1187, in get_freetype_library
    return self.get_library('freetype', os.path.join('objs', '.libs', 'libfreetype.a'), configure_args=['--disable-shared', '--without-zlib'])
  File "/home/joseph/emscripten/tests/runner.py", line 909, in get_library
    return build_library(name, build_dir, output_dir, generated_libs, configure,
  File "/home/joseph/emscripten/tests/runner.py", line 1690, in build_library
    Building.configure(configure + configure_args, env=env, stdout=stdout, stderr=stderr)
  File "/home/joseph/emscripten/tools/shared.py", line 1722, in configure
    run_process(args, stdout=stdout, stderr=stderr, env=env)
  File "/home/joseph/emscripten/tools/shared.py", line 184, in run_process
    ret = subprocess.run(cmd, check=check, input=input, *args, **kw)
  File "/usr/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['sh', './configure', '--disable-shared', '--without-zlib']' returned non-zero exit status 2.

----------------------------------------------------------------------
Ran 1 test in 6.196s

FAILED (errors=1)

@kripken
Copy link
Member

kripken commented Dec 12, 2019

Interesting - @tlively saw that error locally a few days ago, but we don't remember what made it go away...

Perhaps try clearing the cache, or install the latest tot build?

Also, do other tests pass for you locally (like test_hello_world, for example)?

Rework dup syscalls so file descriptors share all flags and also seek position.
@jakogut
Copy link
Contributor Author

jakogut commented Dec 18, 2019

Interesting - @tlively saw that error locally a few days ago, but we don't remember what made it go away...

Perhaps try clearing the cache, or install the latest tot build?

Also, do other tests pass for you locally (like test_hello_world, for example)?

Other tests pass, including wasm0.test_hello_world, wasm0.test_unistd_dup, wasm0.test_nodefs_rw, etc.

Clearing the cache doesn't change anything. I'm using the latest version of the incoming branch of Emscripten with the latest LLVM and Binaryen versions from git.

@tlively
Copy link
Member

tlively commented Dec 18, 2019

Ah, if I remember correctly I saw that error when I had just pulled a new version of Binaryen but forgot to build it. Anyway, there is no guarantee that the latest versions of LLVM, Binaryen, and Emscripten work together correctly (and it is expected that sometimes they do not). I highly recommend using emsdk to install known-good combinations of tools or if you must manage the tools yourself, look at https://chromium.googlesource.com/emscripten-releases/ to see what emsdk's tot tag is shipping.

@sbc100 sbc100 closed this Jan 30, 2020
@sbc100
Copy link
Collaborator

sbc100 commented Jan 30, 2020

The deletion of the incoming branch triggered the automatic closing of this branch. My apologies. This was an unexpected side effect.

Re-opening and re-targeting to master.

@sbc100 sbc100 reopened this Jan 30, 2020
@sbc100 sbc100 changed the base branch from incoming to master January 30, 2020 20:23
@stale
Copy link

stale bot commented Jan 30, 2021

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 30 days. Feel free to re-open at any time if this issue is still relevant.

@stale stale bot added the wontfix label Jan 30, 2021
Base automatically changed from master to main March 8, 2021 23:49
@stale stale bot removed the wontfix label Mar 8, 2021
@trybka
Copy link
Collaborator

trybka commented Aug 12, 2021

Any chance of rebasing/landing this? I just ran into this issue as well.

Copy link
Collaborator

@trybka trybka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM modulo the comment about ERRNO_CODES -> cDefine

@@ -1310,8 +1315,13 @@ var SyscallsLibrary = {
#if ASSERTIONS
assert(!flags);
#endif
if (old.fd === suggestFD) return -{{{ cDefine('EINVAL') }}};
return SYSCALLS.doDup(old.path, old.flags, suggestFD);
if (old.fd === suggestFD) return -ERRNO_CODES.EINVAL;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

switch these to use the cDefine to avoid a dep on ERRNO_CODES which, I believe was removed

@jakogut
Copy link
Contributor Author

jakogut commented Aug 12, 2021

I can see about rebasing this later and applying the above feedback.

@sbc100
Copy link
Collaborator

sbc100 commented Sep 16, 2021

@jakogut do you have time to look at rebasing this change?

@IhorKaravan
Copy link

IhorKaravan commented May 10, 2022

Are you going to resolve this conflict soon to merge changes? I see that changes are approved.

@sbc100
Copy link
Collaborator

sbc100 commented May 10, 2022

I think a large part of this change landed in #14808.. but a rebase to see if there is anything else worth landing seems like a good idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

dup() syscall is wrongly implemented
7 participants