testing with the library "gl" (aka headless-gl) breaks if you have more than one test file #2029

Closed
gre opened this Issue Oct 31, 2016 · 19 comments

Comments

Projects
None yet
9 participants
@gre

gre commented Oct 31, 2016

What is the current behavior?

I have written 2 identical tests.

  • Each test individually passes.
  • Putting the 2 tests in a same file, passes.
  • but if each test is put in 2 files. it breaks.

Repository that reproduce the bug:

https://github.com/gre/jest-multi-tests-gl-issue

What is the expected behavior?

The tests should pass. (it passes in a same file!)

--debug

jest version = 16.0.2
test framework = jasmine2
config = {
  "rootDir": "/Users/gre/Desktop/jest-multi-tests-gl-issue",
  "name": "-Users-gre-Desktop-jest-multi-tests-gl-issue",
  "setupFiles": [],
  "testRunner": "/Users/gre/Desktop/jest-multi-tests-gl-issue/node_modules/jest-jasmine2/build/index.js",
  "scriptPreprocessor": "/Users/gre/Desktop/jest-multi-tests-gl-issue/node_modules/babel-jest/build/index.js",
  "usesBabelJest": true,
  "automock": false,
  "bail": false,
  "browser": false,
  "cacheDirectory": "/var/folders/3q/zjl78zks3bv7_8lfw8r505x40000gn/T/jest",
  "clearMocks": false,
  "coveragePathIgnorePatterns": [
    "/node_modules/"
  ],
  "coverageReporters": [
    "json",
    "text",
    "lcov",
    "clover"
  ],
  "globals": {},
  "haste": {
    "providesModuleNodeModules": []
  },
  "mocksPattern": "__mocks__",
  "moduleDirectories": [
    "node_modules"
  ],
  "moduleFileExtensions": [
    "js",
    "json",
    "jsx",
    "node"
  ],
  "moduleNameMapper": {},
  "modulePathIgnorePatterns": [],
  "noStackTrace": false,
  "notify": false,
  "preset": null,
  "preprocessorIgnorePatterns": [
    "/node_modules/"
  ],
  "resetModules": false,
  "testEnvironment": "jest-environment-jsdom",
  "testPathDirs": [
    "/Users/gre/Desktop/jest-multi-tests-gl-issue"
  ],
  "testPathIgnorePatterns": [
    "/node_modules/"
  ],
  "testRegex": "(/__tests__/.*|\\.(test|spec))\\.jsx?$",
  "testURL": "about:blank",
  "timers": "real",
  "useStderr": false,
  "verbose": null,
  "watch": false,
  "cache": true,
  "watchman": true,
  "testcheckOptions": {
    "times": 100,
    "maxSize": 200
  }
}
 PASS  ./b.test.js
 FAIL  ./a.test.js
  ● foo

    TypeError: framebufferTexture2D(GLenum, GLenum, GLenum, WebGLTexture, GLint)

      at WebGLRenderingContext.framebufferTexture2D (node_modules/gl/webgl.js:2297:11)
      at Object.<anonymous>.test (a.test.js:4:12)
      at process._tickCallback (internal/process/next_tick.js:103:7)

Test Suites: 1 failed, 1 passed, 2 total
Tests:       1 failed, 1 passed, 2 total
Snapshots:   0 total
Time:        1.435s
Ran all test suites.
@thymikee

This comment has been minimized.

Show comment
Hide comment
@thymikee

thymikee Dec 8, 2016

Collaborator

Confirmed, this is really strange. Maybe gl has some issues with running under Node's vm?

Collaborator

thymikee commented Dec 8, 2016

Confirmed, this is really strange. Maybe gl has some issues with running under Node's vm?

@thymikee

This comment has been minimized.

Show comment
Hide comment
@thymikee

thymikee Dec 8, 2016

Collaborator

Although it's most probably not Jest-related (but not 100% sure, maybe we run on some edge case here). So for now I'll close the issue but happy to keep the discussion going and reopen if necessary.

Collaborator

thymikee commented Dec 8, 2016

Although it's most probably not Jest-related (but not 100% sure, maybe we run on some edge case here). So for now I'll close the issue but happy to keep the discussion going and reopen if necessary.

@thymikee thymikee closed this Dec 8, 2016

@gre gre referenced this issue in stackgl/headless-gl Dec 9, 2016

Open

issue with running the library in parallel #91

@cpojer

This comment has been minimized.

Show comment
Hide comment
@cpojer

cpojer Dec 12, 2016

Contributor

Does it work with two files when doing jest -i? We parallelize across processes, so that may be an issue here?

Contributor

cpojer commented Dec 12, 2016

Does it work with two files when doing jest -i? We parallelize across processes, so that may be an issue here?

@thymikee

This comment has been minimized.

Show comment
Hide comment
@thymikee

thymikee Dec 12, 2016

Collaborator

Tested with -i, fails too.

Collaborator

thymikee commented Dec 12, 2016

Tested with -i, fails too.

@cpojer

This comment has been minimized.

Show comment
Hide comment
@cpojer

cpojer Dec 12, 2016

Contributor

That's really odd then because that makes it run in process, serially. It may indeed be a gl issue in this case.

Contributor

cpojer commented Dec 12, 2016

That's really odd then because that makes it run in process, serially. It may indeed be a gl issue in this case.

@mcrawshaw

This comment has been minimized.

Show comment
Hide comment
@mcrawshaw

mcrawshaw Mar 8, 2017

Not sure this will have any impact, but maybe try destroying the gl context at the end of each test (plus the -i arg)...

gl.getExtension('STACKGL_destroy_context').destroy()

Not sure this will have any impact, but maybe try destroying the gl context at the end of each test (plus the -i arg)...

gl.getExtension('STACKGL_destroy_context').destroy()
@gre

This comment has been minimized.

Show comment
Hide comment
@gre

gre Mar 9, 2017

I'm pretty sure my tests aren't leaking a gl context (I mean, at least in my "real" tests, not the one I showcase. but yeah, I can try to add that, i'm pretty sure it won't change)

gre commented Mar 9, 2017

I'm pretty sure my tests aren't leaking a gl context (I mean, at least in my "real" tests, not the one I showcase. but yeah, I can try to add that, i'm pretty sure it won't change)

@gre

This comment has been minimized.

Show comment
Hide comment
@gre

gre Mar 9, 2017

I've updated the showcase with the destroy() and also the latest jest version. the bug still occurs. -i does not help neither.

even tho the tests are identically, only one pass. and a test file that contains twice the test still pass. this does not make sense.

gre commented Mar 9, 2017

I've updated the showcase with the destroy() and also the latest jest version. the bug still occurs. -i does not help neither.

even tho the tests are identically, only one pass. and a test file that contains twice the test still pass. this does not make sense.

@mcrawshaw

This comment has been minimized.

Show comment
Hide comment
@mcrawshaw

mcrawshaw Mar 9, 2017

Yeah, I worked on it all morning... It is bizarre...

I can't run the test a second time within the same Jest process instance. If I quit the jest process and start again its ok.

As you mentioned, destroying contexts doesn't help, you can actually create many, many contexts in a loop without issue.

At the moment it's working because I changed the function that tests for the type error. It's situated in node_modules/gl/webgl.js.

function checkObject (object) {
  return typeof object === 'object' ||
  !object
}

All I have done is loosen the void check from object === void 0 to !object. Now it runs, at least it can create a gl context.

More work to do tomorrow.

mcrawshaw commented Mar 9, 2017

Yeah, I worked on it all morning... It is bizarre...

I can't run the test a second time within the same Jest process instance. If I quit the jest process and start again its ok.

As you mentioned, destroying contexts doesn't help, you can actually create many, many contexts in a loop without issue.

At the moment it's working because I changed the function that tests for the type error. It's situated in node_modules/gl/webgl.js.

function checkObject (object) {
  return typeof object === 'object' ||
  !object
}

All I have done is loosen the void check from object === void 0 to !object. Now it runs, at least it can create a gl context.

More work to do tomorrow.

@pconerly

This comment has been minimized.

Show comment
Hide comment
@pconerly

pconerly Mar 15, 2017

@mcrawshaw I made that change and my yarn tests were able to run successfully multiple times. We're not doing anything more than creating the GL context in our tests, but they were able to run multiple times.

@mcrawshaw I made that change and my yarn tests were able to run successfully multiple times. We're not doing anything more than creating the GL context in our tests, but they were able to run multiple times.

@quasor

This comment has been minimized.

Show comment
Hide comment
@quasor

quasor Apr 6, 2017

@thymikee given the above efforts, shall we reopen this?

quasor commented Apr 6, 2017

@thymikee given the above efforts, shall we reopen this?

@thymikee thymikee reopened this Apr 6, 2017

@thymikee

This comment has been minimized.

Show comment
Hide comment
@thymikee

thymikee Apr 6, 2017

Collaborator

Let's keep this open until we find proper solution or fix the gl lib.

Collaborator

thymikee commented Apr 6, 2017

Let's keep this open until we find proper solution or fix the gl lib.

@thymikee

This comment has been minimized.

Show comment
Hide comment
@thymikee

thymikee May 11, 2017

Collaborator

Closing due to: #3552 (as a workaround use --no-cache flag for now)

Collaborator

thymikee commented May 11, 2017

Closing due to: #3552 (as a workaround use --no-cache flag for now)

@thymikee thymikee closed this May 11, 2017

@gre

This comment has been minimized.

Show comment
Hide comment
@gre

gre May 11, 2017

interesting! thanks

gre commented May 11, 2017

interesting! thanks

@alvinsight

This comment has been minimized.

Show comment
Hide comment
@alvinsight

alvinsight Jul 31, 2017

Hey, @mikolalysenko @mourner
Any chance you could propagate @mcrawshaw 's fix into gl so it doesn't happen again ?
Or confirm why the void check has to be so strict ?
Thanks guys :)

Hey, @mikolalysenko @mourner
Any chance you could propagate @mcrawshaw 's fix into gl so it doesn't happen again ?
Or confirm why the void check has to be so strict ?
Thanks guys :)

@mourner

This comment has been minimized.

Show comment
Hide comment
@mourner

mourner Aug 1, 2017

@alvinsight I'm not sure about that check, but I'd welcome a PR!

mourner commented Aug 1, 2017

@alvinsight I'm not sure about that check, but I'd welcome a PR!

@mcrawshaw

This comment has been minimized.

Show comment
Hide comment
@mcrawshaw

mcrawshaw Aug 2, 2017

I'm not sure on the fix I mentioned above, it was something I came too after tracing the code. When I apply the fix and run it now I receive this error: THREE.WebGLShader: Shader couldn't compile.. Also, it doesn't make any sense why it works.

@gre your repo works when --no-cache is added to the args. I also don't see how this would impact the result. It's just a transpilation cache, there must be more too it.

I think we need to understand just what is going on with the --no-cache flag to work out the best approach to the problem.

I'm not sure on the fix I mentioned above, it was something I came too after tracing the code. When I apply the fix and run it now I receive this error: THREE.WebGLShader: Shader couldn't compile.. Also, it doesn't make any sense why it works.

@gre your repo works when --no-cache is added to the args. I also don't see how this would impact the result. It's just a transpilation cache, there must be more too it.

I think we need to understand just what is going on with the --no-cache flag to work out the best approach to the problem.

@alvinsight

This comment has been minimized.

Show comment
Hide comment
@alvinsight

alvinsight Aug 2, 2017

Ah, dang, good luck @mcrawshaw !
it's weird that THREE.WebGLShader even appears there, are you using three.js ?
three.js is weird so I wouldn't use it, all my test cases work as long as I add --no-cache and they're pretty similar to @gre 's repo.
Good luck again!

Ah, dang, good luck @mcrawshaw !
it's weird that THREE.WebGLShader even appears there, are you using three.js ?
three.js is weird so I wouldn't use it, all my test cases work as long as I add --no-cache and they're pretty similar to @gre 's repo.
Good luck again!

@hujiulong

This comment has been minimized.

Show comment
Hide comment
@hujiulong

hujiulong Apr 12, 2018

@mcrawshaw @mourner @gre

I found the reason, but I don't know how to fix it. I'm not sure this is the bug of gl or jest.

In gl/webgl.js, there is such a piece of code:

var gl = nativeGL.WebGLRenderingContext.prototype
// ...
var _framebufferTexture2D = gl.framebufferTexture2D    // native function
gl.framebufferTexture2D = function framebufferTexture2D(
    target,
    attachment,
    textarget,
    texture,
    level ) {
    // ...
    if ( !checkObject( texture ) ) {
        throw new TypeError( 'framebufferTexture2D(GLenum, GLenum, GLenum, WebGLTexture, GLint)' )
    }
    // ...
}

In another place in gl/webgl.js, there is such a piece of code:

// line 
for ( var i = 0; i < ATTACHMENTS.length; ++i ) {
    _framebufferTexture2D.call(
        context,
        gl.FRAMEBUFFER,
        ATTACHMENTS[ i ],
        gl.TEXTURE_2D,
        0,    //  -> as parameter `texture`
        0 )
}

under normal circumstances, _framebufferTexture2D.call should be executed the native function, however, when running with jest, it is possible to call the overridden method(the js function)

checkObject( texture ) -> checkObject( 0 );
function checkObjectA (object) {
  return typeof object === 'object' ||
  !object
}
function checkObjectB (object) {
  return typeof object === 'object' ||
  (object === void 0)
}

checkObjectA( 0 );  // true
checkObjectB( 0 );  // false

hujiulong commented Apr 12, 2018

@mcrawshaw @mourner @gre

I found the reason, but I don't know how to fix it. I'm not sure this is the bug of gl or jest.

In gl/webgl.js, there is such a piece of code:

var gl = nativeGL.WebGLRenderingContext.prototype
// ...
var _framebufferTexture2D = gl.framebufferTexture2D    // native function
gl.framebufferTexture2D = function framebufferTexture2D(
    target,
    attachment,
    textarget,
    texture,
    level ) {
    // ...
    if ( !checkObject( texture ) ) {
        throw new TypeError( 'framebufferTexture2D(GLenum, GLenum, GLenum, WebGLTexture, GLint)' )
    }
    // ...
}

In another place in gl/webgl.js, there is such a piece of code:

// line 
for ( var i = 0; i < ATTACHMENTS.length; ++i ) {
    _framebufferTexture2D.call(
        context,
        gl.FRAMEBUFFER,
        ATTACHMENTS[ i ],
        gl.TEXTURE_2D,
        0,    //  -> as parameter `texture`
        0 )
}

under normal circumstances, _framebufferTexture2D.call should be executed the native function, however, when running with jest, it is possible to call the overridden method(the js function)

checkObject( texture ) -> checkObject( 0 );
function checkObjectA (object) {
  return typeof object === 'object' ||
  !object
}
function checkObjectB (object) {
  return typeof object === 'object' ||
  (object === void 0)
}

checkObjectA( 0 );  // true
checkObjectB( 0 );  // false
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment