optimizely-sdk test suite has cross-test-file pollution

[jasonkarns]

While researching which file is causing the node process to hang during the test suite (#487 ), I ran each test file individually.

In doing so, I discovered that the lib/plugins/event_dispatcher/index.browser.tests.js test file does not pass successfully when run independently. That means it is dependent (accidentally) on some side effect that another test file is causing.

$ npx mocha -r esm -r lib/tests/exit_on_unhandled_rejection.js lib/plugins/event_dispatcher/index.browser.tests.js


  lib/plugins/event_dispatcher/browser
    APIs
      dispatchEvent
        1) should send a POST request with the specified params
        2) should execute the callback passed to event dispatcher with a post
        3) should execute the callback passed to event dispatcher with a get


  0 passing (10ms)
  3 failing

  1) lib/plugins/event_dispatcher/browser
       APIs
         dispatchEvent
           should send a POST request with the specified params:
     ReferenceError: XMLHttpRequest is not defined
      at dispatchEvent (lib/plugins/event_dispatcher/index.browser.js:31:5)
      at Context.<anonymous> (lib/plugins/event_dispatcher/index.browser.tests.js:50:9)
      at process.topLevelDomainCallback (domain.js:126:23)

  2) lib/plugins/event_dispatcher/browser
       APIs
         dispatchEvent
           should execute the callback passed to event dispatcher with a post:
     ReferenceError: XMLHttpRequest is not defined
      at dispatchEvent (lib/plugins/event_dispatcher/index.browser.js:31:5)
      at Context.<anonymous> (lib/plugins/event_dispatcher/index.browser.tests.js:69:9)
      at process.topLevelDomainCallback (domain.js:126:23)

  3) lib/plugins/event_dispatcher/browser
       APIs
         dispatchEvent
           should execute the callback passed to event dispatcher with a get:
     ReferenceError: XMLHttpRequest is not defined
      at dispatchEvent (lib/plugins/event_dispatcher/index.browser.js:51:5)
      at Context.<anonymous> (lib/plugins/event_dispatcher/index.browser.tests.js:86:9)
      at process.topLevelDomainCallback (domain.js:126:23)

@optimizely/optimizely-sdk version:

tested at 764135b

[jasonkarns]

Looks like there are actually two instances of cross-test pollution.

$ npx mocha -r esm lib/index.browser.umdtests.js 


  javascript-sdk
    APIs
      createInstance
        1) "before each" hook for "should default to INFO when no logLevel is provided"
        2) "after each" hook for "should default to INFO when no logLevel is provided"


  0 passing (9ms)
  2 failing

  1) javascript-sdk
       APIs
         createInstance
           "before each" hook for "should default to INFO when no logLevel is provided":
     ReferenceError: window is not defined
      at Context.<anonymous> (lib/index.browser.umdtests.js:56:19)
      at process.topLevelDomainCallback (domain.js:126:23)

  2) javascript-sdk
       APIs
         createInstance
           "after each" hook for "should default to INFO when no logLevel is provided":
     ReferenceError: window is not defined
      at Context.<anonymous> (lib/index.browser.umdtests.js:64:9)
      at process.topLevelDomainCallback (domain.js:126:23)

[jasonkarns]

Even more scenarios of cross test pollution:

$ npx mocha  -r esm -r lib/tests/exit_on_unhandled_rejection.js lib/index.browser.tests.js lib/index.browser.umdtests.js lib/index.node.tests.js lib/index.react_native.tests.js 

[/snip]

  40 passing (94ms)
  5 failing

  1) javascript-sdk
       APIs
         createInstance
           "before each" hook for "should default to INFO when no logLevel is provided":
     ReferenceError: window is not defined
      at Context.<anonymous> (lib/index.browser.umdtests.js:56:19)
      at process.topLevelDomainCallback (domain.js:126:23)

  2) javascript-sdk
       APIs
         createInstance
           "after each" hook for "should default to INFO when no logLevel is provided":
     ReferenceError: window is not defined
      at Context.<anonymous> (lib/index.browser.umdtests.js:64:9)
      at process.topLevelDomainCallback (domain.js:126:23)

  3) optimizelyFactory
       APIs
         createInstance
           "before each" hook for "should not throw if the provided config is not valid and log an error if logger is passed in":
     TypeError: Attempted to wrap validate which is already wrapped
      at checkWrappedMethod (node_modules/sinon/lib/sinon/util/core/wrap-method.js:39:21)
      at wrapMethod (node_modules/sinon/lib/sinon/util/core/wrap-method.js:85:13)
      at Proxy.stub (node_modules/sinon/lib/sinon/stub.js:58:44)
      at Context.<anonymous> (lib/index.node.tests.js:46:15)
      at process.topLevelDomainCallback (domain.js:126:23)
  --------------
  Error: Stack Trace for original
      at wrapMethod (node_modules/sinon/lib/sinon/util/core/wrap-method.js:110:26)
      at Proxy.stub (node_modules/sinon/lib/sinon/stub.js:58:44)
      at Context.<anonymous> (lib/index.browser.umdtests.js:41:15)
      at process.topLevelDomainCallback (domain.js:126:23)

  4) optimizelyFactory
       APIs
         createInstance
           "after each" hook for "should not throw if the provided config is not valid and log an error if logger is passed in":
     TypeError: _157‍.g.console.error.restore is not a function
      at Context.<anonymous> (lib/index.node.tests.js:53:31)
      at process.topLevelDomainCallback (domain.js:126:23)

  5) javascript-sdk/react-native
       APIs
         createInstance
           when no event dispatcher passed to createInstance
             uses the default event dispatcher:
     TypeError: Cannot read property 'then' of undefined
      at Context.<anonymous> (lib/index.react_native.tests.js:150:39)
      at process.topLevelDomainCallback (domain.js:126:23)

[mjc1283]

Not fixed totally yet.

[jasonkarns]

@mjc1283 HAHA, today I re-learned (for the millionth time) not to use the phrase "partially fixes #N" because github just sees the fixes

[mjc1283]

I tried to reproduce this on current master, but I think it's fixed now (probably due to #489). lib/index.browser.umdtests.js is intended to run in Karma via the "test-umdbrowser" package.json script.