testem fails to connect when running a production environment

[runspired]

happens in both ember-cli 3.4.x and 3.5.x

does not happen in ember-cli ~3.3.0

Output from ember version --verbose && npm --version && yarn --version:

(node:73081) ExperimentalWarning: The fs.promises API is experimental
ember-cli: 3.4.3
http_parser: 2.8.0
node: 10.1.0
v8: 6.6.346.27-node.6
uv: 1.20.2
zlib: 1.2.11
ares: 1.14.0
modules: 64
nghttp2: 1.29.0
napi: 3
openssl: 1.1.0h
icu: 61.1
unicode: 10.0
cldr: 33.0
tz: 2018c
os: darwin x64
6.4.1
1.5.1

[runspired]

(this prevents ember-data from upgrading past 3.3.0)

[runspired]

was able to get passed this by removing broccoli-asset-rev which is responsible for fingerprinting which ember-data does not need. Apps hitting this issue may disable fingerprinting in ember-cli-build when being run via the test command as it causes testem.js to be fingerprinted and not load correctly.

You can detect that the command was test and the environment production via process.env.EMBER_CLI_TEST_COMMAND === true

see

ember-cli/lib/commands/test.js

Line 47 in 6635ab1

process.env.EMBER_CLI_TEST_COMMAND = true;

[rwjblue]

Thank you for reporting and providing a nice work around, I still think we should make this work nicely out of the box.

[ctcpip]

• workaround partially defeats the purpose of testing with the production flag for those of us who use broccoli-asset-rev.
• production tests are broken for a new app (ember new myapp)... this is not the first time this sort of thing has happened, either. is there something we can do to prevent this happening for future releases?
• it would be nice if we could take a look at this issue when addressing this, as it seems to be related: fingerprinting with 'prepend' option breaks production tests #7157

[ctcpip]

there's been a TODO to re-enable the production test for nearly three years 😭

509b010

[step2yeung]

I was made aware of this issue after chatting with @scalvert
I can provide first an explanation of why this is happening:

Testem starts an express server that will serve the assets. When it get a request for /testem.js will concat all the test framework adapter code from testem/public/testem directory with serveTestemClientJs()

app.get('/testem.js', (req, res) => {
      this.serveTestemClientJs(req, res);
    });

This is necessary to setup the socket connection between the browser and the testem server and the hooks to the test framework for reporting.

If you run this in headful chrome or with --server you will actually see a browser is invoked and tests will run on the browser, but there is no test reporting on the terminal. And eventually, it will fail with Browser failed to connect within 30s. testem.js not loaded?

As @runspired has pointed out, testem.js is fingerprinted when a production build is done. So the filename looks something like testem-some-finger-print.js. So when this file is requested by the browser, the express server just serves testem-some-finger-print.js as any normal asset, and won't call serveTestemClientJs().

One potential fix could be recognizing testem-some-finger-print.js on the express server:

app.get(^\/testem.*.js, (req, res) => {
      this.serveTestemClientJs(req, res);
    });

@rwjblue thoughts?

[rwjblue]

Seems fine, but I think it would also be ok to configure broccoli-asset-rev to ignore testem.js and not fingerprint it...

[step2yeung]

That works a well, if anything probably better

[runspired]

For anyone who hits this after separating a production build from the test command for re-use, building with EMBER_CLI_TEST_COMMAND=true will include the necessary test assets and infra. I don't think it's a common use case and certainly it is a good default for production builds to not include these by default, but it manifests the same as this error did.