Intermittent - test-tls-connnect-cnnic.js failures

[mhdawson]

• Version: 4.3.1
• Platform: OxS 64 bit
• Subsystem:net

This test fails intermittently (not all that often) in our build/test environment. I also wonder if we should have a test that connects to a public site.

@shigeki

[Trott]

???? I don't see test-tls-connnect-cnnic.js in our code base. I only see these:

test/parallel/test-tls-connect-given-socket.js
test/parallel/test-tls-connect-no-host.js
test/parallel/test-tls-connect-pipe.js
test/parallel/test-tls-connect-secure-context.js
test/parallel/test-tls-connect-simple.js
test/parallel/test-tls-connect-stream-writes.js
test/pummel/test-tls-connect-memleak.js

[gibfahn]

@Trott It's in internet:

https://github.com/nodejs/node/blob/master/test/internet/test-tls-connnect-cnnic.js

[Trott]

Heh. I didn't find it because it's connnect (with three instances of n) rather than connect.

[shigeki]

@mhdawson Are you running tests on test/internet on CI?
Some tests in test/internet need to have net connections to public servers so that their results are depending on network environments where you run tests. Some discussion was made on #2424 (comment) to let them out of our CI.

Forgive me I miss-spelled two test filenames of connnect. I found it today.

[mhdawson]

We are running them as part of our regression tests for our internal builds. It might be good to document when each set should/should not be used. For the tests in question is it not possible to have them use local resources ?

[shigeki]

All internet tests should be fine as long as networking and remote resources are fine as we expected.
But they are of course out of our controls so that I wrote the comment not to include them in CI.

For the test in your question, you can see test/parallel/test-tls-cnnic-whitelist.js, which is the test to use only local resources. The test of test/internet/test-tls-connnect-cnnic.js is made for the test against the resource we cannot build in our local environment.

All the tests in tests/intenet are inevitable to be flaky because they depend on networking environment and remote resources. Why do you want to run and check them at every your internal builds? I think we can check their regressions only by hands.

[mhdawson]

Thanks for the clarification. We probably did not understand that the internet ones were not meant to be run as part of a CI.

[mhdawson]

I guess I'll close this