Puppeteer tests fail when not on the Internet

[timabbott]

Repurposed to discuss the new puppeteer tests with the same bug.

Apparently, test-js-with-casper fails when one is not on the Internet, fairly early.

I have no idea why -- we don't have any intentional use of the Internet in our casper tests, so this is likely an accident of some sort.

(Structurally, test-backend is designed to block standard Python ways of accessing the Internet to ensure we never introduce use of the Internet in our tests there; we need to do the same thing here)

[zulipbot]

Hello @zulip/server-testing members, this issue was labeled with the "area: testing-infrastructure" label, so you may want to check it out!

[andersk]

tshark suggests that it’s blocked on a DNS request for zulipdev.com. (It also shows periodic DNS requests for puppet, whether or not the tests are running…that’s weird but probably unrelated.)

[timabbott]

Oh, of course, that makes sense. (That's the hostname mapping to 127.0.0.1 we use for some testing). We should be able to fix that.

[YashRE42]

We disabled casper in CI in 1a8798d. We can at least drop the high priority label here?

[timabbott]

Yeah, we can close this; @chdinesh1089 @priyank-p FYI; we should fix the similar issue with puppeteer if present.

[chdinesh1089]

This actually happens with puppeteer too. The problem is with the test server starting. As @andersk told, it is blocked on DNS request.
So, we're getting stuck at while not server_is_up(server, log_file): in test_server_running with server_is_up returning False because we have return requests.get('http://zulipdev.com:9981/accounts/home').status_code == 200 that's doesn't return True when there's no internet.

Do you have any ideas on how to try fixing this?

[timabbott]

I think we could check localhost:9991 there; but then we'll access the zulipdev.com URLs inside the realm creation test (for example).

We could try making it work at least in Vagrant and Droplets by adding DNS entries for *.zulipdev.com in /etc/hosts pointing at 127.0.0.1.

[chdinesh1089]

Adding these to /etc/hosts makes it work.

127.0.0.1       testsubdomain.zulipdev.com
127.0.0.1       zulip.zulipdev.com
127.0.0.1       zulipdev.com

(having *.zulipdev.com isn't detecting all subdomains.)

Thought we'd write code to add those in provision.py script but we don't allow provision to be run as root. Is there a better way than doing it manually to add these?

[timabbott]

We should add these to Vagrantfile, or actually better, to some common code that runs in Vagrant and droplets. @hackerkid @amanagr we don't have a common "dedicated Zulip environment" setup script, but probably should, right?

Maybe we can call it tools/setup/configure-dedicated-zulip-host, and that can call tools/setup/add-zulipdev-to-etc-hosts, which we can also recommend in the test failure output if that line has a network failure.

[timabbott]

That new script is also what we want for:
#15381