Review decision to not support HTTP for NDT7

[nkinkade]

We lately discovered that Fing has been relying on boticelli (multi-stream ndt5) in the neubot experiment. We were not generally aware of this, and as we continued to migrate nodes to the k8s platform, that left fewer and fewer nodes running the legacy neubot experiment on the PLC platform. At some point (that point being yesterday) the number of nodes got so low and geographically distant from Europe that Fing started noticing errors.

Fing's workaround for now is for clients to a do single-stream NDT5 test instead, which was the fallback previously anyway. They have a limitation in their hardware that prevents them from currently considering NDT7 over HTTPS. They could support NDT7 over HTTP, but we don't currently support this mode of operation.

We should reconsider whether we ought to allow both HTTPS and HTTP NDT7 tests. Encryption is quickly becoming the standard for the Internet, but HTTP is still important. Additionally, there are probably very few privacy-related reasons why NDT7 must be HTTPS, since it's all synthetic data anyway.

[pboothe]

Pro

• HTTP is one of the most important internet protocols, and many people rely on it.
• Allows you to run an NDT7 server without knowing how to get a cert
• Allows devices with crappy CPUs to test fast networks

Con

• It can be sniffed in transit, allowing providers to use DPI to distinguish speed test traffic from non-speed-test traffic.
• It splits our dataset into encrypted and unencrypted tests
• It adds complication to the ndt7 codebase

Still don't know what the guaranteed right answer is here.

[nkinkade]

Adding to your first con (about DPI): there is no need to do deep packet inspection for an ISP to distinguish M-Lab speedtest traffic. They only need to do very shallow packet inspection to see that the traffic is destined for an M-Lab host. If an ISP really cared enough to try to distinguish M-Lab traffic on their network, and even further to manipulate it in some way, I would argue that HTTPS wouldn't hinder them.

[pboothe]

The more I think about this, the more I think we should support unencrypted NDT7. In particular, the reason "Allows you to run an NDT7 server without knowing how to get a cert" turns out to be super-important for people who want to use NDT7 as a pedagogical tool (and, btw, they SHOULD want to use it as a pedagogical tool). The fact that supporting educational objectives also aligns with supporting low-powered devices seems to imply that the right thing is to support unencrypted NDT7 tests.

[bassosimone]

We have already implemented this for the official Go client in m-lab/ndt7-client-go#31. There is a pending PR to do this in the server (#222). We may want to follow-up for libndt as well, but that is a slightly longer process perhaps (measurement-kit/libndt#151).

[pboothe]

Decision: we will support unencrypted NDT7