syncing package metadata in poor internet: connection timeout

[Tarang]

The new Meteor 0.9.x build tools work very poorly for people who do not have access to low latency high grade connections (virtually all of Sub saharan Africa, including South Africa).

The inherent problem with the design of DDP is that latency is a huge issue with it. Even if there is no real intention to use it. There are no major data centers on the continent (unlike aws/do).

When connecting via DDP there is a 'handshake' for the initial websocket, then for the second layer on top (if its there, like SockJS via ws emulation), then for the DDP stack itself. Since Meteor has no servers close to Africa it becomes difficult to use.

In addition to latency the connections are very jittery and cut off often.

I see this in my logs:

=> Modified -- restarting.
Error connecting to package server: DDP connection timed out
Warning: could not connect to package server

Error connecting to package server: DDP connection timed out
Warning: could not connect to package server

Error connecting to package server: DDP connection timed out
Warning: could not connect to package server

Error connecting to package server: DDP connection timed out
Warning: could not connect to package server

Error connecting to package server: DDP connection timed out
Warning: could not connect to package server

I just modified my app's code, why would I need a DDP connection anyway? This process takes up to 20 seconds on a fresh simple project. Each simple small file change really adds up and it slowly becomes unrealistic to use Meteor. The old page refresh is much faster.

What I suggest is some kind of work around that allows it to work in this environment.

I've been using Meteor from the start and this really troubles me that I have to experience this.

Keep in mind I have tried this now in two countries (South Africa & Kenya) and I see the impact on both.

[mizzao]

#2789 seems to be the same issue as this.

[alimalibaev]

mizzao, And what do we do now? Say goodbye to Meteor.JS?

[n1mmy]

Thanks for the report, @Tarang. That sounds pretty frustrating. We're looking into it, and we'll see if there is a workaround that makes using meteor easier on slow / high latency connections.

[glasser]

As a note, the timeout in question is actually just for the initial websocket handshake, not for the other handshakes you mention.

[glasser]

Tarang, can you check out 98429c2 and run meteor refresh and tell me if you get the connect timeout?

[matb33]

I've been getting a pretty consistent ~6 timeouts each time I build (I'm doing a bunch of meteor build for cordova testing), and my connection isn't slow (25 down/10 up). I think having a large number of packages may be causing this (I have 97 in this project).

Error connecting to package server: DDP connection timed out
Warning: could not connect to package server

Error connecting to package server: DDP connection timed out
Warning: could not connect to package server

Error connecting to package server: DDP connection timed out
Warning: could not connect to package server

Error connecting to package server: DDP connection timed out
Warning: could not connect to package server

Error connecting to package server: DDP connection timed out
Warning: could not connect to package server

Error connecting to package server: DDP connection timed out
Warning: could not connect to package server

Running on OS X 10.10, Meteor 0.9.4

[glasser]

We just landed a fix that simplifies when refreshes are performed, so you certainly will not see 6 timeouts. (We will be opening known issues tickets to describe future optimizations.) Number of packages shouldn't affect "DDP connection timed out", I think. Maybe there's something else going here, hmm. We'll have an RC out shortly and I'll ask you if it makes it better.

[matb33]

@glasser I'd be happy to test it out, let me know

[n1mmy]

We've put out a release with the changes glasser described. Can you try using --release 1.0-rc.2, please? Hopefully it will be faster for you.

[Tarang]

I just gave this a go. The improvement is quite dramatic & meteor is once more very usable!

I'm not sure whether it was the timeouts because the time taken for meteor to reload has improved too. Were there any other changes other than the timeout?

I'll keep checking my internet's quite stable today (Sunday, so quite expected). But so far it looks really good.

[n1mmy]

Great! Glad to hear it, @Tarang.

There are several changes in 1.0-rc that should help:

1. timeout changes glasser mentioned above
2. fewer package resyncs
3. better error handling when there are resync errors.

Hopefully these changes combined make meteor usable even with slow internet.

[babnik63]

Exactly my case ;-)

[matb33]

@glasser it has made a world of difference -- no more timeouts and much faster to reload as well. Also exciting to be running a RC of 1.0!

[glasser]

Great to hear! Closing this.

[Tarang]

Just thought I'd leave another note. The performance increase is superb with this. It's even better than meteorite. It was incredibly frustrating for a while but now no =)