HTTP+filtered changes fails with ESOCKETTIMEDOUT on large CouchDB changeset

[shimaore]

If CouchDB contains a large changeset (in my case, 170K+ deletions), filtered replication will take some time to process in CouchDB (which is OK), but PouchDB's .changes will fail with an ESOCKETTIMEDOUT error:

$ node -e 'P = require("pouchdb"); db = new P("http://127.0.0.1:5984/provisioning"); db.changes({include_docs:true,filter:"royal-thing/global_numbers",since:1897043}).on("error",function(error){console.log(error.toString());});'
Error: ESOCKETTIMEDOUT

This is repeatable; the error shows up 31 seconds after the start of the request.
(This is PouchDB 3.3.1 against CouchDB 1.4.0. since in the query is at 1897043, while the update_seq of the database is 2020286. CouchDB logs show it running the filter.)

Doing a plain curl at the command prompt with the same parameters as the PouchDB request, curl just waits for the data to come in:

$ curl 'http://127.0.0.1:5984/provisioning/_changes?timeout=25000&include_docs=true&feed=longpoll&filter=royal-thing/global_numbers&since=1897043&limit=25'
{"results":[

I tried playing with ajax.timeout since this is the only timer available, but unsurprisingly it doesn't affect the outcome, since the TCP connection is established (there's no timeout at the network level).

Note: this might be related to the second problem described in #3210

[nolanlawson]

I assume this is Node.js? Can you provide a public version of your database so that we can reproduce?

This seems like some Node-specific error handler that we're not setting. @calvinmetcalf any suggestions?

[nolanlawson]

I feel tempted to just do one of those

process.on('ESOCKETTIMEDOUT', function () {
  // do something
});

But that can't possibly be the "Node way" of handling this kind of error.

[calvinmetcalf]

it's a timeout error you just handle it like a regular error...

[nolanlawson]

So are we supposed to handle this? Or is it the user's responsibility to handle it in an on('error')?

[shimaore]

I guess I wasn't explicit enough about the issue. This isn't about how the error is reported, it's about "there shouldn't be a timeout at that point since CouchDB accepted the connection and it's just taking some time to generate the data". I'll do a quick grep in the code to see if I can figure where that 31 seconds timer comes from.

[shimaore]

Here are some steps to recreate:

• create a database with a design document containing a filter that always return false -- function(doc) { return false; } is enough;

• fill the database with records so that the filter takes some time; on my laptop ~20K records are enough to make this work;

• query the database using the filter:

  PouchDB = require("pouchdb");
  var db = new PouchDB("http://127.0.0.1:5984/foo");
  db.changes({ include_docs:true, filter:"bar/slow", since:1 })
  .on("error",function(error){console.log(error.toString());});
  

The expected outcome is that no changes are reported and the query returns normally.
The observed outcome is that if CouchDB takes more than 31 seconds to complete, the HTTP adapter fails.

curl will wait long enough and report an empty result:

{"results":[

],
"last_seq":19130}

The first line is sent immediately, the last two lines are sent at the end of the filter, for example after 48s in my last run.

[shimaore]

ESOCKETTIMEDOUT is generated by request, apparently using the same timer as for ETIMEDOUT. The value for that timer is 30s, I'm trying to figure out whether it can be modified.

[shimaore]

If I provide an ajax object to new PouchDB, running the code with NODE_DEBUG=request I noticed the first request send by PouchDB has the value set properly, but the subsequent requests have the default (30s) timeout.

[shimaore]

OK, found it. PouchDB's changes has an undocumented timeout option that overrides the one provided in new PouchDB and defaults to 30s.

[shimaore]

@nolanlawson Yes this applies to Node.js and the HTTP adapter, although I suspect similar problems will occur in the browser.

[nolanlawson]

Thanks for all the detailed research. So it seems we should either document this, or make sure that the timeout from new PouchDB is used in changes. Reopening this issue.

[nolanlawson]

Oh wait, you're already on it. Nice! :)

[limoragni]

I had the same error replicating a large pouchdb database (40K docs) to an online couchdb instance. It was replicating just fine, but I got Error: ESOCKETTIMEDOUT(…) after aprox 20 mins. Is this expected behaviour? Is there a way of setting the timeout? And what's curious is that if I try to replicate again it throws that same error but after a couple of secs. And if I do it a third time it gives me this similar error Error: ETIMEDOUT(…)

[nolanlawson]

I believe the timeout is a Node-wide setting, but I'm not sure TBH.

[ronycohen]

Hi, I have the same error with the last Version.
Is there a way to relaunch the change action ?

Unhandled rejection Error: ETIMEDOUT
at onError (/MyApp/node_modules/pouchdb/lib/index.js:425:18)
at Request._callback (/MyApp/node_modules/pouchdb/lib/index.js:463:14)
at self.callback (/MyApp/node_modules/pouchdb/node_modules/request/request.js:198:22)
at Request.emit (events.js:107:17)
at null._onTimeout (/MyApp/node_modules/pouchdb/node_modules/request/request.js:764:12)
at Timer.listOnTimeout (timers.js:119:15)

Here is my config :

  var options = {
                continuous: true,
                live: true,
                since: 0,
                include_docs: true,
                batch_size : 1000,
                filter : 'globalFilter/demandesToBackend'
            };

db.changes(options)
                .on('change', function (change) {
                    logger.info('demande occured ', change);
                })
                .on('error', function(e){
                    logger.error("error in globalRemoteDB.changes : ",e);
                });

[cdaringe]

this definitely threw me for a loop, too. I did not expect the changes timeout to be different than the primary