Zip import fails with: throw new Error('"toString()" failed')

[swapsCAPS]

Config:

{
  "mongoUrl": "mongodb://localhost:27017/gtfs",
  "agencies": [
    {
      "agency_key": "openov",
      "url": "http://gtfs.openov.nl/gtfs/gtfs-openov-nl.zip"
    }
  ]
}

Output

gtfs-import --configPath ./gtfs-import.json

Starting GTFS import for 1 file
openov: Downloading GTFS from http://gtfs.openov.nl/gtfs/gtfs-openov-nl.zipbuffer.js:504
    throw new Error('"toString()" failed');
    ^

Error: "toString()" failed
    at Buffer.toString (buffer.js:504:11)
    at Request.<anonymous> (~/gtfs-importer/node_modules/request/request.js:1145:39)
    at emitOne (events.js:101:20)
    at Request.emit (events.js:191:7)
    at IncomingMessage.<anonymous> (~/gtfs-importer/node_modules/request/request.js:1091:12)
    at Object.onceWrapper (events.js:293:19)
    at emitNone (events.js:91:20)
    at IncomingMessage.emit (events.js:188:7)
    at endReadableNT (_stream_readable.js:974:12)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickCallback (internal/process/next_tick.js:104:9)

When downloading and extracting first it works fine

[brendannee]

Thanks for the report. I tried the config file and it worked with the latest version of node-gtfs 0.9.9

Try updating your global node-gtfs installation to the latest version:

npm update gtfs -g 

Otherwise, it is a large GTFS file so perhaps the download is failing silently?

[swapsCAPS]

Might be a large file issue indeed.
Also had some weird connection timeout issues with very large files in the import.js removeDatabase method.
Mongod CPU usage also stayed up to 300% after throwing the error.
Setting: { connectTimeoutMS: 0, socketTimeoutMS: 0 } in MongoClient.connect fixed it.
Better to just have a longer timeout probably instead of infinite.
But in the large file case, the default 30 seconds timeout MongoClient sets was not enough for some operations to complete.

[brendannee]

Thanks for that suggestion.

I made an update to the mongo connection timeout on import and it was able to handle importing the GTFS file you linked: https://github.com/BlinkTagInc/node-gtfs/releases/tag/0.9.10

You are right that it was in the removing of old data that it was timing out - perhaps I'll come up with a better way of doing this.

[stepri]

I got also this issue, tried several node.js versions. Also when using the path option in config.json I got the same error.