hierarchy-lookup may be the cause of hanging imports

[sevko]

After running several imports via pelias/openaddresses, I noticed that they'd hang whenever I pipe()d data through hierarchy-lookup. It might be the cause of the hanging imports we see in production, and, after some digging, these lines look suspect:

function(){
  // close the adapter when done
  // @TODO: this might not be ideal in some cases
  opts.backends[0].adapter.client.close();
});

As per the through2 documentation, the flush function receives a callback argument that it should invoke before exiting. Adding that made the import exit as expected, but another problem popped up right before it did:

esclient error { [Error: No Living connections] message: 'No Living connections' }

I queried localhost:9200/pelias/_count and came up about 500 records short of what I expected. I suspect that opts.backends[0].adapter.client.close(); globally closes connections to elasticsearch, since commenting that out resolved the problem. If we don't need it, we can get rid of the flush function entirely.

[missinglink]

Let's have a chat about this issue, it's most likely the source of the 'process not exiting' issues and we can probably do something better to avoid it.

[sevko]

This was the problem for openaddresses, which depends on this package, and removing the flush function resolved it. openstreetmap doesn't replicate this code and has a separate issue, unfortunately.