Skip to content
This repository has been archived by the owner on Jan 20, 2022. It is now read-only.

fixes issue #3. remove dependence on pelias-config #4

Closed
wants to merge 1 commit into from

Conversation

ryanramage
Copy link

As suggested, pass the config directly to the elasticsearch.Client.

missinglink added a commit that referenced this pull request Dec 29, 2014
@missinglink
Copy link
Contributor

hey @ryanramage thanks for the PR.

I didn't know anyone else was using this lib, it's quite useful for the streaming bulk indexing in to ES and this repo represents the third iteration of a client to achieve that end.

I'm currently working on a v4 which actually works great but is totally undocumented and really really messy (cleaning it up is near the top of my todo list).
https://github.com/pelias/dbclient

The new client handles back-pressure more elegantly and will be more modular and could potentially serve as a general purpose bulk streaming indexer for a range of datastores via adapters.

I've attached another PR which goes a little further in declaring the change is a breaking API change and throws an error to avoid users getting stung with a misconfigured client.

The settings object and the config object contain different parameters and using the same for both would break existing consumers.

How are you using this lib? is it as part of Pelias or something which is useful independently?

@missinglink
Copy link
Contributor

We've previously discussed internally that we are happy to move this repo out to another org (potentially the mapzen org) if it is useful to the open-source community at large.

If you're interested in collaborating on a general-purpose streaming batch indexer I'd be happy to have a chat about what I've learned so far and we could work together on it, plus maybe add couchdb support?

Last time I spoke to Max Ogden he said that leveldb only has a streaming ReadStream and not a WriteStream, this could also serve to fix that issue for leveldb and the dat project [1].

[1] dat-ecosystem-archive/datproject-discussions#5

@orangejulius
Copy link

We haven't touched this code in a long time, we'd suggest using the aforementioned pelias/dbclient now.

I'm closing this PR, but if this code is actually still really useful to people, let us know and we can figure something out.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants