Fix for bulk indexing error (#20) plus TTL addition

[araddon]

This has unit tests and addresses #20.

Also includes:

• a (unfortunate) breaking api change for adding ttl parameter to bulk indexing.
• exposing the GlobalBulkIndexor

[mattbaird]

Hey Aaron, good change. I don't know what the etiquette is for breaking changes. Should we put something on the readme?

[araddon]

Yes, i feel really bad about the breaking change but didn't really see a way around it. Yes, Let me add something to the Readme. Also, maybe a note about the Search DSL api being still open to changes? I continue to find new features in ES search dsl which to meet require changes.

[mattbaird]

I like that plan. Better to break it early. Thanks. I will merge when you
submit a pr with the updated readme.
On Jul 10, 2013 9:19 AM, "Aaron Raddon" notifications@github.com wrote:

Yes, i feel really bad about the breaking change but didn't really see a
way around it. Yes, Let me add something to the Readme. Also, maybe a note
about the Search DSL api being still open to changes? I continue to find
new features in ES search dsl which to meet require changes.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/21#issuecomment-20753626
.

[dmichael]

@araddon - out of curiosity (and not fully reading the code yet), is it possible to get the responses from the bulk indexer as well? - specifically concerning responses where there errors or for checking for duplicates. Looking forward to this merge - thanks for this work.

[araddon]

@dmichael , I updated the example doc to show usage to inspect the response bytes. However, @mattbaird the http.Response isn't available in the underlying api.DoCommand. What would you think about creating a new api.DoWithResponse?

Also, here is how you would get the http response until that change is made:

indexor := core.NewBulkIndexor(10)
// Create a custom Sendor Func, to allow inspection of response/error
indexor.BulkSendor = func(buf *bytes.Buffer) error {
    req, err := api.ElasticSearchRequest("POST", "/_bulk")
    if err != nil {
        return err
    }
    req.SetBody(buf)
    res, err := http.DefaultClient.Do((*http.Request)(req))
    if err != nil {
        return err
    }
    return err
}
done := make(chan bool)
indexor.Run(done)

[dmichael]

Awesome, thanks for the examples - so whattya think @mattbaird, can we get a merge? The current implementation of bulk in master is not completely functional/robust ... this looks to be a big improvement

[mattbaird]

will do.

On Thu, Jul 11, 2013 at 9:51 AM, David Michael notifications@github.comwrote:

Awesome, thanks for the examples - so whattya think @mattbairdhttps://github.com/mattbaird,
can we get a merge? The current implementation of bulk in master is not
completely functional/robust ...

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/21#issuecomment-20825765
.