Provide method that can observe backup and restore (IObservable)

[Mpdreamz]

Much like the ReIndex() today it would be nice if we expose methods that encapsulates monitoring a backup and restore operation by exposing an IObservable that can report back on status (configurable interval), complete and errors.

[robertlyson]

Sounds interesting. I would like to implement this.

[Mpdreamz]

Go for it @robertlyson :) Have a look at Reindex() for an existing implementation. Keep us posted as well :)

[robertlyson]

Everything was going well until I started implementing the configurable interval feature and now I'm stuck :)

1. My first thought was Thread.Sleep should be enough, but later on I realized it won't be accurate.
2. Use Thread.Timer - it should be much more accurate than sleep, but I haven't found reliable / clear solution how to wait until Timer is done.

So, do you have any advice how to do this part easily? Maybe I can't see something obvious.

[Mpdreamz]

Great news :)

Thread.Sleep is blocking on the calling thread so best to avoid that.

This SO thread has some great usage patterns for using Threading.Timer, I would add that the param thats listed as null is the thread local state something you can use to pass a reference to the timer itself and the current observable.

Another route to explore:

Use Task.Delay(200).ContinueWith(methodThatCallsSuccessErrorOrTaskDelayOnNextAgain)

Have you already pushed your work to your fork?

[robertlyson]

Thanks for your tips @Mpdreamz :)

I wrote tests in the way that blocking current thread wasn't a problem, so I'll change this and maybe solution will pop up.
This topic about Timers looks really interesting, so thanks for this 👍

I guess I can't use Task.Delay, because support for this was added in .Net 4.5, and NEST solution is set for .Net 4.0.

I haven't pushed code to my fork yet.
I want to finish this interval feature first, maybe I'll do this soon.

[robertlyson]

I pushed branch with this feature, but it doesn't work properly yet.

To obtain information about restore status I'm using ElasticClient.RecoveryStatus but the shard stage is always marked as done. I don't think so it's true.

Would be great if you can take a look on SnapshotRestoreObservable test.

Don't know what is causing this issue, advice is much appreciated :)

PS:
Can't make recovery status request with descriptor parameter Human(true).
Serialization exception.

[Mpdreamz]

It's going to be tricky writing a reliable integration tests, what might be best is to create a hefty index and restore it manually capturing the status requests in between and writing a unit test for it.

The unit test and mocking is a bit involved I can totally understand if this is beyond your available free time :) You could open a PR with what you have at the moment (from glancing over github it looks very good so far!) and me or @gmarz could work on adding the tests for it as part of our 1.4 release?

PS: Just offering to unload some work from you by the way, if you feel up for doing the mocking and testing as part of our unit tests let me know I'll gladly pair with you here over skype/hangouts 👍

[Mpdreamz]

Closing this since @robertlyson implemented this as per #1115 ! 👍