Fix idxInSync, WaitUntilIndexesAreSynced, and TestNames

[KyleMaas]

Requires ssbc/go-luigi#4, so be sure to add:

replace github.com/ssbc/go-luigi => ./../go-luigi

...or something like it to go.mod if that pull request hasn't yet been merged.

This fixes #251 and #250 by fixing idxInSync, WaitUntilIndexesAreSynced(), and TestNames(). I ran TestNames over 500 times with zero failures with this in place.

[KyleMaas]

Oh, and the CI tests will fail with this, because they're not going to be using the updated version of Luigi.

[cryptix]

Oh wow! I will try to have a look at this later this week.

re go.mod: once you pushed your working branch you should be able to go get $package@branch and not need a local replace anymore.

[decentral1se]

Incredible @KyleMaas! Yeh this matches up with all our discussion and thinking through this. Pretty damn smooth solution in the end 🤯 Will wait to hear from @cryptix for the final word but this LGTM 👏

[KyleMaas]

Thanks! The first way I built this (which worked) was to pass in idxInSync as a WaitGroup pointer. But then I realized we can't reference count that to later query if indexes are caught up, which we need to gracefully fix this:

#293 (comment)

So I went with the callback method instead, which also opens up more options for other users of Luigi to implement status checking in other ways if they so choose. And now we can do "index busy" reference counting using atomic operations to give a status of how many indexes are out of sync. We can also use this to update the index status codes, so the callback method IMHO is far superior for flexibility.

[cryptix]

Super down to trying this! Thanks for combing through this.

Left a comment on the luigi PR.

[decentral1se]

ssbc/go-luigi#4 has landed, let's rebase & merge here too 👏

I think go get github.com/ssbc/go-luigi@bd28e676fa9902104dac8620f332761346f968bd should summon the go.* file upgrades?

[decentral1se]

Ah sure I can do the luigi work after the merge, thanks for all this work @KyleMaas!

[KyleMaas]

Excellent. Thanks!