Add a button in the web UI to rebuild sindexes

[mlucy]

This depends on #2797 and #2798 .

[neumino]

Just to be sure I'm on the same page as everyone else.

Issue #2797 to duplicate indexes is not making it in 1.14?

indexStatus will return something that I can re-use to rebuild an index. If yes, then I can do two things:

1. drop/recreate the index
2. create a new index, run indexWait, then drop the old index, and use indexRename
3. means that users will lose availability while the index is being rebuilt
4. is kind of fragile. People can close the web interface and not have their index renamed.

[mlucy]

#2797 should make it in.

I think we should do (2). If they close the web interface and re-open it, the issue will still be there, and when they hit resolve again it will rename the index (if it's done building).

[neumino]

Err, sorry I refered to #2797 as something like r.rebuild.

Is there a limit of index that can be built at the same time? Can I rebuild all of them at the same time?

[mlucy]

You can rebuild all of them at the same time, but I think that will slow down the cluster a lot. (@danielmewes, anything to say about that?)

[mlucy]

Rebuilding sequentially is probably better.

[danielmewes]

I agree, rebuilding one at a time is definitely safer.

[neumino]

How do people feel with this implementation?

An issue pops up if some old indexes are found.
A list of all indexes are displayed, and you have one button "rebuild" per index.
Users can rebuild just one index at a time[1].

If you want to recreate all the indexes (sequentially, at once), you need to run a Node/Python/Ruby script.

Sequentially building the indexes on the interface leads to two issues

• it's a non trivial amount of work[2]
• if people close the web interface, once the index being built is available, we won't start building the next one.

With a script, we can dynamically build a query that will sequentially rebuild all indexes.

[1] The reason why we should limit one index at a time is that if the OOM killer kill RethinkDB because too many indexes are being created, restarting RethinkDB will probably lead to another crash.

[2] Mostly because it requires "long" testing

The other solution, is to have the web ui pull data every few minutes and rebuild an index if none is being rebuilt.
If we go for this solution, I plan to use a special key like $reql_index$ to detect indexes being built.

[danielmewes]

I like that approach, because it doesn't do anything unexpected.
Users have full control about what's going on, and we don't get into the problem of having super long running background tasks going on with possibly no way to interrupt them.

On small setups, clicking a button for each index one at a time is very easy.
For large setups running a script sounds acceptable too.

[mlucy]

We should probably write this script for people.

[mlucy]

(Although, one could argue that if we're writing a script for common behavior we should support it out of the box. How hard would it be to add e.g. server support for recreating these indexes sequentially?)

[timmaxw]

RethinkDB currently doesn't have job control features. The advantage of a script is that we can use the OS's job control features. For example, the user can kill the script; and it can print messages describing its process. That's a much nicer UI than anything server-side that we could whip up in a week.

[Tryneus]

I wouldn't be too hard to include something in the python driver alongside the export/import/dump/restore scripts.

[mlucy]

We should do that and make it accessible through the rethinkdb binary the same way "rethinkdb dump" is.

[Tryneus]

I'll go ahead and do that, then.

[Tryneus]

Ok, the script is up in review 1925.

[neumino]

Talked a bit with @coffeemug

We'll just display the issue and a link to the appropriate docs.

The reason is mostly because

• the behavior is not that good - Users have to keep the web interface opened
• it's actually quite a lot of work - Handling the case of multiple web interface connected is pretty tough.

[neumino]

Up in review 1950 assigned to @mglukhovsky
Branch michel_2799_simple

[Tryneus]

The rethindb index-rebuild command and script has been approved and merged to next in commit ab184ee, and cherry-picked into v1.14.x in commit 497950e. Will be in release 1.14.0.

[coffeemug]

@neumino @mglukhovsky -- what's the status of this issue? It's the last one in the 1.14 branch.

[neumino]

I fixed a bug earlier today, and it's back in code review.
Ping @mglukhovsky

[neumino]

Merged in v1.14.x as 8b6bd59