Batched per-db _index_info endpoint - #6103
Open
nickva wants to merge 1 commit into
Open
Conversation
Implement a new `/{db}/_index_info` endpoint to return the status of all the
indexes in the database in a single endpoint. It matches the general pattern of
`_dbs_infos` where we return the db info for all databases in a cluster. Just
like with `_db_infos`, where it doesn't return anything new just a better and
more optimized summary of all dbs, the new endpoint returns all the index
status in a single easy to use endpoint. User could, for example, fetch all the
design docs, parse them, and with an if/else check call a per-index specific
`_info` endpoint such as `/{db}/_design/{ddoc}/_info`,
`/{db}/_design/{ddoc}/_nouveau_info/{index}` etc. to get all this information,
but that's not very ergonomic and it's slower.
One of the main uses of this endpoint is to find out when all the indexes in
the db have been built. Each index returns an `updates_pending` object with the
pending bounds across all copies of each range. With a minimum (the best copy),
maximum (the worst copy) and a count of how many live copies returned a valid
result. This information is more accurate than what we might get with the view
group `_info` calls. It turns out for view groups `_info` we broke the
minimum/preferred/total `updates_pending` stats for view groups. That happened
when we switched the collection to work as a ring (the COUCHDB-3053 dedup fix),
so then only the first responding copy of each range is aggregated, total =
minimum and prefered only counts the responding shards. The new endpoint on the
other hand, waits and aggregates stats from all copies of the index.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implement a new
/{db}/_index_infoendpoint to return the status of all the indexes in the database in a single endpoint. It matches the general pattern of_dbs_infoswhere we return the db info for all databases in a cluster. Just like with_db_infos, where it doesn't return anything new just a better and more optimized summary of all dbs, the new endpoint returns all the index status in a single easy to use endpoint. User could, for example, fetch all the design docs, parse them, and with an if/else check call a per-index specific_infoendpoint such as/{db}/_design/{ddoc}/_info,/{db}/_design/{ddoc}/_nouveau_info/{index}etc. to get all this information, but that's not very ergonomic and it's slower.One of the main uses of this endpoint is to find out when all the indexes in the db have been built. Each index returns an
updates_pendingobject with the pending bounds across all copies of each range. With a minimum (the best copy), maximum (the worst copy) and a count of how many live copies returned a valid result. This information is more accurate than what we might get with the view group_infocalls. It turns out for view groups_infowe broke the minimum/preferred/totalupdates_pendingstats for view groups. That happened when we switched the collection to work as a ring (the COUCHDB-3053 dedup fix), so then only the first responding copy of each range is aggregated, total = minimum and preferred only counts the responding shards. The new endpoint on the other hand, waits and aggregates stats from all copies of the index.