Views don't search on all indexes of a stream #6828
Comments
|
This sounds like the index ranges aren't up to date for that index set. |
|
It does, sadly. Recalculating didn't do anything. |
|
I can confirm this for
|
dennisoelkers
added a commit
that referenced
this issue
Dec 9, 2019
When rotating an index, the following details are calculated and stored in an index range: * the smallest timestamp (oldest document) * the largest timestamp (youngest document) * the set of stream ids contained in documents in this index For the latter, a terms query is used. Unfortunately, no specific size was set for it, so it used the default size of 10. This meant that all index ranges calculated before did not contain more than 10 stream ids. In the current search, this did not lead to imminent problems. The logic used to determine the indices which are used for the search was lenient enough to only consider either the set of stream ids containing the current stream id or the index being managed by the index set of the stream. The new search is stricter and expects the set of stream ids to contain the current stream being searched for. It only checks if the index set of the stream manages the index if it is the deflector. This change is now calculating the stream id set for a give index correctly, by specifying a `size` parameter for the terms aggregation that is hopefully high enough (`Integer.MAX_VALUE`) to handle all cases. Fixes #6828.
10 tasks
bernd
pushed a commit
that referenced
this issue
Dec 10, 2019
…x. (#6957) When rotating an index, the following details are calculated and stored in an index range: * the smallest timestamp (oldest document) * the largest timestamp (youngest document) * the set of stream ids contained in documents in this index For the latter, a terms query is used. Unfortunately, no specific size was set for it, so it used the default size of 10. This meant that all index ranges calculated before did not contain more than 10 stream ids. In the current search, this did not lead to imminent problems. The logic used to determine the indices which are used for the search was lenient enough to only consider either the set of stream ids containing the current stream id or the index being managed by the index set of the stream. The new search is stricter and expects the set of stream ids to contain the current stream being searched for. It only checks if the index set of the stream manages the index if it is the deflector. This change is now calculating the stream id set for a give index correctly, by specifying a `size` parameter for the terms aggregation that is hopefully high enough (`Integer.MAX_VALUE`) to handle all cases. Fixes #6828.
bernd
pushed a commit
that referenced
this issue
Dec 10, 2019
…x. (#6957) When rotating an index, the following details are calculated and stored in an index range: * the smallest timestamp (oldest document) * the largest timestamp (youngest document) * the set of stream ids contained in documents in this index For the latter, a terms query is used. Unfortunately, no specific size was set for it, so it used the default size of 10. This meant that all index ranges calculated before did not contain more than 10 stream ids. In the current search, this did not lead to imminent problems. The logic used to determine the indices which are used for the search was lenient enough to only consider either the set of stream ids containing the current stream id or the index being managed by the index set of the stream. The new search is stricter and expects the set of stream ids to contain the current stream being searched for. It only checks if the index set of the stream manages the index if it is the deflector. This change is now calculating the stream id set for a give index correctly, by specifying a `size` parameter for the terms aggregation that is hopefully high enough (`Integer.MAX_VALUE`) to handle all cases. Fixes #6828. (cherry picked from commit 8e7560b)
|
@dploeger @l0calhost Thank you for the report and information. A fix for this will be included in the upcoming 3.1.4 bug fix release. Please make sure to read the release notes because it requires a manual step to recalculate the index ranges. (there was a bug in calculating them) |
bernd
added a commit
that referenced
this issue
Dec 10, 2019
…x. (#6957) (#6967) When rotating an index, the following details are calculated and stored in an index range: * the smallest timestamp (oldest document) * the largest timestamp (youngest document) * the set of stream ids contained in documents in this index For the latter, a terms query is used. Unfortunately, no specific size was set for it, so it used the default size of 10. This meant that all index ranges calculated before did not contain more than 10 stream ids. In the current search, this did not lead to imminent problems. The logic used to determine the indices which are used for the search was lenient enough to only consider either the set of stream ids containing the current stream id or the index being managed by the index set of the stream. The new search is stricter and expects the set of stream ids to contain the current stream being searched for. It only checks if the index set of the stream manages the index if it is the deflector. This change is now calculating the stream id set for a give index correctly, by specifying a `size` parameter for the terms aggregation that is hopefully high enough (`Integer.MAX_VALUE`) to handle all cases. Fixes #6828. (cherry picked from commit 8e7560b)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When using views to display a graph and selecting a specific stream, the graph seems to stop at the point in time, where the stream's index was rotated.
Expected Behavior
A graph in the view should display all data across all indices of a selected stream.
Current Behavior
The graph only displays data from where the rotation was done.
When no stream is selected, the complete graph is shown
Steps to Reproduce (for bugs)
Your Environment
The text was updated successfully, but these errors were encountered: