Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement index for buffer #24954

Merged
merged 3 commits into from
May 6, 2024
Merged

feat: Implement index for buffer #24954

merged 3 commits into from
May 6, 2024

Commits on May 2, 2024

  1. feat: Implement index for buffer

    This implements an index for the data in the table buffers. For now, by default, it indexes all tags, keeping a mapping of tag key/value pair to the row ids that it has in the buffer. When queries ask for record batches from the table buffer, the filter expression is evaluated to determine if a record batch can be built on the fly using only the row ids that match the index. If we don't have it in the index, the entire record batch from the buffer will be returned.
    
    This also updates the logic in segment state to only request a record batch with the projection. The query executor was updated so that it pushes the filter and projection down to the request to get table chunks.
    
    While implementing this, I believe I uncovered a bug where when limits are hit, a write still attempts to get buffered. I'll log a follow up to look at that.
    pauldix committed May 2, 2024
    Configuration menu
    Copy the full SHA
    bd5b999 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. Configuration menu
    Copy the full SHA
    8bb7760 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    25a1311 View commit details
    Browse the repository at this point in the history