Skip to content

[Bug] Union read policy will never take effects in block reader #41995

@Hastyshell

Description

@Hastyshell

Search before asking

  • I had searched in the issues and found no similar issues.

Version

Sine 2.0.

What's Wrong?

BlockReader::_rowsets_overlapping always return true even if non-overlap, resulting in union read optimization never takes effect.

bool BlockReader::_rowsets_overlapping(const ReaderParams& read_params) {
    std::string cur_max_key;
    const std::vector<RowSetSplits>& rs_splits = read_params.rs_splits;
    for (const auto& rs_split : rs_splits) {
        // version 0-1 of every tablet is empty, just skip this rowset
        if (rs_split.rs_reader->rowset()->version().second == 1) {
            continue;
        }
        if (rs_split.rs_reader->rowset()->num_rows() == 0) {
            continue;
        }
        if (rs_split.rs_reader->rowset()->is_segments_overlapping()) {
            return true;
        }
        std::string min_key;
        bool has_min_key = rs_split.rs_reader->rowset()->min_key(&min_key);
        if (!has_min_key) {
            return true;
        }
        if (min_key <= cur_max_key) {
            return true;
        }
        CHECK(rs_split.rs_reader->rowset()->max_key(&cur_max_key));
    }

    for (const auto& rs_reader : rs_splits) {
        // version 0-1 of every tablet is empty, just skip this rowset
        if (rs_reader.rs_reader->rowset()->version().second == 1) {
            continue;
        }
        if (rs_reader.rs_reader->rowset()->num_rows() == 0) {
            continue;
        }
        if (rs_reader.rs_reader->rowset()->is_segments_overlapping()) {
            return true;
        }
        std::string min_key;
        bool has_min_key = rs_reader.rs_reader->rowset()->min_key(&min_key);
        if (!has_min_key) {
            return true;
        }
        if (min_key <= cur_max_key) {
            return true;
        }
        CHECK(rs_reader.rs_reader->rowset()->max_key(&cur_max_key));
    }
    return false;
}

What You Expected?

Make it correct.

How to Reproduce?

Doing compaction without vertical policy, or query a dataset with progressive increasing keys.

Anything Else?

Not a correctness problem, may gain performance if make it correct (but not for vertical block reader).

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions