Skip to content

Tpch500 poc#29042

Closed
kaka11chen wants to merge 9 commits intoapache:masterfrom
kaka11chen:tpch500-poc
Closed

Tpch500 poc#29042
kaka11chen wants to merge 9 commits intoapache:masterfrom
kaka11chen:tpch500-poc

Conversation

@kaka11chen
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@kaka11chen
Copy link
Contributor Author

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

_file_system(nullptr),
_io_ctx(io_ctx),
_enable_lazy_mat(enable_lazy_mat),
_enable_merge_small_io(true),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: member initializer for '_enable_merge_small_io' is redundant [modernize-use-default-member-init]

Suggested change
_enable_merge_small_io(true),
,

for (size_t i = 0; i < run_length; ++i) {
StringRef& slice = _dict_items[_indexes[dict_index++]];
char* buf_start = const_cast<char*>(slice.data);
uint32_t length = (uint32_t)slice.size;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: use auto when initializing with a cast to avoid duplicating the type name [modernize-use-auto]

Suggested change
uint32_t length = (uint32_t)slice.size;
auto length = (uint32_t)slice.size;


#include "delta_bit_pack_decoder.h"

#include <string.h>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: inclusion of deprecated C++ header 'string.h'; consider using 'cstring' instead [modernize-deprecated-headers]

Suggested change
#include <string.h>
#include <cstring>

}

template <typename T>
Status DeltaBitPackDecoder<T>::_get_internal(T* buffer, int num_values, int* out_num_values) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function '_get_internal' has cognitive complexity of 76 (threshold 50) [readability-function-cognitive-complexity]

Status DeltaBitPackDecoder<T>::_get_internal(T* buffer, int num_values, int* out_num_values) {
                               ^
Additional context

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:104: +1, including nesting penalty of 0, nesting level increased to 1

    if (num_values == 0) {
    ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:109: +1, including nesting penalty of 0, nesting level increased to 1

    while (i < num_values) {
    ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:110: +2, including nesting penalty of 1, nesting level increased to 2

        if (PREDICT_FALSE(_values_remaining_current_mini_block == 0)) {
        ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:111: +3, including nesting penalty of 2, nesting level increased to 3

            if (PREDICT_FALSE(!_block_initialized)) {
            ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:114: +4, including nesting penalty of 3, nesting level increased to 4

                if (i == num_values) {
                ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:122: +5, including nesting penalty of 4, nesting level increased to 5

                    if (_total_value_count != 1) {
                    ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:123: +6, including nesting penalty of 5, nesting level increased to 6

                        RETURN_IF_ERROR(_init_block());
                        ^

be/src/common/status.h:534: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:123: +7, including nesting penalty of 6, nesting level increased to 7

                        RETURN_IF_ERROR(_init_block());
                        ^

be/src/common/status.h:536: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:127: +4, including nesting penalty of 3, nesting level increased to 4

                RETURN_IF_ERROR(_init_block());
                ^

be/src/common/status.h:534: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:127: +5, including nesting penalty of 4, nesting level increased to 5

                RETURN_IF_ERROR(_init_block());
                ^

be/src/common/status.h:536: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:128: +1, nesting level increased to 3

            } else {
              ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:130: +4, including nesting penalty of 3, nesting level increased to 4

                if (_mini_block_idx < _mini_blocks_per_block) {
                ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:131: +5, including nesting penalty of 4, nesting level increased to 5

                    RETURN_IF_ERROR(_init_mini_block(_delta_bit_widths.data()[_mini_block_idx]));
                    ^

be/src/common/status.h:534: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:131: +6, including nesting penalty of 5, nesting level increased to 6

                    RETURN_IF_ERROR(_init_mini_block(_delta_bit_widths.data()[_mini_block_idx]));
                    ^

be/src/common/status.h:536: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:132: +1, nesting level increased to 4

                } else {
                  ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:133: +5, including nesting penalty of 4, nesting level increased to 5

                    RETURN_IF_ERROR(_init_block());
                    ^

be/src/common/status.h:534: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:133: +6, including nesting penalty of 5, nesting level increased to 6

                    RETURN_IF_ERROR(_init_block());
                    ^

be/src/common/status.h:536: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:140: +2, including nesting penalty of 1, nesting level increased to 2

        for (int j = 0; j < values_decode; ++j) {
        ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:141: +3, including nesting penalty of 2, nesting level increased to 3

            if (!_bit_reader->GetValue(_delta_bit_width, buffer + i + j)) {
            ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:145: +2, including nesting penalty of 1, nesting level increased to 2

        for (int j = 0; j < values_decode; ++j) {
        ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:157: +1, including nesting penalty of 0, nesting level increased to 1

    if (PREDICT_FALSE(_total_values_remaining == 0)) {
    ^

be/src/vec/exec/format/parquet/delta_bit_pack_decoder.cpp:158: +2, including nesting penalty of 1, nesting level increased to 2

        if (!_bit_reader->Advance(_delta_bit_width * _values_remaining_current_mini_block)) {
        ^

public:
FixLengthDictDecoder() : BaseDictDecoder() {};
FixLengthDictDecoder(tparquet::Type::type physical_type)
: BaseDictDecoder(), _physical_type(physical_type) {};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: initializer for base class 'std::doris::vectorized::BaseDictDecoder' is redundant [readability-redundant-member-init]

Suggested change
: BaseDictDecoder(), _physical_type(physical_type) {};
: , _physical_type(physical_type) {};


inline DateV2Value<DateV2ValueType> get_value_for_day(int day) const {
int index = day + DAY_BEFORE_EPOCH;
if (LIKELY(index >= 0 && index < DICT_DAYS)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: boolean expression can be simplified by DeMorgan's theorem [readability-simplify-boolean-expr]

        if (LIKELY(index >= 0 && index < DICT_DAYS)) {
            ^
Additional context

be/src/common/compiler_util.h:34: expanded from macro 'LIKELY'

#define LIKELY(expr) __builtin_expect(!!(expr), 1)
                                       ^

}
}

inline DateV2Value<DateV2ValueType> get_value_for_day7(int day) const {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: method 'get_value_for_day7' can be made static [readability-convert-member-functions-to-static]

Suggested change
inline DateV2Value<DateV2ValueType> get_value_for_day7(int day) const {
static inline DateV2Value<DateV2ValueType> get_value_for_day7(int day) {

}
}

inline DateV2Value<DateV2ValueType> get_value_for_day8(int day) const {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: method 'get_value_for_day8' can be made static [readability-convert-member-functions-to-static]

Suggested change
inline DateV2Value<DateV2ValueType> get_value_for_day8(int day) const {
static inline DateV2Value<DateV2ValueType> get_value_for_day8(int day) {

}
}

inline DateV2Value<DateV2ValueType> get_value_for_day2(int day) const {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: method 'get_value_for_day2' can be made static [readability-convert-member-functions-to-static]

Suggested change
inline DateV2Value<DateV2ValueType> get_value_for_day2(int day) const {
static inline DateV2Value<DateV2ValueType> get_value_for_day2(int day) {

return DATE_DAY_OFFSET_ITEMS[day + DAY_BEFORE_EPOCH];
}

inline DateV2Value<DateV2ValueType>& get_value_for_day3(int day) const {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: method 'get_value_for_day3' can be made static [readability-convert-member-functions-to-static]

Suggested change
inline DateV2Value<DateV2ValueType>& get_value_for_day3(int day) const {
static inline DateV2Value<DateV2ValueType>& get_value_for_day3(int day) {

@kaka11chen kaka11chen closed this Jan 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants