Incorrect count() with condition query result for SummingMergeTree.

[sarg]

Sample scenario:

create table tst (
  timestamp DateTime, val Nullable(Int8)
) engine SummingMergeTree partition by toYYYYMM(timestamp) ORDER by (timestamp);

insert into tst values
       ('2018-02-01 00:00:00', 1),
       ('2018-02-02 00:00:00', 2);

select count(1) from tst where timestamp is not null;
-- 2

select count(1) from tst final where timestamp is not null;
-- 0

select * from tst final;
-- 2018-02-01 00:00:00;1
-- 2018-02-02 00:00:00;2

Both FINAL and w/o FINAL count() queries should have the same result - 2.

[4ertus2]

It's a bug. It's look like 'is not null' is not working on not nullable column with MergeTree engine at all.

drop table if exists tst;
create table tst
(
  timestamp DateTime,
  val Nullable(Int8)
) engine MergeTree partition by toYYYYMM(timestamp) ORDER by (timestamp);

insert into tst values ('2018-02-01 00:00:00', 1), ('2018-02-02 00:00:00', 2);

select * from tst;
select count(1) from tst where val is not null;
select count(1) from tst where timestamp is not null;

drop table tst;

2018-02-01 00:00:00     1
2018-02-02 00:00:00     2
2
Received exception from server (version 18.14.2):
Code: 171. DB::Exception: Received from localhost:9000, 127.0.0.1. DB::Exception: Block structure mismatch in MergeTreeThread stream: different columns:
isNotNull(timestamp) UInt8 UInt8(size = 2)
isNotNull(timestamp) UInt8 Const(size = 0, UInt8(size = 1)).

[sarg]

Actually, IS NOT NULL isn't strictly necessary. Any condition will do fine, try where timestamp > '2017-01-01 00:00'

[filimonov]

Confirm.

Selects on same table:

select count() from tst final;
-- ok (2)

 select count() from tst final where timestamp>0;
-- wrong result (0)

 select count() from tst final prewhere timestamp > 0;
-- exeption: Not found column timestamp in block. There are only columns: 0.

select count() from tst final where val>0;
-- ok

select count() from tst final prewhere val>0;
-- segfault(!) 
2018.10.04 11:58:17.508285 [ 1 ] <Error> BaseDaemon: ########################################
2018.10.04 11:58:17.508704 [ 1 ] <Error> BaseDaemon: (from thread 12981) Received signal Segmentation fault (11).
2018.10.04 11:58:17.508729 [ 1 ] <Error> BaseDaemon: Address: NULL pointer.
2018.10.04 11:58:17.508743 [ 1 ] <Error> BaseDaemon: Access: write.
2018.10.04 11:58:17.508756 [ 1 ] <Error> BaseDaemon: Address not mapped to object.
2018.10.04 11:58:17.562872 [ 1 ] <Error> BaseDaemon: 0. clickhouse-server(DB::AggregateFunctionSum<unsigned char, unsigned char, DB::AggregateFunctionSumData<unsigned char> >::insertResultInto(char const*, DB::IColumn&) const+0x1c) [0x427fccc]
2018.10.04 11:58:17.562919 [ 1 ] <Error> BaseDaemon: 1. clickhouse-server(DB::SummingSortedBlockInputStream::insertCurrentRowIfNeeded(std::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&)+0xf1) [0x4e104c1]
2018.10.04 11:58:17.562947 [ 1 ] <Error> BaseDaemon: 2. clickhouse-server(DB::SummingSortedBlockInputStream::merge(std::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, std::priority_queue<DB::SortCursor, std::vector<DB::SortCursor, std::allocator<DB::SortCursor> >, std::less<DB::SortCursor> >&)+0x34d) [0x4e1333d]
2018.10.04 11:58:17.562960 [ 1 ] <Error> BaseDaemon: 3. clickhouse-server(DB::SummingSortedBlockInputStream::readImpl()+0x1c6) [0x4e13b96]
2018.10.04 11:58:17.562970 [ 1 ] <Error> BaseDaemon: 4. clickhouse-server(DB::IProfilingBlockInputStream::read()+0x1fd) [0x436f81d]
2018.10.04 11:58:17.562980 [ 1 ] <Error> BaseDaemon: 5. clickhouse-server(DB::ExpressionBlockInputStream::readImpl()+0x1b) [0x4dcf98b]
2018.10.04 11:58:17.562990 [ 1 ] <Error> BaseDaemon: 6. clickhouse-server(DB::IProfilingBlockInputStream::read()+0x1fd) [0x436f81d]
2018.10.04 11:58:17.563001 [ 1 ] <Error> BaseDaemon: 7. clickhouse-server(DB::ConvertColumnWithDictionaryToFullBlockInputStream::readImpl()+0x23) [0x4a93a33]
2018.10.04 11:58:17.563015 [ 1 ] <Error> BaseDaemon: 8. clickhouse-server(DB::IProfilingBlockInputStream::read()+0x1fd) [0x436f81d]
2018.10.04 11:58:17.563025 [ 1 ] <Error> BaseDaemon: 9. clickhouse-server(DB::ConcatBlockInputStream::readImpl()+0xb5) [0x4a941f5]
2018.10.04 11:58:17.563034 [ 1 ] <Error> BaseDaemon: 10. clickhouse-server(DB::IProfilingBlockInputStream::read()+0x1fd) [0x436f81d]
2018.10.04 11:58:17.563047 [ 1 ] <Error> BaseDaemon: 11. clickhouse-server(DB::Aggregator::execute(std::shared_ptr<DB::IBlockInputStream> const&, DB::AggregatedDataVariants&)+0x223) [0x4e477e3]
2018.10.04 11:58:17.563057 [ 1 ] <Error> BaseDaemon: 12. clickhouse-server(DB::AggregatingBlockInputStream::readImpl()+0x347) [0x4d9aad7]
2018.10.04 11:58:17.563066 [ 1 ] <Error> BaseDaemon: 13. clickhouse-server(DB::IProfilingBlockInputStream::read()+0x1fd) [0x436f81d]
2018.10.04 11:58:17.563076 [ 1 ] <Error> BaseDaemon: 14. clickhouse-server(DB::ExpressionBlockInputStream::readImpl()+0x1b) [0x4dcf98b]
2018.10.04 11:58:17.563085 [ 1 ] <Error> BaseDaemon: 15. clickhouse-server(DB::IProfilingBlockInputStream::read()+0x1fd) [0x436f81d]
2018.10.04 11:58:17.563095 [ 1 ] <Error> BaseDaemon: 16. clickhouse-server(DB::ExpressionBlockInputStream::readImpl()+0x1b) [0x4dcf98b]
2018.10.04 11:58:17.563104 [ 1 ] <Error> BaseDaemon: 17. clickhouse-server(DB::IProfilingBlockInputStream::read()+0x1fd) [0x436f81d]
2018.10.04 11:58:17.563113 [ 1 ] <Error> BaseDaemon: 18. clickhouse-server(DB::AsynchronousBlockInputStream::calculate()+0x45) [0x4365665]
2018.10.04 11:58:17.563121 [ 1 ] <Error> BaseDaemon: 19. clickhouse-server() [0x4365aca]
2018.10.04 11:58:17.563131 [ 1 ] <Error> BaseDaemon: 20. clickhouse-server(ThreadPool::worker()+0x167) [0x5264307]
2018.10.04 11:58:17.563139 [ 1 ] <Error> BaseDaemon: 21. clickhouse-server() [0x572b92f]
2018.10.04 11:58:17.563147 [ 1 ] <Error> BaseDaemon: 22. /lib64/libpthread.so.0(+0x7e25) [0x7f70b8d6fe25]