Cannot use projection when parallel replicas is enabled.

[RinChanNOWWW]

ClickHouse version: the latest codes from master.
I create a test shard with 2 replicas, and create a ReplicatedMergeTree with projection in this test cluster.

CREATE TABLE default.test_proj
(
    `event_date` Date,
    `user_id` UInt32,
    `value` UInt32,
    `category` String,
    PROJECTION p_category_sum
    (
        SELECT
            category,
            sum(value) AS total_value
        GROUP BY category
    )
)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/test_proj', '{replica}')
PARTITION BY toYYYYMM(event_date)
ORDER BY (event_date, user_id)
SETTINGS index_granularity = 8192;

INSERT INTO test_proj VALUES ('2024-06-01', 1, 100, 'A'), ('2024-06-01', 2, 200, 'B'), ('2024-06-02', 3, 150, 'A'),('2024-06-02', 4, 300, 'C');
INSERT INTO test_proj VALUES ('2024-06-01', 1, 100, 'D');
INSERT INTO test_proj VALUES ('2024-06-01', 1, 100, 'F');

I test several cases to use the projection:

-- 1. disable parallel replicas
EXPLAIN
SELECT
    category,
    sum(value) AS total_value
FROM test_proj
GROUP BY category
SETTINGS allow_experimental_parallel_reading_from_replicas = 0;
-- result:
   ┌─explain─────────────────────────────────────┐
1. │ Expression ((Projection + Before ORDER BY)) │
2. │   Aggregating                               │
3. │     Expression                              │
4. │       ReadFromMergeTree (p_category_sum)    │
   └─────────────────────────────────────────────┘
-- 2. enable parallel replicas and enable analyzer
EXPLAIN
SELECT
    category,
    sum(value) AS total_value
FROM test_proj
GROUP BY category
SETTINGS allow_experimental_parallel_reading_from_replicas = 1, enable_analyzer = 1
-- result:
   ┌─explain─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
1. │ Expression ((Project names + Projection))                                                                                                                                                                                           │
2. │   MergingAggregated                                                                                                                                                                                                                 │
3. │     Union                                                                                                                                                                                                                           │
4. │       Aggregating                                                                                                                                                                                                                   │
5. │         Expression ((Before GROUP BY + Change column names to column identifiers))                                                                                                                                                  │
6. │           ReadFromMergeTree (default.test_proj)                                                                                                                                                                                     │
7. │       ReadFromRemoteParallelReplicas (Query: SELECT `__table1`.`category` AS `category`, sum(`__table1`.`value`) AS `total_value` FROM `default`.`test_proj` AS `__table1` GROUP BY `__table1`.`category` Replicas: 127.0.0.1:9010) │
   └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
-- 3. enable parallel replicas and disable analyzer
EXPLAIN
SELECT
    category,
    sum(value) AS total_value
FROM test_proj
GROUP BY category
SETTINGS allow_experimental_parallel_reading_from_replicas = 1, enable_analyzer = 0, parallel_replicas_only_with_analyzer = 0
-- result:
   ┌─explain──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
1. │ Expression ((Projection + Before ORDER BY))                                                                                                                                          │
2. │   MergingAggregated                                                                                                                                                                  │
3. │     ReadFromRemoteParallelReplicas (Query: SELECT `category`, sum(`value`) AS `total_value` FROM `default`.`test_proj` GROUP BY `category` Replicas: 127.0.0.1:9000, 127.0.0.1:9010) │
   └──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Furthermore, I use LLDB to inspect the execution, I found that it cannot reach the line of the second breakpoint in the picture below:

It means the query cannot pass the line of the first breakpoint.

[RinChanNOWWW]

cc @amosbird @nickitat

[nickitat]

The only thing we could promise to fix at the moment is the documentation page.

[RinChanNOWWW]

I modified stateless 01710_force_use_projection to a parallel replicas version, and it cannot pass either.

create table tp (d1 Int32, d2 Int32, eventcnt Int64, projection p (select sum(eventcnt) group by d1)) engine = ReplicatedMergeTree('/clickhouse/tables/{shard}/tp', '{replica}') order by (d1, d2);

set optimize_use_projections = 1, force_optimize_projection = 1, enable_parallel_replicas = 1;

select sum(eventcnt) eventcnt, d1 from tp group by d1;

Result:

Received exception from server (version 25.7.1):
Code: 584. DB::Exception: Received from localhost:9000. DB::Exception: No projection is used when optimize_use_projections = 1 and force_optimize_projection = 1. (PROJECTION_NOT_USED)

[amosbird]

I'll look into fixing this, but first I need some time to understand how parallel replicas work.