Skip to content

Update default limit for dedup function to 10000#589

Merged
yokofly merged 1 commit into
mainfrom
yokofly-patch-1
Mar 3, 2026
Merged

Update default limit for dedup function to 10000#589
yokofly merged 1 commit into
mainfrom
yokofly-patch-1

Conversation

@yokofly
Copy link
Copy Markdown
Contributor

@yokofly yokofly commented Mar 2, 2026

proton:

https://github.com/timeplus-io/proton/blob/b569d87ef64db5718911e37a025a133603aaf08f/src/Processors/Transforms/Streaming/DedupTransform.cpp#L187-L194

timeplusd:

timeplusd client version 3.1.3-rc.0.
Connecting to localhost:8463 as user default.
Connected to timeplusd server version 3.1.3 revision 54473.

timeplusd :) CREATE STREAM IF NOT EXISTS src_dedup_bug (id int, value string);
             CREATE STREAM IF NOT EXISTS tgt_dedup_bug (id int, value string);


timeplusd :) 
             
             
             CREATE MATERIALIZED VIEW mv_dedup_bug INTO tgt_dedup_bug
             AS SELECT id, value
             FROM dedup(src_dedup_bug, id, 3600s);


timeplusd :) INSERT INTO src_dedup_bug (id, value)
             SELECT number, 'batch4' FROM numbers(15000);

0 rows in set. Elapsed: 0.002 sec. Processed 15.00 thousand rows, 120.00 KB (7.76 million rows/s., 62.06 MB/s.)

timeplusd :) 
             SELECT count() as total_rows FROM table(tgt_dedup_bug);

SELECT
  count() AS total_rows
FROM
  table(tgt_dedup_bug)

Query id: ee418e13-966d-4636-afaa-353f5ac334ee

┌─total_rows─┐
│      15000 │
└────────────┘

1 row in set. Elapsed: 0.014 sec. 

timeplusd :) 
             INSERT INTO src_dedup_bug (id, value)
             SELECT number, 'batch3' FROM numbers(15000);

0 rows in set. Elapsed: 0.002 sec. Processed 15.00 thousand rows, 120.00 KB (7.75 million rows/s., 62.00 MB/s.)

timeplusd :) 
             SELECT count() as total_rows FROM table(tgt_dedup_bug);

SELECT
  count() AS total_rows
FROM
  table(tgt_dedup_bug)

Query id: f4544e9e-72f8-42ab-86d0-684787ccc57a

┌─total_rows─┐
│      20000 │
└────────────┘

1 row in set. Elapsed: 0.002 sec. 

timeplusd :) 


@netlify
Copy link
Copy Markdown

netlify Bot commented Mar 2, 2026

Deploy Preview for elastic-elion-a958b6 ready!

Name Link
🔨 Latest commit 342e023
🔍 Latest deploy log https://app.netlify.com/projects/elastic-elion-a958b6/deploys/69a50bfc9b83430008c9b2ed
😎 Deploy Preview https://deploy-preview-589--elastic-elion-a958b6.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@yokofly yokofly requested review from chenziliang and yl-lisen March 2, 2026 04:03
@yokofly yokofly merged commit c720c02 into main Mar 3, 2026
4 checks passed
@yokofly yokofly deleted the yokofly-patch-1 branch March 3, 2026 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants