Skip to content

feat: add pull_checks table to clickhouse#522

Merged
silent-cipher merged 51 commits into
mainfrom
feat/clickhouse-pull-check
May 25, 2026
Merged

feat: add pull_checks table to clickhouse#522
silent-cipher merged 51 commits into
mainfrom
feat/clickhouse-pull-check

Conversation

@silent-cipher
Copy link
Copy Markdown
Collaborator

@silent-cipher silent-cipher commented May 11, 2026

silent-cipher added 30 commits May 1, 2026 00:51
@silent-cipher
chore: upgrade synapse to latest
f0855e2
@silent-cipher
feat: initial pull flow testing job
a51bd33
@silent-cipher
fix: double pull check trigger
e08929e
@silent-cipher
chore: remove pull check from dev-tools
720e227
@silent-cipher
remove pull_checks table from db
a98e8e0
@silent-cipher
refactor: add piece validation and improve logging
a3c7aec
@silent-cipher
fix: logging
2875737
@silent-cipher
fix: prometheus metric
07d21b1
@silent-cipher
feat: add more prometheus metrics
9e57235
@silent-cipher
add tests
f9b0e8d
@silent-cipher
Merge branch 'main' into feat/pull-flow-testing
f23911a
@silent-cipher
add docs
079bc60
@silent-cipher
refactor: remove commit from pull check
5f4e729
@silent-cipher
fix tests after commit removal
e0a4cb7
@silent-cipher
chore: remove unused mock
5b5e2b4
@silent-cipher
fix: observe pull check completion latency once
2c8e57b
@silent-cipher
fix: metric help text
ded9267
@silent-cipher
fix: job calculations
4267cae
@silent-cipher
Merge branch 'main' into feat/pull-flow-testing
ee8a0fd
@silent-cipher
chore: revert back to waitForPullStatus
200dbe9
@silent-cipher
chore: fix lint
f02e4b4
@silent-cipher
feat: add deterministic random data generation and pull piece persist…
540f03d 
…ence
@silent-cipher
chore: cleanup
229bc7a
@silent-cipher
chore: cleanup
a0e13b8
@silent-cipher
docs: update to latest
4f6d2d0
@silent-cipher
chore: remove MAX_BYTES limit
c3d87a6
@silent-cipher
chore: address pr comments
23d5ee7
@silent-cipher
chore: fix lint plus docs
30f2379
@silent-cipher
doc: remove ttl leftovers
b395c98
@silent-cipher
refactor: worker doesn't expose /api/piece/:pieceCid
b9873cd
@BigLep BigLep requested review from SgtPooki and Copilot May 11, 2026 22:36
Copilot AI reviewed May 11, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

Comment thread docs/checks/pull-check.md
Comment thread apps/backend/src/clickhouse/clickhouse.schema.ts Outdated
Comment thread apps/backend/src/clickhouse/clickhouse.schema.ts
Comment thread apps/backend/src/pull-check/pull-check.service.spec.ts Outdated
silent-cipher and others added 3 commits May 12, 2026 11:19
@silent-cipher
feat: add expiresAt field to pull piece entity
65e8a3a
@silent-cipher
refactor: stream piece validation and add size checks
1ba5cd6
@SgtPooki
Merge branch 'main' into feat/pull-flow-testing
dfcefc8 
Resolves #519 (retention double-count) via auto-merge.

Also guards openPullPieceStream against serving expired registrations:
public /api/piece/:pieceCid endpoint now returns 404 when expires_at has
passed, regardless of when sweep runs.
SgtPooki
SgtPooki requested changes May 12, 2026
View reviewed changes
Comment thread apps/backend/src/pull-check/pull-check.service.spec.ts Outdated
Comment thread apps/backend/src/pull-check/pull-check.service.ts
Comment thread docs/checks/pull-check.md
Comment thread apps/backend/src/clickhouse/clickhouse.schema.ts Outdated
@github-project-automation github-project-automation Bot moved this from 🔎 Awaiting review to ⌨️ In Progress in FOC May 12, 2026
@silent-cipher silent-cipher requested a review from SgtPooki May 13, 2026 07:21
@BigLep BigLep moved this from ⌨️ In Progress to 🔎 Awaiting review in FOC May 13, 2026
@github-project-automation github-project-automation Bot moved this from 🔎 Awaiting review to ✔️ Approved by reviewer in FOC May 18, 2026
Base automatically changed from feat/pull-flow-testing to main May 19, 2026 19:24
@rjan90
Copy link
Copy Markdown
Contributor

rjan90 commented May 20, 2026

@silent-cipher looks like this needs to resolve the conflicts, but after that it seems good to merge?

@rjan90
Copy link
Copy Markdown
Contributor

rjan90 commented May 25, 2026

@silent-cipher can you iron out the remaining conflicts and then merge this?

@silent-cipher silent-cipher requested a review from Copilot May 25, 2026 15:49
Copilot AI reviewed May 25, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

@silent-cipher silent-cipher merged commit 4c1a9ca into main May 25, 2026
13 checks passed
@silent-cipher silent-cipher deleted the feat/clickhouse-pull-check branch May 25, 2026 15:52
@github-project-automation github-project-automation Bot moved this from ✔️ Approved by reviewer to 🎉 Done in FOC May 25, 2026
@FilOzzy FilOzzy mentioned this pull request May 25, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@SgtPooki SgtPooki SgtPooki approved these changes

+1 more reviewer

@iand iand iand approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@silent-cipher silent-cipher

Labels

None yet

Projects

FOC
Status: 🎉 Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@silent-cipher @rjan90 @iand @SgtPooki @BigLep @FilOzzy