Skip to content

Handle Telegram forward-to-bot provenance#12

Merged
kossoy merged 2 commits into
mainfrom
feat/folio-4-6-telegram-forward-to-bot-connector
Jun 25, 2026
Merged

Handle Telegram forward-to-bot provenance#12
kossoy merged 2 commits into
mainfrom
feat/folio-4-6-telegram-forward-to-bot-connector

Conversation

@kossoy

@kossoy kossoy commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Refs #6

What changed

  • Map Telegram forwarded media provenance from forward_origin and legacy forward_from_chat into DiscoveredMedia source metadata.
  • Preserve existing Telegram downloaded-item compatibility by storing dedupe keys as <source_chat>:<source_message_id>:<file_unique_id> or <bot_chat>:<message_id>:<file_unique_id> while keeping Source.ExternalID as the source/bot message identity.
  • Parse forward_origin.sender_chat for MessageOriginChat forwards so forwarded group/chat posts keep their original collection provenance.
  • Keep media download resolution by file_id and fixture-backed tests without live credentials.

Verification

git fetch origin
# no output
git rebase origin/main
Current branch feat/folio-4-6-telegram-forward-to-bot-connector is up to date.
./scripts/product-verifier.sh
npm warn Unknown builtin config "globalignorefile". This will stop working in the next major version of npm. See `npm help npmrc` for supported config options.
npm warn deprecated glob@10.5.0: Old versions of glob are not supported, and contain widely publicized security vulnerabilities, which have been fixed in the current version. Please update. Support for old versions may be purchased (at exorbitant rates) by contacting i@izs.me

added 313 packages, and audited 314 packages in 3s

76 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
npm warn allow-scripts 1 package has install scripts not yet covered by allowScripts:
npm warn allow-scripts   esbuild@0.28.1 (postinstall: node install.js)
npm warn allow-scripts
npm warn allow-scripts Run `npm approve-scripts --allow-scripts-pending` to review, or `npm approve-scripts <pkg>` to allow.
npm warn Unknown builtin config "globalignorefile". This will stop working in the next major version of npm. See `npm help npmrc` for supported config options.

> dashboard@0.0.0 build
> tsc -b && vite build

vite v7.3.6 building client environment for production...
transforming...
✓ 906 modules transformed.
rendering chunks...
computing gzip size...
dist/index.html                   0.68 kB │ gzip:  0.35 kB
dist/assets/index-mnmuewfG.css   21.50 kB │ gzip:  4.68 kB
dist/assets/query-Dk8W7wRr.js    34.83 kB │ gzip: 10.31 kB
dist/assets/index-Krc35nC_.js    73.31 kB │ gzip: 14.09 kB
dist/assets/react-DD2qiXMp.js   229.91 kB │ gzip: 73.54 kB
dist/assets/charts-CZ01G27q.js  335.22 kB │ gzip: 99.81 kB
✓ built in 3.80s
# github.com/mattn/go-sqlite3
sqlite3-binding.c: In function 'sqlite3ShadowTableName':
sqlite3-binding.c:123133:9: warning: assignment discards 'const' qualifier from pointer target type [-Wdiscarded-qualifiers]
123133 |   zTail = strrchr(zName, '_');
       |         ^
?    ok-folio/cmd/extractor [no test files]
?    ok-folio/dashboard/node_modules/flatted/golang/pkg/flatted [no test files]
ok   ok-folio/internal/api (cached)
ok   ok-folio/internal/config (cached)
?    ok-folio/internal/dashboard [no test files]
ok   ok-folio/internal/database (cached)
?    ok-folio/internal/exif [no test files]
ok   ok-folio/internal/gallery (cached)
?    ok-folio/internal/photoprism [no test files]
?    ok-folio/internal/provider [no test files]
ok   ok-folio/internal/provider/telegram (cached)
ok   ok-folio/internal/provider/webgallery (cached)
?    ok-folio/internal/scheduler [no test files]
ok   ok-folio/internal/scraper (cached)
ok   ok-folio/internal/testguard (cached)
ok   ok-folio/pkg/retry (cached)

Maestro-Backend: codex (0-9m); codex (9m-end, in_place_respawn)

Greptile Summary

This PR adds Telegram forwarded-media provenance handling. The main changes are:

  • Maps forward_origin metadata into discovered media source fields.
  • Supports legacy forward_from_chat and forward_from_message_id provenance.
  • Uses source chat and source message ids for Telegram dedupe keys when available.
  • Keeps media download resolution based on Telegram file_id.
  • Expands fixture-backed tests for forwarded Telegram media.

Confidence Score: 4/5

Mostly safe to merge after fixing the legacy Telegram forward provenance handling so channel/group source metadata is preserved when the original message id is absent.

The change is narrow and fixture-backed, and the remaining issue is isolated to legacy Telegram forward metadata handling rather than the broader media download path.

internal/provider/telegram/types.go

T-Rex T-Rex Logs

What T-Rex did

  • Ran a focused Telegram provider Go test with an httptest getUpdates response containing forward_from_chat and no forward_from_message_id, and the endpoint returned HTTP 200 OK; the test showed that origin collection_id and collection_name were mapped to the bot chat instead of the forwarded origin channel.
  • Compared pre- and post-test data to verify how IDs and collections map across channel-origin, legacy forward_from_chat, and no-message-id fallbacks.
  • Validated how prefixes are emitted for forward-origin and legacy-forward cases, and confirmed stable media-id suffixes are preserved across scenarios.
  • Verified ResolveMedia behavior by inspecting how test runs resolve the media fixture URL, demonstrating independence from dedupe/source values.

View all artifacts

T-Rex Ran code and verified through T-Rex

Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
internal/provider/telegram/types.go:117-126
**Preserve legacy chat provenance**
When Telegram sends `forward_from_chat` without `forward_from_message_id`, `SourceRef` builds the origin chat ref but never returns it, so the code falls through to the bot message as the source. The legacy message id is optional, so these forwards lose the source channel or group metadata instead of using the origin collection with bot-message dedupe fallback.

Reviews (2): Last reviewed commit: "Fix Telegram forward provenance dedupe" | Re-trigger Greptile

Greptile also left 1 inline comment on this PR.

Context used:

  • Context used - Review OK Folio pull requests as a merge gate. Pri... (source)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d292c78ff6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/provider/telegram/connector.go Outdated
DedupeKey: provider.DedupeKey{
ProviderID: ProviderID,
Value: fmt.Sprintf("%s:%s", externalID, ref.StableID()),
Value: externalID,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve Telegram dedupe compatibility

For installations that have already scraped Telegram media, DownloadedPhoto.URL is stored from stableDedupeKey, and before this change Telegram keys included the media stable ID (<chat>:<message>:<file_unique_id>). This line now emits only <chat>:<message>, while duplicate detection only checks the exact new key for Telegram, so previously kept Telegram items will be treated as new and re-downloaded/re-recorded after upgrade. Keep the old key shape or add a Telegram legacy lookup/migration before changing it.

Useful? React with 👍 / 👎.

Type string `json:"type"`
SenderUser *User `json:"sender_user"`
SenderUserName string `json:"sender_user_name"`
Chat *Chat `json:"chat"`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Parse MessageOriginChat sender_chat

For forwards where Telegram reports forward_origin.type == "chat" (messages originally sent on behalf of a group/chat), the origin payload uses sender_chat, not chat. With only this field mapped, o.Chat stays nil, ForwardOrigin.SourceRef returns false, and the later fallback records the bot/private chat instead of the original collection, so those forwarded media lose their source provenance. Add a sender_chat path before falling back.

Useful? React with 👍 / 👎.

Comment on lines +117 to +126
if ref.MessageID > 0 {
return ref
}
if m.ForwardSenderName != "" {
ref.ChatName = m.ForwardSenderName
}
if m.ForwardSignature != "" {
ref.ChatName = m.ForwardSignature
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Preserve legacy chat provenance
When Telegram sends forward_from_chat without forward_from_message_id, SourceRef builds the origin chat ref but never returns it, so the code falls through to the bot message as the source. The legacy message id is optional, so these forwards lose the source channel or group metadata instead of using the origin collection with bot-message dedupe fallback.

Context Used: Review OK Folio pull requests as a merge gate. Pri... (source)

Artifacts

Repro: focused Go test for legacy forward_from_chat without forward_from_message_id

  • Contains supporting evidence from the run (text/x-go; charset=utf-8).

Repro: verbose go test output showing HTTP 200 OK and dropped origin collection metadata

  • Keeps the command output available without making the summary code-heavy.

View artifacts

T-Rex Ran code and verified through T-Rex

Prompt To Fix With AI
This is a comment left during a code review.
Path: internal/provider/telegram/types.go
Line: 117-126

Comment:
**Preserve legacy chat provenance**
When Telegram sends `forward_from_chat` without `forward_from_message_id`, `SourceRef` builds the origin chat ref but never returns it, so the code falls through to the bot message as the source. The legacy message id is optional, so these forwards lose the source channel or group metadata instead of using the origin collection with bot-message dedupe fallback.

**Context Used:** Review OK Folio pull requests as a merge gate. Pri... ([source](greptile.json))

How can I resolve this? If you propose a fix, please make it concise.

@kossoy kossoy merged commit 33fbd75 into main Jun 25, 2026
2 checks passed
@kossoy kossoy deleted the feat/folio-4-6-telegram-forward-to-bot-connector branch June 25, 2026 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant