Skip to content

fix(knowledge): distinguish duplicate files by type - #2384

Merged
lyingbug merged 2 commits into
Tencent:mainfrom
koishi514-Z:fix/2374-distinguish-file-types
Jul 29, 2026
Merged

fix(knowledge): distinguish duplicate files by type#2384
lyingbug merged 2 commits into
Tencent:mainfrom
koishi514-Z:fix/2374-distinguish-file-types

Conversation

@koishi514-Z

Copy link
Copy Markdown
Contributor

Description

Fixes duplicate-file detection for file uploads with identical content but different file types.

File deduplication now matches on type, file_hash, and file_type, allowing byte-identical files such as .md and .txt to coexist in the same knowledge base while continuing to reject duplicate uploads of the same file type.

Type of Change

  • 🐛 Bug fix
  • ✨ New feature
  • 💥 Breaking change
  • 📚 Documentation update
  • 🎨 Refactor
  • ⚡ Performance improvement
  • 🧪 Test
  • 🔧 Configuration / Build / CI

Related Issue

Fixes #2374

Testing

go test ./internal/application/repository -run '^TestCheckKnowledgeExists_FileHashIsScopedByFileType$' -count=1 -v

Verified:

  • Same hash with different file types (md and txt) is not treated as a duplicate.
  • Same hash with the same file type remains a duplicate.
  • git diff --check passes.

Checklist

  • git diff --check origin/main...HEAD passes
  • Changed source files are formatted
  • Targeted tests for the changed packages/components pass
  • Diff-scoped lint passes where applicable (for Go: golangci-lint run --new-from-rev=origin/main ./...)
  • Full-repository checks were run, or any unrelated/environment-dependent failures are documented above
  • Self-reviewed the code
  • Added/updated tests covering the change
  • Updated related documentation (README, docs/, Swagger annotations, etc.)
  • Breaking changes are clearly called out in the description above

Screenshots / Recordings

no user-visible UI changes.

koishi514-Z and others added 2 commits July 29, 2026 00:40
Use case-insensitive file_type matching, scope the filename fallback by type, and document the dedup contract for datasource sweep handling.
@lyingbug
lyingbug merged commit 5db186a into Tencent:main Jul 29, 2026
1 check passed
@koishi514-Z
koishi514-Z deleted the fix/2374-distinguish-file-types branch July 29, 2026 05:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: 同一知识库中内容相同的txt文件与md文件没办法共存

2 participants