Skip to content

fix: range copy, DM member assignment, SQLite variable limit#665

Merged
rusq merged 3 commits intorusq:masterfrom
Fizmatik:fix/migration-bugs
Apr 17, 2026
Merged

fix: range copy, DM member assignment, SQLite variable limit#665
rusq merged 3 commits intorusq:masterfrom
Fizmatik:fix/migration-bugs

Conversation

@Fizmatik
Copy link
Copy Markdown

Summary

Three bugs found during a real Slack-to-Mattermost migration (31 users, 5M posts, 76K files):

  • source.go: range-value copy bugChannels() iterates with for _, c := range chns, so c.Members = users assigns to a copy of the struct. The original slice element is never updated, leaving all channels with empty Members. Fixed with index-based iteration.

  • index.go: wrong DM memberconvertToDM case 1 (single-member DMs) uses me (the archiving user) instead of ch.Members[0]. This makes every single-member DM appear as a conversation with the current user, rather than the actual participant. Fixed by using the actual member.

  • dedupe.go: SQLite variable limitdeleteChunksByID passes all chunk IDs (147K+ in large workspaces) as bind parameters in a single DELETE ... WHERE ID IN (?) query, exceeding SQLite's SQLITE_MAX_VARIABLE_NUMBER. Fixed with batched deletes (10,000 IDs per query).

Test plan

  • Added regression test for DM case 1 where ch.Members[0] != me
  • Existing tests updated and passing
  • go test ./internal/structures/ ./internal/chunk/backend/dbase/ ./internal/chunk/backend/dbase/repository/ — all pass

Yuriy Tolpygo and others added 2 commits April 15, 2026 21:49
Three bugs found during a real Slack-to-Mattermost migration
(31 users, 5M posts, 76K files):

1. source.go: Channels() uses range-value loop, so assigning Members
   to the loop variable modifies a copy — the original slice element
   is never updated. All channels end up with empty Members.
   Fix: use index-based iteration.

2. index.go: convertToDM case 1 uses `me` (the current workspace user)
   instead of the actual channel member. This makes every single-member
   DM appear as a conversation with the archiving user.
   Fix: use ch.Members[0] instead of me.

3. dedupe.go: deleteChunksByID passes all chunk IDs (147K+ in large
   workspaces) as bind parameters in a single DELETE query, exceeding
   SQLite's SQLITE_MAX_VARIABLE_NUMBER limit.
   Fix: batch deletes in groups of 10,000.
Copy link
Copy Markdown
Owner

@rusq rusq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

return nil, err
}
c.Members = users
chns[i].Members = users
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good catch of the slop! :)

@rusq rusq merged commit bce4e68 into rusq:master Apr 17, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants