-
Notifications
You must be signed in to change notification settings - Fork 311
Description
Bug Description
When a row's primary key column is updated in the database, the Materializer crashes with a KeyError because it tries to look up the new key (post-update) in an index that only contains the old key (pre-update).
Reproduction
Given a table where a column is part of the primary key (e.g., task_user_acl with PK (organization_id, subject_user_id, task_id)):
UPDATE task_user_acl SET task_id = 'new-task-id' WHERE task_id = 'old-task-id';This causes the Materializer to crash:
** (KeyError) key "\"public\".\"task_user_acl\"/\"org-092\"/\"user-0078\"/\"org-100-task-1109\"" not found in:
%{
"\"public\".\"task_user_acl\"/\"org-092\"/\"user-0078\"/\"org-092-task-0013\"" => "org-092-task-0013",
...
}
:erlang.map_get("...new_key...", %{"...old_key..." => ...})
(electric) lib/electric/shapes/consumer/materializer.ex:466: anonymous fn/3 in Electric.Shapes.Consumer.Materializer.apply_changes/2
Root Cause
In materializer.ex, the apply_changes/2 function handles UpdatedRecord by extracting only key (the new key after the update):
%Changes.UpdatedRecord{
key: key,
record: record,
...
},Then it uses key to look up the existing value:
old_value = Map.fetch!(index, key)But when a PK column changes, key != old_key. The index contains entries keyed by old_key, so the lookup fails.
Fix
The fix requires:
- Extract
old_keyfromUpdatedRecordin addition tokey - Handle the case where
old_keyis nil (PK didn't change) by defaulting tokey - Use
old_keywhen looking up/removing from the index - Use
old_keywhen removing from tag indices - Use
key(new key) when inserting into the index and adding to tag indices
%Changes.UpdatedRecord{
key: key,
+ old_key: old_key,
record: record,
move_tags: move_tags,
removed_move_tags: removed_move_tags
},
{{index, tag_indices}, counts_and_events} ->
+ old_key = old_key || key
...
tag_indices =
tag_indices
- |> remove_row_from_tag_indices(key, removed_move_tags)
+ |> remove_row_from_tag_indices(old_key, removed_move_tags)
|> add_row_to_tag_indices(key, move_tags)
if columns_present do
{value, original_string} = cast!(record, state)
- old_value = Map.fetch!(index, key)
+ {old_value, index} = Map.pop!(index, old_key)
index = Map.put(index, key, value)Context
This affects shapes that use subqueries with ACL tables where the ACL table has a composite primary key that includes a foreign key column. When that FK column is updated, the shape's materializer crashes.