Skip to content
This repository has been archived by the owner on Mar 2, 2024. It is now read-only.

duplicate keys #71

Closed
Severin-S opened this issue Dec 15, 2022 · 0 comments · Fixed by #74
Closed

duplicate keys #71

Severin-S opened this issue Dec 15, 2022 · 0 comments · Fixed by #74
Assignees
@sumnerevans

Comments

@Severin-S
Copy link

I just updated the bridge after #65 was merged and found a lot of errors in my logs, each followed by an exception. Everything works completely fine tho (as far as I tested).

$ docker logs mautrix-instagram 2>&1 | grep ERROR
[2022-12-15 18:14:15,514] [ERROR@mau.user.@severin:schueller.xyz] Failed to sync threads. Waiting 30 seconds before retrying sync.
[2022-12-15 18:15:04,356] [ERROR@mau.user.@severin:schueller.xyz] Failed to sync threads. Waiting 30 seconds before retrying sync.
[2022-12-15 18:15:58,411] [ERROR@mau.user.@severin:schueller.xyz] Failed to sync threads. Waiting 30 seconds before retrying sync.
[2022-12-15 18:16:58,861] [ERROR@mau.user.@severin:schueller.xyz] Failed to sync threads. Waiting 30 seconds before retrying sync.
[2022-12-15 18:18:02,422] [ERROR@mau.user.@severin:schueller.xyz] Failed to sync threads. Waiting 30 seconds before retrying sync.
[2022-12-15 18:18:32,425] [ERROR@mau.user.@severin:schueller.xyz] Failed to sync threads five times. Will not retry.
[2022-12-15 18:22:59,773] [ERROR@mau.user.@severin:schueller.xyz] Failed to backfill portal 340282366841710300949128127470726605830: 
[2022-12-15 18:23:00,906] [ERROR@mau.portal.ID1] Failed to store batch message IDs
[2022-12-15 18:23:01,159] [ERROR@mau.portal.ID1] Failed to store batch message IDs
[2022-12-15 18:23:01,433] [ERROR@mau.portal.ID1] Failed to store batch message IDs
[2022-12-15 18:23:01,707] [ERROR@mau.portal.ID1] Failed to store batch message IDs
[2022-12-15 18:23:01,958] [ERROR@mau.portal.ID1] Failed to store batch message IDs
[2022-12-15 18:23:02,185] [ERROR@mau.portal.ID1] Failed to store batch message IDs
[2022-12-15 18:23:02,430] [ERROR@mau.portal.ID1] Failed to store batch message IDs
[2022-12-15 18:23:02,729] [ERROR@mau.portal.ID1] Failed to store batch message IDs
[2022-12-15 18:23:02,989] [ERROR@mau.portal.ID1] Failed to store batch message IDs
[2022-12-15 18:23:23,203] [ERROR@mau.user.@severin:schueller.xyz] Failed to backfill portal ID1: duplicate key value violates unique constraint "message_pkey"
[2022-12-15 18:23:43,489] [ERROR@mau.portal.ID2] Failed to store batch message IDs
[2022-12-15 18:23:43,725] [ERROR@mau.portal.ID2] Failed to store batch message IDs
[2022-12-15 18:23:43,978] [ERROR@mau.portal.ID2] Failed to store batch message IDs
[2022-12-15 18:23:44,196] [ERROR@mau.portal.ID2] Failed to store batch message IDs
[2022-12-15 18:23:44,446] [ERROR@mau.portal.ID2] Failed to store batch message IDs
[2022-12-15 18:23:44,705] [ERROR@mau.portal.ID2] Failed to store batch message IDs
[2022-12-15 18:24:04,756] [ERROR@mau.user.@severin:schueller.xyz] Failed to backfill portal ID2: duplicate key value violates unique constraint "message_pkey"
[2022-12-15 18:24:41,685] [ERROR@mau.portal.ID3] Failed to store batch message IDs
[2022-12-15 18:24:42,267] [ERROR@mau.portal.ID3] Failed to store batch message IDs
[2022-12-15 18:24:42,993] [ERROR@mau.portal.ID3] Failed to store batch message IDs
[2022-12-15 18:24:43,855] [ERROR@mau.portal.ID3] Failed to store batch message IDs
[2022-12-15 18:25:03,906] [ERROR@mau.user.@severin:schueller.xyz] Failed to backfill portal ID3: duplicate key value violates unique constraint "message_pkey"
[2022-12-15 18:26:25,621] [ERROR@mau.user.@severin:schueller.xyz] Failed to backfill portal ID4: duplicate key value violates unique constraint "message_pkey"
[2022-12-15 18:28:10,755] [ERROR@mau.user.@severin:schueller.xyz] Failed to backfill portal ID4: duplicate key value violates unique constraint "message_pkey"

[2022-12-15 18:15:58,411] [ERROR@mau.user.@severin:schueller.xyz] Failed to sync threads. Waiting 30 seconds before retrying sync.
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/mautrix_instagram/user.py", line 823, in run_with_sync_lock
    await func()
  File "/usr/lib/python3.10/site-packages/mautrix_instagram/user.py", line 760, in _backfill_threads
    await self._sync_threads_with_delay(
  File "/usr/lib/python3.10/site-packages/mautrix_instagram/user.py", line 796, in _sync_threads_with_delay
    had_new_messages = await self._sync_thread(thread)
  File "/usr/lib/python3.10/site-packages/mautrix_instagram/user.py", line 612, in _sync_thread
    await portal.send_post_backfill_dummy(
  File "/usr/lib/python3.10/site-packages/mautrix_instagram/portal.py", line 2249, in send_post_backfill_dummy
    await DBMessage(
  File "/usr/lib/python3.10/site-packages/mautrix_instagram/db/message.py", line 49, in insert
    await self.db.execute(
  File "/usr/lib/python3.10/site-packages/mautrix/util/async_db/database.py", line 136, in execute
    return await conn.execute(query, *args, timeout=timeout)
  File "/usr/lib/python3.10/site-packages/mautrix/util/async_db/connection.py", line 38, in wrapper
    ret = await func(self, arg, *args, **kwargs)
  File "/usr/lib/python3.10/site-packages/mautrix/util/async_db/connection.py", line 81, in execute
    return await self.wrapped.execute(query, *args, timeout=timeout)
  File "/usr/lib/python3.10/site-packages/asyncpg/connection.py", line 319, in execute
    _, status, _ = await self._execute(
  File "/usr/lib/python3.10/site-packages/asyncpg/connection.py", line 1658, in _execute
    result, _ = await self.__execute(
  File "/usr/lib/python3.10/site-packages/asyncpg/connection.py", line 1683, in __execute
    return await self._do_execute(
  File "/usr/lib/python3.10/site-packages/asyncpg/connection.py", line 1730, in _do_execute
    result = await executor(stmt, None)
  File "asyncpg/protocol/protocol.pyx", line 201, in bind_execute
asyncpg.exceptions.UniqueViolationError: duplicate key value violates unique constraint "message_pkey"
DETAIL:  Key (item_id, receiver)=(, SHORT_ID) already exists.


[2022-12-15 18:23:44,196] [ERROR@mau.portal.ID2] Failed to store batch message IDs
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/mautrix_instagram/portal.py", line 2214, in _finish_batch
    await DBMessage.bulk_insert(messages)
  File "/usr/lib/python3.10/site-packages/mautrix_instagram/db/message.py", line 66, in bulk_insert
    await conn.copy_records_to_table("message", records=records, columns=columns)
  File "/usr/lib/python3.10/site-packages/mautrix/util/async_db/connection.py", line 38, in wrapper
    ret = await func(self, arg, *args, **kwargs)
  File "/usr/lib/python3.10/site-packages/mautrix/util/async_db/connection.py", line 151, in copy_records_to_table
    return await self.wrapped.copy_records_to_table(
  File "/usr/lib/python3.10/site-packages/asyncpg/connection.py", line 983, in copy_records_to_table
    return await self._protocol.copy_in(
  File "asyncpg/protocol/protocol.pyx", line 529, in copy_in
asyncpg.exceptions.UniqueViolationError: duplicate key value violates unique constraint "message_pkey"
DETAIL:  Key (item_id, receiver)=(ID5, SHORT_ID) already exists.

[2022-12-15 18:28:10,755] [ERROR@mau.user.@severin:schueller.xyz] Failed to backfill portal ID4: duplicate key value violates unique constraint "message_pkey"
DETAIL:  Key (item_id, receiver)=(, SHORT_ID) already exists.
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/mautrix_instagram/user.py", line 314, in _handle_backfill_requests_loop
    await portal.backfill(self, req)
  File "/usr/lib/python3.10/site-packages/mautrix_instagram/portal.py", line 1826, in backfill
    await self.send_post_backfill_dummy(last_message_ig_timestamp)
  File "/usr/lib/python3.10/site-packages/mautrix_instagram/portal.py", line 2249, in send_post_backfill_dummy
    await DBMessage(
  File "/usr/lib/python3.10/site-packages/mautrix_instagram/db/message.py", line 49, in insert
    await self.db.execute(
  File "/usr/lib/python3.10/site-packages/mautrix/util/async_db/database.py", line 136, in execute
    return await conn.execute(query, *args, timeout=timeout)
  File "/usr/lib/python3.10/site-packages/mautrix/util/async_db/connection.py", line 38, in wrapper
    ret = await func(self, arg, *args, **kwargs)
  File "/usr/lib/python3.10/site-packages/mautrix/util/async_db/connection.py", line 81, in execute
    return await self.wrapped.execute(query, *args, timeout=timeout)
  File "/usr/lib/python3.10/site-packages/asyncpg/connection.py", line 319, in execute
    _, status, _ = await self._execute(
  File "/usr/lib/python3.10/site-packages/asyncpg/connection.py", line 1658, in _execute
    result, _ = await self.__execute(
  File "/usr/lib/python3.10/site-packages/asyncpg/connection.py", line 1683, in __execute
    return await self._do_execute(
  File "/usr/lib/python3.10/site-packages/asyncpg/connection.py", line 1730, in _do_execute
    result = await executor(stmt, None)
  File "asyncpg/protocol/protocol.pyx", line 201, in bind_execute
asyncpg.exceptions.UniqueViolationError: duplicate key value violates unique constraint "message_pkey"
DETAIL:  Key (item_id, receiver)=(, SHORT_ID) already exists.

I replaced some IDs as I'm not sure wehter these are sensible information.
Bridge version: mautrix-instagram 0.2.3+dev.6556cbb7
Synapse version: 1.73.0
sumnerevans added a commit that referenced this issue Dec 15, 2022
@sumnerevans
backfill: fix inserting backfill dummy
7bba229 
Related to #71

Signed-off-by: Sumner Evans <sumner@beeper.com>
@sumnerevans sumnerevans self-assigned this Dec 15, 2022
@Severin-S Severin-S mentioned this issue Dec 16, 2022
Closed
@sumnerevans sumnerevans mentioned this issue Dec 16, 2022
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@sumnerevans sumnerevans

Labels
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

backfill: dedup when pulling more pages mautrix/instagram
2 participants
@sumnerevans @Severin-S