Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

synapse.api.errors.StoreError: 404: No row found in background process http_pusher.on_new_receipts since upgrade to v1.70.0 #14300

Closed
jae1911 opened this issue Oct 26, 2022 · 16 comments
Labels
A-Database DB stuff like queries, migrations, new/remove columns, indexes, unexpected entries in the db O-Uncommon Most users are unlikely to come across this or unexpected workflow S-Major Major functionality / product severely impaired, no satisfactory workaround. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues.

Comments

@jae1911
Copy link
Contributor

jae1911 commented Oct 26, 2022

Description

I am getting a synapse.api.errors.StoreError: 404: No row found in the Synapse log since the update to v1.70.0 (see details down there).

It renders clients unable to sync.

Steps to reproduce

  1. Upgrade server to v1.70.0
  2. Open Element
  3. Observe in console error
/sync error M_UNKNOWN: MatrixError: [404] No row found (https://jae.fi/_matrix/client/r0/sync?filter=10&timeout=0&since=s3488824_42934899_20_4245165_737345_762_433303_2724564_0)
e.<computed> @ rageshake.ts:73
(anonymous) @ logger.ts:50
onSyncError @ sync.ts:978
doSync @ sync.ts:821
  1. Observe in Synapse log (see down)

Homeserver

jae.fi

Synapse Version

1.70.0

Installation Method

Docker (matrixdotorg/synapse)

Platform

Docker on Debian 10

Relevant log output

synapse_1           | 2022-10-26 11:52:39,189 - synapse.metrics.background_process_metrics - 242 - ERROR - http_pusher.on_new_receipts-5 - Background process 'http_pusher.on_new_receipts' threw an exception
synapse_1           | Traceback (most recent call last):
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/metrics/background_process_metrics.py", line 240, in run
synapse_1           |     return await func(*args, **kwargs)
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/push/httppusher.py", line 168, in _update_badge
synapse_1           |     badge = await push_tools.get_badge_count(
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/push/push_tools.py", line 39, in get_badge_count
synapse_1           |     await concurrently_execute(get_room_unread_count, joins, 10)
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/util/async_helpers.py", line 235, in concurrently_execute
synapse_1           |     await yieldable_gather_results(
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/util/async_helpers.py", line 291, in yieldable_gather_results
synapse_1           |     raise dfe.subFailure.value from None
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1692, in _inlineCallbacks
synapse_1           |     result = context.run(
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator
synapse_1           |     return g.throw(self.type, self.value, self.tb)
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/util/async_helpers.py", line 227, in _concurrently_execute_inner
synapse_1           |     await maybe_awaitable(func(value))
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/push/push_tools.py", line 33, in get_room_unread_count
synapse_1           |     await store.get_unread_event_push_actions_by_room_for_user(
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1692, in _inlineCallbacks
synapse_1           |     result = context.run(
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator
synapse_1           |     return g.throw(self.type, self.value, self.tb)
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/event_push_actions.py", line 488, in get_unread_event_push_actions_by_room_for_user
synapse_1           |     return await self.db_pool.runInteraction(
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 881, in runInteraction
synapse_1           |     return await delay_cancellation(_runInteraction())
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1692, in _inlineCallbacks
synapse_1           |     result = context.run(
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator
synapse_1           |     return g.throw(self.type, self.value, self.tb)
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 848, in _runInteraction
synapse_1           |     result = await self.runWithConnection(
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 976, in runWithConnection
synapse_1           |     return await make_deferred_yieldable(
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/twisted/python/threadpool.py", line 244, in inContext
synapse_1           |     result = inContext.theWork()  # type: ignore[attr-defined]
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/twisted/python/threadpool.py", line 260, in <lambda>
synapse_1           |     inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/twisted/python/context.py", line 117, in callWithContext
synapse_1           |     return self.currentContext().callWithContext(ctx, func, *args, **kw)
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/twisted/python/context.py", line 82, in callWithContext
synapse_1           |     return func(*args, **kw)
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/twisted/enterprise/adbapi.py", line 282, in _runWithConnection
synapse_1           |     result = func(conn, *args, **kw)
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 969, in inner_func
synapse_1           |     return func(db_conn, *args, **kwargs)
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 710, in new_transaction
synapse_1           |     r = func(cursor, *args, **kwargs)
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/event_push_actions.py", line 523, in _get_unread_counts_by_receipt_txn
synapse_1           |     stream_ordering = self.get_stream_id_for_event_txn(txn, event_id)
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/databases/main/stream.py", line 847, in get_stream_id_for_event_txn
synapse_1           |     return self.db_pool.simple_select_one_onecol_txn(  # type: ignore[call-overload]
synapse_1           |   File "/usr/local/lib/python3.9/site-packages/synapse/storage/database.py", line 1718, in simple_select_one_onecol_txn
synapse_1           |     raise StoreError(404, "No row found")
synapse_1           | synapse.api.errors.StoreError: 404: No row found

Anything else that would be useful to know?

Reverting to previous stable release fixes the problem.

enable_metrics is set to false

@reivilibre reivilibre added the X-Regression Something broke which worked on a previous release label Oct 26, 2022
@reivilibre
Copy link
Contributor

reivilibre commented Oct 26, 2022

I'll look into it. Do you use message retention / purging or otherwise delete old events?

(Vague overview: local_current_membership is pointing to an event_id that does not appear in the events table. v1.70.0's event push actions code now seems to rely on it existing. Unclear why this row would have gone missing; seems unlikely that message retention should remove such important events but can't rule it out.)

@jae1911
Copy link
Contributor Author

jae1911 commented Oct 26, 2022

Yes, I have some rooms that have retention setup.

The config of the server itself looks like this:

redaction_retention_period: 30s
user_ips_max_age: 1m

retention:
  enabled: true
  min_lifetime: 1m
  max_lifetime: 1y
  purge_jobs:
    - longest_max_lifetime: 3d
      interval: 12h
    - shortest_max_lifetime: 3d
      interval: 1d

@reivilibre
Copy link
Contributor

Message retention/purging isn't meant to touch state events (see

should_delete_expr = "state_events.state_key IS NULL"
), so I'm fairly confused as to why someone's join event could have gone missing.

That said, looking at local_current_membership, I spot this comment:

# We create a new table called `local_current_membership` that stores the latest
# membership state of local users in rooms, which helps track leaves/bans/etc
# even if the server has left the room (and so has deleted the room from
# `current_state_events`). This will also include outstanding invites for local
# users for rooms the server isn't in.

so it sounds like it may be intended to track events that don't exist anymore...(?)

Would you mind trying to find the room in question with these missing events and, to your knowledge, seeing if you know anything that happened to the room that may explain why it's unusual in some way? (e.g. maybe you left the room, maybe it's a pending invite, ..?)

For one of the users that is failing to sync, please try running this:

SELECT room_id, event_id, membership FROM local_current_membership LEFT JOIN events USING (event_id) WHERE user_id = '@someone:jae.fi' AND events.type IS NULL;

(What do you see in membership — any non-joins? Know anything special about the rooms?)

Do any of those rooms not exist anymore?

SELECT room_id, event_id, membership FROM local_current_membership LEFT JOIN rooms USING (room_id) WHERE user_id = '@someone:jae.fi' AND rooms.room_version IS NULL;

@jae1911
Copy link
Contributor Author

jae1911 commented Oct 26, 2022

Tested on my own user.

For the first query:

            room_id             |                   event_id                   | membership
--------------------------------+----------------------------------------------+------------
 !ehXvUhWNASUkSLvAGP:matrix.org | $oPMANIq9VuQBnbN6dt4FJC6hadCcrxKiDZDpu20nPJo | join
(1 row)

Room ID resolves to the Synapse Admin room that I left a looong time ago and ended up blocking and unblocking at the time of spam attacks.

Second request yields the same result:

            room_id             |                   event_id                   | membership
--------------------------------+----------------------------------------------+------------
 !ehXvUhWNASUkSLvAGP:matrix.org | $oPMANIq9VuQBnbN6dt4FJC6hadCcrxKiDZDpu20nPJo | join
(1 row)

@reivilibre
Copy link
Contributor

Thanks! That's interesting to know.

@reivilibre
Copy link
Contributor

reivilibre commented Oct 27, 2022

Incidentally, do you have any other tracebacks that look related to sync? The one you've posted seems to be related mostly to push, which I think is still a problem but it may not exactly explain why /sync is failing.
(Some of the code path is possibly the same, but to my eyes it'd have to be generating a sync entry for a room that doesn't exist anymore — which seems odd to the point that I'm struggling to believe that would work before.)

You say you blocked and unblocked the room — did you delete it with the Admin API / shut it down / ... it at any point? (Blocking a room does not remove it from the database. Similarly, I don't think we actually clean up rooms that you've left or even forgotten automatically.)

Any other 'unusual' circumstances? Perhaps the room was lost during restore of a backup or etc. :/

A colleague proposes a hypothesis:

Synapse Admins is a big room. If they tried to join it while the purge was ongoing (before the row from rooms was dropped at the end of the process) then maybe they would end up in the state they are now.

I don't know if that possibly matches up at all?

@jae1911
Copy link
Contributor Author

jae1911 commented Oct 27, 2022

I do not have any different traces than this one.

You say you blocked and unblocked the room — did you delete it with the Admin API / shut it down / ... it at any point? (Blocking a room does not remove it from the database. Similarly, I don't think we actually clean up rooms that you've left or even forgotten automatically.)

Yes, I most probably tried deleting it using the admin endpoint at that time.
The hypothesis could make sense as I unblocked it a while after and tried joining it without success.

Do you have any idea how to at least apply a band-aid to the problem? Like could completely wiping the room from the database could be a solution?

@reivilibre
Copy link
Contributor

Yep, I've been otherwise busy with the release but I'll try and help you sort your database out.

First port of call:
You can try the room purge admin API to delete that room.
I think there's a good chance it won't do anything though, since the row for the room has already been deleted, it might think there's nothing to delete.

If that doesn't work, last resort:
Turn off Synapse. Make a backup (pg_dump will do, as long as you know how to restore it afterwards if it all goes wrong).

Open a psql connection to your Synapse database and run this query to wipe the room entirely from every table in the database:

BEGIN; SET CONSTRAINTS ALL DEFERRED;

 DELETE FROM federation_inbound_events_staging WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM partial_state_events WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM event_search WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM events WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM insertion_events WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM local_current_membership WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM partial_state_rooms WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM blocked_rooms WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM current_state_delta_stream WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM room_stats_state WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM appservice_room_list WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM event_auth WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM event_forward_extremities WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM event_reports WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM event_json WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM insertion_event_extremities WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM pusher_throttle WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM receipts_graph WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM room_aliases WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM room_account_data WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM room_depth WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM room_tags_revisions WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM receipts_linearized WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM room_stats_earliest_token WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM room_tags WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM state_events WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM event_txn_id WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM room_memberships WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM user_directory WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM state_groups WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM state_groups_state WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM users_in_public_rooms WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM insertion_event_edges WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM users_who_share_private_rooms WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM current_state_events WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM device_lists_changes_in_room WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM e2e_room_keys WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM event_auth_chain_to_calculate WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM event_push_summary WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM event_labels WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM room_retention WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM rooms WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM stream_ordering_to_exterm WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM event_edges WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM event_push_actions WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM destination_rooms WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM event_backward_extremities WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM partial_state_rooms_servers WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM room_stats_current WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM event_failed_pull_attempts WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';
 DELETE FROM batch_events WHERE room_id = '!ehXvUhWNASUkSLvAGP:matrix.org';

COMMIT;

(expect this to possibly take a long time and it may not even work)
Now turn on Synapse and see if it works. Check there's nothing obviously broken going on in the logs.

(This was obtained from)

SELECT CONCAT('DELETE FROM ', table_name, ' WHERE room_id = ''!ehXvUhWNASUkSLvAGP:matrix.org'';') FROM information_schema.columns WHERE table_schema = 'public' AND column_name = 'room_id';

@DMRobertson DMRobertson added S-Major Major functionality / product severely impaired, no satisfactory workaround. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues. A-Database DB stuff like queries, migrations, new/remove columns, indexes, unexpected entries in the db O-Uncommon Most users are unlikely to come across this or unexpected workflow and removed X-Regression Something broke which worked on a previous release labels Oct 28, 2022
@jae1911
Copy link
Contributor Author

jae1911 commented Oct 28, 2022

Hey there, your second method worked, thanks a lot for the help!
Turns out the solution was to nuke everything.

@NicolasDerumigny
Copy link

I encountered the same problem, and the same solution seemed to have worked. Thanks!!!

@reivilibre reivilibre removed their assignment Nov 10, 2022
@lsrmg
Copy link

lsrmg commented Nov 11, 2022

I have the same issue updating from 1.69.0 to 1.71.0, but the SELECT statements

SELECT room_id, event_id, membership FROM local_current_membership LEFT JOIN events USING (event_id) WHERE user_id = '@myname:and.server' AND events.type IS NULL;
and
SELECT room_id, event_id, membership FROM local_current_membership LEFT JOIN rooms USING (room_id) WHERE user_id = '@myname AND rooms.room_version IS NULL;

doesn't show a single room where events.type/rooms.room_version is NULL even without the join.

I get the following error. Its like the error from jae1911 but in my case from the background process 'rotate_notifs'

2022-11-11 20:15:50,596 - synapse.metrics.background_process_metrics - 242 - ERROR - rotate_notifs-18 - Background process 'rotate_notifs' threw an exception
Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/metrics/background_process_metrics.py", line 240, in run
    return await func(*args, **kwargs)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/databases/main/event_push_actions.py", line 1052, in _rotate_notifs
    caught_up = await self.db_pool.runInteraction(
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/database.py", line 881, in runInteraction
    return await delay_cancellation(_runInteraction())
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/twisted/internet/defer.py", line 1656, in _inlineCallbacks
    result = current_context.run(
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/twisted/python/failure.py", line 514, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/database.py", line 848, in _runInteraction
    result = await self.runWithConnection(
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/database.py", line 976, in runWithConnection
    return await make_deferred_yieldable(
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/twisted/python/threadpool.py", line 244, in inContext
    result = inContext.theWork()  # type: ignore[attr-defined]
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/twisted/python/threadpool.py", line 260, in <lambda>
    inContext.theWork = lambda: context.call(  # type: ignore[attr-defined]
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/twisted/python/context.py", line 117, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/twisted/python/context.py", line 82, in callWithContext
    return func(*args, **kw)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/twisted/enterprise/adbapi.py", line 282, in _runWithConnection
    result = func(conn, *args, **kw)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/database.py", line 969, in inner_func
    return func(db_conn, *args, **kwargs)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/database.py", line 710, in new_transaction
    r = func(cursor, *args, **kwargs)
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/databases/main/event_push_actions.py", line 1097, in _handle_new_receipts_for_notifs_txn
    old_rotate_stream_ordering = self.db_pool.simple_select_one_onecol_txn(
  File "/opt/venvs/matrix-synapse/lib/python3.9/site-packages/synapse/storage/database.py", line 1718, in simple_select_one_onecol_txn
    raise StoreError(404, "No row found")
synapse.api.errors.StoreError: 404: No row found

@squahtx
Copy link
Contributor

squahtx commented Nov 15, 2022

@lsrmg That looks like it's a different underlying problem Could you file a new issue for your problem?

@DMRobertson DMRobertson changed the title synapse.api.errors.StoreError: 404: No row found since upgrade to v1.70.0 synapse.api.errors.StoreError: 404: No row found in background process http_pusher.on_new_receipts since upgrade to v1.70.0 Dec 8, 2022
@mi-gri
Copy link

mi-gri commented Jan 21, 2023

I'm not sure if this is the same problem, but it may be related.
I tried to migrate my database of synapse 1.75 from sqlite to postgres using the synapse_port_db tool and getting this:

  • synapse.metrics.background_process_metrics - 244 - ERROR - Background process 'rotate_notifs' threw an exception
    Traceback (most recent call last):
    File "/usr/local/lib/python3.8/dist-packages/synapse/metrics/background_process_metrics.py", line 242, in run
    return await func(*args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/synapse/storage/databases/main/event_push_actions.py", line 1436, in _rotate_notifs
    caught_up = await self.db_pool.runInteraction(
    File "/usr/local/lib/python3.8/dist-packages/synapse/storage/database.py", line 882, in runInteraction
    return await delay_cancellation(_runInteraction())
    File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
    File "/usr/lib/python3/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
    File "/usr/local/lib/python3.8/dist-packages/synapse/storage/database.py", line 849, in _runInteraction
    result = await self.runWithConnection(
    File "/usr/local/lib/python3.8/dist-packages/synapse/storage/database.py", line 977, in runWithConnection
    return await make_deferred_yieldable(
    File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 250, in inContext
    result = inContext.theWork()
    File "/usr/lib/python3/dist-packages/twisted/python/threadpool.py", line 266, in
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
    File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
    File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
    File "/usr/lib/python3/dist-packages/twisted/enterprise/adbapi.py", line 306, in _runWithConnection
    compat.reraise(excValue, excTraceback)
    File "/usr/lib/python3/dist-packages/twisted/python/compat.py", line 464, in reraise
    raise exception.with_traceback(traceback)
    File "/usr/lib/python3/dist-packages/twisted/enterprise/adbapi.py", line 297, in _runWithConnection
    result = func(conn, *args, **kw)
    File "/usr/local/lib/python3.8/dist-packages/synapse/storage/database.py", line 970, in inner_func
    return func(db_conn, *args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/synapse/storage/database.py", line 711, in new_transaction
    r = func(cursor, *args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/synapse/storage/databases/main/event_push_actions.py", line 1471, in _handle_new_receipts_for_notifs_txn
    min_receipts_stream_id = self.db_pool.simple_select_one_onecol_txn(
    File "/usr/local/lib/python3.8/dist-packages/synapse/storage/database.py", line 1699, in simple_select_one_onecol_txn
    raise StoreError(404, "No row found")
    synapse.api.errors.StoreError: 404: No row found

@squahtx
Copy link
Contributor

squahtx commented Jan 25, 2023

@mi-gri Could you file a new issue for your problem?

@lsrmg
Copy link

lsrmg commented Jan 25, 2023

Looks like my issue connected to this one (#14454).

@H-Shay
Copy link
Contributor

H-Shay commented Aug 8, 2023

It seems as though the original issue is resolved, I am going to close this - if for some reason we need to re-open it we can do so.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A-Database DB stuff like queries, migrations, new/remove columns, indexes, unexpected entries in the db O-Uncommon Most users are unlikely to come across this or unexpected workflow S-Major Major functionality / product severely impaired, no satisfactory workaround. T-Defect Bugs, crashes, hangs, security vulnerabilities, or other reported issues.
Projects
None yet
Development

No branches or pull requests

8 participants