MSC2314: A Method to Backfill Room State #2314

hawkowl · 2019-10-07T12:19:36Z

Rendered

This API can be tested in matrix-org/synapse#6176 .

Signed-off-by: Amber H. Brown amberb@matrix.org

FCP proposal: #2314 (comment)

proposals/2314-backfill-current-state.md

richvdh · 2019-10-29T13:45:27Z

This API can be accessed provisionally under /_matrix/federation/unstable/state/ in matrix-org/synapse#6176, and is feature flagged by net.atleastfornow.msc2314.

could you update this if it's no longer correct?

hawkowl · 2019-11-01T14:36:34Z

This API can be accessed provisionally under /_matrix/federation/unstable/state/ in matrix-org/synapse#6176, and is feature flagged by net.atleastfornow.msc2314.

could you update this if it's no longer correct?

done, reflecting the current PR

erikjohnston · 2019-11-04T13:48:08Z

proposals/2314-backfill-current-state.md

+
+If a server experiences data loss, it is difficult for it to recover membership in federation without having a user be reinvited, as backfill alone can not easily retrieve the data that is required to operate in a room (namely the current state and auth chain).
+
+This MSC introduces S2S APIs to provide a given room's auth chain and current state events, provided the requesting server is in the room specified.


How do you see this API actually being used? The "current state" isn't really a concept that is used much in s2s, as if the server wants to start participating in the room it needs a set of most recent events and their state (from which the current state can be derived).

Yes, it's asking another server for it's current state, in a situation where it knows it's joined to the room but is missing state (data loss).

But how does it actually use the current state? I don't think the current state is useful in rehydrating a room for example, as you need the extremities and the state at each of them.

hrm, it seems I missed this thread previously. @erikjohnston raises a good point - it is very hard to see how this API could be usefully used.

If we can't figure out a practical use for this change, we should instead reject it, and back out the changes in Synapse and Sytest.

@erikjohnston any further thoughts here?

It does feel like it should be useful, but I can't really see how. If this is mainly useful for rescuing a server that has suffered catastrophic memory lost then I think I'd like to see a step-by-step write up of the process of recovering from that, including any new APIs that need to be added.

Having a one stop "bootstrap this room for me" API might be useful, which returns the extremities, auth chain and state at each event (probably with some clever de-duplication there)

right then. In that case, I'm going to close this MSC, and back out the changes in sytest and synapse.

proposals/2314-backfill-current-state.md

turt2live · 2019-11-19T21:11:52Z

proposals/2314-backfill-current-state.md

+
+Excessively large rooms may cause performance problems for servers implementing this API (including in its v1 incarnation). Discretionary rate limiting of this API may be required.
+
+## Security considerations


servers might also have to worry about malicious attempts to get the server to join rooms. A server that really wants you to join a room could send the server an event over /send, therefore confusing the server into thinking it might be in the room. The victim server would then use this new API, find out some information about the room (possibly a falsified join event) and push the room through as joined. In theory at some point a signature check would fail, but another workaround could be to ask a couple servers in the room (if possible) to verify the state returned by a server.

turt2live

seems generally sensible to me

…org/matrix-spec-proposals#2314

auscompgeek · 2020-05-02T05:10:51Z

So by my understanding, both Synapse and Construct implement this MSC as is on the stable endpoint.

auscompgeek · 2020-05-06T04:43:31Z

It has come to my attention (thanks to @jevolk) that it would be a good idea for /state_ids/ to get a similar treatment. I think it would be good for that to be in this MSC as well.

DMRobertson · 2022-04-04T12:37:24Z

I'm not sure what happened to this, but Synapse's sytest blacklist is written as if this is formally merged into the spec:

# Blacklisted due to https://github.com/matrix-org/matrix-doc/pull/2314 removing
# this requirement from the spec
Inbound federation of state requires event_id as a mandatory paramater

richvdh · 2022-04-04T13:38:35Z

This landed in synapse (matrix-org/synapse#6176), hence the change to the sytest blacklist.

We should either rip it out of synapse, or land it in the spec.

I think the latter is doable, hence:

@mscbot fcp merge

mscbot · 2022-04-04T13:38:37Z

This FCP proposal has been cancelled by #2314 (comment).

Team member @mscbot has proposed to merge this. The next step is review by the rest of the tagged people:

Once at least 75% of reviewers approve (and there are no outstanding concerns), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for information about what commands tagged team members can give me.

proposals/2314-backfill-current-state.md

anoadragon453

One small q, the rest looks good to me.

proposals/2314-backfill-current-state.md

richvdh · 2022-04-14T12:46:23Z

closing this as it doesn't seem to have a real usecase, per #2314 (comment).

turt2live · 2022-04-14T13:00:14Z

@mscbot fcp cancel

@Bubu

Synapse 1.58.0rc1 (2022-04-26) ============================== As of this release, the groups/communities feature in Synapse is now disabled by default. See [\#11584](matrix-org/synapse#11584) for details. As mentioned in [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1580), this feature will be removed in Synapse 1.61. Features -------- - Implement [MSC3383](matrix-org/matrix-spec-proposals#3383) for including the destination in server-to-server authentication headers. Contributed by @Bubu and @jcgruenhage for Famedly. ([\#11398](matrix-org/synapse#11398)) - Docker images and Debian packages from matrix.org now contain a locked set of Python dependencies, greatly improving build reproducibility. ([Board](https://github.com/orgs/matrix-org/projects/54), [\#11537](matrix-org/synapse#11537)) - Enable processing of device list updates asynchronously. ([\#12365](matrix-org/synapse#12365), [\#12465](matrix-org/synapse#12465)) - Implement [MSC2815](matrix-org/matrix-spec-proposals#2815) to allow room moderators to view redacted event content. Contributed by @tulir. ([\#12427](matrix-org/synapse#12427)) - Build Debian packages for Ubuntu 22.04 "Jammy Jellyfish". ([\#12543](matrix-org/synapse#12543)) Bugfixes -------- - Prevent a sync request from removing a user's busy presence status. ([\#12213](matrix-org/synapse#12213)) - Fix bug with incremental sync missing events when rejoining/backfilling. Contributed by Nick @ Beeper. ([\#12319](matrix-org/synapse#12319)) - Fix a long-standing bug which incorrectly caused `GET /_matrix/client/v3/rooms/{roomId}/event/{eventId}` to return edited events rather than the original. ([\#12476](matrix-org/synapse#12476)) - Fix a bug introduced in Synapse 1.27.0 where the admin API for [deleting forward extremities](https://github.com/matrix-org/synapse/blob/erikj/fix_delete_event_response_count/docs/admin_api/rooms.md#deleting-forward-extremities) would always return a count of 1, no matter how many extremities were deleted. ([\#12496](matrix-org/synapse#12496)) - Fix a long-standing bug where the image thumbnails embedded into email notifications were broken. ([\#12510](matrix-org/synapse#12510)) - Fix a bug in the implementation of [MSC3202](matrix-org/matrix-spec-proposals#3202) where Synapse would use the field name `device_unused_fallback_keys`, rather than `device_unused_fallback_key_types`. ([\#12520](matrix-org/synapse#12520)) - Fix a bug introduced in Synapse 0.99.3 which could cause Synapse to consume large amounts of RAM when back-paginating in a large room. ([\#12522](matrix-org/synapse#12522)) Improved Documentation ---------------------- - Fix rendering of the documentation site when using the 'print' feature. ([\#12340](matrix-org/synapse#12340)) - Add a manual documenting config file options. ([\#12368](matrix-org/synapse#12368), [\#12527](matrix-org/synapse#12527)) - Update documentation to reflect that both the `run_background_tasks_on` option and the options for moving stream writers off of the main process are no longer experimental. ([\#12451](matrix-org/synapse#12451)) - Update worker documentation and replace old `federation_reader` with `generic_worker`. ([\#12457](matrix-org/synapse#12457)) - Strongly recommend [Poetry](https://python-poetry.org/) for development. ([\#12475](matrix-org/synapse#12475)) - Add some example configurations for workers and update architectural diagram. ([\#12492](matrix-org/synapse#12492)) - Fix a broken link in `README.rst`. ([\#12495](matrix-org/synapse#12495)) - Add HAProxy delegation example with CORS headers to docs. ([\#12501](matrix-org/synapse#12501)) - Remove extraneous comma in User Admin API's device deletion section so that the example JSON is actually valid and works. Contributed by @olmari. ([\#12533](matrix-org/synapse#12533)) Deprecations and Removals ------------------------- - The groups/communities feature in Synapse is now disabled by default. ([\#12344](matrix-org/synapse#12344)) - Remove unstable identifiers from [MSC3440](matrix-org/matrix-spec-proposals#3440). ([\#12382](matrix-org/synapse#12382)) Internal Changes ---------------- - Preparation for faster-room-join work: start a background process to resynchronise the room state after a room join. ([\#12394](matrix-org/synapse#12394)) - Preparation for faster-room-join work: Implement a tracking mechanism to allow functions to wait for full room state to arrive. ([\#12399](matrix-org/synapse#12399)) - Remove an unstable identifier from [MSC3083](matrix-org/matrix-spec-proposals#3083). ([\#12395](matrix-org/synapse#12395)) - Run CI in the locked [Poetry](https://python-poetry.org/) environment, and remove corresponding `tox` jobs. ([\#12425](matrix-org/synapse#12425), [\#12434](matrix-org/synapse#12434), [\#12438](matrix-org/synapse#12438), [\#12441](matrix-org/synapse#12441), [\#12449](matrix-org/synapse#12449), [\#12478](matrix-org/synapse#12478), [\#12514](matrix-org/synapse#12514), [\#12472](matrix-org/synapse#12472)) - Change Mutual Rooms' `unstable_features` flag to `uk.half-shot.msc2666.mutual_rooms` which matches the current iteration of [MSC2666](matrix-org/matrix-spec-proposals#2666). ([\#12445](matrix-org/synapse#12445)) - Fix typo in the release script help string. ([\#12450](matrix-org/synapse#12450)) - Fix a minor typo in the Debian changelogs generated by the release script. ([\#12497](matrix-org/synapse#12497)) - Reintroduce the list of targets to the linter script, to avoid linting unwanted local-only directories during development. ([\#12455](matrix-org/synapse#12455)) - Limit length of `device_id` to less than 512 characters. ([\#12454](matrix-org/synapse#12454)) - Dockerfile-workers: reduce the amount we install in the image. ([\#12464](matrix-org/synapse#12464)) - Dockerfile-workers: give the master its own log config. ([\#12466](matrix-org/synapse#12466)) - complement-synapse-workers: factor out separate entry point script. ([\#12467](matrix-org/synapse#12467)) - Back out experimental implementation of [MSC2314](matrix-org/matrix-spec-proposals#2314). ([\#12474](matrix-org/synapse#12474)) - Fix grammatical error in federation error response when the room version of a room is unknown. ([\#12483](matrix-org/synapse#12483)) - Remove unnecessary configuration overrides in tests. ([\#12511](matrix-org/synapse#12511)) - Refactor the relations code for clarity. ([\#12519](matrix-org/synapse#12519)) - Add type hints so `docker` and `stubs` directories pass `mypy --disallow-untyped-defs`. ([\#12528](matrix-org/synapse#12528)) - Update `delay_cancellation` to accept any awaitable, rather than just `Deferred`s. ([\#12468](matrix-org/synapse#12468)) - Handle cancellation in `EventsWorkerStore._get_events_from_cache_or_db`. ([\#12529](matrix-org/synapse#12529))

@Bubu

Synapse 1.58.0 (2022-05-03) =========================== As of this release, the groups/communities feature in Synapse is now disabled by default. See [\#11584](matrix-org/synapse#11584) for details. As mentioned in [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1580), this feature will be removed in Synapse 1.61. No significant changes since 1.58.0rc2. Synapse 1.58.0rc2 (2022-04-26) ============================== This release candidate fixes bugs related to Synapse 1.58.0rc1's logic for handling device list updates. Bugfixes -------- - Fix a bug introduced in Synapse 1.58.0rc1 where the main process could consume excessive amounts of CPU and memory while handling sentry logging failures. ([\#12554](matrix-org/synapse#12554)) - Fix a bug introduced in Synapse 1.58.0rc1 where opentracing contexts were not correctly sent to whitelisted remote servers with device lists updates. ([\#12555](matrix-org/synapse#12555)) Internal Changes ---------------- - Reduce unnecessary work when handling remote device list updates. ([\#12557](matrix-org/synapse#12557)) Synapse 1.58.0rc1 (2022-04-26) ============================== Features -------- - Implement [MSC3383](matrix-org/matrix-spec-proposals#3383) for including the destination in server-to-server authentication headers. Contributed by @Bubu and @jcgruenhage for Famedly. ([\#11398](matrix-org/synapse#11398)) - Docker images and Debian packages from matrix.org now contain a locked set of Python dependencies, greatly improving build reproducibility. ([Board](https://github.com/orgs/matrix-org/projects/54), [\#11537](matrix-org/synapse#11537)) - Enable processing of device list updates asynchronously. ([\#12365](matrix-org/synapse#12365), [\#12465](matrix-org/synapse#12465)) - Implement [MSC2815](matrix-org/matrix-spec-proposals#2815) to allow room moderators to view redacted event content. Contributed by @tulir @ Beeper. ([\#12427](matrix-org/synapse#12427)) - Build Debian packages for Ubuntu 22.04 "Jammy Jellyfish". ([\#12543](matrix-org/synapse#12543)) Bugfixes -------- - Prevent a sync request from removing a user's busy presence status. ([\#12213](matrix-org/synapse#12213)) - Fix bug with incremental sync missing events when rejoining/backfilling. Contributed by Nick @ Beeper. ([\#12319](matrix-org/synapse#12319)) - Fix a long-standing bug which incorrectly caused `GET /_matrix/client/v3/rooms/{roomId}/event/{eventId}` to return edited events rather than the original. ([\#12476](matrix-org/synapse#12476)) - Fix a bug introduced in Synapse 1.27.0 where the admin API for [deleting forward extremities](https://github.com/matrix-org/synapse/blob/erikj/fix_delete_event_response_count/docs/admin_api/rooms.md#deleting-forward-extremities) would always return a count of 1, no matter how many extremities were deleted. ([\#12496](matrix-org/synapse#12496)) - Fix a long-standing bug where the image thumbnails embedded into email notifications were broken. ([\#12510](matrix-org/synapse#12510)) - Fix a bug in the implementation of [MSC3202](matrix-org/matrix-spec-proposals#3202) where Synapse would use the field name `device_unused_fallback_keys`, rather than `device_unused_fallback_key_types`. ([\#12520](matrix-org/synapse#12520)) - Fix a bug introduced in Synapse 0.99.3 which could cause Synapse to consume large amounts of RAM when back-paginating in a large room. ([\#12522](matrix-org/synapse#12522)) Improved Documentation ---------------------- - Fix rendering of the documentation site when using the 'print' feature. ([\#12340](matrix-org/synapse#12340)) - Add a manual documenting config file options. ([\#12368](matrix-org/synapse#12368), [\#12527](matrix-org/synapse#12527)) - Update documentation to reflect that both the `run_background_tasks_on` option and the options for moving stream writers off of the main process are no longer experimental. ([\#12451](matrix-org/synapse#12451)) - Update worker documentation and replace old `federation_reader` with `generic_worker`. ([\#12457](matrix-org/synapse#12457)) - Strongly recommend [Poetry](https://python-poetry.org/) for development. ([\#12475](matrix-org/synapse#12475)) - Add some example configurations for workers and update architectural diagram. ([\#12492](matrix-org/synapse#12492)) - Fix a broken link in `README.rst`. ([\#12495](matrix-org/synapse#12495)) - Add HAProxy delegation example with CORS headers to docs. ([\#12501](matrix-org/synapse#12501)) - Remove extraneous comma in User Admin API's device deletion section so that the example JSON is actually valid and works. Contributed by @olmari. ([\#12533](matrix-org/synapse#12533)) Deprecations and Removals ------------------------- - The groups/communities feature in Synapse is now disabled by default. ([\#12344](matrix-org/synapse#12344)) - Remove unstable identifiers from [MSC3440](matrix-org/matrix-spec-proposals#3440). ([\#12382](matrix-org/synapse#12382)) Internal Changes ---------------- - Preparation for faster-room-join work: start a background process to resynchronise the room state after a room join. ([\#12394](matrix-org/synapse#12394)) - Preparation for faster-room-join work: Implement a tracking mechanism to allow functions to wait for full room state to arrive. ([\#12399](matrix-org/synapse#12399)) - Remove an unstable identifier from [MSC3083](matrix-org/matrix-spec-proposals#3083). ([\#12395](matrix-org/synapse#12395)) - Run CI in the locked [Poetry](https://python-poetry.org/) environment, and remove corresponding `tox` jobs. ([\#12425](matrix-org/synapse#12425), [\#12434](matrix-org/synapse#12434), [\#12438](matrix-org/synapse#12438), [\#12441](matrix-org/synapse#12441), [\#12449](matrix-org/synapse#12449), [\#12478](matrix-org/synapse#12478), [\#12514](matrix-org/synapse#12514), [\#12472](matrix-org/synapse#12472)) - Change Mutual Rooms' `unstable_features` flag to `uk.half-shot.msc2666.mutual_rooms` which matches the current iteration of [MSC2666](matrix-org/matrix-spec-proposals#2666). ([\#12445](matrix-org/synapse#12445)) - Fix typo in the release script help string. ([\#12450](matrix-org/synapse#12450)) - Fix a minor typo in the Debian changelogs generated by the release script. ([\#12497](matrix-org/synapse#12497)) - Reintroduce the list of targets to the linter script, to avoid linting unwanted local-only directories during development. ([\#12455](matrix-org/synapse#12455)) - Limit length of `device_id` to less than 512 characters. ([\#12454](matrix-org/synapse#12454)) - Dockerfile-workers: reduce the amount we install in the image. ([\#12464](matrix-org/synapse#12464)) - Dockerfile-workers: give the master its own log config. ([\#12466](matrix-org/synapse#12466)) - complement-synapse-workers: factor out separate entry point script. ([\#12467](matrix-org/synapse#12467)) - Back out experimental implementation of [MSC2314](matrix-org/matrix-spec-proposals#2314). ([\#12474](matrix-org/synapse#12474)) - Fix grammatical error in federation error response when the room version of a room is unknown. ([\#12483](matrix-org/synapse#12483)) - Remove unnecessary configuration overrides in tests. ([\#12511](matrix-org/synapse#12511)) - Refactor the relations code for clarity. ([\#12519](matrix-org/synapse#12519)) - Add type hints so `docker` and `stubs` directories pass `mypy --disallow-untyped-defs`. ([\#12528](matrix-org/synapse#12528)) - Update `delay_cancellation` to accept any awaitable, rather than just `Deferred`s. ([\#12468](matrix-org/synapse#12468)) - Handle cancellation in `EventsWorkerStore._get_events_from_cache_or_db`. ([\#12529](matrix-org/synapse#12529))

hawkowl added 2 commits October 7, 2019 13:18

initial

615f91a

rename to msc id

5cd0a60

hawkowl changed the title ~~MSC-XXXX: A Method to Backfill Room State~~ MSC-2314: A Method to Backfill Room State Oct 7, 2019

hawkowl added the proposal A matrix spec change proposal label Oct 7, 2019

turt2live changed the title ~~MSC-2314: A Method to Backfill Room State~~ MSC2314: A Method to Backfill Room State Oct 7, 2019

turt2live added the proposal-in-review label Oct 7, 2019

turt2live reviewed Oct 7, 2019

View reviewed changes

proposals/2314-backfill-current-state.md Outdated Show resolved Hide resolved

turt2live self-requested a review October 7, 2019 12:27

fix

ae147a6

hawkowl mentioned this pull request Oct 7, 2019

Implementation of MSC2314 matrix-org/synapse#6176

Merged

hawkowl requested a review from a team October 21, 2019 13:17

hawkowl mentioned this pull request Oct 22, 2019

MSC2324: Facilitating early releases of software dependent on spec #2324

Merged

richvdh reviewed Oct 29, 2019

View reviewed changes

proposals/2314-backfill-current-state.md Outdated Show resolved Hide resolved

proposals/2314-backfill-current-state.md Outdated Show resolved Hide resolved

Update to be a backwards-compatible change to the v1 API

1253584

erikjohnston reviewed Nov 4, 2019

View reviewed changes

turt2live reviewed Nov 19, 2019

View reviewed changes

proposals/2314-backfill-current-state.md Outdated Show resolved Hide resolved

turt2live reviewed Nov 19, 2019

View reviewed changes

proposals/2314-backfill-current-state.md Outdated Show resolved Hide resolved

turt2live reviewed Nov 19, 2019

View reviewed changes

turt2live approved these changes Nov 19, 2019

View reviewed changes

wrap, add updates

7d88e56

turt2live added the kind:feature MSC for not-core and not-maintenance stuff label Apr 20, 2020

jevolk added a commit to jevolk/charybdis that referenced this pull request May 1, 2020

modules/federation/state: Implement room_version in response; matrix-…

7854a0e

…org/matrix-spec-proposals#2314

turt2live added the needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. label Jun 8, 2021

turt2live force-pushed the old_master branch from e895827 to dca99ee Compare August 30, 2021 22:34

DMRobertson mentioned this pull request Apr 4, 2022

review the list of blacklisted sytests matrix-org/synapse#10696

Closed

mscbot added disposition-merge proposed-final-comment-period Currently awaiting signoff of a majority of team members in order to enter the final comment period. and removed proposal-in-review labels Apr 4, 2022

turt2live removed the needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. label Apr 4, 2022

erikjohnston added this to Needs idea feedback / initial review in Spec Core Team Backlog via automation Apr 5, 2022

erikjohnston moved this from Needs idea feedback / initial review to Ready for FCP ticks in Spec Core Team Backlog Apr 5, 2022

richvdh reviewed Apr 5, 2022

View reviewed changes

richvdh added 2 commits April 5, 2022 18:20

Minor clarifications

ec68e33

Update proposals/2314-backfill-current-state.md

610f5e3

anoadragon453 reviewed Apr 12, 2022

View reviewed changes

proposals/2314-backfill-current-state.md Outdated Show resolved Hide resolved

richvdh added 2 commits April 13, 2022 07:49

make room_version mandatory

9558b3a

clarification

f50c3c5

richvdh closed this Apr 14, 2022

mscbot added proposal-in-review and removed proposed-final-comment-period Currently awaiting signoff of a majority of team members in order to enter the final comment period. disposition-merge labels Apr 14, 2022

turt2live added abandoned A proposal where the author/shepherd is not responsive and removed proposal-in-review labels Apr 14, 2022

turt2live removed this from Ready for FCP ticks in Spec Core Team Backlog Apr 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MSC2314: A Method to Backfill Room State #2314

MSC2314: A Method to Backfill Room State #2314

hawkowl commented Oct 7, 2019 •

edited by richvdh

Loading

richvdh commented Oct 29, 2019

hawkowl commented Nov 1, 2019

erikjohnston Nov 4, 2019

hawkowl Nov 7, 2019

erikjohnston Nov 27, 2019

richvdh Apr 13, 2022

erikjohnston Apr 13, 2022

richvdh Apr 14, 2022

turt2live Nov 19, 2019

turt2live left a comment

auscompgeek commented May 2, 2020 •

edited

Loading

auscompgeek commented May 6, 2020

DMRobertson commented Apr 4, 2022

richvdh commented Apr 4, 2022

mscbot commented Apr 4, 2022 •

edited

Loading

anoadragon453 left a comment

richvdh commented Apr 14, 2022

turt2live commented Apr 14, 2022


		If a server experiences data loss, it is difficult for it to recover membership in federation without having a user be reinvited, as backfill alone can not easily retrieve the data that is required to operate in a room (namely the current state and auth chain).

		This MSC introduces S2S APIs to provide a given room's auth chain and current state events, provided the requesting server is in the room specified.


		Excessively large rooms may cause performance problems for servers implementing this API (including in its v1 incarnation). Discretionary rate limiting of this API may be required.

		## Security considerations

MSC2314: A Method to Backfill Room State #2314

MSC2314: A Method to Backfill Room State #2314

Conversation

hawkowl commented Oct 7, 2019 • edited by richvdh Loading

richvdh commented Oct 29, 2019

hawkowl commented Nov 1, 2019

erikjohnston Nov 4, 2019

Choose a reason for hiding this comment

hawkowl Nov 7, 2019

Choose a reason for hiding this comment

erikjohnston Nov 27, 2019

Choose a reason for hiding this comment

richvdh Apr 13, 2022

Choose a reason for hiding this comment

erikjohnston Apr 13, 2022

Choose a reason for hiding this comment

richvdh Apr 14, 2022

Choose a reason for hiding this comment

turt2live Nov 19, 2019

Choose a reason for hiding this comment

turt2live left a comment

Choose a reason for hiding this comment

auscompgeek commented May 2, 2020 • edited Loading

auscompgeek commented May 6, 2020

DMRobertson commented Apr 4, 2022

richvdh commented Apr 4, 2022

mscbot commented Apr 4, 2022 • edited Loading

anoadragon453 left a comment

Choose a reason for hiding this comment

richvdh commented Apr 14, 2022

turt2live commented Apr 14, 2022

hawkowl commented Oct 7, 2019 •

edited by richvdh

Loading

auscompgeek commented May 2, 2020 •

edited

Loading

mscbot commented Apr 4, 2022 •

edited

Loading