Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

We need a way to be able to expire data from a HS. (SPEC-141) #447

Open
matrixbot opened this issue Feb 24, 2015 · 9 comments

Comments

@matrixbot
Copy link
Member

commented Feb 24, 2015

This could be implementation-specific, or via a standard admin API or per-room state configuration.

Eitherway, we need a way to tell HSes to:

  • no longer store this room
  • only keep N messages for this room
  • expire more than N messages for this room
  • only keep X days of messages for this room
  • purge state from this room (as opposed to messages)?

(Imported from https://matrix.org/jira/browse/SPEC-141)

(Reported by @ara4n)

@matrixbot

This comment has been minimized.

Copy link
Member Author

commented Feb 24, 2015

Jira watchers: @ara4n

@matrixbot

This comment has been minimized.

Copy link
Member Author

commented Feb 24, 2015

Links exported from Jira:

relates to SPEC-224
relates to #440
relates to SPEC-216
relates to SYN-361
relates to SYN-601

@matrixbot

This comment has been minimized.

Copy link
Member Author

commented Sep 10, 2015

The "No longer store this room" use case is now important given the history of rooms we have left are now returned to clients - we need a way to tell the server "no, really, leave and forget this conversation forever please".

-- @ara4n

@matrixbot

This comment has been minimized.

Copy link
Member Author

commented Sep 10, 2015

(A workaround for this in the short term would be to tag rooms you want to purge as 'purge' or something via SPEC-218 and filter them out on the client, but clearly this should be done serverside)

-- @ara4n

@matrixbot matrixbot changed the title We need a way to be able to expire data from a HS. We need a way to be able to expire data from a HS. (SPEC-141) Oct 31, 2016
@matrixbot matrixbot added the feature label Nov 7, 2016
@richvdh richvdh removed the vector label Oct 13, 2017
@ara4n

This comment has been minimized.

Copy link
Member

commented Jul 3, 2018

Bumping up the priority on this, as it's causing large homeserver admins (e.g. disroot.org) to consider discontinuing the service as their DB keeps growing and reaping it is hard.

In practice, i think what's needed is a way for ops to specify the desired history retention scheme for a given room - and for server admins to specify a maximum history retention in general.

(synapse bug matrix-org/synapse#1480)

@monkz

This comment has been minimized.

Copy link

commented Jul 3, 2018

I like to request a warning of some sort for this feature, so people can export their chatlogs before the culling.

@verymilan

This comment has been minimized.

Copy link

commented Jul 4, 2018

Maybe it's not worth mentioning but i'd like to add that the current way of deleting the chathistory is not very effective in terms of space usage as i ran against many big rooms only keeping 30days with synpurge and after a reindex only got like 20-30G free.
...so a more effective way would be very nice for such a feature.
pg_diskspace-year

@jryans jryans added phase:3 and removed phase:2 labels Sep 27, 2019
@ara4n

This comment has been minimized.

Copy link
Member

commented Oct 10, 2019

This is solved by #1763, which is implemented in matrix-org/synapse#5815. This is currently being ported into mainline synapse.

@MurzNN

This comment has been minimized.

Copy link
Contributor

commented Oct 10, 2019

Does messages removed fully with all metadata (so no one record kept in database about that message was exist before), or only message content?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
9 participants
You can’t perform that action at this time.