Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Feature: Purge local room history. #911

Merged
merged 9 commits into from Jul 7, 2016

Conversation

Projects
None yet
4 participants
Owner

erikjohnston commented Jul 6, 2016

No description provided.

erikjohnston added some commits Jul 4, 2016

@NegativeMjark NegativeMjark commented on an outdated diff Jul 7, 2016

synapse/storage/events.py
+ # state_groups_state
+
+ # First ensure that we're not about to delete all the forward extremeties
+ txn.execute(
+ "SELECT e.event_id, e.depth FROM events as e "
+ "INNER JOIN event_forward_extremities as f "
+ "ON e.event_id = f.event_id "
+ "AND e.room_id = f.room_id "
+ "WHERE f.room_id = ?",
+ (room_id,)
+ )
+ rows = txn.fetchall()
+ max_depth = max(row[0] for row in rows)
+
+ if max_depth <= topological_ordering:
+ raise Exception("topological_ordering is greater than forward extremeties")
@NegativeMjark

NegativeMjark Jul 7, 2016

Contributor

Maybe 400 here rather than 500?
Maybe comment on why this is bad.

Contributor

NegativeMjark commented Jul 7, 2016 edited

What happens when clients try to backfill using tokens from before the purge?

@erikjohnston erikjohnston referenced this pull request in matrix-org/sytest Jul 7, 2016

Merged

Add basic /admin/purge_history tests #260

erikjohnston added some commits Jul 7, 2016

Owner

erikjohnston commented Jul 7, 2016

What happens when clients try to backfill using tokens from before the purge?

It (successfully) returns 0 events. It probably triggers backfill to remote servers, so eventually it would probably return some events once the sever had backpaginated enough.

Contributor

NegativeMjark commented Jul 7, 2016

LGTM

@erikjohnston erikjohnston merged commit 70d650b into develop Jul 7, 2016

10 checks passed

Flake8 + Packaging (Commit) Build #1112 origin/erikj/purge_history succeeded in 37 sec
Details
Flake8 + Packaging (Merged PR) Build finished.
Details
Sytest Dendron (Commit) Build #227 origin/erikj/purge_history succeeded in 6 min 38 sec
Details
Sytest Dendron (Merged PR) Build finished.
Details
Sytest Postgres (Commit) Build #1067 origin/erikj/purge_history succeeded in 6 min 11 sec
Details
Sytest Postgres (Merged PR) Build finished.
Details
Sytest SQLite (Commit) Build #1086 origin/erikj/purge_history succeeded in 5 min 59 sec
Details
Sytest SQLite (Merged PR) Build finished.
Details
Unit Tests (Commit) Build #1151 origin/erikj/purge_history succeeded in 1 min 43 sec
Details
Unit Tests (Merged PR) Build finished.
Details

rubo77 commented Jul 10, 2016 edited

Is this already a solution for #890 - delete old data to clean up database ?

Owner

erikjohnston commented Jul 11, 2016

@rubo77 Yes, though its only a basic, manual per room API atm

Contributor

4nd3r commented Aug 19, 2016 edited

@erikjohnston is this manual per room API documented somewhere or can anyone can give an example?

@rubo77 rubo77 referenced this pull request Aug 19, 2016

Open

Add maintenance vacuum #916

rubo77 commented Aug 19, 2016

In #916 I posted a gist, that could work. If it works, we should add that in the api-rst file

rubo77 commented Aug 19, 2016

@erikjohnston can you take part of my example gist from #916 into that documentation?

https://gist.github.com/rubo77/458eb4808d665e619feeaf556476f73f

Contributor

4nd3r commented Aug 20, 2016

@erikjohnston thanks. does this API also delete uploaded media from disk?

rubo77 commented Aug 20, 2016

I updated to 0.17 now, and now my gist file works fine

rubo77 commented Aug 21, 2016

#1034 will add an example to the API Documentation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment