Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Typed Persistence: Implement deleting snapshots / events #24698

Closed
ktoso opened this Issue Mar 12, 2018 · 11 comments

Comments

Projects
5 participants
@ktoso
Copy link
Member

commented Mar 12, 2018

Currently there is no API for it, and I left ??? in the persistent bahavior for it

@ktoso ktoso changed the title Typed Persistence: Implement deleting snapshots Typed Persistence: Implement deleting snapshots / events Mar 12, 2018

@ktoso

This comment has been minimized.

Copy link
Member Author

commented Mar 12, 2018

Should we expose those on the persistence extension? It does not make much sense to expose those operations as Effect hmmm; I guess it could be done, felt a bit weird though.

People seem to often ask for "delete an actors state", we could offer this.

@johanandren

This comment has been minimized.

Copy link
Member

commented Apr 5, 2018

I think it should be an API separate from the typed persistent actors, as it is a separate concern. Konrad suggested something like JournalCleanup().deletePersistenceId(…, until...) which I think sounds promising.

@patriknw

This comment has been minimized.

Copy link
Member

commented Aug 17, 2018

Also remember to add the seqnr check when implementing deletes of events, #25487

@patriknw patriknw moved this from Backlog to Ready for production backlog in Akka Typed Dec 11, 2018

@helena helena self-assigned this Dec 15, 2018

@helena helena moved this from Ready for production backlog to In Progress in Akka Typed Jan 8, 2019

@patriknw

This comment has been minimized.

Copy link
Member

commented Mar 4, 2019

Would be nice to have an api that supported sequenced cleanup

  • save snapshot
  • delete old snapshots (keep a few old snapshots around)
  • delete events prior to snapshots

For example with snapshotting every 100 events and keeping two snapshots

  • save events 1-100
  • save snapshot at 100
  • save events 101-200
  • save snapshot at 200
  • save events 201-300
  • save snapshot at 300
  • delete events <= 100
  • delete snapshots <= 100
  • save events 301-400
  • save snapshot at 400
  • delete events <= 200
  • delete snapshots <= 200

We already have the saving of snapshots every N events in place and I think the deletion of old events and snapshots should be there as well since they are triggered by the SaveSnapshotSuccess. It would be completely automatic and the only API would be some way of specifying how many snapshots to keep around and maybe a way to NOT delete events at all but only delete snapshots.

We could also have something external like the suggested JournalCleanup, but I think that is lower priority.

@helena

This comment has been minimized.

Copy link
Member

commented Mar 4, 2019

@patriknw It seemed as though the sharding logic of keepNrOfBatches with snapshotAfter is useful in a more general sense WRT the snapshot algo, moving forward? Which is related to why I thought we may prefer to move the duplicated function to consolidated one in persistence which then typed could use. Thoughts?

@patriknw

This comment has been minimized.

Copy link
Member

commented Mar 4, 2019

I think we should add this in Persistence Typed only. Good with some added value to encourage usage of new stuff. Trying to share that between the two different implementations will add more complexity than it is worth.

@patriknw

This comment has been minimized.

Copy link
Member

commented Mar 4, 2019

and note that the usage from sharding will go away since we will probably deprecate/remove the persistence mode of sharding, in favor of only maintaining ddata mode

@tunesmith

This comment has been minimized.

Copy link

commented Mar 11, 2019

Hi, can you share more explanation on the reasoning behind deprecating/removing the persistence mode of sharding? I was under the impression that sharding persistence scaled out better than ddata.

@patriknw

This comment has been minimized.

Copy link
Member

commented Mar 11, 2019

@helena helena moved this from In Progress to Reviewing in Akka Typed Mar 16, 2019

johanandren added a commit that referenced this issue Mar 18, 2019

@johanandren johanandren added this to the 2.5.22 milestone Mar 18, 2019

@johanandren

This comment has been minimized.

Copy link
Member

commented Mar 18, 2019

I'm not sure if we can close this or if we should keep it open until the follow up PRs are in?

@helena helena removed the 3 - in progress label Mar 18, 2019

@helena helena closed this Mar 18, 2019

Akka Typed automation moved this from Reviewing to Done Mar 18, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.