New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partitioning support for Event Store #770

Open
jokokko opened this Issue May 23, 2017 · 5 comments

Comments

Projects
None yet
6 participants
@jokokko
Collaborator

jokokko commented May 23, 2017

To avoid table bloat with the Marten Event Store, events table (mt_events by default) could be partitioned using PostgreSQL table partitioning.

While the table can already be manually partitioned after creation with table inheritance in PG 9.x, PG 10 will introduce native table partitioning with new prerequisites (and limitations): https://www.postgresql.org/docs/10/static/ddl-partitioning.html Given that "It is not possible to turn a regular table into a partitioned table or vice versa.", it would be extremely useful to be able to define a partitioning scheme upfront as Marten creates the necessary tables.

@cocowalla

This comment has been minimized.

Contributor

cocowalla commented Jul 7, 2017

It would be great to also support this on arbitrary tables when using Marten as a document database, not just as an Event Store.

@jokokko

This comment has been minimized.

Collaborator

jokokko commented Jul 8, 2017

Indeed. But if it's implemented for Event Store tables (well, mainly mt_events), should not be too big a step to generalize for all tables. The hardest thing likely is to offer a flexible way to define the actual partitioning scheme. Then again, being able to do it via Marten would be a huge advantage, as, afaik, tables cannot be partitioned afterwards (talking about native partitioning in PG 10).

@turbo

This comment has been minimized.

turbo commented Sep 21, 2018

I'm relying on Marten for most persistence in my app, but there are two massive tables that are still raw PGSQL because they need partitioning supports (by tenant, timestamp). It would be nice to handle this in Marten.

Although I'd urge you to wait for PG11, which will have quite a few partition improvements.

@oskardudycz

This comment has been minimized.

Collaborator

oskardudycz commented Sep 24, 2018

@turbo currently you can use Conjoined Tenancy that will at least give you possibility to split it by tenant (also stream and events table). I'll have a look on the PG partitioning in PG 10.

@isengrim613

This comment has been minimized.

isengrim613 commented Nov 15, 2018

Seems like PG11 has been released
https://www.postgresql.org/about/news/1894/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment