New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for multi tenancy at documentStore level #435

Closed
valeriob opened this Issue Jun 19, 2016 · 12 comments

Comments

Projects
None yet
8 participants
@valeriob

valeriob commented Jun 19, 2016

One of the main advantages of using a "schemales db server" is that it makes multi tenant applications easier. To achieve that i think there should be an OpenSession ovarload that takes a database name as parameter.

@hellfirehd

This comment has been minimized.

Show comment
Hide comment
@hellfirehd

hellfirehd Jun 19, 2016

How do you envision that being implemented? Are you thinking the DocumentStore would manage multiple connection pools?

At the session level you might be able to have a "BucketId" which would be added to every document/event. NEventStore does this and uses the "default" bucket if one is not specified.

hellfirehd commented Jun 19, 2016

How do you envision that being implemented? Are you thinking the DocumentStore would manage multiple connection pools?

At the session level you might be able to have a "BucketId" which would be added to every document/event. NEventStore does this and uses the "default" bucket if one is not specified.

@valeriob

This comment has been minimized.

Show comment
Hide comment
@valeriob

valeriob Jun 19, 2016

I started a project similar to marten a few years ago, just it's not a framework, just an alternative implementation of ravendb IDocumentStore and IDocumentSession for the project i'm working on. It behaves the same way, so documentStore can access the whole sql db instance, and document session is opened on a particular database.
The connection string points to the main database, and than on every http request it opens the right documentSession for the tenant being targeted.

What do you mean by connection pools ? AdoNet providers already implement connection pooling.

valeriob commented Jun 19, 2016

I started a project similar to marten a few years ago, just it's not a framework, just an alternative implementation of ravendb IDocumentStore and IDocumentSession for the project i'm working on. It behaves the same way, so documentStore can access the whole sql db instance, and document session is opened on a particular database.
The connection string points to the main database, and than on every http request it opens the right documentSession for the tenant being targeted.

What do you mean by connection pools ? AdoNet providers already implement connection pooling.

@jeremydmiller jeremydmiller modified the milestone: 1.1 Jun 20, 2016

@JohnCampionJr

This comment has been minimized.

Show comment
Hide comment
@JohnCampionJr

JohnCampionJr Jun 21, 2016

Contributor

I think an important question is multi-tenancy via database-per-tenant or shared database.

Personally, my usage needs shared database, giving the ability to scale as needed, so I'd like to see the ability to apply a filter to all queries so ensuring tenant isolation can be done at the IDocumentSession level, kind of like the dynamic filtering in EF.

Contributor

JohnCampionJr commented Jun 21, 2016

I think an important question is multi-tenancy via database-per-tenant or shared database.

Personally, my usage needs shared database, giving the ability to scale as needed, so I'd like to see the ability to apply a filter to all queries so ensuring tenant isolation can be done at the IDocumentSession level, kind of like the dynamic filtering in EF.

@CoreyKaylor

This comment has been minimized.

Show comment
Hide comment
@CoreyKaylor

CoreyKaylor Jul 20, 2016

Contributor

@JohnCampionJr I think if you want to restrict it you would be better off to use row level security in PG.

@valeriob How do you envision dealing with the multi-tenancy aspect within the document store, schema diffs, etc if you are isolating via database per tenant?

Contributor

CoreyKaylor commented Jul 20, 2016

@JohnCampionJr I think if you want to restrict it you would be better off to use row level security in PG.

@valeriob How do you envision dealing with the multi-tenancy aspect within the document store, schema diffs, etc if you are isolating via database per tenant?

@valeriob

This comment has been minimized.

Show comment
Hide comment
@valeriob

valeriob Jul 21, 2016

@CoreyKaylor i'd leave it out of the library, you con use the same code running on different schema if they are backward compatible or just spin multiple instances of the application and upgrading the tenants incrementally.
This goes hand in hand with a very powerfull feature of ravendb that is non destructive updates of the json document, where they system does not deletes what it does not knows.

valeriob commented Jul 21, 2016

@CoreyKaylor i'd leave it out of the library, you con use the same code running on different schema if they are backward compatible or just spin multiple instances of the application and upgrading the tenants incrementally.
This goes hand in hand with a very powerfull feature of ravendb that is non destructive updates of the json document, where they system does not deletes what it does not knows.

@jeremydmiller jeremydmiller changed the title from Support for multi tenacity at documentStore level to Support for multi tenancy at documentStore level Aug 4, 2016

@jeremydmiller

This comment has been minimized.

Show comment
Hide comment
@jeremydmiller

jeremydmiller Aug 24, 2016

Contributor

My thoughts on this this morning:

  • I think we keep this out of Marten itself, but change a couple things to make it easier to do yourself
  • For the case of switching on a different database per client, I'd personally handle that outside of Marten with something that can just fetch the correct IDocumentStore for the current tenant, then work from there. I'd do it w/ IoC tricks myself, but it doesn't require that.
  • If you want to just use some kind of TenantId to mark documents as belonging to a tenant, I think we can make the "write" side of this pretty easy if you can easily inject a custom IDocumentSessionListener into the calls to IDocumentStore.OpenSession(). We don't support that today, but that would be a pretty easy change. That would allow users to automatically tag matching entities with the current tenant id in "BeforeCommit" hooks.
  • You might opt into a decorator around IDocumentSession to handle filtering on tenants in Linq queries and also to limit users from loading, deleting, or persisting documents from the wrong tenant. That might be the thing that does force us into native support for multi-tenancy in Marten.
Contributor

jeremydmiller commented Aug 24, 2016

My thoughts on this this morning:

  • I think we keep this out of Marten itself, but change a couple things to make it easier to do yourself
  • For the case of switching on a different database per client, I'd personally handle that outside of Marten with something that can just fetch the correct IDocumentStore for the current tenant, then work from there. I'd do it w/ IoC tricks myself, but it doesn't require that.
  • If you want to just use some kind of TenantId to mark documents as belonging to a tenant, I think we can make the "write" side of this pretty easy if you can easily inject a custom IDocumentSessionListener into the calls to IDocumentStore.OpenSession(). We don't support that today, but that would be a pretty easy change. That would allow users to automatically tag matching entities with the current tenant id in "BeforeCommit" hooks.
  • You might opt into a decorator around IDocumentSession to handle filtering on tenants in Linq queries and also to limit users from loading, deleting, or persisting documents from the wrong tenant. That might be the thing that does force us into native support for multi-tenancy in Marten.
@jeremydmiller

This comment has been minimized.

Show comment
Hide comment
@jeremydmiller

jeremydmiller Feb 1, 2017

Contributor

Alright, so let's assume that this does go into Marten itself. Here's my thoughts this morning:

  • Can we get away with saying that multi-tenancy is going to be done via separate databases only,
    or do we also need to support an option to do it by schema? The mechanics are actually very
    different and the database per tenant is easier in this case.

  • Do we support the creation of databases on the fly for missing tenants? I assume yes depending
    upon the AutoCreate mode

  • This will definitely be in a Marten 2.0 because it's going to cause API changes

Contributor

jeremydmiller commented Feb 1, 2017

Alright, so let's assume that this does go into Marten itself. Here's my thoughts this morning:

  • Can we get away with saying that multi-tenancy is going to be done via separate databases only,
    or do we also need to support an option to do it by schema? The mechanics are actually very
    different and the database per tenant is easier in this case.

  • Do we support the creation of databases on the fly for missing tenants? I assume yes depending
    upon the AutoCreate mode

  • This will definitely be in a Marten 2.0 because it's going to cause API changes

@migajek

This comment has been minimized.

Show comment
Hide comment
@migajek

migajek Feb 1, 2017

Contributor

I'd go the database-per-tenant way, it's more clean approach and in fact doesn't introduce "multitenancy" concept at all, only the extension points to make it possible.

I think the most important yet completely sufficient API surface change would be adding a string databaseName argument to the DocumentStore.OpenSession method

Contributor

migajek commented Feb 1, 2017

I'd go the database-per-tenant way, it's more clean approach and in fact doesn't introduce "multitenancy" concept at all, only the extension points to make it possible.

I think the most important yet completely sufficient API surface change would be adding a string databaseName argument to the DocumentStore.OpenSession method

@phinett

This comment has been minimized.

Show comment
Hide comment
@phinett

phinett Feb 1, 2017

I can only speak for myself, but I've always used database per tenant on past projects so this is fine with me. I prefer the clean separation of data.

Auto creating the database would fit in with same concept as auto creating tables etc, so yes.

phinett commented Feb 1, 2017

I can only speak for myself, but I've always used database per tenant on past projects so this is fine with me. I prefer the clean separation of data.

Auto creating the database would fit in with same concept as auto creating tables etc, so yes.

@pblachut

This comment has been minimized.

Show comment
Hide comment
@pblachut

pblachut Mar 21, 2017

@migajek: I agree that it would be a really nice feature but as far as I know current connection implementation handling (inside DocumentStore based on NpgsqlConnection) does not allow it. It enforces us to specifiy database at the moment of DocumentStore creation.
I'm not sure how it looks if we would like to handle it by using scheme per tenant in single database but I'm sure that it would break the API contract. Maybe it can be done by another implementation of document session with extended interface?

@jeremydmiller: I think before Marten 2.0. it would be possible to add feature which would do 'ioc tricks' as you have mentioned. It'll use database per tenant approach. It can be some kind of document store registry which would be a singleton and would keep all already created tenant document stores (each tenant document store would be created only once). I'll try to add some POC of this solution.

pblachut commented Mar 21, 2017

@migajek: I agree that it would be a really nice feature but as far as I know current connection implementation handling (inside DocumentStore based on NpgsqlConnection) does not allow it. It enforces us to specifiy database at the moment of DocumentStore creation.
I'm not sure how it looks if we would like to handle it by using scheme per tenant in single database but I'm sure that it would break the API contract. Maybe it can be done by another implementation of document session with extended interface?

@jeremydmiller: I think before Marten 2.0. it would be possible to add feature which would do 'ioc tricks' as you have mentioned. It'll use database per tenant approach. It can be some kind of document store registry which would be a singleton and would keep all already created tenant document stores (each tenant document store would be created only once). I'll try to add some POC of this solution.

@jeremydmiller

This comment has been minimized.

Show comment
Hide comment
@jeremydmiller

jeremydmiller Mar 21, 2017

Contributor

@pblachut IoC tricks and effectively a different DocumentStore per tenant where the only thing different is the connection string would definitely work today -- but it's gonna be a lot more efficient to do something more formal within Marten 2.0 to avoid having to create more than one copy of a lot of the underlying objects.

I think the IoC thing would work fine if you had a relatively low number of tenants.

Contributor

jeremydmiller commented Mar 21, 2017

@pblachut IoC tricks and effectively a different DocumentStore per tenant where the only thing different is the connection string would definitely work today -- but it's gonna be a lot more efficient to do something more formal within Marten 2.0 to avoid having to create more than one copy of a lot of the underlying objects.

I think the IoC thing would work fine if you had a relatively low number of tenants.

@jeremydmiller

This comment has been minimized.

Show comment
Hide comment
@jeremydmiller

jeremydmiller May 6, 2017

Contributor

I've split this one out to #749, #750, and #752

Contributor

jeremydmiller commented May 6, 2017

I've split this one out to #749, #750, and #752

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment