-
Notifications
You must be signed in to change notification settings - Fork 424
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New architecture does not seem to allow centralized django.contrib.auth #43
Comments
Hi Ted, even though the docs says auth is required on 2013/5/1 Ted Chen notifications@github.com
Bernardo Pires Carneiro |
Bernardo, --- On Wed, 5/1/13, Bernardo Pires Carneiro notifications@github.com wrote: From: Bernardo Pires Carneiro notifications@github.com Hi Ted, even though the docs says auth is required on should actually work without it. What error are you getting when running ` sync_schemas`? 2013/5/1 Ted Chen notifications@github.com
Bernardo Pires Carneiro — |
Previously you did need auth, contenttypes and sites, but now you shouldn't 2013/5/1 Ted Chen notifications@github.com
Bernardo Pires Carneiro |
Bernardo, --- On Wed, 5/1/13, Bernardo Pires Carneiro notifications@github.com wrote: From: Bernardo Pires Carneiro notifications@github.com Previously you did need auth, contenttypes and sites, but now you shouldn't need it any longer because the tenant and public schemas are pretty much completely independent now when being synced. Only TENANT_APPS get synced to the tenants and only SHARED_APPS are synced to public. Was the issue you were seeing with the DB using the latest django-tenant-schemas version? If yes, I'd love if you could reproduce this error and we can work on it together. I want this repo to be as flexible as possible! 2013/5/1 Ted Chen notifications@github.com
Bernardo Pires Carneiro — |
Hey Ted, did you get everything working? Bernardo |
Bernardo, Thanks for getting back to me. I did a pip install of django-tenant-schemas last week. Initially, I thought everything was fine. However, I encountered errors doing a manage.py sync_schemas and upoon the creation of a new schema (using the tenantmixin). In settings.py, I included contenttypes and auth in 'SHARED_APPS' only. I have copied the traces for the sync_schema and database error (when I tried creating a new schema) below. Apparently, the new schema was successfully created, but I encountered a database error. The sync_schema error suggests to me that there is a problem if contenttypes is in SHARED_APPS (and not in TENANT_APPS). Also, I speculate that the database error that I encountered is related, Would appreciate your view! Regards, Ted sync_schemas error trace:Bernardo, There could be an issue. I ran a python manage.py sync_schemas and I ran into this problem. It seemed to do a sync with my public schema but ran into a problem when sync'ing in my tenant, which I named "demo". Here's the trace. Could it be that you still require django_content_type in your tenant_apps? I placed it only in my shared_apps: (v_productcentral)ted@productcentralhost:~/productcentral$ python manage.py sync_schemas Database error trace is here:DatabaseError at /createtenant_verify/ /home/ted/.virtualenvs/v_productcentral/local/lib/python2.7/site-packages/django/core/handlers/base.py |
Hi Ted, many thanks for the report and the traces! I have encountered this error before and you are completely right. Because content_types is listed on Bernardo |
I ran into the same issue with "auth_user". I.e. "DatabaseError: relation "auth_user" does not exist" when django.contrib.auth was only in SHARED_APPS as per the latest Documentation. I got past that by adding auth to TENANT_APPS. And then the same error was reported for "django_site". So in the end I had to add the following to tenant apps in addition to the actual tenant app: 'django.contrib.contenttypes', Might be that I have to add more apps to TENANT_APPS at some point but for now I got sync_schemas to go through with these. |
I'm facing the same problem, but in my case I only need to add the django.contrib.auth to the TENANT_APPS. Why does it happen? If I do that I'll have 2 user tables (one public and one per tenant), how is the authentication managed? which table does the work? |
Hi Bernardo, My purpose is creating a multitenant application able to serve several companies. I've been reading a lot of documentation and for me, the best approach is using multi-schemas, that's what make me arrive to your framework.
In my tenant schemas I need to have some models with ForeigKey to Agent Model, which must be in the public schema. First of all, is it possible to have relations between public and tenant specific schemas? At the moment I'm not able to get that working, I have the following content in INSTALLED_APPS: SHARED_APPS = (
) TENANT_APPS = ( INSTALLED_APPS = SHARED_APPS + TENANT_APPS The second question that I have is which is the best way to create the public Tenant. As far as it is mandatory prior to acces to any url, i'd like to create it automatically the first time that I run the server. Which is the best place to do that? Everything runs ok with the public schema, but when I create a new Company (tenant model) the syncronization of the new schema works (it creates all the tables) but django fails with the following stack trace: http://dpaste.com/1225585/ So it seems that need the auth application also in TENANT_APPS, but why? Please @bcarneiro can you help me with that? |
Hello all, I will look into this issue as soon as I have the time. Bernardo
|
I think this is happening because if you put One quick and hacky "solution" is to put auth into both SHARED_APPS and TENANT_APPS and ALTER TABLE so primary keys point to public schema instead of the current schema, and DELETE Another way to explicitly modify the SQL queries FOREIGN KEYS during sync for app models that have FOREIGN KEYS to SHARED_APPS models to point to public schema. |
I just reproduced this issue by simply following the tutorial (only There may be a way to override the signal/events to stop this behavior, but 2013/8/14 Walkman notifications@github.com
Bernardo Pires Carneiro |
I think we are talking about the same thing, I just wasn't clear enough so please clarify this, because I think I'm ready with the solution, just need to make sure it's right 😄 So, let's suppose CREATE TABLE "django_admin_log" (
"id" serial NOT NULL PRIMARY KEY,
"action_time" timestamp with time zone NOT NULL,
"user_id" integer NOT NULL REFERENCES "auth_user" ("id") DEFERRABLE INITIALLY DEFERRED,
"content_type_id" integer REFERENCES "django_content_type" ("id") DEFERRABLE INITIALLY DEFERRED,
"object_id" text,
"object_repr" varchar(200) NOT NULL,
"action_flag" smallint CHECK ("action_flag" >= 0) NOT NULL,
"change_message" text NOT NULL
)
; but because
So one possible solution is to modify the query, and make an explicit reference to the |
I keep running into this problem of not being able to write to the admin log (and other similar situations) as well. It seems to happen when I get my application in an odd state. I often find myself dropping tables and re-syncing them in the earlier stages of development (rather than doing migrations all the time, since they can get tricky). I've found that it's usually because I do the sync / south stuff in the wrong order. At any rate, thus far I've found the django admin stuff to be the most tricky part of getting this module working, so glad to see this part of the application get some attention. |
@walkman, I'm not sure if manually patching the query is the best way to solve this. I think one of the cool things we do on this app is not having to edit any query, we simply set the One soluton might be to patch Just to make it clear, what exactly are you trying to achieve here? Just that it works with these settings or to allow |
If you put an app to |
Ok, so what you want to do is to allow a tenant app to reference a shared What does your code do when I have Personally I think a better solution would be to have a special foreign What do you guys think? If you put an app to SHARED_APPS, it means you want every other application — |
You are right, I didn't think about this case, but the logically expected behavior is to reference tables in the schema, not on public, right? It's easy to do either.
We should approach this topic consistently, I mean if the default way with this app to make things public or tenant by SETTINGS, we should stick to it. (Other approach could be setting these properties in models SHARED_MODELS could be FORCED_PUBLIC_MODELS or FORCED_PUBLIC_APPS, whatever you suggest or we agree on if you like this idea.
Actually, PUBLIC_SCHEMA_URLCONF messes things up quite a bit, because if a user put something into |
Yes, the default should be referencing tables on the tenant, not on public. Personally I'm not a big fan of adding another settings dicitonary to specify which models should be referenced on public. This is the purpose of your Wouldn't a |
Yes, but one-by-one. So you can put an app into TENANT_APPS, but only one model from that app into the public schema.
If I understand correctly, it should be specified in the Model, but I thought the whole point of this app is that Django Models are not aware that they are using a multi-schema database, and no modification needed for any app. So if we would make a new field
That's why I think a setting would be better. |
* Added shared_models list to connection * Switched DatabaseCreation to the new schema-aware implementation * Removed example.com creation signal from tenant creation
* Added shared_models list to connection * Switched DatabaseCreation to the new schema-aware implementation * Removed example.com creation signal from tenant creation
I might have found a solution to make |
Hey @surfer26th @vmaatta @Barnalite and others interested, I've just pushed a new branch called |
I'm afraid I'm not using tenant-schemas in anything right now so I can't be Ville On 2 Nov 2013, at 15:02, Bernardo Pires notifications@github.com wrote: Hey guys @surfer26th https://github.com/surfer26th I've just pushed a new branch called experimental-appcache. I did get — |
I've found another solution for this problem, this time using a database router. The branch is called DATABASE_ROUTERS = (
'tenant_schemas.routers.TenantSyncRouter',
) This is a much better solution than manually patching |
Man, |
* Added shared_models list to connection * Switched DatabaseCreation to the new schema-aware implementation * Removed example.com creation signal from tenant creation
I played with Django database routers quite a bit and finished my shared_models feature. Here are my thoughts: However my implementation in I think it's very common scenario that someone want a centralized User table, but want different Groups, even Permissions for every tenant, that's why I think shared_models is an essential feature. |
* Added shared_models list to connection * Switched DatabaseCreation to the new schema-aware implementation * Removed example.com creation signal from tenant creation
* Added shared_models list to connection * Switched DatabaseCreation to the new schema-aware implementation * Removed example.com creation signal from tenant creation
Just a quick question walkman, how are the permissions for the users going |
Anything can be created anywhere, you specify it with the settings. For example: |
Sounds awesome, I'll try it as soon as I can. :) 2013/11/21 Kiss György notifications@github.com
Bernardo Pires Carneiro |
I am sorry that I have not responded earlier. I've been very busy getting my second release out, which is a SaaS. Tenant Schemas is a hugely strategically important component of my platform. I cannot emphasize that enough. My Tenant Schema fork is something I scraped off of Bernardo's github page well over a year ago. My SaaS has been operating online for a few months, albeit with a few users. However, with my second release, I expect to get a lot more users. Because of MY need for centralized auth, I have had to stick with the old version of Tenant Schemas and I have not been able to get South working. With my expectation that I will continue to evolve my SaaS with a much larger number of users in 2014, I have a recurring nightmare that I will not be able to manage my site without getting database migrations (i.e. South) working. Therein lies my renewed interest in helping move this forward. I thought that it would be important to portray my Use Case, since I think it is a common one for SaaS. My need for Centralized User: How have I managed up to now: After I do the first syncdb (which populates Public with all models (tables)), I run a script that deletes the tables that were created in Public, but that are needed only in each Tenant. From that point on, whenever a Tenant is created (by a user),Tenant-create is invoked (using TenantMixin) and the tenant models (and ONLY the tenant-specific models) are created properly in the tenant. This seems to be working well. I have tried to get South working, but (using my old Tenant Schemas fork) could not get the South table installed in both Public and each tenant schema. So, up to now, I haven't been using South. The good news is that I have proven that I can have centralized User administration and that my shared apps (in Public) are working fine in combination with my Tenant apps. I do NOT do a foreign-key from any of my tenant models to any model in Public, like Barnalite is trying to do. My Contemplated Next Steps: (1) in settings.py, place 'django.contrib.contenttypes', 'django.contrib.auth' into BOTH Shared Apps and Tenant Apps If this works with my App, and if South migration works, then this should provide me a solution that I will be able to live with. Bernardo, Walkman – Do either of you have any insights about the merits/dismerits of my plan? Also, would Walkman's "Shared Models" work for my Use Case, and should I be using that instead, even though I don't think I need all the capabilities that Shared_Models has to offer? I sincerely appreciate all the hard work on this strategically important code. Ted [Also, BTW my app uses Celery for async tasks, and I have proven that Celery workers spawned from a tenant app can work. I do this by setting the connection at the top of the Task and closing the connection at the bottom
at the bottom, I do this:
|
Bernardo, I have re-read this thread multiple times, but just began to take a serious look at your experimental-db-router branch. For my needs, at least, it looks promising. I will play around with it. I am concerned about Walkman's last comment about South not working with his SHARED_MODELS. Ted |
If you don't need to share only one model from an app, the router approach is just perfect. And simple. |
Walkman -- Thank you for your insight on the router approach. I wanted to share with you my approach on permissions, since it may be applicable to you. In looking at the Use Case you are trying to do -- assigning permissions in Tenant and foreign Key to Users -- I do the same thing, I believe, but have all that in Public Apps. Let me be more specific: I have created a Membership model (in Public Apps) that foreign key's to the Tenant model and also to User (both of which are also in Public). Membership model designates which users are assigned to which Tenant. The account creator for each Tenant is the de facto admin for the Tenant, and is the person who invites members to his/her Tenant (schema) and assigns permissions. Permissions is a field within the Membership model. In contrast, the apps and models that I have placed in Tenant_apps are the ones that contain absolutely proprietary user data (that the user uploads from their ERP systems) and that absolutely must be "protected" and isolated from those of other Tenants. Also, by placing those models in each Tenant, it greatly improves the manageability of things for me and reduces the risk of a really bad thing happening to the data. I cannot imagine needing to manage humongous tables consisting of every Tenants' large chunks of proprietary data and the fragmentation problems that would invariably occur and all the management challenges and nightmares. This is where the technology strategy of using Postgresql schemas and Django Tenant-Schema plays such an important role. Ted |
Bernardo -- I have been playing around with the experimental_db_router branch for my SaaS on my development machine. That's my code that requires centralized auth, as described above. Everything seems to be operating well so far. I am able to place all these ONLY in SHARED_APPS: ('django.contrib.auth','django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.messages','django.contrib.staticfiles', 'django.contrib.admin') Also, I have successfully created the south_migrationhistory tables in both the 'Public' schema and in a Tenant schema (by adding South in SHARED_APPS and TENANT_APPS and by doing a manage.py sync_schemas). Since I have existing tables in my SaaS in both Public as well as multiple Tenant schemas, and since I need to convert to South, I would like to add to the tenant_schemas management commands a command to Convert_to_South. I will create it (using the migrate_schemas command as a template) and will try it with my development system. If it appears to work, I will submit a pull request to add convert_to_south_schemas.py. I still have more verification to do, in addition to creating and running the convert_to_south_schemas command. Hopefully, I will be able to report back status early this week. |
Continued to work with the experimental_db_router branch on my development environment. Good news: I was able to create a new Tenant on-the-fly (using TenantMixin) and using Django 1.5.5 (I can't use Django 1.6, since it creates errors with my software which I don't have time to sort out right now). Also, the south_migrationhistory table in my new Tenant was properly faked. Another Good News: I created a command to Convert_to_South, as discussed above. It worked properly on my existing models within an existing Tenant! I will create a pull-request to add the command with a caution: I used as a template the migrate_schemas command, and to be honest, I don't have a clue about parts of the code. Not So Good News (or warning for people): I initially tried to create a Tenant with my software using Django 1.4.5. But I encountered a database error (trace shown below). I tried modifying the tenant-schema commands to try to isolate the problem, but to no avail. The thing that worked was moving up to Django 1.5.5. My trace is as follows: DatabaseError at /createtenant_verify/ /home/ted/Envs/v_dev/local/lib/python2.7/site-packages/django/core/handlers/base.py in get_response /home/ted/Envs/v_dev/local/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py in execute |
Reading the beginning of this issue/thread (which is still open) one gets the idea that centralized auth is not possible. But then reading the end it seemed that it was a work in progress; but the last post is almost a year ago. What is the status at the moment? |
Would allowing the user to change the search order of private vs public schemas help this problem at all? #205 |
This should now be possible thanks to the database router. I've added more tests ensuring this works. Foreign keys constraints however do not get created. Let me know how this goes for you guys! |
@bernardopires Thanks for this. The TenantSyncRouter probably needs allow_migrate(), I saw some warnings about this when migrating to 1.7. From the 1.7 release notes:
|
I guess we'll need to keep both because of 1.6, thanks for the heads up tpyo! Do you want to send a pull request that adds allow_migrate() performing exactly like allow_syncdb()? |
Happy to: #217 |
Bernardo - I forked your experimental branch (experimental db router) that you created in late 2013 ("fixed sync_schemas and migrate_schemas not taking arguments") and have been using it with Django 1.5.5 on my production server with success since. I have done quite a few migrations in development and in production successfully. As I am updating/upgrading my software, I hope to get back in sync with your master. Thank you for your continued work on this and specifically for closing this issue out! Thanks to tpyo for the warning on Django 1.7 and migrations. I will not upgrade to Django 1.7, but hope to upgrade to 1.6. |
Hi Ted @surfer26th, I'm now using the router in the master so you should have no problems upgrading! Let me know if you have any issues. Thanks for your continued suppport! |
It's been awhile since I inspected tenant_schemas. I've been using a branch successfully in my SaaS. I wanted to check out the latest commit. In doing so, however, the latest software seems to not work with my requirements.
I require that django.contrib.auth exists in public, since I want centralized authorization and billing (my stripe-based billing app needs to join in with the auth_user table). However, it seems that the latest tenant_schema code requires that auth also be placed in TENANT_APPS (in addition to optionally in SHARED_APPS). Otherwise, I get a DB error when doing sync_schemas. However, when I do that and successfully do a sync_schemas, my users (for which there exists an auth table in public) are not authorized to login into a tenant. I suppose I would need to somehow synchronize the user in each tenant with auth_user in public. But that would be too unwieldy.
I would like to use the latest commit. With my current branch of tenant_schemas (forked 9 months ago), whenever I do a syncdb, I need to delete tables in public that should only be in each tenant. Your latest commit seems to have a solution for that using sync_db.
Any thoughts?
The text was updated successfully, but these errors were encountered: