Dynamically creating and switching tenants on the fly.

[RustComet]

This is more of a question than an issue.

I have a project that is integrating with Yammer, and where a user can sign in and if there is no schema for their current Yammer organisation then it will create one, and pull in their Yammer connections.

This all seems to work fine in dev, and I cannot recreate my issue no matter how hard I try. But things change when I get to the staging server.

Essentially these are the steps that a user goes through:

1. User hits the yammer oauth and is asked to sign up
2. Yammer callback is initiated and the are redirected back to yammer.mydomain.com/callback_path with yammer being and excluded subdomain.
3. This then makes the call to the oauth controller which searches the users and subdomains to see if they exist
4. If they do not, then it creates a new tenant, and adds the user (along with their connections to that tenant).
5. User is logged in, and they can start using the site.

The first time I ran this it worked fine. But when going through testing I can across an intermittent error.

G::UndefinedTable: ERROR:  relation "admin_users" does not exist at character 13
: INSERT INTO "admin_users" ("created_at", "current_sign_in_at", "current_sign_in_ip", "email", "encrypted_password", "last_sign_in_at", "last_sign_in_ip", "receive_notifications", "remember_created_at", "reset_password_sent_at", "reset_password_token", "sign_in_count", "updated_at") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13) RETURNING "id"
PG::InFailedSqlTransaction: ERROR:  current transaction is aborted, commands ignored until end of transaction block
:           SELECT COUNT(*)
          FROM pg_namespace
          WHERE nspname = 'somethingelse'

PG::InFailedSqlTransaction: ERROR:  current transaction is aborted, commands ignored until end of transaction block
: SET search_path TO "public"
PG::InFailedSqlTransaction: ERROR:  current transaction is aborted, commands ignored until end of transaction block
:           SELECT COUNT(*)
          FROM pg_namespace
          WHERE nspname = 'public'

PG::InFailedSqlTransaction: ERROR:  current transaction is aborted, commands ignored until end of transaction block
: SET search_path TO "public"
   (0.6ms)  ROLLBACK
ActiveRecord::StatementInvalid: PG::InFailedSqlTransaction: ERROR:  current transaction is aborted, commands ignored until end of transaction block
: SET search_path TO "public"

For a while I could not even figure out how to reliably reproduce this error. But I have managed to narrow it down.

From the rails console if I call the User.from_omniauth call, which is the one that sets off the steps above, it will work the first time. But then if I delete that tenant and try again it will throw that error.

However…

If I call reload! then the make the call again it will work correctly as expected.

This is all fine for the console where I know I can call reload. However when running the app on the staging server I get the error above intermittently. I believe it has something to do with the way I am calling Apartment::Tenant.switch and Apartment::Database.process but after playing with this for a few days I have been unable to make progress.

I know that it is a slight edge case. But for the product to enter production I will need to be able to resolve.

Is there any way that this sort of creation could break apartment?

[RustComet]

UPDATE:
After a lot of trying and failing (and just a little bit of swearing at the computer) I have found a workaround.

Just before starting the creation of the tenant, I have made a call to restart connections

Tenant.restart_connections

  def self.restart_connections
    ActiveRecord::Base.connection_pool.disconnect!
    ActiveSupport.on_load(:active_record) do
      config = Rails.application.config.database_configuration[Rails.env]
      config['reaping_frequency'] = ENV['DB_REAP_FREQ'] || 10 # seconds
      config['pool']              = ENV['DB_POOL'] || 5
      ActiveRecord::Base.establish_connection(config)
    end
  end

This just makes sure that the DB is looking in the right place and the connections are all correct before beginning the transaction.

Hope this helps somebody else that comes across this edge case.

[hesalx]

I know this error.
I once wrote some tenant system on Postgres. The problem is here https://github.com/influitive/apartment/blob/v0.25.2/lib/apartment/adapters/abstract_adapter.rb#L74ract_adapter.rb#L74
When an error occurs, rails will rollback a transaction in a very end of request. Thought, the method with ensure is called somewhere in the middle, as the ensure itself. Thus, SET search_path ... is called inside a broken transaction. The actual error becomes very self-describing now.

The straightforward solution I have found is to execute ROLLBACK on exception. It is safe to execute even outside of transaction, it will do nothing in this case.

  def within_schema
    connection.schema_search_path = new_schema
    yield # business staff
  rescue
    ActiveRecord::Base.connection.execute 'ROLLBACK'
  ensure
    connection.schema_search_path = previous_schema
  end

I do not know side-effects of this solution and can not guarantee it is right and perfect. However is works in my circumstances and allow test cases pass.

[tonyfung99]

Sorry I would like to ask about the handling on creating the tenant. The yammer subdomain is one of the "fake" tenant (which means it included all the functions and data and can work as an instance) that in charge of creating new tenant? Or it is set of specific controllers used to create tenant?

I am looking for a way to do the "tenant generate console". The reason I asked that is I cannot find a way to exclude part of the controllers out of apartment. Please correct me If i am wrong.