Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

schema_format :sql should behave like schema_format :ruby #2948

Merged
merged 1 commit into from

13 participants

Alex Tambellini Sam Goldman Oleg Dashevskii John Weathers Tony Collen Nick Gauthier Mike Subelsky Gopal Patel Juan M. Cuello Derek Prior Ankur Sethi Morgan Christiansson Aaron Patterson
Alex Tambellini

This commit adds a db:structure:load task that is run instead of
db:schema:load when schema_format is set to :sql. This patch also removes
the prefixing of the structure.sql files to mimic the use of a single
schema.rb file. The patch originates from github issue #715.

Oleg Dashevskii
be9 commented

+1

Tony Collen

+1 here as well, although it looks like the patch doesn't apply cleanly against master. Any chance to get a more up-to-date patch?

Alex Tambellini

@tonyc, just updated the patch, 50c2ca7 should apply against master now.

Tony Collen
Tony Collen

Playing around with this patch a bit, I don't think it's quite right. When I run "rake," I get errors about duplicate constraints and tables. Rails.env within the tasks is set to "development", so it's trying to pipe structure.sql into my development database for tests, which is wrong, since we're running tests. We can't just hardcode the task to use abcs['test']['database'] though, because then we lose the ability to manually run "rake db:structure:load" and have the structure sucked into the development DB.

Tony Collen

@atambo, I've made some changes that seem to work for me so far, at least while running 'rake' . I'm going to test it with other various tasks to see if it's working correctly.

Edit: Here's a squashed changeset with mine applied atop of @atambo's:

tonyc/rails@e40a0f3

Tony Collen

FWIW, a coworker and I have been using my squashed changeset in a fork of Rails for a number of weeks now without issues, and I'm fairly happy with how it's working so far.

Alex Tambellini atambo schema_format :sql should behave like schema_format :ruby
This commit adds a db:structure:load task that is run instead of
db:schema:load when schema_format is set to :sql. This patch also removes
the prefixing of the structure.sql files to mimic the use of a single
schema.rb file. The patch originates from github issue #715.
15fb430
Alex Tambellini

I've cherry-picked @tonyc's changes.

Aaron Patterson tenderlove merged commit 43821bf into from
Tony Collen

Thanks! <3 <3

Nick Gauthier

OMG!

Mike Subelsky

this is awesome, will definitely save me time/pain in a constraint-heavy app I'm working on

Gopal Patel

Will this be backported to a 3.1.x release?

Juan M. Cuello

I think this commit introduces some issues.

Previously to these changes, when I executed rake db:test:clone_structure
the structure was dumped (to structure.sql) and then loaded again no matter whether
the config.active_record.schema_format was set to :sql or :ruby. That was OK
because I'm explicitly saying that I want the structure to be cloned.

With these changes, when I call now the same task, although the structure is
dumped, the loaded schema depends on the value of the
config.active_record.schema_format. So If I have it set to :ruby and then I
call rake db:test:clone_structure the loaded file is schema.rb and not the
structure.sql how it is supposed to be according to the name of the task and
its description.

The same thing happens when I call rake db:test:clone. When called,
a new schema.rb is created but the file used to load the schema depends on
config.active_record.schema_format. If I have it set to :sql, a new schema.rb
is created but the structure.sql file is used to create the database. So it is
not consistent.

I didn't create an issue/pull-request because I wanted to discuss it here first,
but in case we decide to make some changes, I think we could do something like this:

Juanmcuello@88a4aea

Derek Prior

If I submitted a pull request that applied cleanly to 3.1-stable, would it be accepted there? I'd love to see these changes make it into the next 3.1 patch while we await 3.2.0 final.

Juan M. Cuello

@derekprior, this was already fixed in #4036. If you want the fix to be included in next release I think you will have to create a pull request to backport it.

Ankur Sethi

@atambo I am on 3.2.3 and this doesn't work.

This is in my application.rb: config.active_record.schema_format = :sql

This is the rake trace:

** Invoke db:setup (first_time)
** Invoke db:schema:load_if_ruby (first_time)
** Invoke db:create (first_time)
** Invoke db:load_config 
** Execute db:create
** Execute db:schema:load_if_ruby
** Invoke db:schema:load (first_time)

In the console:

ActiveRecord::Base.schema_format
=> :sql

I also confirmed that the application.rb is loaded by the rake task.

Alex Tambellini

@amalagaura, it looks like db:setup is broken when using schema_format :sql. For now you should be able to use db:create db:migrate db:seed db:reset db:drop etc, just not db:setup.

Morgan Christiansson morganchristiansson commented on the diff
activerecord/lib/active_record/railties/databases.rake
((31 lines not shown))
else
raise "Task not supported by '#{abcs[Rails.env]["adapter"]}'"
end
if ActiveRecord::Base.connection.supports_migrations?
- File.open("#{Rails.root}/db/#{Rails.env}_structure.sql", "a") { |f| f << ActiveRecord::Base.connection.dump_schema_information }

postgres pg_dump doesn't reset it's schema so these INSERT statements will fail if the search_path has not been set to include public at end of migration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Juan M. Cuello

Not sure if it's what you mean, but I think pull request #4132 already fixed it.

See also this line in master.

Morgan Christiansson

Awesome, yes that should have fixed it.

Can't wait for the next rails release for this change - as currently migrating 12 databases with parallel tests in our CI takes 75 seconds (67% of the total setup cost) and this should reduce that quite a lot.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Nov 4, 2011
  1. Alex Tambellini

    schema_format :sql should behave like schema_format :ruby

    atambo authored
    This commit adds a db:structure:load task that is run instead of
    db:schema:load when schema_format is set to :sql. This patch also removes
    the prefixing of the structure.sql files to mimic the use of a single
    schema.rb file. The patch originates from github issue #715.
This page is out of date. Refresh to see the latest.
Showing with 66 additions and 38 deletions.
  1. +66 −38 activerecord/lib/active_record/railties/databases.rake
104 activerecord/lib/active_record/railties/databases.rake
View
@@ -151,6 +151,7 @@ db_namespace = namespace :db do
ActiveRecord::Migration.verbose = ENV["VERBOSE"] ? ENV["VERBOSE"] == "true" : true
ActiveRecord::Migrator.migrate(ActiveRecord::Migrator.migrations_paths, ENV["VERSION"] ? ENV["VERSION"].to_i : nil)
db_namespace["schema:dump"].invoke if ActiveRecord::Base.schema_format == :ruby
+ db_namespace["structure:dump"].invoke if ActiveRecord::Base.schema_format == :sql
end
namespace :migrate do
@@ -174,6 +175,7 @@ db_namespace = namespace :db do
raise 'VERSION is required' unless version
ActiveRecord::Migrator.run(:up, ActiveRecord::Migrator.migrations_paths, version)
db_namespace['schema:dump'].invoke if ActiveRecord::Base.schema_format == :ruby
+ db_namespace['structure:dump'].invoke if ActiveRecord::Base.schema_format == :sql
end
# desc 'Runs the "down" for a given migration VERSION.'
@@ -182,6 +184,7 @@ db_namespace = namespace :db do
raise 'VERSION is required' unless version
ActiveRecord::Migrator.run(:down, ActiveRecord::Migrator.migrations_paths, version)
db_namespace['schema:dump'].invoke if ActiveRecord::Base.schema_format == :ruby
+ db_namespace['structure:dump'].invoke if ActiveRecord::Base.schema_format == :sql
end
desc 'Display status of migrations'
@@ -222,6 +225,7 @@ db_namespace = namespace :db do
step = ENV['STEP'] ? ENV['STEP'].to_i : 1
ActiveRecord::Migrator.rollback(ActiveRecord::Migrator.migrations_paths, step)
db_namespace['schema:dump'].invoke if ActiveRecord::Base.schema_format == :ruby
+ db_namespace['structure:dump'].invoke if ActiveRecord::Base.schema_format == :sql
end
# desc 'Pushes the schema to the next version (specify steps w/ STEP=n).'
@@ -229,10 +233,14 @@ db_namespace = namespace :db do
step = ENV['STEP'] ? ENV['STEP'].to_i : 1
ActiveRecord::Migrator.forward(ActiveRecord::Migrator.migrations_paths, step)
db_namespace['schema:dump'].invoke if ActiveRecord::Base.schema_format == :ruby
+ db_namespace['structure:dump'].invoke if ActiveRecord::Base.schema_format == :sql
end
# desc 'Drops and recreates the database from db/schema.rb for the current environment and loads the seeds.'
- task :reset => [ 'db:drop', 'db:setup' ]
+ task :reset => :environment do
+ db_namespace["drop"].invoke
+ db_namespace["setup"].invoke
+ end
# desc "Retrieves the charset for the current environment's database"
task :charset => :environment do
@@ -285,7 +293,12 @@ db_namespace = namespace :db do
end
desc 'Create the database, load the schema, and initialize with the seed data (use db:reset to also drop the db first)'
- task :setup => [ 'db:create', 'db:schema:load', 'db:seed' ]
+ task :setup => :environment do
+ db_namespace["create"].invoke
+ db_namespace["schema:load"].invoke if ActiveRecord::Base.schema_format == :ruby
+ db_namespace["structure:load"].invoke if ActiveRecord::Base.schema_format == :sql
+ db_namespace["seed"].invoke
+ end
desc 'Load the seed data from db/seeds.rb'
task :seed => 'db:abort_if_pending_migrations' do
@@ -360,81 +373,96 @@ db_namespace = namespace :db do
case abcs[Rails.env]['adapter']
when /mysql/, 'oci', 'oracle'
ActiveRecord::Base.establish_connection(abcs[Rails.env])
- File.open("#{Rails.root}/db/#{Rails.env}_structure.sql", "w+") { |f| f << ActiveRecord::Base.connection.structure_dump }
+ File.open("#{Rails.root}/db/structure.sql", "w:utf-8") { |f| f << ActiveRecord::Base.connection.structure_dump }
when /postgresql/
ENV['PGHOST'] = abcs[Rails.env]['host'] if abcs[Rails.env]['host']
- ENV['PGPORT'] = abcs[Rails.env]["port"].to_s if abcs[Rails.env]['port']
+ ENV['PGPORT'] = abcs[Rails.env]['port'].to_s if abcs[Rails.env]['port']
ENV['PGPASSWORD'] = abcs[Rails.env]['password'].to_s if abcs[Rails.env]['password']
+ ENV['PGUSER'] = abcs[Rails.env]['username'].to_s if abcs[Rails.env]['username']
search_path = abcs[Rails.env]['schema_search_path']
unless search_path.blank?
search_path = search_path.split(",").map{|search_path_part| "--schema=#{search_path_part.strip}" }.join(" ")
end
- `pg_dump -i -U "#{abcs[Rails.env]['username']}" -s -x -O -f db/#{Rails.env}_structure.sql #{search_path} #{abcs[Rails.env]['database']}`
+ `pg_dump -i -s -x -O -f db/structure.sql #{search_path} #{abcs[Rails.env]['database']}`
raise 'Error dumping database' if $?.exitstatus == 1
when /sqlite/
dbfile = abcs[Rails.env]['database'] || abcs[Rails.env]['dbfile']
- `sqlite3 #{dbfile} .schema > db/#{Rails.env}_structure.sql`
+ `sqlite3 #{dbfile} .schema > db/structure.sql`
when 'sqlserver'
- `smoscript -s #{abcs[Rails.env]['host']} -d #{abcs[Rails.env]['database']} -u #{abcs[Rails.env]['username']} -p #{abcs[Rails.env]['password']} -f db\\#{Rails.env}_structure.sql -A -U`
+ `smoscript -s #{abcs[Rails.env]['host']} -d #{abcs[Rails.env]['database']} -u #{abcs[Rails.env]['username']} -p #{abcs[Rails.env]['password']} -f db\\structure.sql -A -U`
when "firebird"
set_firebird_env(abcs[Rails.env])
db_string = firebird_db_string(abcs[Rails.env])
- sh "isql -a #{db_string} > #{Rails.root}/db/#{Rails.env}_structure.sql"
+ sh "isql -a #{db_string} > #{Rails.root}/db/structure.sql"
else
raise "Task not supported by '#{abcs[Rails.env]["adapter"]}'"
end
if ActiveRecord::Base.connection.supports_migrations?
- File.open("#{Rails.root}/db/#{Rails.env}_structure.sql", "a") { |f| f << ActiveRecord::Base.connection.dump_schema_information }

postgres pg_dump doesn't reset it's schema so these INSERT statements will fail if the search_path has not been set to include public at end of migration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ File.open("#{Rails.root}/db/structure.sql", "a") { |f| f << ActiveRecord::Base.connection.dump_schema_information }
end
end
- end
-
- namespace :test do
- # desc "Recreate the test database from the current schema.rb"
- task :load => 'db:test:purge' do
- ActiveRecord::Base.establish_connection(ActiveRecord::Base.configurations['test'])
- ActiveRecord::Schema.verbose = false
- db_namespace['schema:load'].invoke
- end
- # desc "Recreate the test database from the current environment's database schema"
- task :clone => %w(db:schema:dump db:test:load)
+ # desc "Recreate the databases from the structure.sql file"
+ task :load => [:environment, :load_config] do
+ env = ENV['RAILS_ENV'] || 'test'
- # desc "Recreate the test databases from the development structure"
- task :clone_structure => [ 'db:structure:dump', 'db:test:purge' ] do
abcs = ActiveRecord::Base.configurations
- case abcs['test']['adapter']
+ case abcs[env]['adapter']
when /mysql/
- ActiveRecord::Base.establish_connection(:test)
+ ActiveRecord::Base.establish_connection(abcs[env])
ActiveRecord::Base.connection.execute('SET foreign_key_checks = 0')
- IO.readlines("#{Rails.root}/db/#{Rails.env}_structure.sql").join.split("\n\n").each do |table|
+ IO.readlines("#{Rails.root}/db/structure.sql").join.split("\n\n").each do |table|
ActiveRecord::Base.connection.execute(table)
end
when /postgresql/
- ENV['PGHOST'] = abcs['test']['host'] if abcs['test']['host']
- ENV['PGPORT'] = abcs['test']['port'].to_s if abcs['test']['port']
- ENV['PGPASSWORD'] = abcs['test']['password'].to_s if abcs['test']['password']
- `psql -U "#{abcs['test']['username']}" -f "#{Rails.root}/db/#{Rails.env}_structure.sql" #{abcs['test']['database']} #{abcs['test']['template']}`
+ ENV['PGHOST'] = abcs[env]['host'] if abcs[env]['host']
+ ENV['PGPORT'] = abcs[env]['port'].to_s if abcs[env]['port']
+ ENV['PGPASSWORD'] = abcs[env]['password'].to_s if abcs[env]['password']
+ ENV['PGUSER'] = abcs[env]['username'].to_s if abcs[env]['username']
+ `psql -f "#{Rails.root}/db/structure.sql" #{abcs[env]['database']} #{abcs[env]['template']}`
when /sqlite/
- dbfile = abcs['test']['database'] || abcs['test']['dbfile']
- `sqlite3 #{dbfile} < "#{Rails.root}/db/#{Rails.env}_structure.sql"`
+ dbfile = abcs[env]['database'] || abcs[env]['dbfile']
+ `sqlite3 #{dbfile} < "#{Rails.root}/db/structure.sql"`
when 'sqlserver'
- `sqlcmd -S #{abcs['test']['host']} -d #{abcs['test']['database']} -U #{abcs['test']['username']} -P #{abcs['test']['password']} -i db\\#{Rails.env}_structure.sql`
+ `sqlcmd -S #{abcs[env]['host']} -d #{abcs[env]['database']} -U #{abcs[env]['username']} -P #{abcs[env]['password']} -i db\\structure.sql`
when 'oci', 'oracle'
- ActiveRecord::Base.establish_connection(:test)
- IO.readlines("#{Rails.root}/db/#{Rails.env}_structure.sql").join.split(";\n\n").each do |ddl|
+ ActiveRecord::Base.establish_connection(abcs[env])
+ IO.readlines("#{Rails.root}/db/structure.sql").join.split(";\n\n").each do |ddl|
ActiveRecord::Base.connection.execute(ddl)
end
when 'firebird'
- set_firebird_env(abcs['test'])
- db_string = firebird_db_string(abcs['test'])
- sh "isql -i #{Rails.root}/db/#{Rails.env}_structure.sql #{db_string}"
+ set_firebird_env(abcs[env])
+ db_string = firebird_db_string(abcs[env])
+ sh "isql -i #{Rails.root}/db/structure.sql #{db_string}"
else
- raise "Task not supported by '#{abcs['test']['adapter']}'"
+ raise "Task not supported by '#{abcs[env]['adapter']}'"
+ end
+ end
+ end
+
+ namespace :test do
+ # desc "Recreate the test database from the current schema.rb"
+ task :load => 'db:test:purge' do
+ ActiveRecord::Base.establish_connection(ActiveRecord::Base.configurations['test'])
+ ActiveRecord::Schema.verbose = false
+ db_namespace["schema:load"].invoke if ActiveRecord::Base.schema_format == :ruby
+
+ begin
+ old_env, ENV['RAILS_ENV'] = ENV['RAILS_ENV'], 'test'
+ db_namespace["structure:load"].invoke if ActiveRecord::Base.schema_format == :sql
+ ensure
+ ENV['RAILS_ENV'] = old_env
end
+
end
+ # desc "Recreate the test database from the current environment's database schema"
+ task :clone => %w(db:schema:dump db:test:load)
+
+ # desc "Recreate the test databases from the structure.sql file"
+ task :clone_structure => [ "db:structure:dump", "db:test:load" ]
+
# desc "Empty the test database"
task :purge => :environment do
abcs = ActiveRecord::Base.configurations
Something went wrong with that request. Please try again.