Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic support for adding and removing foreign keys #15606

Merged
merged 18 commits into from Jun 26, 2014

Conversation

@senny
Copy link
Member

senny commented Jun 10, 2014

We had this as a GSoC idea but there was no accepted student to work on it. Because I was really looking forward to see this in rails/rails I made a first draft.

This patch adds basic support for adding and removing foreign keys using the migration DSL:

add_foreign_key :from, :to # will add the key to `to_id`
add_foreign_key :from, :to, column: "fk_id" # with specific column

add_foreign_key :from, :to, on_delete: :restrict # to specify ON DELETE RESTRICT
add_foreign_key :from, :to, on_delete: :cascade # to specify ON DELETE CASCADE
add_foreign_key :from, :to, on_delete: :nullify # to specify ON DELETE SET NULL

add_foreign_key :from, :to, on_update: :cascade # to specify ON UPDATE CASCADE
add_foreign_key :from, :to, on_update: :nullify # to specify ON UPDATE SET NULL

add_foreign_key :from, :to, name: "my_fk" # with specific name
add_foreign_key :from, :to, primary_key: "special_pk" # will reference `special_pk` from `:to`

Removing a foreign key works mostly the same:

remove_foreign_key :from, :to # will remove foreign key on `to_id`
remove_foreign_key :from, column: "fk_id" # will remove foreign key on `fk_id`
remove_foreign_key :from, name: "my_fk" # will remove foreign key named `my_fk`

Foreign keys are dumped after the indexes to schema.rb. The options name and column are always dumped. dependent only when set.

add_foreign_key is reversible, remove_foreign_key is not.

There is a connection#supports_foreign_keys? method to determine wether the adapter does support the API.

TODO:

  • Write documentation
  • Decide wether add-/remove_foreign_key should be no-op when supports_foreign_keys? is false
  • Make sure it works with automatic test schema maintenance (#15394)
  • Schema dumping must ensure the tables are created in the right order
  • Handle long identifiers.
  • rename dependent to on_delete and introduce on_update
  • rename when a table / column is renamed. (because of digest names, this is no longer needed)

Future Work (not this PR):

  • Support foreign keys when creating a table (This would allow SQLite to use FK's as well)

This patch was hugely inspired by foreigner. Credits to @matthuhiggins ❤️

@rafaelfranca
Copy link
Member

rafaelfranca commented Jun 10, 2014

Thank you. One thing to make work is the automatic test schema could you check if it is working now? I know it doesn't with foreigner

@fxn
Copy link
Member

fxn commented Jun 10, 2014

I have not checked the patch but something in the example code caught my attention.

Foreign keys have ON DELETE CASCADE, and ON UPDATE CASCADE. While :dependent resonates to associations, I see a few drawbacks:

  • It is suspicious that there's only one possible value (other than nil).
  • It doesn't generalize to ON UPDATE CASCADE.

What about two explicit flags on_delete_cascade: true, and on_update_cascade?

Coincidentally I have heavily used ON UPDATE CASCADE recently in a database merge where I had to move IDs.

@senny
Copy link
Member Author

senny commented Jun 10, 2014

@fxn the current implementation does not allow for ON UPDATE specifications. The ON DELETE case is broader than the examples:

          def dependency_sql(dependency)
            case dependency
              when :nullify then "ON DELETE SET NULL"
              when :delete  then "ON DELETE CASCADE"
              when :restrict then "ON DELETE RESTRICT"
              else ""
            end
          end

A simple boolean is not enough. This also varies per adapter. PostgreSQL for example has a default of NO ACTION.

@fxn
Copy link
Member

fxn commented Jun 10, 2014

Ah, I see didn't remember that.

Why isn't ON UPDATE supported?

@senny
Copy link
Member Author

senny commented Jun 10, 2014

@fxn because I did not implement it yet 😅

@fxn
Copy link
Member

fxn commented Jun 10, 2014

Haha :).

So yes, at the database schema level on_update and on_delete seem to be a better vocabulary to me than :dependent. Or you could have both, but if the DSL has only :dependent I see no equivalent choice for ON UPDATE.

@senny
Copy link
Member Author

senny commented Jun 10, 2014

@fxn agreed. Will update accordingly.

@matthuhiggins
Copy link

matthuhiggins commented Jun 10, 2014

I might as well chime in!

  • I regret using :dependent as the option name. Since the option was named after has_many's option of the same name, it created expectations for users that it would work the same. :on_delete is much better.
  • I always turned down any support for on_update because I viewed foreign keys as a database constraint rather than a way to change the behavior of the application. foreigner did support this through :options: https://github.com/matthuhiggins/foreigner#database-specific-options
  • Similar to above, there will always be database specific constraint types, so you'll probably need the :options option regardless. (Example: matthuhiggins/foreigner#136)
  • This is your first and last time to decide on the default constraint name when no name is provided. "foreign_key_#{from_name}_on_#{column_names * '_'}" better mirrors the naming convention used by add_index.
  • Since add_foreign_key supplants the use of add_index in many cases, users will expect the foreign key methods to work much like the index migration methods. This goes for things such as creating a foreign key during create_table, along with renaming foreign keys.
  • Related to above, it's important to note that adding a foreign key to a column without an index will introduce a new index to that column. This results in some annoyances:
change_table :widgets do |t|
  t.foreign_key :factories
end
# widgets.factory_id now has a foreign key and an index.

change_table :widgets do |t|
  t.remove_foreign_key :factories
end
# widgets still has an index on factory_id
  • The whole sqlite thing is probably worth addressing as a different beast. I have not checked recently, but I believe that the only way to add a foreign key to an existing sqlite table is to drop and recreate the table. I'm sure that this could be done automatically, but it would probably convoluted.

Anyways, those are the things I learned over the last 5 years of trying to keep foreigner as simple as possible. You can get a general gist of problems that you'll need to address or say no to by looking at https://github.com/matthuhiggins/foreigner/issues.

@senny
Copy link
Member Author

senny commented Jun 10, 2014

@matthuhiggins thanks so much for sharing your insight. Will investigate further to provide a sound starting point. I mostly agree with your points, I'm skeptical about the fk naming though. We had many issues because identifiers that got too long. I'd like to stick to a short naming when possible. Any thoughts on that?

@fxn
Copy link
Member

fxn commented Jun 10, 2014

Regarding ON UPDATE my take is that databases are who define what are FK constraints for. It's their field rather than ours.

FKs protect you from orphan records in the common case, but also allow you to define what to do when something happens to the parent record. In the use case I mentioned before, I need to issue dozens of statements like this:

UPDATE foos SET id = id + delta;

and ON UPDATE CASCADE is the blessed way to keep the FKs in children tables in sync automatically.

I believe that if we add support for FKs then we need to adapt to whatever FKs mean to database writers.

@matthuhiggins
Copy link

matthuhiggins commented Jun 10, 2014

Your current naming is completely fine too. Yes, I am aware of the naming limits, in particular with postgres. It is less likely to occur with foreign keys, since it is on only one column rather than multiple. Your choice is completely fine (hey now, it was my choice too). There is also fk_#{table_name}_on_#{column_name}.

Great work btw. I hope that some form of this makes it in.

@matthuhiggins
Copy link

matthuhiggins commented Jun 10, 2014

@fxn That is completely reasonable. I was stating where I personally drew the line, and therefore did not support. Similarly, I never supported composite foreign keys. Per your philosophy, should composite foreign keys be supported?

@senny Make sure that this issue does not exist with your new implementation: matthuhiggins/foreigner#110

@senny
Copy link
Member Author

senny commented Jun 10, 2014

@matthuhiggins thanks for the issue reference. I think this is the behavior @rafaelfranca was talking about. As we now have automatic schema maintenance for test runs, this is even more important. I haven't settled on a solution yet but will investigate dropping the database and not the tables.

@fxn
Copy link
Member

fxn commented Jun 10, 2014

@matthuhiggins Oh absolutely, regading your feedback. Just to be clear I was also talking from the point of view of a team member thinking about a new feature in Rails, without implying anything about your choices in foreigner, since that is your baby and I totally respect your choices there, couldn't be otherwise :).

Regarding composite FKs, I have never used them. In principle I'd say yes from the point of view of Active Record. If there's support for FKs I'd expect support for multiple columns. But then it also depends on practical matters, does it complicate the interface, is it worthwhile? So I'd say yes unless there are practical trade-offs that suggest not to support it.

In the case of ON UPDATE what I see is that it is totally analogous to ON DELETE. I imagine the implementation should be similar, and indeed treating both as equal (as they are in SQL) may even influence naming/interface.

@egilburg
Copy link
Contributor

egilburg commented Jun 11, 2014

we already use foreigner in our code. is this patch fully compatible with existing code that uses foreigner's syntax?

@egilburg
Copy link
Contributor

egilburg commented Jun 11, 2014

add_foreign_key is reversible, remove_foreign_key is not.

why not? remove_column is reversible in rails 4 provided it was defined with the params that add_column would usually require

@egilburg
egilburg reviewed Jun 11, 2014
View changes
activerecord/lib/active_record/connection_adapters/abstract/schema_statements.rb Outdated
options[:column] ||= foreign_key_column_for(to_table)
primary_key = options.fetch(:primary_key, "id")

options = {

This comment has been minimized.

Copy link
@egilburg

egilburg Jun 11, 2014

Contributor

foreigner accepts arbitrary options like add_foreign_key(:comments, :posts, options: 'ON UPDATE DEFERRED'). WDYT of supporting that?

This comment has been minimized.

Copy link
@senny

senny Jun 11, 2014

Author Member

I will probably add something of that sort but for now, I'd like to get the basic functionality working and merged.

@egilburg
Copy link
Contributor

egilburg commented Jun 11, 2014

And of course, much ❤️ for proposing this to core Rails.

I always thought that "getting started quickly" and "convention over configuration" doesn't mean "throw basic sane db design out the window". Not having db-level foreign key constraints and only relying on validations/assocs just means a world of hurt when you have to deal with production maintenance, changing application code, data migrations, etc.

So IMO this is a much needed patch and should also have docs that encourage it's use to the same tune as indexes and null constraints.

@egilburg
Copy link
Contributor

egilburg commented Jun 11, 2014

Also, one thing that bit me with foreigner is that rename_table, rename_column and friends doesn't rename existing foreign key names. This is especially painful when a db refactor results in a completely different table being named what another table was before. Is this something to consider tracking?

@senny
Copy link
Member Author

senny commented Jun 11, 2014

@fxn Why do you think we need to provide a complete solution for FK's? When I look at other methods from the migration DSL it becomes obvious that we only support a small subset of what would be possible. You can always drop down to execute.

I'm not opposed to build to a more complete solution but I think we should be shipping this as soon as we hit 80 / 20.

@fxn
Copy link
Member

fxn commented Jun 11, 2014

@senny note that my answer had some ifs :).

@senny
Copy link
Member Author

senny commented Jun 11, 2014

@egilburg there is basic compatibility between the current implementation and foreigner. The name of the options might vary (currently we have :on_delete and not :dependent). You should be able to migrate over though:

1.) remove foreigner
2.) remove legacy migrations
3.) dump a new schema.rb to have the correct syntax reflected

I will look into the renaming stuff. I know we do this for indexes.

@senny senny changed the title WIP: Basic support for adding and removing foreign keys Basic support for adding and removing foreign keys Jun 18, 2014
@matthewd
matthewd reviewed Jun 19, 2014
View changes
activerecord/lib/active_record/connection_adapters/abstract/schema_creation.rb Outdated
when :nullify then "ON #{action} SET NULL"
when :cascade then "ON #{action} CASCADE"
when :restrict then "ON #{action} RESTRICT"
end

This comment has been minimized.

Copy link
@matthewd

matthewd Jun 19, 2014

Member

else ArgumentError ?

@matthewd
matthewd reviewed Jun 19, 2014
View changes
activerecord/lib/active_record/connection_adapters/abstract/schema_creation.rb Outdated
@@ -18,11 +18,28 @@ def visit_AddColumn(o)
add_column_options!(sql, column_options(o))
end

def visit_AddForeignKey(o)
sql = <<-SQL
ADD CONSTRAINT #{o.name}

This comment has been minimized.

Copy link
@matthewd

matthewd Jun 19, 2014

Member

We should probably quote this

@matthewd
matthewd reviewed Jun 19, 2014
View changes
activerecord/lib/active_record/connection_adapters/postgresql/schema_statements.rb Outdated
WHERE c.contype = 'f'
AND t1.relname = #{quote(table_name)}
AND t3.nspname = ANY (current_schemas(false))
ORDER BY c.conname

This comment has been minimized.

Copy link
@matthewd

matthewd Jun 19, 2014

Member

If we're not going to support multi-column FKs, would it be more polite to skip them entirely, rather than dumping a half-complete depiction?

@matthewd
matthewd reviewed Jun 19, 2014
View changes
activerecord/lib/active_record/schema_dumper.rb Outdated
@@ -102,6 +103,10 @@ def tables(stream)
end
table(tbl, stream)
end

sorted_tables.each do |tbl|

This comment has been minimized.

Copy link
@matthewd

matthewd Jun 19, 2014

Member

Is it worth a comment here to clarify why the FKs can't follow their respective tables, like indexes?

This comment has been minimized.

Copy link
@senny

senny Jun 20, 2014

Author Member

Whom would that comment target? Do you see a situation where the user cares about that?

@matthewd
matthewd reviewed Jun 19, 2014
View changes
activerecord/lib/active_record/schema_dumper.rb Outdated
'primary_key: ' + foreign_key.primary_key.inspect,
'name: ' + foreign_key.name.inspect
]
parts << ('on_delete: ' + foreign_key.on_delete.inspect) if foreign_key.on_delete

This comment has been minimized.

Copy link
@matthewd

matthewd Jun 19, 2014

Member

Missing on_update?

@matthewd
matthewd reviewed Jun 19, 2014
View changes
activerecord/lib/active_record/schema_dumper.rb Outdated
'add_foreign_key ' + remove_prefix_and_suffix(foreign_key.from_table).inspect,
remove_prefix_and_suffix(foreign_key.to_table).inspect,
'column: ' + foreign_key.column.inspect,
'primary_key: ' + foreign_key.primary_key.inspect,

This comment has been minimized.

Copy link
@matthewd

matthewd Jun 19, 2014

Member

Do we really need to always include this?

@VanTanev
Copy link

VanTanev commented Jun 23, 2014

One thing possibly worth considering: composite key foreign keys.

@rafaelfranca rafaelfranca merged commit a5b3f37 into rails:master Jun 26, 2014
1 check was pending
1 check was pending
continuous-integration/travis-ci The Travis CI build is in progress
Details
rafaelfranca added a commit that referenced this pull request Jun 26, 2014
Basic support for adding and removing foreign keys
@rafaelfranca
Copy link
Member

rafaelfranca commented Jun 26, 2014

:shipit:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Really awesome work as usual 👏 👏 👏 👏

@fnando
Copy link
Contributor

fnando commented Jun 26, 2014

<3 <3 <3

Thanks a lot, @senny!

@fabianoleittes
Copy link

fabianoleittes commented Jun 26, 2014

Good work @senny, tks! 👍

@shamanime
Copy link
Contributor

shamanime commented Jun 26, 2014

Nicely done! 👍 :shipit:

@chancancode chancancode added the JRuby label Jun 26, 2014
@senny
Copy link
Member Author

senny commented Jun 26, 2014

thanks guys ❤️

@thibaudgg
Copy link
Contributor

thibaudgg commented Jun 27, 2014

Nice, really useful. Thanks from Switzerland @senny!

@tilsammans
Copy link
Contributor

tilsammans commented Jun 27, 2014

Well done, love this. If anything can we make sure all the work by @matthuhiggins is being incorporated. I use Foreigner in all apps, would be best to have a smooth transition between the two.

@senny
Copy link
Member Author

senny commented Jun 27, 2014

@tilsammans as described above you can transition to this. There will be small differences in the API and especially in the naming of foreign keys.

A lot of this PR is based on the work of @matthuhiggins and his work on foreigner. Hopefully we can improve this first draft down the line. ❤️

@dosire
Copy link
Contributor

dosire commented Jun 27, 2014

Thanks for getting this into rails Yves! <3

@robin850 robin850 added PostgreSQL and removed JRuby labels Jun 27, 2014
@robin850
Copy link
Member

robin850 commented Jun 27, 2014

Awesome work here! ❤️

@matthuhiggins
Copy link

matthuhiggins commented Jun 27, 2014

Great stuff. I gladly relinquish my technical debt to Rails!

@Krule
Copy link

Krule commented Jul 12, 2014

Finally! Thank you. ❤️

@senny senny deleted the senny:ar/foreign_key_support branch Jan 27, 2015
yahonda added a commit to yahonda/oracle-enhanced that referenced this pull request Jun 22, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

You can’t perform that action at this time.