Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change Default Primary Keys to BIGINT #26266

Merged
merged 2 commits into from Dec 5, 2016
Merged

Conversation

jmccartie
Copy link
Contributor

@jmccartie jmccartie commented Aug 24, 2016

Friends don't let friends use INT as a primary key.

— Schneems (@schneems) May 13, 2016

Summary

Per a conversation with @sgrif: changes default primary keys from Integer to BIGINT for both Postgresql and MySQL. Leaves behavior alone for SQLite since this database does not provide support for BIGINT primary keys.

Other Information

For obvious reasons, this also requires foreign keys to change from integer to bigints. As a result the test suite's schema.rb has been change in the necessary places.

I'll squash and add a CHANGELOG entry once the rest looks ok...

@rails-bot
Copy link

Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @kaspth (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

This repository is being automatically checked for code quality issues using Code Climate. You can see results for this analysis in the PR status below. Newly introduced issues should be fixed before a Pull Request is considered ready to review.

Please see the contribution instructions for more information.

@sgrif sgrif assigned sgrif and unassigned kaspth Aug 24, 2016
@sgrif
Copy link
Contributor

sgrif commented Aug 24, 2016

We'll need to make sure that migrations written against 5.0 and later don't have the type of their primary key changed.

@jmccartie
Copy link
Contributor Author

@sgrif Can you recommend a best-practice for ensuring that? Have we done something like that in the past that I can learn from?

@sgrif
Copy link
Contributor

sgrif commented Aug 24, 2016

@jmccartie
Copy link
Contributor Author

@sgrif Thanks.

@jmccartie
Copy link
Contributor Author

@sgrif So override the necessary methods inside class V5_0 ?

@rafaelfranca
Copy link
Member

rafaelfranca commented Aug 24, 2016

This is a dup of #24962. While this implementation may be more complete it is really a bad OSS etiquette to finish other people PR without given them a chance to finish it or proper credits.

My recommendation:

  1. pull Make pg adapter use bigserial for pk by default #24962 commits in this PR.
  2. Work on top of that commits

@jmccartie
Copy link
Contributor Author

@rafaelfranca Ah - thanks for showing me that. I searched for open PR's, but that didn't turn up for me.

I work at Heroku with @rwz, so I'll cycle around with him and see if we can tag team this.

@rafaelfranca
Copy link
Member

👍

@jmccartie
Copy link
Contributor Author

I've now entered into the 3rd layer of git rebase hell...

@rafaelfranca
Copy link
Member

I've now entered into the 3rd layer of git rebase hell...

lol. This is really tough.

@kaspth
Copy link
Contributor

kaspth commented Aug 24, 2016

r? @sgrif

@schneems
Copy link
Member

Looks like tests are passing now

# Since version 5.1 Postgres adapter uses bigserial type for primary
# keys by default. This compat layer makes old migrations utilize
# serial type instead, the way it used to work before 5.1
if connection.adapter_name == "PostgreSQL"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to handle MySQL as well. Is it possible for us to implement this in an adapter agnostic way?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I can make that change.

On Thu, Aug 25, 2016 at 10:13 AM, Sean Griffin notifications@github.com
wrote:

In activerecord/lib/active_record/migration/compatibility.rb
#26266 (comment):

@@ -103,6 +103,16 @@ def index_name_for_remove(table_name, options = {})
end

   class V5_0 < V5_1
  •    def create_table(table_name, options = {})
    
  •      # Since version 5.1 Postgres adapter uses bigserial type for primary
    
  •      # keys by default. This compat layer makes old migrations utilize
    
  •      # serial type instead, the way it used to work before 5.1
    
  •      if connection.adapter_name == "PostgreSQL"
    

This needs to handle MySQL as well. Is it possible for us to implement
this in an adapter agnostic way?


You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/rails/rails/pull/26266/files/de485a4379a265218f1bbea900c08f45e39a93fd#r76285358,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAIJEYj45MhqMMLNBPIkwwMhoRSdHCBuks5qjc1DgaJpZM4JrgwZ
.

@jmccartie
Copy link
Contributor Author

@sgrif Added a test for MySQL compatibility. The logic still queues off the adapter name, though. We could move that logic into a default on each adapter, but then we have some sort of "legacy_primary_key" method on the adapter itself, which I liked much less than the ternary.

Thoughts?

@sgrif
Copy link
Contributor

sgrif commented Aug 26, 2016

Can we just make the PG adapter do the right thing when the PK type is integer?

@jmccartie
Copy link
Contributor Author

jmccartie commented Aug 26, 2016

@sgrif I'm not sure. Both adapters need the logic since we need to ensure MySQL uses integer (and not bigint) and Postgresql uses serial (and not bigserial)

@sgrif
Copy link
Contributor

sgrif commented Aug 26, 2016

How about just :integer?

On Thu, Aug 25, 2016 at 11:04 PM Jon McCartie notifications@github.com
wrote:

@sgrif https://github.com/sgrif Something like this?

options[:id] ||= connection.class::LEGACY_PK_TYPE


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#26266 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABdWK2-R2Rj-aRqV77WLaJaf0AC5MeBYks5qjlfYgaJpZM4JrgwZ
.

jrafanie added a commit to jrafanie/activerecord-id_regions that referenced this pull request Apr 11, 2019
Rails 5.1 changed the default primary key type from integer to
bigint/bigserial in:

rails/rails#26266

With this change, they modified the default compatibility layer for
migration versions less than 5.1 to default to integer if not provided.
Since we want our default to be bigint/bigserial for those old migrations,
we need to prepend our module on the migration class when Rails tries to
find it in the compatilibity stack.

Also suppress messages for migrations in specs.
jrafanie added a commit to jrafanie/activerecord-id_regions that referenced this pull request Apr 23, 2019
Rails 5.1+ changed the default primary key type from integer to bigserial in:
rails/rails#26266

With this change, they modified the default compatibility layer for migration
versions less than 5.1 to default to integer if not provided.  Since we need
bigserial ids for region support, we need 4.2 and 5.0 versioned migrations
to also use bigserial id columns unless they explicitly say id => false
for tables without id columns.
jrafanie added a commit to jrafanie/activerecord-id_regions that referenced this pull request Apr 26, 2019
Rails 5.1+ changed the default primary key type from integer to bigserial in:
rails/rails#26266

With this change, they modified the default compatibility layer for migration
versions less than 5.1 to default to integer if not provided.  Since we need
bigserial ids for region support, we need 4.2 and 5.0 versioned migrations
to also use bigserial id columns unless they explicitly say id => false
for tables without id columns.
jrafanie added a commit to jrafanie/activerecord-id_regions that referenced this pull request Apr 26, 2019
Rails 5.1+ changed the default primary key type from integer to bigserial in:
rails/rails#26266

With this change, they modified the default compatibility layer for migration
versions less than 5.1 to default to integer if not provided.  Since we need
bigserial ids for region support, we need 4.2 and 5.0 versioned migrations
to also use bigserial id columns unless they explicitly say id => false
for tables without id columns.
jrafanie added a commit to jrafanie/activerecord-id_regions that referenced this pull request Apr 29, 2019
Rails 5.1+ changed the default primary key type from integer to bigserial in:
rails/rails#26266

With this change, they modified the default compatibility layer for migration
versions less than 5.1 to default to integer if not provided.  Since we need
bigserial ids for region support, we need 4.2 and 5.0 versioned migrations
to also use bigserial id columns unless they explicitly say id => false
for tables without id columns.
rbns added a commit to rbns/the-idb that referenced this pull request Jun 10, 2020
rails 5.1 defaults to bigint rowids [1]. this lets foreign table
constraints fail, as the types of the compared columns doesn't
match anymore. int in the referencing column, bigint in the
referred column.

this patches schema.rb to use int id columns for the tables. [2]

[1] rails/rails#26266

[2] https://ridingtheclutch.com/post/160099545985/rails-51-new-default-bigint-primary-key-sort-of
rbns added a commit to rbns/the-idb that referenced this pull request Jun 10, 2020
rails 5.1 defaults to bigint rowids [1]. this lets foreign table
constraints fail, as the types of the compared columns doesn't
match anymore. int in the referencing column, bigint in the
referred column.

this patches schema.rb to use int id columns for the tables. [2]

[1] rails/rails#26266

[2] https://ridingtheclutch.com/post/160099545985/rails-51-new-default-bigint-primary-key-sort-of
rbns added a commit to rbns/the-idb that referenced this pull request Jun 10, 2020
rails 5.1 defaults to bigint rowids [1]. this lets foreign table
constraints fail, as the types of the compared columns doesn't
match anymore. int in the referencing column, bigint in the
referred column.

this patches schema.rb to use int id columns for the tables. [2]

[1] rails/rails#26266

[2] https://ridingtheclutch.com/post/160099545985/rails-51-new-default-bigint-primary-key-sort-of
Hamms added a commit to code-dot-org/code-dot-org that referenced this pull request Oct 21, 2020
The default type was changed in rails/rails#26266, so to preserve existing functionality we now have to explicitly say that our current table deviate from the default.
@synth
Copy link

synth commented Feb 24, 2021

Is there official documentation on this?

This is causing a headache for our Rails upgrade because all our primary key types and foreign keys are int, but schema.rb doesn't specify the primary key type, so after upgrade, any database that's created from schema.rb (eg tests and local dev) will have primary keys with bigint but foreign keys that are int and thus will error out due to the mismatch.

Updating the keys is not straightforward because there is a chicken/egg situation. If you add a change_column migration to update the primary keys, you'll still get an error in your migrations when the foreign key migration executes (which is before the latest migration to update the type). And obviously editing earlier migrations has its own problems.

A solution here would be to NOT have implicit primary key types which change with Rails versions, but it's not clear how to manage that in this context of upgrading Rails.

UPDATE: If you're upgrading from Rails 5.0.x or earlier, delete and regenerate your schema.rb files. It looks like implicit primary key types was fixed :)

manno added a commit to frab/frab that referenced this pull request Dec 3, 2021
manno added a commit to frab/frab that referenced this pull request Dec 5, 2021
Previously we assumed that `db/schema.rb` is database agnostic.
This is not exactly true in all regards.

* rails/rails#26209 (comment)

Our commited `schema.rb` can be used to `db:setup` all databases.
running `db:schema:dump` or `db:migrate`, will turn the file into a
database specific one.
Formerly that wasn't obvious with the sqlite adapter, but now it dumps
foreign key information into the schema.
The previous check cannot be used anymore.

This is unfortunate, since frab's migrations are so old, that you cannot
`db:create db:migrate` a fresh system.

Not sure if related, but we will probably see more problems with old
frab installations on MySQL, due to the 2016 bigint change. These become
especially problematic, when using foreign key constraints:

* (MySQL switched to bigint)[rails/rails#26266]
* rails/rails#43168

Changing all IDs and references to bigint on MySQL is probably not
possible from migrations and should be done by the admin. A more
extensive solution than this is needed:
#660 (comment)
manno added a commit to frab/frab that referenced this pull request Dec 18, 2021
Previously we assumed that `db/schema.rb` is database agnostic.
This is not exactly true in all regards.

* rails/rails#26209 (comment)

Our commited `schema.rb` can be used to `db:setup` all databases.
running `db:schema:dump` or `db:migrate`, will turn the file into a
database specific one.
Formerly that wasn't obvious with the sqlite adapter, but now it dumps
foreign key information into the schema.
The previous check cannot be used anymore.

This is unfortunate, since frab's migrations are so old, that you cannot
`db:create db:migrate` a fresh system.

Not sure if related, but we will probably see more problems with old
frab installations on MySQL, due to the 2016 bigint change. These become
especially problematic, when using foreign key constraints:

* (MySQL switched to bigint)[rails/rails#26266]
* rails/rails#43168

Changing all IDs and references to bigint on MySQL is probably not
possible from migrations and should be done by the admin. A more
extensive solution than this is needed:
#660 (comment)
manno added a commit to frab/frab that referenced this pull request Dec 18, 2021
Previously we assumed that `db/schema.rb` is database agnostic.
This is not exactly true in all regards.

* rails/rails#26209 (comment)

Our commited `schema.rb` can be used to `db:setup` all databases.
running `db:schema:dump` or `db:migrate`, will turn the file into a
database specific one.
Formerly that wasn't obvious with the sqlite adapter, but now it dumps
foreign key information into the schema.
The previous check cannot be used anymore.

This is unfortunate, since frab's migrations are so old, that you cannot
`db:create db:migrate` a fresh system.

Not sure if related, but we will probably see more problems with old
frab installations on MySQL, due to the 2016 bigint change. These become
especially problematic, when using foreign key constraints:

* (MySQL switched to bigint)[rails/rails#26266]
* rails/rails#43168

Changing all IDs and references to bigint on MySQL is probably not
possible from migrations and should be done by the admin. A more
extensive solution than this is needed:
#660 (comment)
tagliala added a commit to ifad/chronomodel that referenced this pull request Apr 25, 2022
History tables store a representation of the current record itself, so
the number of historical records will be always greater than or equal
to the current ones.

When the number of records grow, there is less space for historical ids,
but it is logic to expect the opposite (more records, more need for historical versions)

Given also that starting from Rails 5.1 ids are stored as `bigint`, this
commit changes the default hid type to bigserial

Close #153

Ref:
- rails/rails#26266
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet