Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce limit on table names length #45136

Merged
merged 1 commit into from
Jun 13, 2022

Conversation

fatkodima
Copy link
Member

Fixes #45130

For long table names, to generate primary key index name, postgres truncates table name even further and adds _pkey suffix and this all should fit into 63 bytes.

Active Record already has constants for different name limits https://github.com/rails/rails/blob/main/activerecord/lib/active_record/connection_adapters/abstract/database_limits.rb and even checks index names in several places, like

validate_index_length!(table_name, new_name)
and several more places (but not everywhere).

I suggest to implement similar checks for table/(foreign key)/etc names, or at least for table names, as a followup in a separate PR, taking into account backwards compatibility with older migration files.

cc @nvasilevski

@@ -382,7 +382,7 @@ def rename_table(table_name, new_name)
execute "ALTER TABLE #{quote_table_name(table_name)} RENAME TO #{quote_table_name(new_name)}"
pk, seq = pk_and_sequence_for(new_name)
if pk
idx = "#{table_name}_pkey"
idx = "#{table_name[0, index_name_length - '_pkey'.size]}_pkey"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a nitpick and totally up to you but I really find the "#{table_name}_pkey" very readable and was wondering if we could somehow keep it, something like:

Suggested change
idx = "#{table_name[0, index_name_length - '_pkey'.size]}_pkey"
table_name_max_length = index_name_length - '_pkey'.size
truncated_table_name = table_name[0,table_name_max_length]
idx = "#{truncated_table_name}_pkey"

Not sure about the truncated_table_name name though as it's not always truncated, perhaps limited_table_name?


def test_renaming_table_with_long_name
table_name = "a" * (connection.max_identifier_length + 1)
connection.create_table table_name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not necessary and might be personal preference but perhaps it would be reasonable to add an assertion or just a check like flunk("wasn't truncated") unless <how can we check for truncation> to make sure that truncation actually happens. Generally we are pretty confident because of relying on max_identifier_length but what worries me is that the test isn't going to fail if max_identifier_length accidentally changes to something like 10, or an actual database limit becomes higher like 128 but the test still going to be green

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For postgres, we dynamically get a config (compared to hardcoded value for of 64 for other adapters)

# Returns the configured supported identifier length supported by PostgreSQL
def max_identifier_length
@max_identifier_length ||= query_value("SHOW max_identifier_length", "SCHEMA").to_i
end

Wont' this be enough to detect changes?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, "go to definition" confused me and I didn't check the method definition for postgresql adapter 🙈
Nvm my comment, thanks for pointing out!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we dynamically get a config (compared to hardcoded value for of 64 for other adapters)

This kinda confirm my hunch that we shouldn't automatically truncate, otherwise a migration could produce different index names in dev and prod if the postgres is configured differently.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, and it could also produce different table names, but it will be able to correctly determine this truncated name in both envs.

@byroot
Copy link
Member

byroot commented May 21, 2022

I'm a bit on the fence here, truncating index names makes for weird results. Maybe it would make more sense to raise an error instead, with a message indicating that the default name is too long and that a custom name must be set?

@fatkodima
Copy link
Member Author

It would be also hard to tell which index name is too long, since dev and prod can have different values for this configuration.

@byroot
Copy link
Member

byroot commented May 21, 2022

since dev and prod can have different values for this configuration.

It's fine. If that's the case the migration will fail in one env, as long as the error message is clear we're good I think.

I prefer for us to bail out saying we don't know what to do, rather than risking doing the wrong thing.

@fatkodima
Copy link
Member Author

  1. So what do you think about my initial suggestion in the description:

I suggest to implement similar checks for table/(foreign key)/etc names, or at least for table names?

  1. And should we take into account backwards compatibility with older migration files?

@byroot
Copy link
Member

byroot commented May 21, 2022

  1. So what do you think about my initial suggestion in the description:

👍

And should we take into account backwards compatibility with older migration files?

It would be safer yes.

@byroot byroot closed this May 26, 2022
@byroot byroot reopened this May 26, 2022
@byroot
Copy link
Member

byroot commented May 26, 2022

🤦 sorry wrong PR...

@fatkodima
Copy link
Member Author

@byroot I am having troubles on how to make these changes backwards compatible.
For example, if we add the check for table_name to create_table, how to make it compatible in compatibility.rb without duplicating all the code for create_table (except the new check for name length) or without hacks similar to this

options[:_uses_legacy_reference_index_name] = true
?

@byroot
Copy link
Member

byroot commented Jun 12, 2022

without hacks similar to this

I'd say try this way, we'll see how it looks. If it's a bit too disgusting we can fallback to copying as last resort. Duplication a bit annoying for future maintenance, but not the end of the world.

@fatkodima
Copy link
Member Author

@byroot Updated.
Index names length is already validated for all adapters. I added only checks for table names.
Other identifiers, like column names, foreign keys, check constraints etc probably should be also checked.
But since for foreign keys and check constraints the auto generated names are already truncated and it is very unlikely that the user will specify a long name manually, I think checking only index names and table names should be enough.

@fatkodima fatkodima changed the title Fix rename_table to handle long names for PostgreSQL Enforce limit on table names length Jun 12, 2022
@byroot byroot merged commit 5b54da5 into rails:main Jun 13, 2022
willnet added a commit to willnet/globalize that referenced this pull request Oct 6, 2023
CI fails with rails 7.1 like following

```
/home/runner/work/globalize/globalize/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.0/lib/active_record/connection_adapters/abstract/schema_statements.rb:1778:in `validate_table_length!': Table name 'migrated_with_mega_ultra_super_long_model_name_with_more_then_sixty_characters' is too long; the limit is 64 characters (ArgumentError)

            raise ArgumentError, "Table name '#{table_name}' is too long; the limit is #{table_name_length} characters"
```

We need to skip creating a long name table with rails 7.1 due to rails/rails#45136
willnet added a commit to willnet/globalize that referenced this pull request Oct 6, 2023
CI fails with rails 7.1 like following

```
/home/runner/work/globalize/globalize/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.0/lib/active_record/connection_adapters/abstract/schema_statements.rb:1778:in `validate_table_length!': Table name 'migrated_with_mega_ultra_super_long_model_name_with_more_then_sixty_characters' is too long; the limit is 64 characters (ArgumentError)

            raise ArgumentError, "Table name '#{table_name}' is too long; the limit is #{table_name_length} characters"
```

We need to skip creating a long name table with rails 7.1 due to rails/rails#45136
willnet added a commit to willnet/globalize that referenced this pull request Oct 6, 2023
CI fails with rails 7.1 like following

```
/home/runner/work/globalize/globalize/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.0/lib/active_record/connection_adapters/abstract/schema_statements.rb:1778:in `validate_table_length!': Table name 'migrated_with_mega_ultra_super_long_model_name_with_more_then_sixty_characters' is too long; the limit is 64 characters (ArgumentError)

            raise ArgumentError, "Table name '#{table_name}' is too long; the limit is #{table_name_length} characters"
```

We need to skip creating a long name table with rails 7.1 due to rails/rails#45136
willnet added a commit to willnet/globalize that referenced this pull request Oct 6, 2023
CI fails with rails 7.1 like following

```
/home/runner/work/globalize/globalize/vendor/bundle/ruby/3.2.0/gems/activerecord-7.1.0/lib/active_record/connection_adapters/abstract/schema_statements.rb:1778:in `validate_table_length!': Table name 'migrated_with_mega_ultra_super_long_model_name_with_more_then_sixty_characters' is too long; the limit is 64 characters (ArgumentError)

            raise ArgumentError, "Table name '#{table_name}' is too long; the limit is #{table_name_length} characters"
```

We need to skip creating a long name table with rails 7.1 due to rails/rails#45136
tagliala added a commit to ifad/chronomodel that referenced this pull request Nov 11, 2023
tagliala added a commit to ifad/chronomodel that referenced this pull request Nov 11, 2023
tagliala added a commit to ifad/chronomodel that referenced this pull request Nov 11, 2023
owst added a commit to bambooengineering/activerecord7-redshift-adapter that referenced this pull request Jan 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

rename_table does not work for tables with very long table names when using PostgreSQL
3 participants