Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PostgreSQL 10 new relkind for partitioned tables #31336

Merged
merged 8 commits into from Jul 27, 2018
Merged

Conversation

ys
Copy link
Contributor

@ys ys commented Dec 5, 2017

Starting with PostgreSQL 10, we can now have partitioned tables natively

Summary

Because of the new partition type for tables in PostgreSQL 10, table_exists? is not finding the table for natively partition tables.

A new p type has been introduced.

Starting with PostgreSQL 10, we can now have partitioned tables natively
@rails-bot
Copy link

Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @eileencodes (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

This repository is being automatically checked for code quality issues using Code Climate. You can see results for this analysis in the PR status below. Newly introduced issues should be fixed before a Pull Request is considered ready to review.

Please see the contribution instructions for more information.

Copy link
Member

@eileencodes eileencodes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added one comment and otherwise I think this looks good. Are there any tests around this code that would be helpful to update?

@@ -708,7 +708,7 @@ def add_options_for_index_columns(quoted_columns, **options)

def data_source_sql(name = nil, type: nil)
scope = quoted_scope(name, type: type)
scope[:type] ||= "'r','v','m'" # (r)elation/table, (v)iew, (m)aterialized view
scope[:type] ||= "'r','v','m', 'p'" # (r)elation/table, (v)iew, (m)aterialized view
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the comment to include p?

@ys
Copy link
Contributor Author

ys commented Dec 7, 2017

Thanks @eileencodes
For the tests, they would need to be postgres 10 specific. I haven't found much tests around postgres native functions...
Maybe I haven't looked at them correctly.

@matthewd
Copy link
Member

matthewd commented Dec 7, 2017

This is a bit of a drive-by comment, because I haven't actually read through to check this is where you need to look, but I think you want https://github.com/rails/rails/tree/master/activerecord/test/cases/adapters/postgresql and https://github.com/rails/rails/blob/master/activerecord/test/cases/view_test.rb.

The former is where all the PG-specific tests go; the latter is, I believe, basically testing the 'v' (and 'm') part of this expression. So I think we want a partitioned_table_test.rb, in the style of view_test.rb, and in the postgresql/ directory.

I don't know of a specific one off the top of my head, but I expect you'll find a number of version gates on tests inside postgresql/, too.

@ys
Copy link
Contributor Author

ys commented Dec 7, 2017

Added a test for table_exists? but Rails has no part running postgreSQL 10, should I just update the postgreSQL version in .travis.yml or duplicate one of the entries?

@rafbm
Copy link
Contributor

rafbm commented Jan 30, 2018

Wouldn’t it also make sense to handle 'p' along with 'r' here?

def quoted_scope(name = nil, type: nil)
schema, name = extract_schema_qualified_name(name)
type = \
case type
when "BASE TABLE"
"'r'"
when "VIEW"
"'v','m'"
when "FOREIGN TABLE"
"'f'"
end

@ys
Copy link
Contributor Author

ys commented Feb 1, 2018

@rafbm Probably! I am not familiar with a lot of this :D


require "cases/helper"

class PostgreSQLPartitionsTest < ActiveRecord::PostgreSQLTestCase
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test should be run only when this feature is supported. What about to add method to PostgreSQLTestCase checking if feature is available? You can get current version via ActiveRecord::Base.connection.postgresql_version.

def postgresql_version     
  ActiveRecord::Base.connection.postgresql_version
end

def supports_partitioned_tables?
  postgresql_version >= 100000
end

in test:

def test_partitions_table_exists
  skip unless supports_partitioned_tables?
  ...
end

This is similar how adapter supported features are detected, but it is probably not worth it to introduce this feature into adapter, since it is used only for tests now.

Also direct version check is possible as used in

skip unless ActiveRecord::Base.connection.postgresql_version >= 90300

@ys
Copy link
Contributor Author

ys commented Apr 4, 2018

I forgot about this PR and feel a bit overwhelmed by the things to do to test this and what is really impacted by that change.

Would love to get some more help on this.

@ys
Copy link
Contributor Author

ys commented Apr 4, 2018

Looking at all the code linked is not really helping me as I have no idea of the impact beyond this

@yahonda
Copy link
Member

yahonda commented Jun 8, 2018

Hi, I'm interested in this pull request then I am attempting to update a test for this.
yahonda@ee6dd9d

Somehow, it looks like this table has not created. It is likely this transaction has not committed.
Will update when I have some more information but it would be appreciated if someone gave us some advice about DDL transaction differences between partitioned tables and non-partitioned tables.

Thank you.

@yahonda
Copy link
Member

yahonda commented Jun 11, 2018

I have updated this pull request to also select 'p' for "BASE TABLE" and add a test case for this change. yahonda@717d5a3
Now all unit tests are green.

@ys Would you consider to cherry-pick my commit to complete this pull request?

to support PostgreSQL 10 partition tables
@ys
Copy link
Contributor Author

ys commented Jun 12, 2018

@eileencodes Just added the tests that @yahonda wrote!

Btw, Yahonda, thanks so much for this!

@yahonda
Copy link
Member

yahonda commented Jun 13, 2018

Thanks for the cherry-pick and push. I have found my commit needs fixed. Pushed two additional commits to make CI green.

Thanks

@yahonda
Copy link
Member

yahonda commented Jun 14, 2018

Thanks. It looks good now. one CI failure https://travis-ci.org/rails/rails/jobs/392120838 is not related to this pull request, I think.

@kamipo kamipo merged commit 36f28fd into rails:master Jul 27, 2018
kamipo pushed a commit that referenced this pull request Jul 27, 2018
* PostgreSQL 10 new relkind for partitioned tables

Starting with PostgreSQL 10, we can now have partitioned tables natively

* Add comment

* Remove extra space

* Add test for partition table in postgreSQL10

* Select 'p' for "BASE TABLE" and add a test case
to support PostgreSQL 10 partition tables

* Address RuboCop offense

* Addressed incorrect `postgresql_version`

Fixes #33008.

[Yannick Schutz & Yasuo Honda & Ryuta Kamizono]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants