Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pg): PG10 identity columns and preliminary PG11 support #294

Merged
merged 17 commits into from
Sep 8, 2018
Merged

feat(pg): PG10 identity columns and preliminary PG11 support #294

merged 17 commits into from
Sep 8, 2018

Conversation

mattbretl
Copy link
Contributor

Fixes #244

For columns generated always as identity, the associated GraphQL input field should be omitted.

For columns generated by default as identity, the associated GraphQL input field should be nullable.

@mattbretl
Copy link
Contributor Author

Tests fail on PG <10 because I added identity column syntax to the kitchen sink. Should I add a new schema (e?) that's only tested if the PG version is 10+?

@benjie
Copy link
Member

benjie commented Sep 8, 2018

Call the schema pg10, but yes - that's a good idea 👍 The schemas currently have no names for legacy reasons.

Copy link
Member

@benjie benjie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome; thanks for taking this on!

I've included some feedback, I'd also like to see mutation tests on the new fields - to do so add a test file called something like packages/postgraphile-core/__tests__/fixtures/mutations/pg10.identity-columns.graphql

Add the PG10 schema here:

https://github.com/graphile/graphile-build/blob/9b651f18a6c5bf9e03793f82fb9a4fb9ec71d24b/packages/postgraphile-core/__tests__/integration/mutations.test.js#L35-L38

And add a fileName.startsWith("pg10.") clause here:

https://github.com/graphile/graphile-build/blob/9b651f18a6c5bf9e03793f82fb9a4fb9ec71d24b/packages/postgraphile-core/__tests__/integration/mutations.test.js#L72-L73

with console.log("Skipping test as PG version is less than 10"); return; or similar if the version number is pre 10.

Hopefully doing this in a new schema will mean the snapshot changes should be eradicated (there'll only be some new snapshots instead).

Oh, also please add a schema test so we can spot if the id column randomly comes back at some point (or if it exists in patch / similar input types).

You can copy this file for the schema test, just change the schema list to pg10 👍

https://github.com/graphile/graphile-build/blob/master/packages/postgraphile-core/__tests__/integration/schema/defaultOptions.test.js

@@ -82,6 +82,7 @@ export interface PgAttribute {
typeModifier: number;
isNotNull: boolean;
hasDefault: boolean;
identity: string | void;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Be more explicit: identity: '' | 'a' | 'd' | void;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correction: identity: '' | 'a' | 'd'; - I suggest we remove the void option.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also please mirror this in the PgIntrospectionPlugin.js types 👍

const introspectionQuery = await readFile(INTROSPECTION_PATH, "utf8");
const {
rows: [{ server_version_num: serverVersionNum }],
} = await pgClient.query("show server_version_num;");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wondered if this might already be available on pgClient as part of the handshake, but given this is in the node-postgres tests:

https://github.com/brianc/node-postgres/blob/1cbd507b8ca0416b9be0252cc930762e716ba60f/test/integration/test-helper.js#L18-L22

I'm going to assume not. However I do note that they explicitly parseInt, so we should probably do that also.

@@ -1,9 +1,10 @@
-- @see https://www.postgresql.org/docs/9.5/static/catalogs.html
function makeIntrospectionQuery(serverVersionNum) {
return `-- @see https://www.postgresql.org/docs/9.5/static/catalogs.html
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use the syntax:

  return `\
-- @see ...

The backslash escapes the newline so it has no effect - the resulting string will be identical but it's easier to read (and minimises differences).

@@ -157,6 +158,7 @@ with
nullif(att.atttypmod, -1) as "typeModifier",
att.attnotnull as "isNotNull",
att.atthasdef as "hasDefault",
${serverVersionNum >= 100000 ? `att.attidentity as "identity",` : ``}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use straight quotes here since there's no interpolation (stylistic reasons only): 100000 ? 'att.att....",' : ''}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I think it might be best to always define the columns rather than making void an option; so this should be something like:

${serverVersionNum >= 100000 ? 'att.attidentity' : "''"} as "identity",

This also makes the expected column names easier to scan.

@@ -1,9 +1,10 @@
-- @see https://www.postgresql.org/docs/9.5/static/catalogs.html
function makeIntrospectionQuery(serverVersionNum) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please convert this file to TypeScript, to do so I think you only need to change this declaration:

export function makeIntrospectionQuery(serverVersionNum: number): string {

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait I'm getting confused; graphile-build-pg is not in TypeScript yet. Instead, enable Flow with // @flow comment at the top and the same types (but no export keyword).


module.exports = {
makeIntrospectionQuery,
};
Copy link
Member

@benjie benjie Sep 8, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore.

@@ -69,15 +70,15 @@ with
pro.proisagg = false and
pro.proiswindow = false and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to graphile/crystal#796 these two lines causes PostGraphile to fail to start on PG11; it would be awesome if you could fix this in this PR also, but no worries if not - I can add it after 👍

According to the PG11 docs there's a new attribute prokind:

f for a normal function, p for a procedure, a for an aggregate function, or w for a window function

Seems prokind did not exist before PG11, so it'd be something like:

${serverVersionNum >= 110000 ? "pro.prokind = 'f'" : 'pro.proisagg = false and pro.proiswindow = false'} and

I'm not sure if that's all that's required to support PG11.


To run PG11 without installing you can use Docker:

docker run -it --rm -p 127.0.0.1:54321:5432 postgres:11

To connect, you can do:

psql postgres://postgres@localhost:54321/postgres

For the tests

export TEST_DATABASE_URL="postgres://postgres@localhost:54321/postgres"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests now pass with PG11 🎉

@benjie
Copy link
Member

benjie commented Sep 8, 2018

Looking good; let me know when you're ready for me to re-review 👍

@mattbretl
Copy link
Contributor Author

All requested changes have been made, apart from adding mutation tests

@benjie
Copy link
Member

benjie commented Sep 8, 2018

Awesome; I'll review now 👍

Copy link
Member

@benjie benjie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spot on 👍 Just the tests to go 🎉

@benjie
Copy link
Member

benjie commented Sep 8, 2018

Looking really good; thanks for doing this Matt!

I can look at adding tests for this towards the end of the week if you don't have any more time. I also want to figure out how to get travis to test against PG9, PG10 and PG11 before merging - but again, I'm happy for you to leave that with me - let me know 👍 🙌🙌🙌

Copy link
Member

@benjie benjie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

I'd like to have travis run PG11 tests before merge, but the code itself seems good to go - great work! 🙏

@mattbretl
Copy link
Contributor Author

No experience with travis, but I'll give it a shot..

@mattbretl
Copy link
Contributor Author

AFAICT (https://github.com/travis-ci/apt-source-safelist/blob/master/ubuntu.json#L422), there's no way to run PG11 on travis without switching to sudo: required and increasing boot times to 20-50s. If you're willing to do that, I can give it a shot.

@benjie
Copy link
Member

benjie commented Sep 8, 2018

Okay; drop the travis commit and we'll merge as-is and test manually for now. I'm not keen for CI to be any slower.

This reverts commit ac7d49a.
@benjie benjie changed the title Identity columns feat(sql): identity columns and preliminary PG11 support Sep 8, 2018
@benjie benjie changed the title feat(sql): identity columns and preliminary PG11 support feat(pg): identity columns and preliminary PG11 support Sep 8, 2018
@benjie benjie changed the title feat(pg): identity columns and preliminary PG11 support feat(pg): PG10 identity columns and preliminary PG11 support Sep 8, 2018
@benjie benjie merged commit 1e040ba into graphile:master Sep 8, 2018
@benjie
Copy link
Member

benjie commented Sep 8, 2018

🎉 Merged. Looks like there's going to be an RC5 after all! 😂

@mattbretl mattbretl deleted the identity-columns branch September 8, 2018 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants