Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transactional migration (#70) #73

Merged

Conversation

phips28
Copy link
Contributor

@phips28 phips28 commented Apr 7, 2018

Issue: #70

Migration: use pg transaction to avoid inconsistency. In case one query fails, or the node process restarts during the query is running, it will automatically rollback.

Param poolSize: also use max as valid config parameter (its the underlying pg-pool param)

Tests: changed timeouts to work with my remote database; unfortunately the speedTest.js doesnt work with my remote database 👎 but works on travis 👍

…ry fails, or the node process restarts during the query is running, it will automatically rollback.

poolSize: also use `max` as valid config parameter (its the underlying pg-pool param)
tests: changed timeouts to work with remote database; unfortunately the speedTest.js doesnt work with my remote database
@phips28 phips28 changed the title Transactional migration Transactional migration (#71) Apr 7, 2018
@coveralls
Copy link

Coverage Status

Coverage remained the same at 100.0% when pulling 4a51f41 on phips28:transactional-migration into 8d38a15 on timgit:master.

2 similar comments
@coveralls
Copy link

Coverage Status

Coverage remained the same at 100.0% when pulling 4a51f41 on phips28:transactional-migration into 8d38a15 on timgit:master.

@coveralls
Copy link

Coverage Status

Coverage remained the same at 100.0% when pulling 4a51f41 on phips28:transactional-migration into 8d38a15 on timgit:master.

@coveralls
Copy link

coveralls commented Apr 7, 2018

Coverage Status

Coverage remained the same at 100.0% when pulling e85d04d on phips28:transactional-migration into 8d38a15 on timgit:master.

@phips28 phips28 mentioned this pull request Apr 7, 2018
@phips28 phips28 changed the title Transactional migration (#71) Transactional migration (#70) Apr 7, 2018
@timgit
Copy link
Owner

timgit commented Apr 7, 2018

@phips28 Thanks for this! 🙇

btw, I downgraded bluebird to a dev dependency, which actually introduces a use case that Travis didn't catch since it would only show up with the --production arg or NODE_ENV var.

Also, all the commands don't need an extra semicolon, do they? join(';') should take care of that, right?

@phips28
Copy link
Contributor Author

phips28 commented Apr 7, 2018

Also, all the commands don't need an extra semicolon, do they? join(';') should take care of that, right?

Yes thats true, I first added the ; in every query, but then I realized that the migration.js would need the ; too at the end adn I was too lazy to edit all. So I used the join(';').

I removed it

Copy link
Owner

@timgit timgit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor things. I was able to make these changes and get the tests working again. I'd prefer to not merge with all the tests blowing up. Bad optics as they say :).

One more thing. I'd really feel better about this if we created a mock migration to force a failure and then confirm it wasn't committed.

Thanks again

@@ -2,6 +2,7 @@ const assert = require('assert');
const plans = require('./plans');
const migrations = require('./migrations');
const schemaVersion = require('../version.json').schema;
const Promise = require('bluebird');
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this

// use transaction, in case one query fails, it will automatically rollback to avoid inconsistency
let queryInTransaction = `
BEGIN;
${plans.create(this.config.schema).join(';')}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add semicolon after the join() to terminate last command

let queryInTransaction = `
BEGIN;
${plans.create(this.config.schema).join(';')}
${plans.insertVersion(this.config.schema).replace('$1', `'${schemaVersion}'`)}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add semicolon after this as well to terminate command

// use transaction, in case one query fails, it will automatically rollback to avoid inconsistency
let queryInTransaction = `
BEGIN;
${migration.commands.join(';')}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add semicolon after the join() to terminate last command


});

it('connection count does not exceed configured pool size with `max`', function(finished){
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to move this into a config test (does poolSize == max) instead of duplicating the entire connection pooling test

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added new tests, do you like it that way? should we remove the pooling test?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I'll go ahead and merge and take a stab at the migration failure rolllback test

@timgit timgit changed the base branch from master to transactional-migration April 12, 2018 14:14
@timgit timgit merged commit 7f2eb45 into timgit:transactional-migration Apr 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants