SQL Migrations Not Working?

[dschinkel]

So I know you wrote a test to test that ported sqlite migration code, saw that test in one of your commits. But when I try this in my app, it's simply not working:

InMemoryDB.ts

const inMemoryDb = newDb();
inMemoryDb.public.none(fs.readFileSync(path.resolve(__dirname, 'migrations/001-initial.sql'), 'utf8'));
(async () =>await inMemoryDb.public.migrate())();
const skills = inMemoryDb.public.many("select * from skills where name = 'Swift'");
console.log('skills', skills); // prints []

the migrations folder is at the same level as InMemoryDB.ts and in that folder is:

001-initial.sql
002-add-skills.sql

001-initial.sql - definitely adds a skills table seeded with some initial skills

002-add-skills.sql

INSERT INTO skills VALUES ('Swift', 0);
INSERT INTO skills VALUES ('Objective-C', 0);

no idea here, thoughts?

[oguimbal]

No need to read 001-initial.sql, the migration will do that for you.

Moreover, the line (async () =>await inMemoryDb.public.migrate())(); is effectively scheduling a migration, but triggers it "for later" (it dettaches a promise... use a linter, and you will get a warning on this line). Dettached promises is a pattern that should be avoided as much as possible (and which is, in this case, the cause of your problem).

Thus, the 4th line is effectively executed BEFORE the migration.

This should work:

const inMemoryDb = newDb();
await inMemoryDb.public.migrate();
const skills = inMemoryDb.public.many("select * from skills where name = 'Swift'");
console.log('skills', skills);

You'll have to execute this from an async method, though.

[dschinkel]

Got it working.

What doesn't work is just calling migrate() bare without specifying a path even though it's supposed to be smart enough to find the migrations folder if it's in the same directory, without specifying a migrationsPath. But... for some reason it can't. It sets the path to undefined if you don't givce it a valid migrationsPath

The migration code also didn't report back when there was an undefined migration path...so it just ate that damn error (or didn't error at all) which is why I saw nothing resolving or happening which is a hugely unhelpful to anyone trying to use this module. I'm adding behavior to improve that, see my last reply above.

I know that your existing test did specify a path migrationsPath: path.resolve('./src/tests/migrate'),. So I ended up doing something similar with __dirname via migrationsPath: path.join(__dirname, 'migrations'), in my own code. Once I did that, stuff started working.

While it works, their migration code is a complete hack job. I want to help you improve their code.

So...I'm going to do a PR with:

• examples of it working as in an ES6 Singleton module (basically how I got my stuff working)
• adding better user feedback when migrate fails for config reasons
• adding a bit more test coverage to to the migrate.ts file you ported over from the sqlite-node repo

Later on:

• I wanna get more tests around that ported migrate code garbage and also refactor it to be cleaner (Clean Code). Together we can do better than they did

[dschinkel]

My New Tests

@oguimbal I added a couple tests in the test/migrate folder and some new behavior to your repo in my fork which I'll submit as a PR.

I don't see my new tests or migrate tests being run though after running them via yarn test for some reason. It's not picking them up in the test run.

pg-mem Tests

Also why pre-compile the tests? Can't you just run them on your tests folder, straight .ts. I never run my tests on compiled tests to js, it's unnecessary to setup all the precompiling of them with webpack. I honestly just wanna run tests on the .ts files using a simple mocha command, mocha runs fast by its very nature, you don't need to compile the tests physically like this, compile them on the fly using babel or something

TS

I also noticed that when I opened your project (I'm using WebStorm), TS is complaining about the migrate code. Is there any way I can turn that off? I mean you must have to be getting those errors too? The sqlite migrate code is a complete mess (I know you didn't write that and mostly ported it and tweaked it, but just saying)

[oguimbal]

Tests

Some kind of building (transpiling, actually) is necessary given that it's TS, so no, it is not possible to run mocha directly on ts files. When you do that on your side, I suppose that you're using ts-node or something equivalent, which is transpiling TS for you. That's also what does Webpack. Kind of an equivalent way to do the same thing.

Moreover, the current setup of pg-mem tests is using webpack because it's way faster to code that way for me. That's a matter of choice.

Ts compilation error

I dont see this problem on my side ...

[dschinkel]

it is not possible to run mocha directly on ts files

yes I'm using ts-node in conjunction with a tsconfig in my test folder.

I still have the problem whereas when I run your yarn test it's not picking up my tests that I added to even an existing spec of yours though for some reason.