Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upcoming release: 0.6.0 #252

merged 111 commits into from Jun 4, 2014

Upcoming release: 0.6.0 #252

merged 111 commits into from Jun 4, 2014


Copy link

@tgriesser tgriesser commented Apr 22, 2014

Still need to fix up the migrations and clean up a few other things, but I wanted to create a ticket that details the new minor release so people can take a look and give any feedback, because the internal changes are pretty significant and exciting.

The library will still take the "Batteries Included" approach: calling Knex.initialize(config) will make sure everything is taken care of for you. The entire library has been refactored such that a "client" object is now the central piece of the library, with a set of constructors conditionally attached, independent to each client instance (i.e. client.QueryBuilder):

Always loaded:

  • QueryBuilder
  • QueryCompiler
  • Formatter
  • Transaction
  • Raw

Added conditionally if you've specified a connection:

  • Runner
  • Pool

Added conditionally if any of the schema methods are called:

  • SchemaBuilder
  • SchemaCompiler
  • TableBuilder
  • TableCompiler
  • ColumnBuilder
  • ColumnCompiler

...and in SQLite3, where anything related to DDL is a complete mess:

  • SQLite3_DDL

Added conditionally if the migration utilities are used:

  • Migrator

It helps makes things super obvious as to where features should be added, and what the model should be when adding new clients. Also, since each client has independent constructors, it's totally fine to write plugins/modify things as needed without potentially running into issues with others' instances.

Some of the main issues that have come up deal with raw queries and the fact that you can't specify bound parameters when injecting raw statements within a knex query. The solution to this was pretty in-depth, it involved rewriting most of the structure of the entire builder, each call to a query builder pushes the arguments onto a statements array, rather than try to add the bindings as the methods are called.

This allows us to do things like:

knex.whereIn('id', knex.raw('(select * from table where id = ?)', [binding])).orWhere('id', 22)

or to re-use queries by placing them in other queries (see #162 for a neat use):

var q1 ='*').from('table').where('id', binding);
knex.whereIn('id', q1).orWhere('id', 22);

This works because everything now follows a "Builder" -> "Compiler" sequence, where the query is first built, and then "compiled", where the compiler uses its own formatter object, keeping track of the position of the bindings as the query is compiled rather than as it is built.

This new structure also make it super straightforward on how to add new features, for example facilities for view creation. In fact I've taken the liberty of putting a viewbuilder.js & viewcompiler.js in the schema directory, they're empty, anyone can feel free to take that one on :)

Also, as you may have noticed, all methods can be chained from the knex object, and don't need to start with knex(tableName) though that behavior is still supported, knex.insert(values).into(table) or knex.table(name).update(values) or all work as you'd expect, making query composition a nicer experience.

There are also major number of unit tests added, adapted from laravel's suite :)

The new structure should also make it really obvious how to add additional adapters, @tybenz if you want to take a look for mssql, that'd be great. There's also a number of unit tests to start you off that are commented out from laravel's suite.

Other features:
  • an explicit connection can be passed for any query #56
  • drop column / alter column for sqlite3 (quite a pain)
  • all schema actions are run sequentially on the same connection if chained, e.g.:
  • schema actions can now be wrapped in a transaction
  • .references(tableName.columnName) as shorthand for .references(columnName).inTable(tableName)
  • .join('table.column', 'otherTable.column') as shorthand for .join('table.column', '=', 'otherTable.column')
  • Streams should also be supported for selects, passing through to the streaming capabilities of node-mysql and node-postgres... haven't tried them yet though
  • join's with the trailing argument, e.g. join('', '=', 'otherTable.table_id', 'left outer') are now .leftOuterJoin('', '=', 'otherTable.table_id') or more succinctly .leftOuterJoin('', 'otherTable.table_id')
  • whereRaw, orWhereRaw, havingRaw, orHavingRaw are now just .where(knex.raw(rawQuery, bindings)), etc.
Still left to-to:
  • Tests for stream
  • Fix the migration utilities, figure out a better api / best practice of doing those
  • Standardize debugging things with the new runner #167, errors, how to show all queries executed from a single action
  • Ensuring all of the recent bugfixes surrounding operators are preserved, a few more tests/tickets to add things for
  • Check that websql works
  • Get a browser build together; one for websql, one with everything for demo'ing query building in the browser... which will be a cool feature for the docs.

A simple bench against building a simple query with a few where's showed about a 4x improvement from 0.5

Interested to hear any questions/thoughts people have on this, feel free to take a look through and let me know what you think. Hoping to get this out by the end of the month.

/cc @tkellen and @neilk

Copy link

@tkellen tkellen commented May 27, 2014

/me waits with baited breath.

Copy link
Member Author

@tgriesser tgriesser commented May 27, 2014

Alright. Forget whatever's "left", I'm just going to get migrations working, docs and cut a release by the end of the day, and then we can just 0.6.x features from there.

Copy link

@johanneslumpe johanneslumpe commented May 27, 2014

Very nice! Next up is getting Bookshelf to work with 0.6.x right?

Sent from my iPhone

On May 27, 2014, at 7:16 PM, Tim Griesser wrote:

Alright. Forget whatever's "left", I'm just going to get migrations working, docs and cut a release by the end of the day, and then we can just 0.6.x features from there.

Reply to this email directly or view it on GitHub.

Copy link
Member Author

@tgriesser tgriesser commented May 27, 2014

Copy link

@tkellen tkellen commented May 27, 2014

@tgriesser awesome. As soon as this stuff is live I'm gonna swing through every project we're using knex/bookshelf on and update them.

@tgriesser tgriesser merged commit 6b85f1d into master Jun 4, 2014
@tgriesser tgriesser deleted the 0.6.0-alpha branch Jun 4, 2014
Copy link

coolaj86 commented on bd31479 Jun 11, 2014

Possibly unhandled error: syntax error at or near "["

The json values must be quoted and escaped "'" + JSON.stringify(value).replace("'", "''") + "'"

Is ' always the quote / escape character in SQL or does it vary between MySQL / PostgreSQL / SQLite?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants