Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Forced order by ID in sql.js causing 40+ second queries for single row queries. #169
On tables with 100 million+ rows, when querying for a single row using loopback it'll insert an "order by id" forcing the entire table read.
Steps to reproduce
I am using a postgres server, but it is for pretty much all sql queries.
We use this curl command to query the loopback API (I know that I can use findOne instead, but it'll still go to
Loopback generates the following SQL statement
Because hash is non-unique, and can have thousands of rows, (in this case, 115,000) this takes 40.632s on my server to complete.
Loopback should instead generate the following SQL statement:
resulting in a query time of on my server of 12ms.
Link to reproduction sandbox
If you really need me to setup a reproduction sandbox I can, but the problem should be pretty apparent.
link to offending code: https://github.com/strongloop/loopback-connector/blob/master/lib/sql.js#L1388
You could say that this is "working as intended" and that its the database structure is bad/wrong/whatever, but there is no reason why the connector should be dictating an order on all queries.
This functionality should be defined in the default scope of the datasource and model, not built into the base loopback connector.
@SippieCup thanks for bringing up the issue. I agree that
We currently have a community PR ongoing: strongloop/loopback-connector-postgresql#417. I think it's trying to fix the issue you mentioned. It's proposing to have flag
Problem: If order field is not specified, pk will be the default order field in the generated query
It would would make code unreadable, breaking on model/database changes, and would be adding another magnitude of complexity. But I guess it does "work." If thats the solution strongloop chooses, I'll just continue using my forked connector where I just removed the offending code entirely, but I'd feel sorry for other users.
I see two functional solutions: Adding a flag as you have stated and in the PR, or removing the