tickets/DM-16821

[natelust]

Format sql query debug message

[andy-slac]

Did you try compile_kwargs={"literal_binds": True} like here: https://docs.sqlalchemy.org/en/rel_1_2/faq/sqlexpressions.html?highlight=literal_binds#how-do-i-render-sql-expressions-as-strings-possibly-with-bound-parameters-inlined

[natelust]

There might be some variant on that that works, but that compile_kwargs was producing something, that was not a valid sqllite query because of the True, that compiles to true, and sqlite only wants it to be "1" or 1

[andy-slac]

Do you want it in exactly the same format as can be executed against database? Why? I doubt that you approach can produce actual executable statement (especially for other backends besides sqlite), just imagine that you have :param1 and :param10 in your compiled statement.

You can also try pass engine object as the first argument to compile() (together with compile_kwargs) but I doubt that it will improve things as rendering with literal_binds happens in sqlalchemy itself, no in engine.

[TallJimbo]

The use case here was trying to debug why the query was returning no results, for which it actually was useful to paste the query into a SQLite terminal and fiddle with it (i.e. running subqueries individually, removing various restrictions) to understand what was going wrong.

I think I'm gathering that we may not want to change the logging to always do this insertion, but we should think about other ways to do that kind of debugging.

[natelust]

Andy I take your point as a good one, but like Jim said, it would be nice to be able to copy and paste into databases to debug. I see this working with either:

1. switching to a more robust regular expression for substitution to avoid issues like you highlight or,
2. Simply also print out the dictionary of parameter strings, which would not be copy pastable, but perhaps would be enough for an interested party to do their own formatting prior to database execution,

[andy-slac]

Dumping query with parameters dictionary is probably easiest to implement. I agree that we do need better diagnostics for users to understand what happens in preflight (e.g. why returned QGraph is empty). Messing with SQL might work for expert users but for others something more friendly should be implemented. Probably adding an extra option for preflight to exercise different sub-queries and to dump their statistics might be useful.

[natelust]

Check the latest push and comments on Jira, I was able to resolve this in a way that should make everyone happy.