Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature flag causes problems with Number filters in Native query #16218

Closed
dpsutton opened this issue May 24, 2021 · 1 comment
Closed

Feature flag causes problems with Number filters in Native query #16218

dpsutton opened this issue May 24, 2021 · 1 comment
Assignees
@dpsutton
Labels
Priority:P1 Security holes w/o exploit, crashing, setup/upgrade, login, broken common features, correctness Querying/Native The SQL/native query editor Querying/Parameters & Variables Filter widgets, field filters, variables etc. .Regression Bugs that were previously fixed and/or bugs unintentionally shipped with new features. Type:Bug Product defects
Milestone
0.39.3

Comments

@dpsutton
Copy link
Contributor

This is part 2 from #15981

Separated so can be marked in the release.

Native query > Sample Dataset > select * from ORDERS where QUANTITY = {{number_filter}} - set filter type to "Number" and set widget to 20 and run query.

https://user-images.githubusercontent.com/1447303/117578054-d0fdda80-b0ec-11eb-872d-6bb4a0ef0c98.png

Errors with notmatches-some-precondition?

Full stacktrace 
2021-05-09 17:33:43,121 ERROR middleware.catch-exceptions :: Error processing query: null
{:database_id 4,
 :started_at #t "2021-05-09T17:33:42.709908+02:00[Europe/Copenhagen]",
 :via
 [{:status :failed,
   :class clojure.lang.ExceptionInfo,
   :error "Error determining value for parameter",
   :stacktrace
   ["--> driver.common.parameters.values$fn__78952$value_for_tag__78957$fn__78958.invoke(values.clj:323)"
    "driver.common.parameters.values$fn__78952$value_for_tag__78957.invoke(values.clj:316)"
    "driver.common.parameters.values$fn__78976$query__GT_params_map__78981$fn__78985$iter__78987__78991$fn__78992.invoke(values.clj:338)"
    "driver.common.parameters.values$fn__78976$query__GT_params_map__78981$fn__78985.invoke(values.clj:337)"
    "driver.common.parameters.values$fn__78976$query__GT_params_map__78981.invoke(values.clj:328)"
    "driver.sql$fn__80857.invokeStatic(sql.clj:44)"
    "driver.sql$fn__80857.invoke(sql.clj:40)"
    "query_processor.middleware.parameters.native$expand_inner.invokeStatic(native.clj:39)"
    "query_processor.middleware.parameters.native$expand_inner.invoke(native.clj:30)"
    "query_processor.middleware.parameters$expand_one.invokeStatic(parameters.clj:50)"
    "query_processor.middleware.parameters$expand_one.invoke(parameters.clj:41)"
    "query_processor.middleware.parameters$expand_all$replace_48286__48287.invoke(parameters.clj:59)"
    "mbql.util.match.impl$replace_in_collection$iter__17982__17986$fn__17987.invoke(impl.cljc:44)"
    "mbql.util.match.impl$replace_in_collection.invokeStatic(impl.cljc:43)"
    "mbql.util.match.impl$replace_in_collection.invoke(impl.cljc:38)"
    "query_processor.middleware.parameters$expand_all$replace_48286__48287.invoke(parameters.clj:59)"
    "query_processor.middleware.parameters$expand_all.invokeStatic(parameters.clj:59)"
    "query_processor.middleware.parameters$expand_all.invoke(parameters.clj:53)"
    "query_processor.middleware.parameters$expand_all.invokeStatic(parameters.clj:56)"
    "query_processor.middleware.parameters$expand_all.invoke(parameters.clj:53)"
    "query_processor.middleware.parameters$expand_parameters.invokeStatic(parameters.clj:77)"
    "query_processor.middleware.parameters$expand_parameters.invoke(parameters.clj:73)"
    "query_processor.middleware.parameters$fn__48302$substitute_parameters_STAR___48307$fn__48308.invoke(parameters.clj:82)"
    "query_processor.middleware.parameters$fn__48302$substitute_parameters_STAR___48307.invoke(parameters.clj:79)"
    "query_processor.middleware.parameters$substitute_parameters$fn__48328.invoke(parameters.clj:111)"
    "query_processor.middleware.resolve_referenced$resolve_referenced_card_resources$fn__46258.invoke(resolve_referenced.clj:79)"
    "query_processor.middleware.expand_macros$expand_macros$fn__46879.invoke(expand_macros.clj:155)"
    "query_processor.middleware.add_timezone_info$add_timezone_info$fn__44803.invoke(add_timezone_info.clj:15)"
    "query_processor.middleware.splice_params_in_response$splice_params_in_response$fn__49320.invoke(splice_params_in_response.clj:32)"
    "query_processor.middleware.resolve_database_and_driver$resolve_database_and_driver$fn__48531$fn__48535.invoke(resolve_database_and_driver.clj:31)"
    "driver$do_with_driver.invokeStatic(driver.clj:60)"
    "driver$do_with_driver.invoke(driver.clj:56)"
    "query_processor.middleware.resolve_database_and_driver$resolve_database_and_driver$fn__48531.invoke(resolve_database_and_driver.clj:25)"
    "query_processor.middleware.fetch_source_query$resolve_card_id_source_tables$fn__47119.invoke(fetch_source_query.clj:274)"
    "query_processor.middleware.store$initialize_store$fn__49329$fn__49330.invoke(store.clj:11)"
    "query_processor.store$do_with_store.invokeStatic(store.clj:44)"
    "query_processor.store$do_with_store.invoke(store.clj:38)"
    "query_processor.middleware.store$initialize_store$fn__49329.invoke(store.clj:10)"
    "query_processor.middleware.validate$validate_query$fn__49374.invoke(validate.clj:10)"
    "query_processor.middleware.normalize_query$normalize$fn__47255.invoke(normalize_query.clj:22)"
    "query_processor.middleware.add_rows_truncated$add_rows_truncated$fn__44649.invoke(add_rows_truncated.clj:35)"
    "query_processor.middleware.results_metadata$record_and_return_metadata_BANG_$fn__49305.invoke(results_metadata.clj:147)"
    "query_processor.middleware.constraints$add_default_userland_constraints$fn__46500.invoke(constraints.clj:42)"
    "query_processor.middleware.process_userland_query$process_userland_query$fn__48417.invoke(process_userland_query.clj:135)"
    "query_processor.middleware.catch_exceptions$catch_exceptions$fn__46440.invoke(catch_exceptions.clj:173)"
    "query_processor.reducible$async_qp$qp_STAR___37978$thunk__37979.invoke(reducible.clj:103)"
    "query_processor.reducible$async_qp$qp_STAR___37978.invoke(reducible.clj:109)"
    "query_processor.reducible$sync_qp$qp_STAR___37987$fn__37990.invoke(reducible.clj:135)"
    "query_processor.reducible$sync_qp$qp_STAR___37987.invoke(reducible.clj:134)"
    "query_processor$process_userland_query.invokeStatic(query_processor.clj:239)"
    "query_processor$process_userland_query.doInvoke(query_processor.clj:235)"
    "query_processor$fn__49464$process_query_and_save_execution_BANG___49473$fn__49476.invoke(query_processor.clj:251)"
    "query_processor$fn__49464$process_query_and_save_execution_BANG___49473.invoke(query_processor.clj:243)"
    "query_processor$fn__49508$process_query_and_save_with_max_results_constraints_BANG___49517$fn__49520.invoke(query_processor.clj:263)"
    "query_processor$fn__49508$process_query_and_save_with_max_results_constraints_BANG___49517.invoke(query_processor.clj:256)"
    "api.dataset$run_query_async$fn__55721.invoke(dataset.clj:56)"
    "query_processor.streaming$streaming_response_STAR_$fn__55700$fn__55701.invoke(streaming.clj:72)"
    "query_processor.streaming$streaming_response_STAR_$fn__55700.invoke(streaming.clj:71)"
    "async.streaming_response$do_f_STAR_.invokeStatic(streaming_response.clj:65)"
    "async.streaming_response$do_f_STAR_.invoke(streaming_response.clj:63)"
    "async.streaming_response$do_f_async$fn__16071.invoke(streaming_response.clj:84)"],
   :error_type :invalid-query,
   :ex-data
   {:tag
    {:id "733da179-cdc7-87bd-864d-dfb51c87657a", :name "number_filter", :display-name "Number filter", :type :number},
    :type :schema.core/error}}
  {:status :failed,
   :class clojure.lang.ExceptionInfo,
   :error "Error building query parameter map",
   :stacktrace
   ["--> driver.common.parameters.values$fn__78976$query__GT_params_map__78981$fn__78985.invoke(values.clj:346)"
    "driver.common.parameters.values$fn__78976$query__GT_params_map__78981.invoke(values.clj:328)"
    "driver.sql$fn__80857.invokeStatic(sql.clj:44)"
    "driver.sql$fn__80857.invoke(sql.clj:40)"
    "query_processor.middleware.parameters.native$expand_inner.invokeStatic(native.clj:39)"
    "query_processor.middleware.parameters.native$expand_inner.invoke(native.clj:30)"
    "query_processor.middleware.parameters$expand_one.invokeStatic(parameters.clj:50)"
    "query_processor.middleware.parameters$expand_one.invoke(parameters.clj:41)"
    "query_processor.middleware.parameters$expand_all$replace_48286__48287.invoke(parameters.clj:59)"
    "mbql.util.match.impl$replace_in_collection$iter__17982__17986$fn__17987.invoke(impl.cljc:44)"
    "mbql.util.match.impl$replace_in_collection.invokeStatic(impl.cljc:43)"
    "mbql.util.match.impl$replace_in_collection.invoke(impl.cljc:38)"
    "query_processor.middleware.parameters$expand_all$replace_48286__48287.invoke(parameters.clj:59)"
    "query_processor.middleware.parameters$expand_all.invokeStatic(parameters.clj:59)"
    "query_processor.middleware.parameters$expand_all.invoke(parameters.clj:53)"
    "query_processor.middleware.parameters$expand_all.invokeStatic(parameters.clj:56)"
    "query_processor.middleware.parameters$expand_all.invoke(parameters.clj:53)"
    "query_processor.middleware.parameters$expand_parameters.invokeStatic(parameters.clj:77)"
    "query_processor.middleware.parameters$expand_parameters.invoke(parameters.clj:73)"
    "query_processor.middleware.parameters$fn__48302$substitute_parameters_STAR___48307$fn__48308.invoke(parameters.clj:82)"
    "query_processor.middleware.parameters$fn__48302$substitute_parameters_STAR___48307.invoke(parameters.clj:79)"
    "query_processor.middleware.parameters$substitute_parameters$fn__48328.invoke(parameters.clj:111)"
    "query_processor.middleware.resolve_referenced$resolve_referenced_card_resources$fn__46258.invoke(resolve_referenced.clj:79)"
    "query_processor.middleware.expand_macros$expand_macros$fn__46879.invoke(expand_macros.clj:155)"
    "query_processor.middleware.add_timezone_info$add_timezone_info$fn__44803.invoke(add_timezone_info.clj:15)"
    "query_processor.middleware.splice_params_in_response$splice_params_in_response$fn__49320.invoke(splice_params_in_response.clj:32)"
    "query_processor.middleware.resolve_database_and_driver$resolve_database_and_driver$fn__48531$fn__48535.invoke(resolve_database_and_driver.clj:31)"
    "driver$do_with_driver.invokeStatic(driver.clj:60)"
    "driver$do_with_driver.invoke(driver.clj:56)"
    "query_processor.middleware.resolve_database_and_driver$resolve_database_and_driver$fn__48531.invoke(resolve_database_and_driver.clj:25)"
    "query_processor.middleware.fetch_source_query$resolve_card_id_source_tables$fn__47119.invoke(fetch_source_query.clj:274)"
    "query_processor.middleware.store$initialize_store$fn__49329$fn__49330.invoke(store.clj:11)"
    "query_processor.store$do_with_store.invokeStatic(store.clj:44)"
    "query_processor.store$do_with_store.invoke(store.clj:38)"
    "query_processor.middleware.store$initialize_store$fn__49329.invoke(store.clj:10)"
    "query_processor.middleware.validate$validate_query$fn__49374.invoke(validate.clj:10)"
    "query_processor.middleware.normalize_query$normalize$fn__47255.invoke(normalize_query.clj:22)"
    "query_processor.middleware.add_rows_truncated$add_rows_truncated$fn__44649.invoke(add_rows_truncated.clj:35)"
    "query_processor.middleware.results_metadata$record_and_return_metadata_BANG_$fn__49305.invoke(results_metadata.clj:147)"
    "query_processor.middleware.constraints$add_default_userland_constraints$fn__46500.invoke(constraints.clj:42)"
    "query_processor.middleware.process_userland_query$process_userland_query$fn__48417.invoke(process_userland_query.clj:135)"
    "query_processor.middleware.catch_exceptions$catch_exceptions$fn__46440.invoke(catch_exceptions.clj:173)"
    "query_processor.reducible$async_qp$qp_STAR___37978$thunk__37979.invoke(reducible.clj:103)"
    "query_processor.reducible$async_qp$qp_STAR___37978.invoke(reducible.clj:109)"
    "query_processor.reducible$sync_qp$qp_STAR___37987$fn__37990.invoke(reducible.clj:135)"
    "query_processor.reducible$sync_qp$qp_STAR___37987.invoke(reducible.clj:134)"
    "query_processor$process_userland_query.invokeStatic(query_processor.clj:239)"
    "query_processor$process_userland_query.doInvoke(query_processor.clj:235)"
    "query_processor$fn__49464$process_query_and_save_execution_BANG___49473$fn__49476.invoke(query_processor.clj:251)"
    "query_processor$fn__49464$process_query_and_save_execution_BANG___49473.invoke(query_processor.clj:243)"
    "query_processor$fn__49508$process_query_and_save_with_max_results_constraints_BANG___49517$fn__49520.invoke(query_processor.clj:263)"
    "query_processor$fn__49508$process_query_and_save_with_max_results_constraints_BANG___49517.invoke(query_processor.clj:256)"
    "api.dataset$run_query_async$fn__55721.invoke(dataset.clj:56)"
    "query_processor.streaming$streaming_response_STAR_$fn__55700$fn__55701.invoke(streaming.clj:72)"
    "query_processor.streaming$streaming_response_STAR_$fn__55700.invoke(streaming.clj:71)"
    "async.streaming_response$do_f_STAR_.invokeStatic(streaming_response.clj:65)"
    "async.streaming_response$do_f_STAR_.invoke(streaming_response.clj:63)"
    "async.streaming_response$do_f_async$fn__16071.invoke(streaming_response.clj:84)"],
   :error_type :invalid-query,
   :ex-data
   {:type :schema.core/error,
    :tags
    {"number_filter"
     {:id "733da179-cdc7-87bd-864d-dfb51c87657a",
      :name "number_filter",
      :display-name "Number filter",
      :type :number}},
    :params [{:type :string/=, :value ["20"], :target [:variable [:template-tag "number_filter"]]}]}}],
 :error_type :invalid-query,
 :json_query
 {:database 4,
  :native
  {:template-tags
   {:number_filter
    {:id "733da179-cdc7-87bd-864d-dfb51c87657a",
     :name "number_filter",
     :display-name "Number filter",
     :type "number",
     :default nil}},
   :query "select * from ORDERS where QUANTITY = {{number_filter}}"},
  :type "native",
  :parameters [{:type "string/=", :value ["20"], :target ["variable" ["template-tag" "number_filter"]]}],
  :middleware {:js-int-to-string? true, :add-default-userland-constraints? true}},
 :status :failed,
 :class clojure.lang.ExceptionInfo,
 :stacktrace
 ["--> driver.common.parameters.values$fn__78814$value__GT_number__78819.invoke(values.clj:225)"
  "driver.common.parameters.values$fn__78912$parse_value_for_type__78917$fn__78918.invoke(values.clj:294)"
  "driver.common.parameters.values$fn__78912$parse_value_for_type__78917.invoke(values.clj:282)"
  "driver.common.parameters.values$fn__78952$value_for_tag__78957$fn__78958.invoke(values.clj:321)"
  "driver.common.parameters.values$fn__78952$value_for_tag__78957.invoke(values.clj:316)"
  "driver.common.parameters.values$fn__78976$query__GT_params_map__78981$fn__78985$iter__78987__78991$fn__78992.invoke(values.clj:338)"
  "driver.common.parameters.values$fn__78976$query__GT_params_map__78981$fn__78985.invoke(values.clj:337)"
  "driver.common.parameters.values$fn__78976$query__GT_params_map__78981.invoke(values.clj:328)"
  "driver.sql$fn__80857.invokeStatic(sql.clj:44)"
  "driver.sql$fn__80857.invoke(sql.clj:40)"
  "query_processor.middleware.parameters.native$expand_inner.invokeStatic(native.clj:39)"
  "query_processor.middleware.parameters.native$expand_inner.invoke(native.clj:30)"
  "query_processor.middleware.parameters$expand_one.invokeStatic(parameters.clj:50)"
  "query_processor.middleware.parameters$expand_one.invoke(parameters.clj:41)"
  "query_processor.middleware.parameters$expand_all$replace_48286__48287.invoke(parameters.clj:59)"
  "mbql.util.match.impl$replace_in_collection$iter__17982__17986$fn__17987.invoke(impl.cljc:44)"
  "mbql.util.match.impl$replace_in_collection.invokeStatic(impl.cljc:43)"
  "mbql.util.match.impl$replace_in_collection.invoke(impl.cljc:38)"
  "query_processor.middleware.parameters$expand_all$replace_48286__48287.invoke(parameters.clj:59)"
  "query_processor.middleware.parameters$expand_all.invokeStatic(parameters.clj:59)"
  "query_processor.middleware.parameters$expand_all.invoke(parameters.clj:53)"
  "query_processor.middleware.parameters$expand_all.invokeStatic(parameters.clj:56)"
  "query_processor.middleware.parameters$expand_all.invoke(parameters.clj:53)"
  "query_processor.middleware.parameters$expand_parameters.invokeStatic(parameters.clj:77)"
  "query_processor.middleware.parameters$expand_parameters.invoke(parameters.clj:73)"
  "query_processor.middleware.parameters$fn__48302$substitute_parameters_STAR___48307$fn__48308.invoke(parameters.clj:82)"
  "query_processor.middleware.parameters$fn__48302$substitute_parameters_STAR___48307.invoke(parameters.clj:79)"
  "query_processor.middleware.parameters$substitute_parameters$fn__48328.invoke(parameters.clj:111)"
  "query_processor.middleware.resolve_referenced$resolve_referenced_card_resources$fn__46258.invoke(resolve_referenced.clj:79)"
  "query_processor.middleware.expand_macros$expand_macros$fn__46879.invoke(expand_macros.clj:155)"
  "query_processor.middleware.add_timezone_info$add_timezone_info$fn__44803.invoke(add_timezone_info.clj:15)"
  "query_processor.middleware.splice_params_in_response$splice_params_in_response$fn__49320.invoke(splice_params_in_response.clj:32)"
  "query_processor.middleware.resolve_database_and_driver$resolve_database_and_driver$fn__48531$fn__48535.invoke(resolve_database_and_driver.clj:31)"
  "driver$do_with_driver.invokeStatic(driver.clj:60)"
  "driver$do_with_driver.invoke(driver.clj:56)"
  "query_processor.middleware.resolve_database_and_driver$resolve_database_and_driver$fn__48531.invoke(resolve_database_and_driver.clj:25)"
  "query_processor.middleware.fetch_source_query$resolve_card_id_source_tables$fn__47119.invoke(fetch_source_query.clj:274)"
  "query_processor.middleware.store$initialize_store$fn__49329$fn__49330.invoke(store.clj:11)"
  "query_processor.store$do_with_store.invokeStatic(store.clj:44)"
  "query_processor.store$do_with_store.invoke(store.clj:38)"
  "query_processor.middleware.store$initialize_store$fn__49329.invoke(store.clj:10)"
  "query_processor.middleware.validate$validate_query$fn__49374.invoke(validate.clj:10)"
  "query_processor.middleware.normalize_query$normalize$fn__47255.invoke(normalize_query.clj:22)"
  "query_processor.middleware.add_rows_truncated$add_rows_truncated$fn__44649.invoke(add_rows_truncated.clj:35)"
  "query_processor.middleware.results_metadata$record_and_return_metadata_BANG_$fn__49305.invoke(results_metadata.clj:147)"
  "query_processor.middleware.constraints$add_default_userland_constraints$fn__46500.invoke(constraints.clj:42)"
  "query_processor.middleware.process_userland_query$process_userland_query$fn__48417.invoke(process_userland_query.clj:135)"
  "query_processor.middleware.catch_exceptions$catch_exceptions$fn__46440.invoke(catch_exceptions.clj:173)"
  "query_processor.reducible$async_qp$qp_STAR___37978$thunk__37979.invoke(reducible.clj:103)"
  "query_processor.reducible$async_qp$qp_STAR___37978.invoke(reducible.clj:109)"
  "query_processor.reducible$sync_qp$qp_STAR___37987$fn__37990.invoke(reducible.clj:135)"
  "query_processor.reducible$sync_qp$qp_STAR___37987.invoke(reducible.clj:134)"
  "query_processor$process_userland_query.invokeStatic(query_processor.clj:239)"
  "query_processor$process_userland_query.doInvoke(query_processor.clj:235)"
  "query_processor$fn__49464$process_query_and_save_execution_BANG___49473$fn__49476.invoke(query_processor.clj:251)"
  "query_processor$fn__49464$process_query_and_save_execution_BANG___49473.invoke(query_processor.clj:243)"
  "query_processor$fn__49508$process_query_and_save_with_max_results_constraints_BANG___49517$fn__49520.invoke(query_processor.clj:263)"
  "query_processor$fn__49508$process_query_and_save_with_max_results_constraints_BANG___49517.invoke(query_processor.clj:256)"
  "api.dataset$run_query_async$fn__55721.invoke(dataset.clj:56)"
  "query_processor.streaming$streaming_response_STAR_$fn__55700$fn__55701.invoke(streaming.clj:72)"
  "query_processor.streaming$streaming_response_STAR_$fn__55700.invoke(streaming.clj:71)"
  "async.streaming_response$do_f_STAR_.invokeStatic(streaming_response.clj:65)"
  "async.streaming_response$do_f_STAR_.invoke(streaming_response.clj:63)"
  "async.streaming_response$do_f_async$fn__16071.invoke(streaming_response.clj:84)"],
 :context :ad-hoc,
 :error (not (matches-some-precondition? nil)),
 :row_count 0,
 :running_time 0,
 :ex-data {:type :schema.core/error, :value nil, :error (not (matches-some-precondition? nil))},
 :data {:rows [], :cols []}}

2021-05-09 17:33:43,124 DEBUG middleware.log :: POST /api/dataset 202 [ASYNC: completed] 419.5 ms (4 DB calls) App DB connections: 0/13 Jetty threads: 3/50 (3 idle, 0 queued) (102 total active threads) Queries in flight: 0 (0 queued)
@dpsutton dpsutton added Type:Bug Product defects Priority:P1 Security holes w/o exploit, crashing, setup/upgrade, login, broken common features, correctness Querying/Parameters & Variables Filter widgets, field filters, variables etc. Querying/Native The SQL/native query editor .Regression Bugs that were previously fixed and/or bugs unintentionally shipped with new features. labels May 24, 2021
@dpsutton dpsutton added this to the 0.39.3 milestone May 24, 2021
@dpsutton
Copy link
Contributor Author

Closed by #15992

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dpsutton dpsutton

Labels
Priority:P1 Security holes w/o exploit, crashing, setup/upgrade, login, broken common features, correctness Querying/Native The SQL/native query editor Querying/Parameters & Variables Filter widgets, field filters, variables etc. .Regression Bugs that were previously fixed and/or bugs unintentionally shipped with new features. Type:Bug Product defects
Projects
None yet
Milestone
0.39.3
Development

No branches or pull requests
1 participant
@dpsutton