Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

X-Ray failure with message "Cannot determine source table..." #38618

Closed
markbastian opened this issue Feb 9, 2024 · 0 comments · Fixed by #38624
Closed

X-Ray failure with message "Cannot determine source table..." #38618

markbastian opened this issue Feb 9, 2024 · 0 comments · Fixed by #38624
Assignees
Labels
Priority:P2 Average run of the mill bug .Team/DashViz Dashboard and Viz team Type:Bug Product defects
Milestone

Comments

@markbastian
Copy link
Contributor

markbastian commented Feb 9, 2024

To reproduce:

  • X-Ray the Reviews table in the sample database
  • Note that "Product → Price over time" is broken
  • Click the "Product → Price over time" to focus on the query to see the same behavior

See @albertoperdomo's related examples in this Slack thread.

Visual repro:
xraybroke

When this happens, you will see a message like so in the Metabase logs:

 :error "Cannot determine the source table or query for Field clause [:field 59 {:base-type :type/Float}]",

Here's an example broken query:

(qp/process-query
  {:database 1,
   :query {:aggregation [["sum" ["field" 59 {:base-type "type/Float"}]] 
                                        ["avg" ["field" 59 {:base-type "type/Float"}]]],
           :breakout [["field" 69 {:base-type "type/DateTime", :temporal-unit "month"}]],
           :source-table 8},
   :type "query"})
2024-02-09 16:02:29,815 WARN middleware.fix-bad-references :: Bad :field clause [:field 59 {:base-type :type/Float}] for field "PRODUCTS.PRICE" at [:aggregation :sum]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected.
2024-02-09 16:02:29,816 WARN middleware.fix-bad-references :: Bad :field clause [:field 59 {:base-type :type/Float}] for field "PRODUCTS.PRICE" at [:aggregation :avg]: clause should have a :join-alias. Unable to infer an appropriate join. Query may not work as expected.
Execution error (ExceptionInfo) at metabase.query-processor.util.add-alias-info/fn$&f (add_alias_info.clj:193).
Cannot determine the source table or query for Field clause [:field 59 {:base-type :type/Float}]

The issue seems to be that when the breakout is added to the query in the x-ray code, it does not add in a :source-field or other join information since field 59 (PRICE) is from the products table, but we're querying from table 8 (REVIEWS).

When manually creating the question, the right query is:

{:database 1,
 :type     "query",
 :query    {:source-table 8,
            :aggregation  [["sum" ["field" 59 {:base-type "type/Float", :source-field 71}]]
                           ["avg" ["field" 59 {:base-type "type/Float", :source-field 71}]]],
            :breakout     [["field" 69 {:base-type "type/DateTime", :temporal-unit "month"}]]}}

Field 71 is product id.

@markbastian markbastian added the .Task Not a part of any Epic, used by the Task Issue Template label Feb 9, 2024
@markbastian markbastian self-assigned this Feb 9, 2024
@markbastian markbastian added Type:Bug Product defects .Team/DashViz Dashboard and Viz team Priority:P2 Average run of the mill bug and removed .Task Not a part of any Epic, used by the Task Issue Template labels Feb 9, 2024
markbastian added a commit that referenced this issue Feb 9, 2024
When aggregates are populated that reference other tables, either a `:source-field` must be provided in the field metadata or a `:join-alias` needs to be provided. The former is preferred.

Prior to this PR, the first situation in the below simplified case could happen:

```clojure
;;Simplified case
;; busted
(qp/process-query
  {:database 1,
   ;; 59 is "PRODUCTS" -> "PRICE" which is not in the "REVIEWS" table
   :query    {:aggregation  [["sum" ["field" 59 {:base-type "type/Float"}]]]
              ;; "REVIEWS"
              :source-table 8},
   :type     "query"})

; Works
(qp/process-query
  {:database 1,
   ;; The `:source-field` metadata fixes the above query
   :query    {:aggregation  [["sum" ["field" 59 {:base-type    "type/Float"
                                                 ;; "REVIEWS" -> "PRODUCT_ID"
                                                 :source-field 71}]]]
              :source-table 8},
   :type     "query"})
```

This PR updates `metabase.automagic-dashboards.interestin/ground-metric` to use the `->reference` function for fields when building the `decoder` for transforming field names to references, which properly adds sources for external fields.

Fixes #38618
markbastian added a commit that referenced this issue Feb 12, 2024
* source-field populated in aggregates with foreign refs

When aggregates are populated that reference other tables, either a `:source-field` must be provided in the field metadata or a `:join-alias` needs to be provided. The former is preferred.

Prior to this PR, the first situation in the below simplified case could happen:

```clojure
;;Simplified case
;; busted
(qp/process-query
  {:database 1,
   ;; 59 is "PRODUCTS" -> "PRICE" which is not in the "REVIEWS" table
   :query    {:aggregation  [["sum" ["field" 59 {:base-type "type/Float"}]]]
              ;; "REVIEWS"
              :source-table 8},
   :type     "query"})

; Works
(qp/process-query
  {:database 1,
   ;; The `:source-field` metadata fixes the above query
   :query    {:aggregation  [["sum" ["field" 59 {:base-type    "type/Float"
                                                 ;; "REVIEWS" -> "PRODUCT_ID"
                                                 :source-field 71}]]]
              :source-table 8},
   :type     "query"})
```

This PR updates `metabase.automagic-dashboards.interestin/ground-metric` to use the `->reference` function for fields when building the `decoder` for transforming field names to references, which properly adds sources for external fields.

Fixes #38618

* Updating `grounded-metrics-test` to reflect updated field referencing.
github-actions bot pushed a commit that referenced this issue Feb 12, 2024
* source-field populated in aggregates with foreign refs

When aggregates are populated that reference other tables, either a `:source-field` must be provided in the field metadata or a `:join-alias` needs to be provided. The former is preferred.

Prior to this PR, the first situation in the below simplified case could happen:

```clojure
;;Simplified case
;; busted
(qp/process-query
  {:database 1,
   ;; 59 is "PRODUCTS" -> "PRICE" which is not in the "REVIEWS" table
   :query    {:aggregation  [["sum" ["field" 59 {:base-type "type/Float"}]]]
              ;; "REVIEWS"
              :source-table 8},
   :type     "query"})

; Works
(qp/process-query
  {:database 1,
   ;; The `:source-field` metadata fixes the above query
   :query    {:aggregation  [["sum" ["field" 59 {:base-type    "type/Float"
                                                 ;; "REVIEWS" -> "PRODUCT_ID"
                                                 :source-field 71}]]]
              :source-table 8},
   :type     "query"})
```

This PR updates `metabase.automagic-dashboards.interestin/ground-metric` to use the `->reference` function for fields when building the `decoder` for transforming field names to references, which properly adds sources for external fields.

Fixes #38618

* Updating `grounded-metrics-test` to reflect updated field referencing.
metabase-bot bot added a commit that referenced this issue Feb 12, 2024
* source-field populated in aggregates with foreign refs

When aggregates are populated that reference other tables, either a `:source-field` must be provided in the field metadata or a `:join-alias` needs to be provided. The former is preferred.

Prior to this PR, the first situation in the below simplified case could happen:

```clojure
;;Simplified case
;; busted
(qp/process-query
  {:database 1,
   ;; 59 is "PRODUCTS" -> "PRICE" which is not in the "REVIEWS" table
   :query    {:aggregation  [["sum" ["field" 59 {:base-type "type/Float"}]]]
              ;; "REVIEWS"
              :source-table 8},
   :type     "query"})

; Works
(qp/process-query
  {:database 1,
   ;; The `:source-field` metadata fixes the above query
   :query    {:aggregation  [["sum" ["field" 59 {:base-type    "type/Float"
                                                 ;; "REVIEWS" -> "PRODUCT_ID"
                                                 :source-field 71}]]]
              :source-table 8},
   :type     "query"})
```

This PR updates `metabase.automagic-dashboards.interestin/ground-metric` to use the `->reference` function for fields when building the `decoder` for transforming field names to references, which properly adds sources for external fields.

Fixes #38618

* Updating `grounded-metrics-test` to reflect updated field referencing.

Co-authored-by: Mark Bastian <markbastian@gmail.com>
@markbastian markbastian added this to the 0.48.6 milestone Feb 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority:P2 Average run of the mill bug .Team/DashViz Dashboard and Viz team Type:Bug Product defects
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant