Custom columns not appearing in result set when selecting subset of fields

[dpsutton]

Create a question on the sample dataset on Orders, and create a custom column called "adjective" who's formula is case([Total] > 100, "expensive", "cheap")

The custom column appears in the results when all fields on Orders are selected, but does not appear when only a subset of fields are selected:

NOTE: most important is release-x.42.x It is possible this bug does not manifest in master. Will need a e2e test when we have narrowed and identified.

[dpsutton]

I think this is a FE bug. The query when selecting all fields is:

;; selecting all fields omits a fields entry
{:database 1
 :query {:source-table 2
         :expressions {:adjective ["case"
                                   [[[">"
                                      ["field" 7 nil]
                                      100]
                                     "expensive"]]
                                   {:default "cheap"}]}}
 :type "query"
 :parameters []}

Since there are no fields, the fields it computes all of the fields involved in the query. However, the query sent over from the FE when you select only a subset of fields:

;; selecting a subset includes only the fields from the orders table and not the custom expression
{:database 1
 :query {:source-table 2
         :expressions {:adjective ["case"
                                   [[[">"
                                      ["field" 7 nil]
                                      100]
                                     "expensive"]]
                                   {:default "cheap"}]}
         :fields [["field" 9 nil]
                  ["field" 7 nil]
                  ["field" 4 nil]]}
 :type "query"
 :parameters []}

Note the lack of the custom column in the fields to select. As far as I can tell the backend is behaving correctly and not returning the custom column.

the correct query that should be sent from the FE is the following query. Note there is no possible way for the backend to anticipate which fields should be selected once you send a subset of them. You must include all fields you want to select.

;; the correct query when you add the custom column to the query after selecting the subset of columns
{:database 1
 :query {:source-table 2
         :fields [[:field 9 nil]
                  [:field 7 nil]
                  [:field 2 nil]
                  [:expression "adjective"]]
         :expressions {:adjective ["case"
                                   [[[:> [:field 7 nil] 100]
                                     "expensive"]]
                                   {:default "cheap"}]}}
 :type :query}

If you instead select a subset of fields and then add the custom column, the FE constructs the correct query. So the bug is the transition.

Scenarios:

• don't specify fields, add custom field, then select subset of fields -> Busted. This is because there is no fields list to keep track of in the select * case, and when it introduces the fields to keep track of it doesn't account for the custom field
• specify subset of fields, then add custom field. In this code path it is aware of a fields list and adds the custom field to that list and the correct query is created.

[flamber]

Duplicate of #12395