Some questions with old field dimensions or changed columns cannot be viewed, because of Javascript loop

[mlazowik]

Describe the bug
After updating from v0.40.5 to v0.41.1 some of the existing questions do not load in the single question view (including notebook). When opening the page gets stuck on "Loading" (the spinner does continue animating), browser rendering thread runs at over 100% cpu, no actions on the page work (hover, click, even right click for browser menu. I can't even open another url in the same tab after this, the only thing I can do is close the tab.

Logs
It's clearly a browser-side issue, I've tried getting a performance trace of what is going on, but so far no luck – I can start profiling before loading the page, but when I stop profiling it doesn't finish processing the profile. I'll be trying more.

To Reproduce
Here's the tricky part I'm really not sure. I haven't found any regularity in which questions load and which don't, and to be honest it's not that easy to do. I put more hope in the results of the profile, or maybe I could also prepare anonymized results of API calls from the page load.

Expected behavior
Question loads.

Screenshots

Information about your Metabase Installation:

• Your browser and the version: Chrome 94.0.4606.81, Safari Technology Preview Release 133 (Safari 15.4, WebKit 16613.1.2.2)
• Your operating system: macOS 11.6 (20G165)
• Your databases: Postgres
• Metabase version: 0.41.1
• Metabase hosting environment: Docker
• Metabase internal database: Postgres

Severity
It's blocking the update, for now I've reverted.

Additional context
I know that currently this might be completely unreproducible, I'm opening now so that maybe others with the same problem can chip in, I'll try to get better info later.

[mlazowik]

Diff of /api/card/<number> between v40 and v41:

michal@whoooosh ~/t/met> diff v40.json v41.json 
10a11
>       "effective_type": "type/DateTime",
28,29c29,30
<             "earliest": "2021-04-23T00:00:00Z",
<             "latest": "2021-04-23T00:00:00Z"
---
>             "earliest": "2020-12-15T00:00:00Z",
>             "latest": "2020-12-15T00:00:00Z"
38c39
<       "semantic_type": null,
---
>       "effective_type": "type/Float",
42a44
>       "semantic_type": null,
45c47
<           "distinct-count": 14,
---
>           "distinct-count": 15,
51,55c53,57
<             "q1": 86.8532206969377,
<             "q3": 99.12948857453753,
<             "max": 99.69378827646544,
<             "sd": 7.340990057442863,
<             "avg": 93.12121021742163
---
>             "q1": 87.83460434642436,
>             "q3": 98.9928654187574,
>             "max": 99.69485614646905,
>             "sd": 7.097002915884887,
>             "avg": 93.27214043928817
77a80
>   "last_query_start": "2021-10-22T23:04:09.193682Z",
78a82
>   "average_query_time": 37599.333333333336,
80c84,85
<   "updated_at": "2021-09-27T19:57:29.656203Z",
---
>   "moderation_reviews": [],
>   "updated_at": "2021-10-22T23:07:18.650171Z",
303,304c308
<       "expression",
<       null
---
>       "expression"

[mlazowik]

Ok, after waiting for devtools for a few minutes they did load the profile after all. I got following console errors:

react-dom.production.min.js:209 RangeError: Maximum call stack size exceeded
    at _createPredicateIndexFinder.js:11
    at At (find.js:7)
    at Function.Tt [as findWhere] (findWhere.js:7)
    at n.value (Dimension.js:560)
    at f (StructuredQuery.js:1143)
    at Array.filter (<anonymous>)
    at n.value (StructuredQuery.js:1156)
    at i.value (Join.js:587)
    at i.value (Join.js:596)
    at i.value (Join.js:615)

react-dom.production.min.js:209 RangeError: Maximum call stack size exceeded
    at n.value (StructuredQuery.js:577)
    at i.value (Join.js:55)
    at i.value (MBQLClause.js:109)
    at i.value (Join.js:587)
    at i.value (Join.js:596)
    at i.value (Join.js:615)
    at i.value (Join.js:615)
    at i.value (Join.js:615)
    at i.value (Join.js:615)
    at i.value (Join.js:615)

app-main.bundle.js?318bf132ebe65a2a0fcf:2 Uncaught (in promise) RangeError: Maximum call stack size exceeded
    at n.value (StructuredQuery.js:577)
    at i.value (Join.js:55)
    at i.value (MBQLClause.js:109)
    at i.value (Join.js:587)
    at i.value (Join.js:596)
    at i.value (Join.js:615)
    at i.value (Join.js:615)
    at i.value (Join.js:615)
    at i.value (Join.js:615)
    at i.value (Join.js:615)

Uncaught Error: Minified React error #188; visit https://reactjs.org/docs/error-decoder.html?invariant=188 for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
    at rt (react-dom.production.min.js:48)
    at react-dom.production.min.js:49
    at it (react-dom.production.min.js:50)
    at Object.t.findDOMNode (react-dom.production.min.js:289)
    at s.value (ExplicitSize.jsx:31)
    at s.<anonymous> (ExplicitSize.jsx:101)
    at u (throttle.js:16)

The profile page does not render reliably, but when it worked it did look something like infinite recursion.

[mlazowik]

I guess join.clean() at https://github.com/metabase/metabase/blob/v0.41.1/frontend/src/metabase-lib/lib/queries/structured/Join.js#L615 is infinite. It looks like saved questions used in this query might also matter? I'll get API responses for their metadata.

[flamber]

Hi @mlazowik
Are you able to provide the output of /api/card/123, where 123 is the question ID? (only redact as little as possible)
I'm guessing it is something specific metadata about the question, which is causing this, but I'm not seeing anything from the small log snippets, which could help me understand where to look or how to reproduce.
(I can see @andre-sca has upvoted this issue, so if you have any information, which could help, then please chip in)

[andre-sca]

Hi @flamber. I upvoted it because I'm also having the same issue after I update metabase. I had to go back to the v0.40.5 for it to work properly. My dashboards all work normally and show the questions. But if I try to open a question to edit it, I also get stuck at the loading page for a solid 5 minutes, and I also can't open my metabase in a new page. Unfortunalety I didn't find any new information about what is causing this... I'm also somewhat new to metabase and debbuging. If you need any information and can provide me a way to get it, I'll be glad to do my best to help!

[kulyk]

Indeed, this is most likely caused by join.clean() from #17708

There is something about their query metadata/structure that causes an infinite loop. @mfazekas @andre-sca it would be very helpful if you could share an anonymised /api/card/:id response for cards having this issue. If you've downgraded to Metabase < x.41.x, you can just access it from your browser's dev tools Network tab. If you're on x.41.x, you can use this curl command:

# replace `http://localhost:3000` with your Metabase instance address

curl -H "Cookie: $YOUR_COOKIE" http://localhost:3000/api/card/$QUESTION_ID

This should return a JSON describing the question. You can use a cookie from your browser, here's an example of me copying and using a cookie from my local Metabase:

Screen.Recording.2021-10-25.at.11.29.50.mov

[flamber]

Seems like it's a combination of old field dimensions and new cleanup code in 0.41, which causes this.

Reproduce (there's likely a couple of different ways to reproduce this):

1. On 0.37.8, create a question like this, where the Custom Column is coalesce([User ID], [People - User → ID])
   
2. Save question and upgrade to 0.41.0 and try to view the question

/api/card/X output

{
  "description": null,
  "archived": false,
  "collection_position": null,
  "table_id": 11,
  "result_metadata": [
    {
      "base_type": "type/BigInteger",
      "display_name": "coalesce",
      "name": "coalesce",
      "special_type": "type/FK",
      "fingerprint": {
        "global": {
          "distinct-count": 929,
          "nil%": 0
        }
      }
    },
    {
      "base_type": "type/BigInteger",
      "display_name": "Count",
      "name": "count",
      "special_type": "type/Quantity",
      "fingerprint": {
        "global": {
          "distinct-count": 40,
          "nil%": 0
        },
        "type": {
          "type/Number": {
            "min": 1,
            "q1": 4.9350211112798945,
            "q3": 15.144096695892173,
            "max": 46,
            "sd": 7.293757322525821,
            "avg": 10.744558991981672
          }
        }
      }
    },
    {
      "base_type": "type/BigInteger",
      "display_name": "People → ID",
      "name": "ID",
      "description": "A unique identifier given to each user.",
      "special_type": "type/PK",
      "fingerprint": null
    },
    {
      "base_type": "type/Text",
      "display_name": "People → Address",
      "name": "ADDRESS",
      "description": "The street address of the account’s billing address",
      "fingerprint": {
        "global": {
          "distinct-count": 2490,
          "nil%": 0
        },
        "type": {
          "type/Text": {
            "percent-json": 0,
            "percent-url": 0,
            "percent-email": 0,
            "percent-state": 0,
            "average-length": 20.85
          }
        }
      },
      "special_type": null
    },
    {
      "base_type": "type/Text",
      "display_name": "People → Email",
      "name": "EMAIL",
      "description": "The contact email for the account.",
      "special_type": "type/Category",
      "fingerprint": {
        "global": {
          "distinct-count": 2500,
          "nil%": 0
        },
        "type": {
          "type/Text": {
            "percent-json": 0,
            "percent-url": 0,
            "percent-email": 1,
            "percent-state": 0,
            "average-length": 24.1824
          }
        }
      }
    },
    {
      "base_type": "type/Text",
      "display_name": "People → Password",
      "name": "PASSWORD",
      "description": "This is the salted password of the user. It should not be visible",
      "fingerprint": {
        "global": {
          "distinct-count": 2500,
          "nil%": 0
        },
        "type": {
          "type/Text": {
            "percent-json": 0,
            "percent-url": 0,
            "percent-email": 0,
            "percent-state": 0,
            "average-length": 36
          }
        }
      },
      "special_type": null
    },
    {
      "base_type": "type/Text",
      "display_name": "People → Name",
      "name": "NAME",
      "description": "The name of the user who owns an account",
      "special_type": "type/Name",
      "fingerprint": {
        "global": {
          "distinct-count": 2499,
          "nil%": 0
        },
        "type": {
          "type/Text": {
            "percent-json": 0,
            "percent-url": 0,
            "percent-email": 0,
            "percent-state": 0,
            "average-length": 13.532
          }
        }
      }
    },
    {
      "base_type": "type/Text",
      "display_name": "People → City",
      "name": "CITY",
      "description": "The city of the account’s billing address",
      "special_type": "type/City",
      "fingerprint": {
        "global": {
          "distinct-count": 1966,
          "nil%": 0
        },
        "type": {
          "type/Text": {
            "percent-json": 0,
            "percent-url": 0,
            "percent-email": 0,
            "percent-state": 0.002,
            "average-length": 8.284
          }
        }
      }
    },
    {
      "base_type": "type/Float",
      "display_name": "People → Longitude",
      "name": "LONGITUDE",
      "description": "This is the longitude of the user on sign-up. It might be updated in the future to the last seen location.",
      "special_type": "type/Longitude",
      "fingerprint": {
        "global": {
          "distinct-count": 2491,
          "nil%": 0
        },
        "type": {
          "type/Number": {
            "min": -166.5425726,
            "max": -67.96735199999999,
            "avg": -95.18741780363999,
            "sd": 15.399698968175663,
            "q1": -101.58350792373135,
            "q3": -84.65289348288829
          }
        }
      }
    },
    {
      "base_type": "type/Text",
      "display_name": "People → State",
      "name": "STATE",
      "description": "The state or province of the account’s billing address",
      "special_type": "type/State",
      "fingerprint": {
        "global": {
          "distinct-count": 49,
          "nil%": 0
        },
        "type": {
          "type/Text": {
            "percent-json": 0,
            "percent-url": 0,
            "percent-email": 0,
            "percent-state": 1,
            "average-length": 2
          }
        }
      }
    },
    {
      "base_type": "type/Text",
      "display_name": "People → Source",
      "name": "SOURCE",
      "description": "The channel through which we acquired this user. Valid values include: Affiliate, Facebook, Google, Organic and Twitter",
      "special_type": "type/Source",
      "fingerprint": {
        "global": {
          "distinct-count": 5,
          "nil%": 0
        },
        "type": {
          "type/Text": {
            "percent-json": 0,
            "percent-url": 0,
            "percent-email": 0,
            "percent-state": 0,
            "average-length": 7.4084
          }
        }
      }
    },
    {
      "base_type": "type/Date",
      "display_name": "People → Birth Date",
      "name": "BIRTH_DATE",
      "description": "The date of birth of the user",
      "special_type": "type/Birthdate",
      "unit": "default",
      "fingerprint": {
        "global": {
          "distinct-count": 2308,
          "nil%": 0
        },
        "type": {
          "type/DateTime": {
            "earliest": "1958-04-26T06:00:00.000Z",
            "latest": "2000-04-03T06:00:00.000Z"
          }
        }
      }
    },
    {
      "base_type": "type/Text",
      "display_name": "People → Zip",
      "name": "ZIP",
      "description": "The postal code of the account’s billing address",
      "special_type": "type/ZipCode",
      "fingerprint": {
        "global": {
          "distinct-count": 2234,
          "nil%": 0
        },
        "type": {
          "type/Text": {
            "percent-json": 0,
            "percent-url": 0,
            "percent-email": 0,
            "percent-state": 0,
            "average-length": 5
          }
        }
      }
    },
    {
      "base_type": "type/Float",
      "display_name": "People → Latitude",
      "name": "LATITUDE",
      "description": "This is the latitude of the user on sign-up. It might be updated in the future to the last seen location.",
      "special_type": "type/Latitude",
      "fingerprint": {
        "global": {
          "distinct-count": 2491,
          "nil%": 0
        },
        "type": {
          "type/Number": {
            "min": 25.775827,
            "max": 70.6355001,
            "avg": 39.87934670484002,
            "sd": 6.390832341883712,
            "q1": 35.302705923023126,
            "q3": 43.773802584662
          }
        }
      }
    },
    {
      "base_type": "type/DateTime",
      "display_name": "People → Created At",
      "name": "CREATED_AT",
      "description": "The date the user record was created. Also referred to as the user’s \"join date\"",
      "special_type": "type/CreationTimestamp",
      "unit": "default",
      "fingerprint": {
        "global": {
          "distinct-count": 2500,
          "nil%": 0
        },
        "type": {
          "type/DateTime": {
            "earliest": "2016-04-20T03:35:18.752Z",
            "latest": "2019-04-19T20:06:27.300Z"
          }
        }
      }
    }
  ],
  "can_write": true,
  "database_id": 4,
  "enable_embedding": false,
  "collection_id": null,
  "query_type": "query",
  "name": "Test 18630",
  "last_query_start": null,
  "dashboard_count": 0,
  "average_query_time": null,
  "creator_id": 1,
  "moderation_reviews": [],
  "updated_at": "2021-10-25T10:36:58.242",
  "made_public_by_id": null,
  "embedding_params": null,
  "cache_ttl": null,
  "dataset_query": {
    "database": 4,
    "query": {
      "source-query": {
        "source-table": 11,
        "joins": [
          {
            "fields": "all",
            "source-table": 12,
            "condition": [
              "=",
              [
                "field",
                80,
                null
              ],
              [
                "field",
                92,
                {
                  "join-alias": "People - User"
                }
              ]
            ],
            "alias": "People - User"
          }
        ],
        "expressions": {
          "coalesce": [
            "coalesce",
            [
              "field",
              80,
              null
            ],
            [
              "field",
              92,
              {
                "join-alias": "People - User"
              }
            ]
          ]
        },
        "aggregation": [
          [
            "count"
          ]
        ],
        "breakout": [
          [
            "expression",
            "coalesce"
          ]
        ]
      },
      "joins": [
        {
          "fields": "all",
          "source-table": 12,
          "condition": [
            "=",
            [
              "field",
              "coalesce",
              {
                "base-type": "type/Float"
              }
            ],
            [
              "field",
              92,
              {
                "join-alias": "People"
              }
            ]
          ],
          "alias": "People"
        }
      ]
    },
    "type": "query"
  },
  "id": 955,
  "display": "table",
  "visualization_settings": {
    "table.pivot_column": "SOURCE",
    "table.cell_column": "count"
  },
  "collection": null,
  "created_at": "2021-10-25T10:36:58.242",
  "public_uuid": null
}

[mlazowik]

I had it saved to a file already, anonymized it now and pasted here: https://pastebin.com/j3E0Mzm7

Is this enough or do the metadata queries to saved questions this is based on matter too? I'd need to spin up v41 again to get those from that version.

[kulyk]

@mlazowik thank you! This should be enough, no need to spin up 41 again

[kulyk]

@flamber thank you for the repro steps, I'll look into the issue now