Query is resubmitting to trino in every hour

[Ratan-jeet]

While Running a query from Apache Superset which is taking time more than one hour
Superset is re triggering the same query again to the trino and old query is also running simultaneously

Is there any configuration to change/ add / update for the same

[dosubot[bot]]

Hey @Ratan-jeet! I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this issue together!

Yes, you can change the schema_cache_timeout and table_cache_timeout configurations in Apache Superset to prevent a query from being resubmitted to Trino every hour. These settings control the duration of the metadata caching timeout for schemas and tables, respectively. By setting these timeouts, you can control how often the metadata is refreshed, which can help in preventing frequent resubmission of queries.

Here is the relevant part of the configuration:

<StyledInputContainer>
  <div className="control-label">{t('Schema cache timeout')}</div>
  <div className="input-container">
    <input
      type="number"
      name="schema_cache_timeout"
      value={extraJson?.metadata_cache_timeout?.schema_cache_timeout || ''}
      placeholder={t('Enter duration in seconds')}
      onChange={onExtraInputChange}
      data-test="schema-cache-timeout-test"
    />
  </div>
  <div className="helper">
    {t(
      'Duration (in seconds) of the metadata caching timeout for schemas of ' +
        'this database. If left unset, the cache never expires.',
    )}
  </div>
</StyledInputContainer>
<StyledInputContainer>
  <div className="control-label">{t('Table cache timeout')}</div>
  <div className="input-container">
    <input
      type="number"
      name="table_cache_timeout"
      value={extraJson?.metadata_cache_timeout?.table_cache_timeout || ''}
      placeholder={t('Enter duration in seconds')}
      onChange={onExtraInputChange}
      data-test="table-cache-timeout-test"
    />
  </div>
  <div className="helper">
    {t(
      'Duration (in seconds) of the metadata caching timeout for tables of ' +
        'this database. If left unset, the cache never expires. ',
    )}
  </div>
</StyledInputContainer>

You can set these values to a higher duration to reduce the frequency of queries being resubmitted ^[1].

Additionally, if you want to limit the timed refresh on a dashboard, you can add the timed_refresh_immune_slices key to the dashboard JSON Metadata field to exclude specific slices from the timed refresh process:

{
   "filter_immune_slices": [],
    "expanded_slices": {},
    "filter_immune_slice_fields": {},
    "timed_refresh_immune_slices": [324]
}

In the example above, if a timed refresh is set for the dashboard, then every slice except 324 will be automatically re-queried on schedule ^[2].

Moreover, you can also configure the SCHEDULED_QUERIES setting in the superset/config.py file to manage the scheduling of queries from SQL Lab. This allows you to fine-tune how and when queries are executed, potentially reducing unnecessary query resubmissions.

# Configuration for scheduling queries from SQL Lab.
SCHEDULED_QUERIES: dict[str, Any] = {}

By adjusting these configurations, you can better control the frequency of query resubmissions to Trino ^[3].

To continue talking to Dosu, mention @dosu.

[virtualb0x]

Hi there. We have the same problem, but it seems to spawn new query every 5 mins. Even after superset shows to user it has reached timeout of query, queries stay in trino and are trying to be finished.

It seems to be actual for superset stand wicgh has global asynq queries off. Version 6.0.1
any ideas?