Alerts & Reports - CSV as Message Content resulting HTTP 500

[nivdann]

I am trying to understand why there is an internal server error when creating a new alert that will send a chart as CSV file to email.

Superset Config

 ROW_LIMIT = 5000
 ENABLE_PROXY_FIX = True
 ALERT_REPORTS_NOTIFICATION_DRY_RUN = False
FEATURE_FLAGS = {
     "DYNAMIC_PLUGINS": True,
     "ALLOW_ADHOC_SUBQUERY": True,
     "ENABLE_TEMPLATE_PROCESSING": True,
     "DASHBOARD_VIRTUALIZATION": True,
     "DRILL_BY": True,
     "GLOBAL_ASYNC_QUERIES": False,
     "ALERT_REPORTS": True
}
SMTP_HOST = os.getenv("SMTP_HOST")
SMTP_STARTTLS = ast.literal_eval(os.getenv("SMTP_STARTTLS"))
SMTP_SSL = ast.literal_eval(os.getenv("SMTP_SSL"))
SMTP_SSL_SERVER_AUTH = ast.literal_eval(os.getenv("SMTP_SSL_SERVER_AUTH"))
SMTP_PORT = os.getenv("SMTP_PORT")
SMTP_MAIL_FROM = os.getenv("SMTP_MAIL_FROM")
SMTP_USER = ""
SMTP_PASSWORD = ""


from celery.schedules import crontab

class CeleryConfig:
    broker_url = f"redis://{env('REDIS_HOST')}:{env('REDIS_PORT')}/0"
    imports = (
        "superset.sql_lab",
        "superset.tasks.cache",
        "superset.tasks.scheduler",
    )
    result_backend = f"redis://{env('REDIS_HOST')}:{env('REDIS_PORT')}/0"
    worker_prefetch_multiplier = 10
    task_acks_late = True
    task_annotations = {
        "sql_lab.get_sql_results": {
            "rate_limit": "100/s",
        },
    }
    beat_schedule = {
        "reports.scheduler": {
            "task": "reports.scheduler",
            "schedule": crontab(minute="*", hour="*"),
        },
        "reports.prune_log": {
            "task": "reports.prune_log",
            "schedule": crontab(minute=0, hour=0),
        },
    }
CELERY_CONFIG = CeleryConfig


LOG_LEVEL = 'DEBUG'
SCREENSHOT_LOCATE_WAIT = 100
SCREENSHOT_LOAD_WAIT = 600
EMAIL_PAGE_RENDER_WAIT = 60
WEBDRIVER_BASEURL = "http://{{ template "superset.fullname" . }}:{{ .Values.service.port }}"
WEBDRIVER_BASEURL_USER_FRIENDLY = "http://superset:8088"


SUPERSET_WEBSERVER_TIMEOUT = 300


from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

capabilities = DesiredCapabilities.FIREFOX.copy()
capabilities['proxy'] = {
    "proxyType": "DIRECT" # Direct connection, no proxy
}
WEBDRIVER_CONFIGURATION = {"capabilities": capabilities}
WEBDRIVER_OPTION_ARGS = ["--headless"]

SECRET_KEY = os.getenv("SUPERSET_SECRET_KEY")

I using helm to deploy to my EKS running AWS.
Using Firefox and geckodriver (defaults).

Alerts and Snapshots work and are sent to my email as expected. The only thing left that is not working is CSV charts.

Worker logs

[2024-05-16 12:32:00,085: INFO/ForkPoolWorker-1] Scheduling alert test122 eta: 2024-05-16 12:32:00
Executing alert/report, task id: 053e10a5-4907-4c85-acfb-cf1cf9f1df53, scheduled_dttm: 2024-05-16T12:32:00
[2024-05-16 12:32:00,089: INFO/ForkPoolWorker-1] Executing alert/report, task id: 053e10a5-4907-4c85-acfb-cf1cf9f1df53, scheduled_dttm: 2024-05-16T12:32:00
session is validated: id 13, executionid: 053e10a5-4907-4c85-acfb-cf1cf9f1df53
[2024-05-16 12:32:00,089: INFO/ForkPoolWorker-1] session is validated: id 13, executionid: 053e10a5-4907-4c85-acfb-cf1cf9f1df53
Running report schedule 053e10a5-4907-4c85-acfb-cf1cf9f1df53 as user admin
[2024-05-16 12:32:00,129: INFO/ForkPoolWorker-1] Running report schedule 053e10a5-4907-4c85-acfb-cf1cf9f1df53 as user admin
Query for test122 took 1508.18 ms
[2024-05-16 12:32:01,685: INFO/ForkPoolWorker-1] Query for test122 took 1508.18 ms
Getting chart from http://superset:8088/api/v1/chart/2/data/?format=csv&type=post_processed&force=true as user admin
[2024-05-16 12:32:01,699: INFO/ForkPoolWorker-1] Getting chart from http://superset:8088/api/v1/chart/2/data/?format=csv&type=post_processed&force=true as user admin
header_data in notifications for alerts and reports {'notification_type': 'Alert', 'notification_source': <ReportSourceFormat.CHART: 'chart'>, 'notification_format': 'CSV', 'chart_id': 2, 'dashboard_id': None, 'owners': [Superset Admin]}, taskid, 053e10a5-4907-4c85-acfb-cf1cf9f1df53
[2024-05-16 12:32:01,789: INFO/ForkPoolWorker-1] header_data in notifications for alerts and reports {'notification_type': 'Alert', 'notification_source': <ReportSourceFormat.CHART: 'chart'>, 'notification_format': 'CSV', 'chart_id': 2, 'dashboard_id': None, 'owners': [Superset Admin]}, taskid, 053e10a5-4907-4c85-acfb-cf1cf9f1df53
Report sent to email, notification content is None
[2024-05-16 12:32:01,957: INFO/ForkPoolWorker-1] Report sent to email, notification content is None
A downstream exception occurred while generating a report: 053e10a5-4907-4c85-acfb-cf1cf9f1df53. Failed generating csv HTTP Error 500: Internal Server Error
Traceback (most recent call last):
File "/app/superset/commands/report/execute.py", line 258, in _get_csv_data
csv_data = get_chart_csv_data(chart_url=url, auth_cookies=auth_cookies)
File "/app/superset/utils/csv.py", line 92, in get_chart_csv_data
response = opener.open(chart_url)
File "/usr/local/lib/python3.10/urllib/request.py", line 525, in open
response = meth(req, response)
File "/usr/local/lib/python3.10/urllib/request.py", line 634, in http_response
response = self.parent.error(
File "/usr/local/lib/python3.10/urllib/request.py", line 563, in error
return self._call_chain(*args)
File "/usr/local/lib/python3.10/urllib/request.py", line 496, in _call_chain
result = func(*args)
File "/usr/local/lib/python3.10/urllib/request.py", line 643, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 500: Internal Server Error

Please, help here...

[sfirke]

I am experiencing this too today. Did you solve this? EDIT: I see you opened an issue for this, #28595 -- I will engage there instead.