How to compute and cache a screenshot using rest api in superset

[pritypriya25]

I want to get the chart by hitting the API. I read the documentation and found that I can do this by /chart/{pk}/cache_screenshot/
But I am unable to know what is the value of pk over here. It is an integer value and provides a path but I am unable to get the value. Can someone guide me on how can I get the value?

[zhaoyongjie]

Hi, @pritypriya25 , the PK is chart id(aka slice id), you can get this id(PK) from the chart list.

1. clicking Charts then hover on the chart that you want to compute.
2. recording the number between "%3A" and "%7d".
3. filling the number to the API .

[pritypriya25]

thank you @zhaoyongjie . Now I know what is the slice id. But still I can't get my output from hitting the api chart/{pk}/cache_screenshot.

[YangJianFei]

@pritypriya25 ,I want know , how to use access_token? can Screenshot?

[pritypriya25]

@YangJianFei you can generate the token through POST api/v1/security​/login. You can find out the request body on the official superset website https://superset.apache.org/docs/rest-api

[isandyawan]

Hi, have you get a solution? i face same issue...

[Hvitgar]

Solving this requires multiple steps. First, you have to adjust several values in your superset_config.py:
Enable the following feature flags if you want to see thumbnails for your dashboards:

FEATURE_FLAGS = {
    "THUMBNAILS": True,
    "THUMBNAILS_SQLA_LISTENERS": True,
}

provide valid cache configs for CACHE_CONFIG, DATA_CACHE_CONFIG and THUMBNAIL_CACHE_CONFIG, for testing purpose a local FileSystemCache is sufficient:

CACHE_CONFIG = {
    'CACHE_TYPE': 'FileSystemCache',
    'CACHE_DEFAULT_TIMEOUT': 60 * 60 * 24,
    'CACHE_THRESHOLD': CACHE_THRESHOLD,  # integer, limit of items in the cache
    'CACHE_DIR': BASE_CACHE_DIR,  # the directory where the cache should be stored
}

additionally, configure the RESULTS_BACKEND (once again, for testing FileSystemCache is sufficient):

from cachelib.file import FileSystemCache
import os

RESULTS_BACKEND = FileSystemCache(
                    cache_dir=os.path.join(BASE_CACHE_DIR, 'results/'),
                    threshold=CACHE_THRESHOLD,
                    default_timeout=CACHE_DEFAULT_TIMEOUT
                    )

Next, you need to provide a celery config (the one below is copied from the official superset documentation without any modifications):

class CeleryConfig(object):
    BROKER_URL = 'redis://localhost:6379/0'
    CELERY_IMPORTS = (
        'superset.sql_lab',
        'superset.tasks',
        'superset.tasks.thumbnails'
    )
    CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
    CELERYD_LOG_LEVEL = 'DEBUG'
    CELERYD_PREFETCH_MULTIPLIER = 10
    CELERY_ACKS_LATE = True
    CELERY_ANNOTATIONS = {
        'sql_lab.get_sql_results': {
            'rate_limit': '100/s',
        },
        'email_reports.send': {
            'rate_limit': '1/s',
            'time_limit': 120,
            'soft_time_limit': 150,
            'ignore_result': True,
        },
    }
CELERY_CONFIG = CeleryConfig

last steps in the superset_config.py:

WEBDRIVER_TYPE = "firefox"
WEBDRIVER_BASEURL = "http://localhost:8088"  # or whatever URL your superset instance is running on, the webdriver will use this URL to connect to superset in order to take a screenshot
THUMBNAIL_SELENIUM_USER = "admin"  # any user with permissions to view all charts is sufficient

now, in your superset environment, make sure you have a redis server running and listening on port 6379 (otherwise, adjust the redis port in the settings above). Also, you will need firefox and geckodriver.

Make sure that your redis server can write data to the disk by connecting to it via redis-cli and running the following two commands:

CONFIG SET dir /app/superset_home/redis  # where the database will be stored, the directory must exist
CONFIG SET dbfilename redis.rdb  # or any other filename

Start celery using celery --app=superset.tasks.celery_app:app worker --pool=prefork -O fair -c 4

Now, you should be able to take screenshots and see thumbnails of your dashboards

Note: I am not sure whether all settings are actually needed, this is just the setup that worked for me.

[pritypriya25]

@Hvitgar .. Thanks for providing the solution. Actually I am cloning the git repo and running the superset. Could you please provide the file path even because I can't find the file in the repository ?

[Hvitgar]

the file is not in the repository, but you can simply create it yourself and put it in your PYTHONPATH environment variable as described here. This way, you will override the defaults provided here

[isandyawan]

Thank you @Hvitgar for your solution, i have implement it, but when i access screenshot url return 404 not found. Below is log from superset worker:
superset_worker | [2021-09-06 09:08:54,659: WARNING/ForkPoolWorker-1] Failed at generating thumbnail Message: Reached error page: about:neterror?e=dnsNotFound&u=http%3A//l27.0.0.1%3A8088/login/&c=UTF-8&d=We%20can%E2%80%99t%20connect%20to%20the%20server%20at%20l27.0.0.1. superset_worker | Traceback (most recent call last): superset_worker | File "/app/superset/utils/screenshots.py", line 152, in compute_and_cache superset_worker | payload = self.get_screenshot(user=user, window_size=window_size) superset_worker | File "/app/superset/utils/screenshots.py", line 74, in get_screenshot superset_worker | self.screenshot = driver.get_screenshot(self.url, self.element, user) superset_worker | File "/app/superset/utils/webdriver.py", line 112, in get_screenshot superset_worker | driver = self.auth(user) superset_worker | File "/app/superset/utils/webdriver.py", line 87, in auth superset_worker | driver, user superset_worker | File "/app/superset/utils/machine_auth.py", line 53, in authenticate_webdriver superset_worker | driver.get(headless_url("/login/")) superset_worker | File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get superset_worker | self.execute(Command.GET, {'url': url}) superset_worker | File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute superset_worker | self.error_handler.check_response(response) superset_worker | File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response superset_worker | raise exception_class(message, screen, stacktrace) superset_worker | selenium.common.exceptions.WebDriverException: Message: Reached error page: about:neterror?e=dnsNotFound&u=http%3A//l27.0.0.1%3A8088/login/&c=UTF-8&d=We%20can%E2%80%99t%20connect%20to%20the%20server%20at%20l27.0.0.1.
I also try to test webdriver from superset container and i got it :

`

webdriver Firefox ()
Traceback (most recent call last):
File "", line 1, in module>
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 174, in init File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in init
keep_alive=True)
self.start_session (capabilities, browser profile)
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session response = self.execute(Command.NEW SESSION, parameters)
File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error handler.check_response (response) File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception class (message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message: Connection refused (os error 111)
`

[Hvitgar]

hey @isandyawan, I assume that your error is related to this line in the superset_config.py:

WEBDRIVER_BASEURL = "http://localhost:8088"  # or whatever URL your superset instance is running on, the webdriver will use this URL to connect to superset in order to take a screenshot

For taking the screenshot, a browser session accesses the URL above, renders the chart and than produces a screenshot so you have to make sure that superset is accessible on the URL above (it might be a DNS name or IP if hosted publicly, or localhost if on your machine). Also, make sure that the port is correct. Superset offers a health check under /health which returns 'OK' if successful, so you can check your configuration with curl <URL>:<PORT>/health, in my example using curl localhost:8088/health returns 'OK'.

If you are still not able to solve it, please give me more information about the way you are running superset. Are you using docker? Local installation? Cloud?

[isandyawan]

hey @isandyawan, I assume that your error is related to this line in the superset_config.py:

WEBDRIVER_BASEURL = "http://localhost:8088"  # or whatever URL your superset instance is running on, the webdriver will use this URL to connect to superset in order to take a screenshot

For taking the screenshot, a browser session accesses the URL above, renders the chart and than produces a screenshot so you have to make sure that superset is accessible on the URL above (it might be a DNS name or IP if hosted publicly, or localhost if on your machine). Also, make sure that the port is correct. Superset offers a health check under /health which returns 'OK' if successful, so you can check your configuration with curl <URL>:<PORT>/health, in my example using curl localhost:8088/health returns 'OK'.

If you are still not able to solve it, please give me more information about the way you are running superset. Are you using docker? Local installation? Cloud?

Hi @Hvitgar , thanks for your answer, i was try to change WEBDRIVER_BASEURL with IP but it cannot work too. I also check using 'curl localhost:8088/health ' and it return 'OK'. I using docker to running superset

[Hvitgar]

did you do the curl inside your container?If not, please try again from the container as the port inside the container might be different to the one you are using on the system running the container

[pritypriya25]

@Hvitgar.. Thanks for the solution. It's working for me. Now I am getting the screenshot and image URL.

[Chamawix]

This issue is not working well anymore, at least on my configuration, I made it work using the conf file from #20756

[vinodtrekea]

I'm able to cache the screenshot but not able to get the image using the image_url that I got with cache response. How do I use image_url to get the image of chart?

Made superset_config.py file using this here

[aprendizlaura]

@vinodtrekea
I have the same issue

[vinodtrekea]

I have initiated the discussion on this here