Earth Engine initialization error: authentication token not getting detected from Streamlit app's Secrets #41

pskoulgi · 2022-05-19T11:30:53Z

First off, thank you so much for this excellent free and open resource! Your efforts here are much appreciated! I am currently looking to build a simple Earth Engine app and am using this repo to learn how to do so.

The issue

I am getting a "Earth Engine client library not initialized. Run ee.Initialize()" error on my web app.

Although I have generated and saved the Earth Engine authentication token in the app's "Secrets", as you have described in your blog post, the app seems to not pick it up from there.

How do I fix this?

What I have done

I began with trying to replicate this multi-app (https://streamlit.geemap.org/) on my Streamlit account, so I forked this repo and published it as-is on my Streamlit.

Error as seen on the web app

But when I select an app working with Earth Engine (eg. Google Earth Engine (GEE), Awesome GEE Community Datasets), I get the following error.

ee.ee_exception.EEException: This app has encountered an error. The original error message is redacted to prevent data leaks. Full error details have been recorded in the logs (if you're on Streamlit Cloud, click on 'Manage app' in the lower right of your app).
Traceback:

File "/home/appuser/venv/lib/python3.7/site-packages/streamlit/scriptrunner/script_runner.py", line 475, in _run_script
    exec(code, module.__dict__)
File "/app/streamlit-geospatial/app.py", line 47, in <module>
    apps.run()
File "/app/streamlit-geospatial/multiapp.py", line 64, in run
    functions[titles.index(title)]()
File "/app/streamlit-geospatial/apps/gee.py", line 121, in app
    nlcd()
File "/app/streamlit-geospatial/apps/gee.py", line 37, in nlcd
    Map.addLayer(getNLCD(year), {}, "NLCD " + year)
File "/app/streamlit-geospatial/apps/gee.py", line 22, in getNLCD
    dataset = ee.ImageCollection("USGS/NLCD_RELEASES/2016_REL")
File "/home/appuser/venv/lib/python3.7/site-packages/ee/computedobject.py", line 32, in __call__
    return type.__call__(cls, *args, **kwargs)
File "/home/appuser/venv/lib/python3.7/site-packages/ee/imagecollection.py", line 38, in __init__
    self.initialize()
File "/home/appuser/venv/lib/python3.7/site-packages/ee/imagecollection.py", line 72, in initialize
    super(ImageCollection, cls).initialize()
File "/home/appuser/venv/lib/python3.7/site-packages/ee/collection.py", line 32, in initialize
    apifunction.ApiFunction.importApi(cls, 'Collection', 'Collection')
File "/home/appuser/venv/lib/python3.7/site-packages/ee/apifunction.py", line 183, in importApi
    cls.initialize()
File "/home/appuser/venv/lib/python3.7/site-packages/ee/apifunction.py", line 154, in initialize
    signatures = data.getAlgorithms()
File "/home/appuser/venv/lib/python3.7/site-packages/ee/data.py", line 1055, in getAlgorithms
    call = _get_cloud_api_resource().projects().algorithms().list(
File "/home/appuser/venv/lib/python3.7/site-packages/ee/data.py", line 285, in _get_cloud_api_resource
    'Earth Engine client library not initialized. Run `ee.Initialize()`')

Error as seen in "Manage App" log, from inside my Streamlit account

The most recent log in "Manage App" ends with the following—I'm guessing the app is not finding the authentication token eventhough I have generated and save it in the app's "Secrets".

(I have purposely truncated the authentication web link below.)

Installing collected packages: click
  Attempting uninstall: click
    Found existing installation: click 8.0.4
    Uninstalling click-8.0.4:
      Successfully uninstalled click-8.0.4
Successfully installed click-8.0.0
WARNING: You are using pip version 22.0.3; however, version 22.1 is available.
You should consider upgrading via the '/home/appuser/venv/bin/python -m pip install --upgrade pip' command.
[manager] Python dependencies were installed from /app/streamlit-geospatial/requirements.txt using pip.
Check if streamlit is installed
Streamlit is already installed
[manager] Processed dependencies!
To authorize access needed by Earth Engine, open the following URL in a web browser and follow the instructions. If the web browser does not start automatically, please manually browse the URL below.
    https://code.earthengine.google.com/client-auth?scopes=https%3A//www.googleapis.com/auth/earthengine%20https%3A//www.googleapis.com/auth/ ...
The authorization workflow will generate a code, which you should paste in the box below. 
2022-05-19 10:54:04.502 Uncaught app exception
Traceback (most recent call last):
  File "/home/appuser/venv/lib/python3.7/site-packages/streamlit/scriptrunner/script_runner.py", line 475, in _run_script
    exec(code, module.__dict__)
  File "/app/streamlit-geospatial/app.py", line 47, in <module>
    apps.run()
  File "/app/streamlit-geospatial/multiapp.py", line 64, in run
    functions[titles.index(title)]()
  File "/app/streamlit-geospatial/apps/gee_datasets.py", line 182, in app
    eval(datasets[category][dataset])
  File "<string>", line 1, in <module>
  File "/app/streamlit-geospatial/apps/gee_datasets.py", line 75, in global_mangrove_watch
    gmw2007 = ee.FeatureCollection("projects/sat-io/open-datasets/GMW/GMW_2007_v2")
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/computedobject.py", line 32, in __call__
    return type.__call__(cls, *args, **kwargs)
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/featurecollection.py", line 44, in __init__
    self.initialize()
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/featurecollection.py", line 91, in initialize
    super(FeatureCollection, cls).initialize()
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/collection.py", line 32, in initialize
    apifunction.ApiFunction.importApi(cls, 'Collection', 'Collection')
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/apifunction.py", line 183, in importApi
    cls.initialize()
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/apifunction.py", line 154, in initialize
    signatures = data.getAlgorithms()
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/data.py", line 1055, in getAlgorithms
    call = _get_cloud_api_resource().projects().algorithms().list(
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/data.py", line 285, in _get_cloud_api_resource
    'Earth Engine client library not initialized. Run `ee.Initialize()`')
ee.ee_exception.EEException: Earth Engine client library not initialized. Run `ee.Initialize()`
2022-05-19 10:54:05.079 Uncaught app exception
Traceback (most recent call last):
  File "/home/appuser/venv/lib/python3.7/site-packages/streamlit/scriptrunner/script_runner.py", line 475, in _run_script
    exec(code, module.__dict__)
  File "/app/streamlit-geospatial/app.py", line 47, in <module>
    apps.run()
  File "/app/streamlit-geospatial/multiapp.py", line 64, in run
    functions[titles.index(title)]()
  File "/app/streamlit-geospatial/apps/gee_datasets.py", line 182, in app
    eval(datasets[category][dataset])
  File "<string>", line 1, in <module>
  File "/app/streamlit-geospatial/apps/gee_datasets.py", line 75, in global_mangrove_watch
    gmw2007 = ee.FeatureCollection("projects/sat-io/open-datasets/GMW/GMW_2007_v2")
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/computedobject.py", line 32, in __call__
    return type.__call__(cls, *args, **kwargs)
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/featurecollection.py", line 44, in __init__
    self.initialize()
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/featurecollection.py", line 91, in initialize
    super(FeatureCollection, cls).initialize()
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/collection.py", line 32, in initialize
    apifunction.ApiFunction.importApi(cls, 'Collection', 'Collection')
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/apifunction.py", line 183, in importApi
    cls.initialize()
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/apifunction.py", line 154, in initialize
    signatures = data.getAlgorithms()
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/data.py", line 1055, in getAlgorithms
    call = _get_cloud_api_resource().projects().algorithms().list(
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/data.py", line 285, in _get_cloud_api_resource
    'Earth Engine client library not initialized. Run `ee.Initialize()`')
ee.ee_exception.EEException: Earth Engine client library not initialized. Run `ee.Initialize()`

The text was updated successfully, but these errors were encountered:

giswqs · 2022-05-19T13:12:36Z

Were you able to test the app locally?

pskoulgi · 2022-05-19T13:22:05Z

I have not tried that, I went straight from forking to publishing on Streamlit. I'm new to this, I didn't think that step was necessary.

Will now try that step and report back.

giswqs · 2022-05-19T13:23:43Z

If you did not it try it locally, how did you get the Earth Engine token? Make sure the app works fine locally before deploying it to the cloud.

pskoulgi · 2022-05-20T10:42:32Z

I tried it locally, and it worked (screenshot of the local URL on a browser tab below).

[I think I got the app to work on the cloud—by using the token stored on my local machine instead of the one generated by following the link the build log on Streamlit takes me to. But, strangely, "client not initialized" error in the build log persists. More details below.]

Before trying it locally, I got the Earth Engine token by following the authorization link that came up in the "Manage app" build log for the app in my Streamlit. This is in the log I pasted in my message above, see screenshot below where I have highlighted it.

It did not give me the box to paste the token into, like it said it would. I went ahead and saved the token into the app's "Secrets". I thought it'll get picked up on rebooting the app, but that never happened, I get shown a (new) authorization link each time in the build log and I get the same errors.

On seeing that the app runs fine locally, I also took the token in my local machine (in my .config/earthengine/credentials) and saved that into my "Secrets" on Streamlit (they were not the same). With this, the app has started working.

But, strangely, the build log in "Manage app" on Streamlit hasn't stopped saying "Earth Engine client library not initialized. Run ee.Initialize()"

With the app now started to work, perhaps I can ignore this message and move on—or would that be inadvisible, any suggestions?

The last few lines of the log copied over, are as follows:

WARNING: You are using pip version 22.0.3; however, version 22.1 is available.
You should consider upgrading via the '/home/appuser/venv/bin/python -m pip install --upgrade pip' command.
[manager] Python dependencies were installed from /app/streamlit-geospatial/requirements.txt using pip.
Check if streamlit is installed
Streamlit is already installed
[manager] Processed dependencies!
2022-05-20 08:40:56.344 Uncaught app exception
Traceback (most recent call last):
  File "/home/appuser/venv/lib/python3.7/site-packages/streamlit/scriptrunner/script_runner.py", line 475, in _run_script
    exec(code, module.__dict__)
  File "/app/streamlit-geospatial/app.py", line 47, in <module>
    apps.run()
  File "/app/streamlit-geospatial/multiapp.py", line 64, in run
    functions[titles.index(title)]()
  File "/app/streamlit-geospatial/apps/gee_datasets.py", line 182, in app
    eval(datasets[category][dataset])
  File "<string>", line 1, in <module>
  File "/app/streamlit-geospatial/apps/gee_datasets.py", line 75, in global_mangrove_watch
    gmw2007 = ee.FeatureCollection("projects/sat-io/open-datasets/GMW/GMW_2007_v2")
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/computedobject.py", line 32, in __call__
    return type.__call__(cls, *args, **kwargs)
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/featurecollection.py", line 44, in __init__
    self.initialize()
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/featurecollection.py", line 91, in initialize
    super(FeatureCollection, cls).initialize()
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/collection.py", line 32, in initialize
    apifunction.ApiFunction.importApi(cls, 'Collection', 'Collection')
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/apifunction.py", line 183, in importApi
    cls.initialize()
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/apifunction.py", line 154, in initialize
    signatures = data.getAlgorithms()
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/data.py", line 1055, in getAlgorithms
    call = _get_cloud_api_resource().projects().algorithms().list(
  File "/home/appuser/venv/lib/python3.7/site-packages/ee/data.py", line 285, in _get_cloud_api_resource
    'Earth Engine client library not initialized. Run `ee.Initialize()`')
ee.ee_exception.EEException: Earth Engine client library not initialized. Run `ee.Initialize()`

giswqs · 2022-05-20T13:11:48Z

Yes, you need to use the token from .config/earthengine/credentials. Do NOT use the token you copy from the browser. It will not work.

Also, use geemap.ee_initialize() or geemap.Map() to initialize Earth Engine. Never use ee.Initialize() with a streamlit web app. It can't detect the secret and the initialization will fail.

As long as the web app is working properly, you can ignore the warning message.

pskoulgi · 2023-10-30T11:25:00Z

After a gap of ~18 months, I sync'd with the source repo's master branch to pick up updates/changes. I am now seeing a ee.ee_exception.EEException: This app has encountered an error. error. Before this sync, I was getting a ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. in the "Manage App" logs on Streamlit, but thankfully the sync has made that go away, and I suspect the current issue has something to do with authentication again, so I'm coming back and reopening this issue..

I have checked that the app runs locally as well as when I run it in a GitHub Codespace (I have added my EARTHENGINE_TOKEN to the Codespace). I am unable to tell why the EE exception error is happening only on Streamlit Cloud. Help, please?

Links:

Branch of my fork of this repo, for my app
app deployed on Streamlit Cloud

Screenchots to show app is running fine for me locally and on Codespaces:

giswqs · 2023-10-30T12:12:50Z

The authentication method has never changed.

pskoulgi · 2023-10-31T11:55:33Z

If possible, it would be very helpful if could you suggest where I could look for why my app is not working on the cloud (Streamlit) alone. I'm unable to tell, and I'm not very proficient at this.

The most recent logs on "Manage App" on Streamlit is as follows. It is essentially the same as what appears in the map panel in the cloud app.

The line 106 it points to used to work, neither it nor the asset has changed.
It asks to run ee.Initialize(), but I had learnt (from earlier in this thread) that as long as the app runs I can ignore this.

ee.ee_exception.EEException: Earth Engine client library not initialized. Run `ee.Initialize()`
2023-10-31 11:40:14.944 Uncaught app exception
Traceback (most recent call last):
  File "/home/adminuser/venv/lib/python3.8/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 534, in _run_script
    exec(code, module.__dict__)
  File "/mount/src/streamlit-geospatial/app.py", line 131, in <module>
    app()
  File "/mount/src/streamlit-geospatial/app.py", line 106, in app
    oneLabelsCollapsedForViz = ee.Image("projects/ee-open-natural-ecosystems/assets/publish/onesWith7Classes/labelWithUncertAppliedONEStates")
  File "/home/adminuser/venv/lib/python3.8/site-packages/ee/computedobject.py", line 28, in __call__
    return type.__call__(cls, *args, **kwargs)
  File "/home/adminuser/venv/lib/python3.8/site-packages/ee/image.py", line 52, in __init__
    self.initialize()
  File "/home/adminuser/venv/lib/python3.8/site-packages/ee/image.py", line 98, in initialize
    apifunction.ApiFunction.importApi(cls, 'Image', 'Image')
  File "/home/adminuser/venv/lib/python3.8/site-packages/ee/apifunction.py", line 195, in importApi
    cls.initialize()
  File "/home/adminuser/venv/lib/python3.8/site-packages/ee/apifunction.py", line 160, in initialize
    signatures = data.getAlgorithms()
  File "/home/adminuser/venv/lib/python3.8/site-packages/ee/data.py", line 1388, in getAlgorithms
    _get_cloud_projects()
  File "/home/adminuser/venv/lib/python3.8/site-packages/ee/data.py", line 297, in _get_cloud_projects
    raise ee_exception.EEException(
ee.ee_exception.EEException: Earth Engine client library not initialized. Run `ee.Initialize()`

Thank you!

giswqs closed this as completed May 20, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Earth Engine initialization error: authentication token not getting detected from Streamlit app's Secrets #41

Earth Engine initialization error: authentication token not getting detected from Streamlit app's Secrets #41

pskoulgi commented May 19, 2022

giswqs commented May 19, 2022

pskoulgi commented May 19, 2022

giswqs commented May 19, 2022

pskoulgi commented May 20, 2022

giswqs commented May 20, 2022

pskoulgi commented Oct 30, 2023 •

edited

giswqs commented Oct 30, 2023

pskoulgi commented Oct 31, 2023

Earth Engine initialization error: authentication token not getting detected from Streamlit app's Secrets #41

Earth Engine initialization error: authentication token not getting detected from Streamlit app's Secrets #41

Comments

pskoulgi commented May 19, 2022

The issue

What I have done

Error as seen on the web app

Error as seen in "Manage App" log, from inside my Streamlit account

giswqs commented May 19, 2022

pskoulgi commented May 19, 2022

giswqs commented May 19, 2022

pskoulgi commented May 20, 2022

giswqs commented May 20, 2022

pskoulgi commented Oct 30, 2023 • edited

giswqs commented Oct 30, 2023

pskoulgi commented Oct 31, 2023

pskoulgi commented Oct 30, 2023 •

edited