android: SSL certificate issues

[magowiz]

Hi,
I don't know if it is a bug or mine misconfiguration : I'm trying to use meteostat package to get climate normals in my mobile application, that is built on kivymd/kivy and I use buildozer to create an APK package that runs on android. In this mobile version while getting normals I get SSL verification error.

Log

01-22 18:09:44.336  2002  2032 I python  : [INFO   ] [Video       ] Provider: null(['video_ffmpeg', 'video_ffpyplayer'] ignored)
01-22 18:09:45.601  2002  2032 I python  : [INFO   ] [GL          ] NPOT texture support is available
01-22 18:09:47.381  2002  2032 I python  :  InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.open-elevation.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
01-22 18:09:47.663  2002  2032 I python  : [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)
01-22 18:09:47.663  2002  2032 I python  :  Traceback (most recent call last):
01-22 18:09:47.663  2002  2032 I python  :    File "/srv/buildozer_build_dir/android/app/main.py", line 832, in <module>
01-22 18:09:47.664  2002  2032 I python  :    File "/srv/buildozer_build_dir/android/platform/build-armeabi-v7a_x86_64_x86_arm64-v8a/build/python-installs/travel_lint/x86/kivy/app.py", line 954, in run
01-22 18:09:47.664  2002  2032 I python  :    File "/srv/buildozer_build_dir/android/platform/build-armeabi-v7a_x86_64_x86_arm64-v8a/build/python-installs/travel_lint/x86/kivy/app.py", line 949, in _run_prepare
01-22 18:09:47.664  2002  2032 I python  :    File "kivy/_event.pyx", line 731, in kivy._event.EventDispatcher.dispatch
01-22 18:09:47.664  2002  2032 I python  :    File "/srv/buildozer_build_dir/android/app/main.py", line 684, in on_start
01-22 18:09:47.664  2002  2032 I python  :    File "/srv/buildozer_build_dir/android/app/ui/extra_widgets.py", line 189, in set_weather_location
01-22 18:09:47.664  2002  2032 I python  :    File "/srv/buildozer_build_dir/android/app/ui/extra_widgets.py", line 54, in set_weather_location
01-22 18:09:47.665  2002  2032 I python  :    File "/srv/buildozer_build_dir/android/app/lib/weather/weather_getter.py", line 319, in get_normals_in_dates
01-22 18:09:47.665  2002  2032 I python  :  UnboundLocalError: local variable 'data' referenced before assignment
01-22 18:09:47.665  2002  2032 I python  : Python for android ended.

Sample Code

weather.py

from geopy.geocoders import Nominatim
from meteostat import Normals, Point

def _locating_place(place, country) -> Point:
        geolocator = Nominatim(user_agent='myapplication')
        location = geolocator.geocode(f"{place} {country}")
        coord = {'lon': float(location.raw['lon']),
                 'lat': float(location.raw['lat'])}
        coord = Point(coord['lat'], coord['lon'])
        return coord
        
def get_normals(place: str, country: str):
        coord = self._locating_place(place, country)
        # Get Normals data
        data = Normals(coord)
        data = data.fetch()

Main file of the application: main.py

import certifi
import os

os.environ['SSL_CERT_FILE'] = certifi.where()

class MyApp(MDApp):
    def build(self):
          self.root = Builder.load_file('design.kv')
          weather.get_normals('Berlin', 'Germany')
          return self.root

if __name__ == "__main__":
    MyApp().run()

Questions

I noticed your library is using urllib, how can I debug this request done internally and, for example, getting the URL? Is there a way I can disable certificate verification until I fix my app/environment?

[clampr]

Hi,

Good question. I'm not sure if/how to disable verification globally. Did you try this already?

pip install –upgrade certifi

[magowiz]

Yes I built it from scratch, I was able to log, from inside my code, certifi package version, is the same I get when running from desktop.
I did some searches and read meteostat code, I saw that you use pandas to fetch csv, read_csv method, in more recent versions you can pass urllib options (pandas uses urllib) to download via http, so you can say for example which is SSL context. I was trying to do this in a fork I did from your package but no luck, since buildozer can install pandas 1.0.3 which is too old for that storage_options parameter, if I try to force installation of a more recent version of pandas I've got errors. I thought also to split the read csv in two steps:

• use requests get to fetch the file and store it locally, with requests is quite easier to set SSL context or disable verification
• use read_csv to read local file
  Anyhow I wasn't able to understand what should I do to do it properly because sometimes files gets read properly, other times I get bad gzip.

I thought that a version using requests could also interest also meteostat package maintainers, since it is an almost standard de-facto library.