Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: support NumPy 2.0 #145

Merged
merged 1 commit into from
Jul 5, 2024
Merged

fix: support NumPy 2.0 #145

merged 1 commit into from
Jul 5, 2024

Conversation

jgoizueta
Copy link
Member

Issue

    Received: "Command failed: carto snowflake upload --file_url https://storage.googleapis.com/carto_test_tables/raster/medium_cog.tif --database CI --schema CI_TMP_UPLOADS --table TEST_23C23309B13257C3A0381475 --account sxa81489.us-east-1·
    Traceback (most recent call last):
      File \"/usr/local/bin/carto\", line 5, in <module>
        from raster_loader.cli import main
      File \"/usr/local/lib/python3.9/dist-packages/raster_loader/__init__.py\", line 3, in <module>
        from raster_loader.io.bigquery import (
      File \"/usr/local/lib/python3.9/dist-packages/raster_loader/io/bigquery.py\", line 10, in <module>
        from raster_loader.io.common import (
      File \"/usr/local/lib/python3.9/dist-packages/raster_loader/io/common.py\", line 34, in <module>
        \"float16\": np.NaN,
      File \"/usr/local/lib/python3.9/dist-packages/numpy/__init__.py\", line 397, in __getattr__
        raise AttributeError(
    AttributeError: `np.NaN` was removed in the NumPy 2.0 release. Use `np.nan` instead.

@Jesus89
Copy link
Member

Jesus89 commented Jul 5, 2024

Raster-loader is compatible with Python +3.8, let's check compatibility with numpy 2 before merging this

@jgoizueta
Copy link
Member Author

OK, but independently of reviewing the versions of our requirements, we should merge this because np.nan works with also with NumPy < 2.0

@jgoizueta
Copy link
Member Author

With the current dependencies we get NumPy 2.0 for Python 3.9 and NumPy 1.24.4 for Python 3.8:

$ python3 --version
Python 3.9.6
$ python3 -m venv env39
$ source env39/bin/activate
$ pip install raster-loader[bigquery,snowflake]==0.7.0
...
$ pip show numpy | grep Version:
Version: 2.0.0
$ export GOOGLE_APPLICATION_CREDENTIALS=/Users/jgoizueta/Downloads/_service_accounts/cartodb-data-engineering-team-3fff8948ff04.json
$ carto bigquery upload \
  --file_url "https://storage.googleapis.com/carto_test_tables/raster/qa-valid/cog_rgb_24mb_overviews.tiff" \
  --band 1  --band 2 --band 3 --band 4 \
  --project cartodb-data-engineering-team \
  --dataset jgoizueta_imports_tmp \
  --table testpy39
...  
 AttributeError: `np.NaN` was removed in the NumPy 2.0 release. Use `np.nan` instead.
 $ deactivate

$ /usr/local/Cellar/python@3.8/3.8.19/bin/python3.8 --version
Python 3.8.19
$ /usr/local/Cellar/python@3.8/3.8.19/bin/python3.8 -m venv env38
$ source env38/bin/activate
$ pip install raster-loader[bigquery,snowflake]==0.7.0
...
$ pip show numpy | grep Version:
Version: 1.24.4
$ carto bigquery upload \
  --file_url "https://storage.googleapis.com/carto_test_tables/raster/qa-valid/cog_rgb_24mb_overviews.tiff" \
  --band 1  --band 2 --band 3 --band 4 \
  --project cartodb-data-engineering-team \
  --dataset jgoizueta_imports_tmp \
  --table testpy38
...
Updating labels...
Done.
Raster file uploaded to Google BigQuery  

@jgoizueta jgoizueta changed the title fix for NumPy 2.0 fix: support NumPy 2.0 Jul 5, 2024
@jgoizueta jgoizueta merged commit e7ac4be into main Jul 5, 2024
4 checks passed
@jgoizueta jgoizueta deleted the hotfix-np-nan branch July 5, 2024 10:38
@jgoizueta jgoizueta mentioned this pull request Jul 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants