Skip to content

Conversation

@seisman
Copy link
Member

@seisman seisman commented Nov 25, 2025

This PR adds a test for geopandas with non-ASCII characters, which doesn't work on Windows (https://github.com/GenericMappingTools/pygmt/actions/runs/19654941294/job/56289538628?pr=4224):

..\pygmt\tests\test_geopandas.py:289: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\pygmt\helpers\decorators.py:591: in new_module
    return module_func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
..\pygmt\src\info.py:100: in info
    with lib.virtualfile_in(check_kind="vector", data=data) as vintbl:
C:\Users\runneradmin\micromamba\envs\pygmt\Lib\contextlib.py:137: in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
..\pygmt\helpers\tempfile.py:157: in tempfile_from_geojson
    geojson.to_file(**ogrgmt_kwargs)
C:\Users\runneradmin\micromamba\envs\pygmt\Lib\site-packages\geopandas\geodataframe.py:1637: in to_file
    _to_file(self, filename, driver, schema, index, **kwargs)
C:\Users\runneradmin\micromamba\envs\pygmt\Lib\site-packages\geopandas\io\file.py:731: in _to_file
    _to_file_pyogrio(df, filename, driver, schema, crs, mode, metadata, **kwargs)
C:\Users\runneradmin\micromamba\envs\pygmt\Lib\site-packages\geopandas\io\file.py:793: in _to_file_pyogrio
    pyogrio.write_dataframe(df, filename, driver=driver, metadata=metadata, **kwargs)
C:\Users\runneradmin\micromamba\envs\pygmt\Lib\site-packages\pyogrio\geopandas.py:710: in write_dataframe
    write(
C:\Users\runneradmin\micromamba\envs\pygmt\Lib\site-packages\pyogrio\raw.py:723: in write
    ogr_write(
pyogrio\\_io.pyx:2585: in pyogrio._io.ogr_write
    ???
pyogrio\\_io.pyx:2575: in pyogrio._io.ogr_write
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <encodings.cp1252.Codec object at 0x00000273C9A3F110>, input = '\u0641\u064a\u062c\u064a'
errors = 'strict'

    def encode(self,input,errors='strict'):
>       return codecs.charmap_encode(input,errors,encoding_table)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-3: character maps to <undefined>

This PR fixes the issue by setting encoding="UTF-8" when writing geopandas to ORG_GMT file.

@seisman seisman marked this pull request as ready for review November 25, 2025 01:20
@seisman seisman added the bug Something isn't working label Nov 25, 2025
@seisman seisman added this to the 0.18.0 milestone Nov 25, 2025
@seisman seisman changed the title Add a test for geopandas with non-ASCII characters geopandas: Fix the UnicodeEncodeError on Windows for geopandas.GeoDataFrame with non-ASCII characters Nov 25, 2025
@seisman seisman changed the title geopandas: Fix the UnicodeEncodeError on Windows for geopandas.GeoDataFrame with non-ASCII characters geopandas: Fix UnicodeEncodeError on Windows for geopandas.GeoDataFrame with non-ASCII characters Nov 25, 2025
@seisman seisman added the needs review This PR has higher priority and needs review. label Nov 25, 2025
@seisman
Copy link
Member Author

seisman commented Nov 25, 2025

@yvonnefroehlich This PR should fix the issue you had in GenericMappingTools/pygmt-paper-figures#6 (comment).

gdf = gpd.GeoDataFrame(
{
"name_ascii": ["Fiji"],
"name_utf8": ["فيجي"], # Arabic
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can confirm that this is Fiji in Arabic 😆

Co-authored-by: Wei Ji <23487320+weiji14@users.noreply.github.com>
@seisman seisman added final review call This PR requires final review and approval from a second reviewer and removed needs review This PR has higher priority and needs review. labels Nov 25, 2025
@seisman seisman merged commit f9f343d into main Nov 26, 2025
6 checks passed
@seisman seisman deleted the geopandas/nonascii branch November 26, 2025 00:54
@seisman seisman removed the final review call This PR requires final review and approval from a second reviewer label Nov 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants