Let PyGMT work with the conda GMT package on Windows #434
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-Authored-By: Dongdong Tian <seisman.info@gmail.com>
Here we need to set `error` to True first, so if libnames is empty, than we raise the GMTCLibNotFoundError error. On Windows, libnames is a list of possible DLL names, "gmt.dll", "gmt_w64.dll", and "gmt_w32.dll". If any one of them is found, then we set `error` to False and break to for loop. If none of them are found, then we raise the GMTCLibNotFoundError error: ``` Error loading the GMT shared library 'gmt.dll, gmt_w64.dll, gmt_w32.dll'. ``` The error message above can be improved, but it's not a big issue.
To make PyGMT work with conda's GMT package on Windows, we have to set two environmental variables: - GMT_LIBRARY_PATH: `C:\Miniconda\envs\testing\Library\bin` - GMT_SHAREDIR: `C:\Miniconda\envs\testing\Library\share\gmt`
This was referenced May 19, 2020
|
Awesome detective work @seisman! I've had a look through the changes and they look fine. There seems to be a lot of timeout tests, and a couple of minor png image differences for ================================== FAILURES ===================================
__________________ [doctest] pygmt.clib.conversion._as_array __________________
221
222 Examples
223 --------
224
225 >>> import pandas as pd
226 >>> x_series = pd.Series(data=[1, 2, 3, 4])
227 >>> x_array = _as_array(x_series)
228 >>> type(x_array)
229 <class 'numpy.ndarray'>
230 >>> x_array
Expected:
array([1, 2, 3, 4])
Got:
array([1, 2, 3, 4], dtype=int64)
C:\Miniconda\envs\testing\lib\site-packages\pygmt\clib\conversion.py:230: DocTestFailure
_____________ [doctest] pygmt.clib.conversion.dataarray_to_matrix _____________
041 ------
042 GMTInvalidInput
043 If the grid has more than two dimensions or variable grid spacing.
044
045 Examples
046 --------
047
048 >>> from pygmt.datasets import load_earth_relief
049 >>> # Use the global Earth relief grid with 1 degree spacing (60')
050 >>> grid = load_earth_relief(resolution='60m')
UNEXPECTED EXCEPTION: FileNotFoundError("File '@earth_relief_60m' not found.",)
Traceback (most recent call last):
File "C:\Miniconda\envs\testing\lib\doctest.py", line 1330, in __run
compileflags, 1), test.globs)
File "<doctest pygmt.clib.conversion.dataarray_to_matrix[1]>", line 1, in <module>
File "C:\Miniconda\envs\testing\lib\site-packages\pygmt\datasets\earth_relief.py", line 38, in load_earth_relief
fname = which("@earth_relief_{}".format(resolution), download="u")
File "C:\Miniconda\envs\testing\lib\site-packages\pygmt\helpers\decorators.py", line 183, in new_module
return module_func(*args, **kwargs)
File "C:\Miniconda\envs\testing\lib\site-packages\pygmt\modules.py", line 143, in which
raise FileNotFoundError("File '{}' not found.".format(fname))
FileNotFoundError: File '@earth_relief_60m' not found.
C:\Miniconda\envs\testing\lib\site-packages\pygmt\clib\conversion.py:50: UnexpectedException
---------------------------- Captured stderr call -----------------------------
earth_relief_60m: Download file from the GMT data server [data set size is 106K].
earth_relief_60m: Earth Relief at 60x60 arc minutes obtained by Gaussian Cartesian filtering (111 km fullwidth) of SRTM15+V2 [Tozer et al., 2019].
gmtwhich [ERROR]: Libcurl Error: Timeout was reached
gmtwhich [ERROR]: You can turn remote file download off by setting GMT_DATA_SERVER_LIMIT = 0.
gmtwhich [ERROR]: File earth_relief_60m.grd not found!
___________ [doctest] pygmt.clib.conversion.kwargs_to_ctypes_array ____________
263 Returns
264 -------
265 ctypes_value : ctypes array or None
266
267 Examples
268 --------
269
270 >>> import ctypes as ct
271 >>> value = kwargs_to_ctypes_array('bla', {'bla': [10, 10]}, ct.c_int*2)
272 >>> type(value)
Expected:
<class 'pygmt.clib.conversion.c_int_Array_2'>
Got:
<class 'pygmt.clib.conversion.c_long_Array_2'>
C:\Miniconda\envs\testing\lib\site-packages\pygmt\clib\conversion.py:272: DocTestFailure
_____________________________ test_basemap_polar ______________________________
Error: Image dimensions did not match.
Expected shape: (1958, 1822)
D:\a\1\s\tmp-test-dir-with-unique-name\results\tmpuvl6y42z\baseline-test_basemap_polar.png
Actual shape: (1958, 1821)
D:\a\1\s\tmp-test-dir-with-unique-name\results\tmpuvl6y42z\test_basemap_polar.png
______________________ test_blockmedian_input_dataframe _______________________
def test_blockmedian_input_dataframe():
"""
Run blockmedian by passing in a pandas.DataFrame as input
"""
> dataframe = load_sample_bathymetry()
C:\Miniconda\envs\testing\lib\site-packages\pygmt\tests\test_blockmedian.py:20:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
C:\Miniconda\envs\testing\lib\site-packages\pygmt\datasets\tutorial.py:80: in load_sample_bathymetry
fname = which("@tut_ship.xyz", download="c")
C:\Miniconda\envs\testing\lib\site-packages\pygmt\helpers\decorators.py:183: in new_module
return module_func(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
fname = '@tut_ship.xyz', kwargs = {'G': 'c'}
tmpfile = <pygmt.helpers.tempfile.GMTTempFile object at 0x0000014FBA288898>
arg_str = '@tut_ship.xyz -Gc ->C:\\Users\\VSSADM~1\\AppData\\Local\\Temp\\pygmt-4ll01yrg.txt'
lib = <pygmt.clib.session.Session object at 0x0000014FBA2889B0>, path = ''
@fmt_docstring
@use_alias(G="download")
def which(fname, **kwargs):
"""
Find the full path to specified files.
Reports the full paths to the files given through *fname*. We look for
the file in (1) the current directory, (2) in $GMT_USERDIR (if defined),
(3) in $GMT_DATADIR (if defined), or (4) in $GMT_CACHEDIR (if defined).
*fname* can also be a downloadable file (either a full URL, a
`@file` special file for downloading from the GMT Site Cache, or
`@earth_relief_*` topography grids). In these cases, use option *download*
to set the desired behavior. If *download* is not used (or False), the file
will not be found.
Full option list at :gmt-docs:`gmtwhich.html`
{aliases}
Parameters
----------
fname : str
The file name that you want to check.
G : bool or str
If the file is downloadable and not found, we will try to download the
it. Use True or 'l' (default) to download to the current directory. Use
'c' to place in the user cache directory or 'u' user data directory
instead.
Returns
-------
path : str
The path of the file, depending on the options used.
Raises
------
FileNotFoundError
If the file is not found.
"""
with GMTTempFile() as tmpfile:
arg_str = " ".join([fname, build_arg_string(kwargs), "->" + tmpfile.name])
with Session() as lib:
lib.call_module("which", arg_str)
path = tmpfile.read().strip()
if not path:
> raise FileNotFoundError("File '{}' not found.".format(fname))
E FileNotFoundError: File '@tut_ship.xyz' not found.
C:\Miniconda\envs\testing\lib\site-packages\pygmt\modules.py:143: FileNotFoundError
---------------------------- Captured stderr call -----------------------------
gmtwhich [ERROR]: Libcurl Error: Timeout was reached
gmtwhich [ERROR]: You can turn remote file download off by setting GMT_DATA_SERVER_LIMIT = 0.
gmtwhich [ERROR]: File tut_ship.xyz not found!
______________________________ test_japan_quakes ______________________________
def test_japan_quakes():
"Check that the dataset loads without errors"
> data = load_japan_quakes()
C:\Miniconda\envs\testing\lib\site-packages\pygmt\tests\test_datasets.py:20:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
C:\Miniconda\envs\testing\lib\site-packages\pygmt\datasets\tutorial.py:27: in load_japan_quakes
fname = which("@tut_quakes.ngdc", download="c")
C:\Miniconda\envs\testing\lib\site-packages\pygmt\helpers\decorators.py:183: in new_module
return module_func(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
fname = '@tut_quakes.ngdc', kwargs = {'G': 'c'}
tmpfile = <pygmt.helpers.tempfile.GMTTempFile object at 0x0000014FBA381F98>
arg_str = '@tut_quakes.ngdc -Gc ->C:\\Users\\VSSADM~1\\AppData\\Local\\Temp\\pygmt-60z1jjtv.txt'
lib = <pygmt.clib.session.Session object at 0x0000014FBA381C88>, path = ''
@fmt_docstring
@use_alias(G="download")
def which(fname, **kwargs):
"""
Find the full path to specified files.
Reports the full paths to the files given through *fname*. We look for
the file in (1) the current directory, (2) in $GMT_USERDIR (if defined),
(3) in $GMT_DATADIR (if defined), or (4) in $GMT_CACHEDIR (if defined).
*fname* can also be a downloadable file (either a full URL, a
`@file` special file for downloading from the GMT Site Cache, or
`@earth_relief_*` topography grids). In these cases, use option *download*
to set the desired behavior. If *download* is not used (or False), the file
will not be found.
Full option list at :gmt-docs:`gmtwhich.html`
{aliases}
Parameters
----------
fname : str
The file name that you want to check.
G : bool or str
If the file is downloadable and not found, we will try to download the
it. Use True or 'l' (default) to download to the current directory. Use
'c' to place in the user cache directory or 'u' user data directory
instead.
Returns
-------
path : str
The path of the file, depending on the options used.
Raises
------
FileNotFoundError
If the file is not found.
"""
with GMTTempFile() as tmpfile:
arg_str = " ".join([fname, build_arg_string(kwargs), "->" + tmpfile.name])
with Session() as lib:
lib.call_module("which", arg_str)
path = tmpfile.read().strip()
if not path:
> raise FileNotFoundError("File '{}' not found.".format(fname))
E FileNotFoundError: File '@tut_quakes.ngdc' not found.
C:\Miniconda\envs\testing\lib\site-packages\pygmt\modules.py:143: FileNotFoundError
---------------------------- Captured stderr call -----------------------------
gmtwhich [ERROR]: Libcurl Error: Timeout was reached
gmtwhich [ERROR]: You can turn remote file download off by setting GMT_DATA_SERVER_LIMIT = 0.
gmtwhich [ERROR]: File tut_quakes.ngdc not found!
__________________________________ test_logo __________________________________
Error: Image files did not match.
RMS Value: 7.485386103399013
Expected:
D:\a\1\s\tmp-test-dir-with-unique-name\results\tmpxl74dzfk\baseline-test_logo.png
Actual:
D:\a\1\s\tmp-test-dir-with-unique-name\results\tmpxl74dzfk\test_logo.png
Difference:
D:\a\1\s\tmp-test-dir-with-unique-name\results\tmpxl74dzfk\test_logo-failed-diff.png
Tolerance:
2
_____________________________ test_logo_on_a_map ______________________________
Error: Image files did not match.
RMS Value: 2.40664730778399
Expected:
D:\a\1\s\tmp-test-dir-with-unique-name\results\tmpjakdon0f\baseline-test_logo_on_a_map.png
Actual:
D:\a\1\s\tmp-test-dir-with-unique-name\results\tmpjakdon0f\test_logo_on_a_map.png
Difference:
D:\a\1\s\tmp-test-dir-with-unique-name\results\tmpjakdon0f\test_logo_on_a_map-failed-diff.png
Tolerance:
2
______________________________ test_pygmtscraper ______________________________
@pytest.mark.skipif(sphinx_gallery is None, reason="requires sphinx-gallery")
def test_pygmtscraper():
"Make sure the scraper finds the figures and removes them from the pool."
showed = [fig for fig in SHOWED_FIGURES]
for _ in range(len(SHOWED_FIGURES)):
SHOWED_FIGURES.pop()
try:
fig = Figure()
fig.coast(region="BR", projection="M6i", land="gray", frame=True)
fig.show()
assert len(SHOWED_FIGURES) == 1
assert SHOWED_FIGURES[0] is fig
scraper = PyGMTScraper()
with TemporaryDirectory() as tmpdir:
conf = {"src_dir": "meh"}
fname = os.path.join(tmpdir, "meh.png")
block_vars = {"image_path_iterator": (i for i in [fname])}
assert not os.path.exists(fname)
> scraper(None, block_vars, conf)
C:\Miniconda\envs\testing\lib\site-packages\pygmt\tests\test_sphinx_gallery.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
C:\Miniconda\envs\testing\lib\site-packages\pygmt\sphinx_gallery.py:35: in __call__
return figure_rst(image_names, gallery_conf["src_dir"])
C:\Miniconda\envs\testing\lib\site-packages\sphinx_gallery\scrapers.py:266: in figure_rst
for figure_path in figure_list]
C:\Miniconda\envs\testing\lib\site-packages\sphinx_gallery\scrapers.py:264: in <listcomp>
figure_paths = [os.path.relpath(figure_path, sources_dir)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
path = 'C:\\Users\\VSSADM~1\\AppData\\Local\\Temp\\tmp1pxf_t82\\meh.png'
start = 'meh'
def relpath(path, start=None):
"""Return a relative version of a path"""
path = os.fspath(path)
if isinstance(path, bytes):
sep = b'\\'
curdir = b'.'
pardir = b'..'
else:
sep = '\\'
curdir = '.'
pardir = '..'
if start is None:
start = curdir
if not path:
raise ValueError("no path specified")
start = os.fspath(start)
try:
start_abs = abspath(normpath(start))
path_abs = abspath(normpath(path))
start_drive, start_rest = splitdrive(start_abs)
path_drive, path_rest = splitdrive(path_abs)
if normcase(start_drive) != normcase(path_drive):
raise ValueError("path is on mount %r, start on mount %r" % (
> path_drive, start_drive))
E ValueError: path is on mount 'C:', start on mount 'D:'
C:\Miniconda\envs\testing\lib\ntpath.py:584: ValueError
_______________________ test_surface_with_outfile_param _______________________
def test_surface_with_outfile_param():
"""
Run surface with the -Goutputfile.nc parameter
"""
ship_data = load_sample_bathymetry()
data = ship_data.values # convert pandas.DataFrame to numpy.ndarray
try:
output = surface(
data=data, spacing="5m", region=[245, 255, 20, 30], outfile=TEMP_GRID
)
assert output is None # check that output is None since outfile is set
assert os.path.exists(path=TEMP_GRID) # check that outfile exists at path
grid = xr.open_dataarray(TEMP_GRID)
assert isinstance(grid, xr.DataArray) # check that netcdf grid loaded properly
finally:
> os.remove(path=TEMP_GRID)
E PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Miniconda\\envs\\testing\\lib\\site-packages\\pygmt\\tests\\data\\tmp_grid.nc'
C:\Miniconda\envs\testing\lib\site-packages\pygmt\tests\test_surface.py:96: PermissionError
_________________________ test_surface_short_aliases __________________________
def test_surface_short_aliases():
"""
Run surface using short aliases -I for spacing, -R for region, -G for
outfile
"""
ship_data = load_sample_bathymetry()
data = ship_data.values # convert pandas.DataFrame to numpy.ndarray
try:
output = surface(data=data, I="5m", R=[245, 255, 20, 30], G=TEMP_GRID)
assert output is None # check that output is None since outfile is set
assert os.path.exists(path=TEMP_GRID) # check that outfile exists at path
grid = xr.open_dataarray(TEMP_GRID)
assert isinstance(grid, xr.DataArray) # check that netcdf grid loaded properly
finally:
> os.remove(path=TEMP_GRID)
E PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Miniconda\\envs\\testing\\lib\\site-packages\\pygmt\\tests\\data\\tmp_grid.nc'
C:\Miniconda\envs\testing\lib\site-packages\pygmt\tests\test_surface.py:114: PermissionError
_________________________________ test_which __________________________________
def test_which():
"Make sure which returns file paths for @files correctly without errors"
for fname in "tut_quakes.ngdc tut_bathy.nc".split():
> cached_file = which("@{}".format(fname), download="c")
C:\Miniconda\envs\testing\lib\site-packages\pygmt\tests\test_which.py:15:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
C:\Miniconda\envs\testing\lib\site-packages\pygmt\helpers\decorators.py:183: in new_module
return module_func(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
fname = '@tut_quakes.ngdc', kwargs = {'G': 'c'}
tmpfile = <pygmt.helpers.tempfile.GMTTempFile object at 0x0000014FBA534E80>
arg_str = '@tut_quakes.ngdc -Gc ->C:\\Users\\VSSADM~1\\AppData\\Local\\Temp\\pygmt-8d7_p_jk.txt'
lib = <pygmt.clib.session.Session object at 0x0000014FBA534E48>, path = ''
@fmt_docstring
@use_alias(G="download")
def which(fname, **kwargs):
"""
Find the full path to specified files.
Reports the full paths to the files given through *fname*. We look for
the file in (1) the current directory, (2) in $GMT_USERDIR (if defined),
(3) in $GMT_DATADIR (if defined), or (4) in $GMT_CACHEDIR (if defined).
*fname* can also be a downloadable file (either a full URL, a
`@file` special file for downloading from the GMT Site Cache, or
`@earth_relief_*` topography grids). In these cases, use option *download*
to set the desired behavior. If *download* is not used (or False), the file
will not be found.
Full option list at :gmt-docs:`gmtwhich.html`
{aliases}
Parameters
----------
fname : str
The file name that you want to check.
G : bool or str
If the file is downloadable and not found, we will try to download the
it. Use True or 'l' (default) to download to the current directory. Use
'c' to place in the user cache directory or 'u' user data directory
instead.
Returns
-------
path : str
The path of the file, depending on the options used.
Raises
------
FileNotFoundError
If the file is not found.
"""
with GMTTempFile() as tmpfile:
arg_str = " ".join([fname, build_arg_string(kwargs), "->" + tmpfile.name])
with Session() as lib:
lib.call_module("which", arg_str)
path = tmpfile.read().strip()
if not path:
> raise FileNotFoundError("File '{}' not found.".format(fname))
E FileNotFoundError: File '@tut_quakes.ngdc' not found.
C:\Miniconda\envs\testing\lib\site-packages\pygmt\modules.py:143: FileNotFoundError
---------------------------- Captured stderr call -----------------------------
gmtwhich [ERROR]: Libcurl Error: Timeout was reached
gmtwhich [ERROR]: You can turn remote file download off by setting GMT_DATA_SERVER_LIMIT = 0.
gmtwhich [ERROR]: File tut_quakes.ngdc not found!
============================== warnings summary ===============================
C:\Miniconda\envs\testing\lib\site-packages\win32\lib\pywintypes.py:2
C:\Miniconda\envs\testing\lib\site-packages\win32\lib\pywintypes.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative usesI'll be heading in to uni tomorrow and should have access to a Windows computer to test this branch a bit better. But we should definitely merge this in soon and tidy up all the stray pieces. |
It seems Azure Pipelines's Linux and Windows agents cannot access any external websites, including GMT's server. However, macOS agents don't have the problem. The GMT repository also has the same problem and we have to cache all the remote files (e.g.,
That would be great! FYI, I finally find why GMT can't guess its share directory location. See the bug report GenericMappingTools/gmt#3353. |
Ah okay, I was working on consolidating all our Linux/MacOS/Windows tests on Github Actions, not sure if that will fix it, since Azure and Github are both Microsoft owned 😆 I could make that happen once this PR is merged, and work on #345 too. Edit: Same with Github Actions, MacOS works but Linux/Windows doesn't. Might be a DNS related issue? Closest thing I can find is https://stackoverflow.com/questions/58034882/vstest-unit-tests-ping-of-a-dns-server-8-8-8-8-fails-when-running-within-azur,
Good to know, I think we just set |
weiji14
reviewed
May 19, 2020
|
Ok, these are the 5 test failures I'm getting locally on Windows: ================================================================================================== FAILURES ==================================================================================================
_________________________________________________________________________________ [doctest] pygmt.clib.conversion._as_array __________________________________________________________________________________
221
222 Examples
223 --------
224
225 >>> import pandas as pd
226 >>> x_series = pd.Series(data=[1, 2, 3, 4])
227 >>> x_array = _as_array(x_series)
228 >>> type(x_array)
229 <class 'numpy.ndarray'>
230 >>> x_array
Expected:
array([1, 2, 3, 4])
Got:
array([1, 2, 3, 4], dtype=int64)
H:\github\pygmt\pygmt\clib\conversion.py:230: DocTestFailure
___________________________________________________________________________ [doctest] pygmt.clib.conversion.kwargs_to_ctypes_array ___________________________________________________________________________
263 Returns
264 -------
265 ctypes_value : ctypes array or None
266
267 Examples
268 --------
269
270 >>> import ctypes as ct
271 >>> value = kwargs_to_ctypes_array('bla', {'bla': [10, 10]}, ct.c_int*2)
272 >>> type(value)
Expected:
<class 'pygmt.clib.conversion.c_int_Array_2'>
Got:
<class 'pygmt.clib.conversion.c_long_Array_2'>
H:\github\pygmt\pygmt\clib\conversion.py:272: DocTestFailure
_____________________________________________________________________________________________ test_pygmtscraper ______________________________________________________________________________________________
@pytest.mark.skipif(sphinx_gallery is None, reason="requires sphinx-gallery")
def test_pygmtscraper():
"Make sure the scraper finds the figures and removes them from the pool."
showed = [fig for fig in SHOWED_FIGURES]
for _ in range(len(SHOWED_FIGURES)):
SHOWED_FIGURES.pop()
try:
fig = Figure()
fig.coast(region="BR", projection="M6i", land="gray", frame=True)
fig.show()
assert len(SHOWED_FIGURES) == 1
assert SHOWED_FIGURES[0] is fig
scraper = PyGMTScraper()
with TemporaryDirectory() as tmpdir:
conf = {"src_dir": "meh"}
fname = os.path.join(tmpdir, "meh.png")
block_vars = {"image_path_iterator": (i for i in [fname])}
assert not os.path.exists(fname)
> scraper(None, block_vars, conf)
pygmt\tests\test_sphinx_gallery.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pygmt\sphinx_gallery.py:35: in __call__
return figure_rst(image_names, gallery_conf["src_dir"])
C:\Users\leongw1\AppData\Local\Continuum\miniconda3\envs\pygmt\lib\site-packages\sphinx_gallery\scrapers.py:266: in figure_rst
for figure_path in figure_list]
C:\Users\leongw1\AppData\Local\Continuum\miniconda3\envs\pygmt\lib\site-packages\sphinx_gallery\scrapers.py:266: in <listcomp>
for figure_path in figure_list]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
path = 'C:\\Users\\leongw1\\AppData\\Local\\Temp\\tmp36122_v1\\meh.png', start = 'meh'
def relpath(path, start=None):
"""Return a relative version of a path"""
path = os.fspath(path)
if isinstance(path, bytes):
sep = b'\\'
curdir = b'.'
pardir = b'..'
else:
sep = '\\'
curdir = '.'
pardir = '..'
if start is None:
start = curdir
if not path:
raise ValueError("no path specified")
start = os.fspath(start)
try:
start_abs = abspath(normpath(start))
path_abs = abspath(normpath(path))
start_drive, start_rest = splitdrive(start_abs)
path_drive, path_rest = splitdrive(path_abs)
if normcase(start_drive) != normcase(path_drive):
raise ValueError("path is on mount %r, start on mount %r" % (
> path_drive, start_drive))
E ValueError: path is on mount 'C:', start on mount 'H:'
C:\Users\leongw1\AppData\Local\Continuum\miniconda3\envs\pygmt\lib\ntpath.py:562: ValueError
-------------------------------------------------------------------------------------------- Captured stderr call --------------------------------------------------------------------------------------------
psconvert [ERROR]: Error opening HKLM key
psconvert [ERROR]: Error opening HKLM key
______________________________________________________________________________________ test_surface_with_outfile_param _______________________________________________________________________________________
def test_surface_with_outfile_param():
"""
Run surface with the -Goutputfile.nc parameter
"""
ship_data = load_sample_bathymetry()
data = ship_data.values # convert pandas.DataFrame to numpy.ndarray
try:
output = surface(
data=data, spacing="5m", region=[245, 255, 20, 30], outfile=TEMP_GRID
)
assert output is None # check that output is None since outfile is set
assert os.path.exists(path=TEMP_GRID) # check that outfile exists at path
grid = xr.open_dataarray(TEMP_GRID)
assert isinstance(grid, xr.DataArray) # check that netcdf grid loaded properly
finally:
> os.remove(path=TEMP_GRID)
E PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'H:\\github\\pygmt\\pygmt\\tests\\data\\tmp_grid.nc'
pygmt\tests\test_surface.py:96: PermissionError
_________________________________________________________________________________________ test_surface_short_aliases _________________________________________________________________________________________
def test_surface_short_aliases():
"""
Run surface using short aliases -I for spacing, -R for region, -G for
outfile
"""
ship_data = load_sample_bathymetry()
data = ship_data.values # convert pandas.DataFrame to numpy.ndarray
try:
output = surface(data=data, I="5m", R=[245, 255, 20, 30], G=TEMP_GRID)
assert output is None # check that output is None since outfile is set
assert os.path.exists(path=TEMP_GRID) # check that outfile exists at path
grid = xr.open_dataarray(TEMP_GRID)
assert isinstance(grid, xr.DataArray) # check that netcdf grid loaded properly
finally:
> os.remove(path=TEMP_GRID)
E PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'H:\\github\\pygmt\\pygmt\\tests\\data\\tmp_grid.nc'
pygmt\tests\test_surface.py:114: PermissionError
========================================================================================== short test summary info ===========================================================================================
FAILED H:\github\pygmt\pygmt\clib\conversion.py::pygmt.clib.conversion._as_array
FAILED H:\github\pygmt\pygmt\clib\conversion.py::pygmt.clib.conversion.kwargs_to_ctypes_array
FAILED H:\github\pygmt\pygmt\tests\test_sphinx_gallery.py::test_pygmtscraper - ValueError: path is on mount 'C:', start on mount 'H:'
FAILED H:\github\pygmt\pygmt\tests\test_surface.py::test_surface_with_outfile_param - PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'H:\\gi...
FAILED H:\github\pygmt\pygmt\tests\test_surface.py::test_surface_short_aliases - PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'H:\\github\...
================================================================================= 5 failed, 246 passed in 151.48s (0:02:31) ==================================================================================2 are with |
|
I think this PR is good to merge, unless you want to add more tests. |
So that the file is closed properly for deletion later.
weiji14
approved these changes
May 20, 2020
There was a problem hiding this comment.
Ok, the test_surface.py errors are gone, and somehow the .clib ones disappeared on my system (Edit: it only fails when I run pygmt.test() from ipython, not in the terminal).
The pygmtscraper test seems to be failing because my stuff is on H:\ instead of C:\. Same with Azure where they use D:\ instead of C:\. One possible solution is to change this line:
pygmt/pygmt/tests/test_sphinx_gallery.py
Line 31 in 81ed8f4
to this:
with TemporaryDirectory(dir=os.getcwd()) as tmpdir:But I'm happy to let that slide for now 😄
|
Merged. We can fix other failing tests later. |
5 tasks
5 tasks
13 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of proposed changes
This PR fixes the PyGMT crash with the conda GMT package on Windows. It's a follow-up of RP #313. The commit history and the changed files in PR #313 is a little messy, so I cherry-picked 4 related commits from PR #313 and then work on top of it.
To make PyGMT work on Windows, users have to manually add two environmental variables:
C:\Miniconda\envs\pygmt\Library\binC:\Miniconda\envs\pygmt\Library\share\gmtctypes.CDLLcannot find the libraries provided by conda, possibly because conda doesn't add the library path into Windows' library search path. Thus, we have to add the GMT_LIBRARY_PATH variable to tell pygmt (i.e., ctypes.CDLL) where to find the gmt library file.GMT needs to know the path of its share directory to work. To determine the share directory, GMT checks a few environmental variables (GMT6_SHAREDIR, GMT5_SHAREDIR and GMT_SHARE). If none of them are defined, then GMT checks the variable GMT_SHARE_DIR which is set during building GMT. Since the conda GMT package was not built on users' computer, thus the GMT_SHARED_DIR directory doesn't exist at all. Then GMT tries to guess the share directory path based on the relative locations among the library, bin and share directories.
The guessing function works well on macOS and Linux, but not on Windows, and that function crashes.
PyGMT works well with the official GMT Windows installers, simply because the installers add the GMT6_SHAREDIR variable automatically. I believe it's a GMT bug, but we may need more time to find the exact crash location. Currently, the simplest workaround is adding the GMT_SHAREDIR variable manually.
The Windows CI jobs now work well, except 11 failing tests. These failing tests will be addressed in other PRs.
Fixes #46, #353. Closes #313.
Reminders
make formatandmake checkto make sure the code follows the style guide.doc/api/index.rst.