Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Entry points for gdal-utils package #5282

Closed
wants to merge 14 commits into from

Conversation

maphew
Copy link
Contributor

@maphew maphew commented Feb 10, 2022

What does this PR do?

Modifies setup.py so that when pip install gdal-utils is run all of the .py files in ./scripts are turned into native platform executable scripts that call the .py file using the appropriate interpreter. Although developed to improve the experience on Windows the implementation is cross platform (due to upstream magic by setuptools and/or distutils) and also is better for linux (c.f. https://click.palletsprojects.com/en/8.0.x/setuptools/?highlight=entry_points).

Benefits for the user is that after install the gdal python utility scripts are available from the command line immediately, just like all the first class binary utiltiies, e.g. gdalinfo, gdal_translate, and without needing to use .py extension with the name. Gdal edit is just gdal_edit.

What are related issues/pull requests?

gdal-utils: improve command line in Windows and ditch .bat wrappers #5281

Tasklist

  • implement and test locally
  • Add test case(s)
  • Add documentation
  • Review
  • Adjust for comments
  • All CI builds and checks have passed

Environment

  • OS: Microsoft Windows [Version 10.0.19042.804]
  • Compiler: n/a

maphew and others added 14 commits February 2, 2022 12:33
As per "[gdal-dev] gdal-utils: About GDAL Python Utilities" thread,
an overview and intro page for the gdal-utils sub project.
I don't know if this is the right place for this info. Feel free to
move/edit/adjust.
Co-authored-by: Even Rouault <even.rouault@spatialys.com>
Co-authored-by: Even Rouault <even.rouault@spatialys.com>
Co-authored-by: Even Rouault <even.rouault@spatialys.com>
Co-authored-by: Even Rouault <even.rouault@spatialys.com>
sorry for the noisy commit history, I didn't understand how using the
Github resolve conversation and commit buttons worked.
Adapted from https://stackoverflow.com/questions/23708231/git-shallow-clone-clone-depth-misses-remote-branches
Note important amendments from comments (don't use single quote, also
limit fetch depth).
Fixes:
```
---------------------------
gdalinfo.exe - System Error
---------------------------
The code execution cannot proceed because VCRUNTIME140D.dll was not found. Reinstalling the program may fix this problem.
```
As per "gdal-utils: improve command line in Windows and ditch .bat wrappers OSGeo#5281"
@maphew
Copy link
Contributor Author

maphew commented Feb 10, 2022

I don't know how to write tests, especially so when it comes to setup.py. Sorry, but I'm going to need help with this part!

This is how it works on my laptop.

Dev environment:

conda create -n geo 
conda activate geo
conda install -c conda-forge gdal 
git clone ...
(geo) CMD C:\Users\mhwilkie\code\gdal\swig\python\gdal-utils
» pip install -e .
Obtaining file:///C:/Users/mhwilkie/code/gdal/swig/python/gdal-utils
  Preparing metadata (setup.py) ... done
Requirement already satisfied: gdal in c:\tools\miniconda3\envs\geo\lib\site-packages (from gdal-utils==3.3.0.7) (3.4.1)
Installing collected packages: gdal-utils
  Running setup.py develop for gdal-utils
Successfully installed gdal-utils-3.3.0.7

(geo) CMD C:\Users\mhwilkie\code\gdal\swig\python\gdal-utils
» gdal_edit
Usage: gdal_edit [--help-general] [-ro] [-a_srs srs_def]
                 [-a_ullr ulx uly lrx lry] [-a_ulurll ulx uly urx ury llx lly]
                 [-tr xres yres] [-unsetgt] [-unsetrpc] [-a_nodata value] [-unsetnodata]
                 [-offset value] [-scale value] [-units value]
                 [-colorinterp_X red|green|blue|alpha|gray|undefined]*
                 [-unsetstats] [-stats] [-approx_stats]
                 [-setstats min max mean stddev]
                 [-gcp pixel line easting northing [elevation]]*
                 [-unsetmd] [-oo NAME=VALUE]* [-mo "META-TAG=VALUE"]*  datasetname

Edit in place various information of an existing GDAL dataset.
» pip uninstall gdal-utils
Found existing installation: gdal-utils 3.3.0.7
Uninstalling gdal-utils-3.3.0.7:
  Would remove:
    c:\tools\miniconda3\envs\geo\lib\site-packages\gdal-utils.egg-link
    c:\tools\miniconda3\envs\geo\scripts\gdal2tiles-script.py
    c:\tools\miniconda3\envs\geo\scripts\gdal2tiles.exe
    c:\tools\miniconda3\envs\geo\scripts\gdal2xyz-script.py
    c:\tools\miniconda3\envs\geo\scripts\gdal2xyz.exe
    c:\tools\miniconda3\envs\geo\scripts\gdal_calc-script.py
    c:\tools\miniconda3\envs\geo\scripts\gdal_calc.exe
    c:\tools\miniconda3\envs\geo\scripts\gdal_edit-script.py
    c:\tools\miniconda3\envs\geo\scripts\gdal_edit.exe
    c:\tools\miniconda3\envs\geo\scripts\gdal_fillnodata-script.py
    c:\tools\miniconda3\envs\geo\scripts\gdal_fillnodata.exe
    c:\tools\miniconda3\envs\geo\scripts\gdal_merge-script.py
    c:\tools\miniconda3\envs\geo\scripts\gdal_merge.exe
    c:\tools\miniconda3\envs\geo\scripts\gdal_pansharpen-script.py
    c:\tools\miniconda3\envs\geo\scripts\gdal_pansharpen.exe
    c:\tools\miniconda3\envs\geo\scripts\gdal_polygonize-script.py
    c:\tools\miniconda3\envs\geo\scripts\gdal_polygonize.exe
    c:\tools\miniconda3\envs\geo\scripts\gdal_proximity-script.py
    c:\tools\miniconda3\envs\geo\scripts\gdal_proximity.exe
    c:\tools\miniconda3\envs\geo\scripts\gdal_retile-script.py
    c:\tools\miniconda3\envs\geo\scripts\gdal_retile.exe
    c:\tools\miniconda3\envs\geo\scripts\gdal_sieve-script.py
    c:\tools\miniconda3\envs\geo\scripts\gdal_sieve.exe
    c:\tools\miniconda3\envs\geo\scripts\gdalattachpct-script.py
    c:\tools\miniconda3\envs\geo\scripts\gdalattachpct.exe
    c:\tools\miniconda3\envs\geo\scripts\gdalcompare-script.py
    c:\tools\miniconda3\envs\geo\scripts\gdalcompare.exe
    c:\tools\miniconda3\envs\geo\scripts\gdalmove-script.py
    c:\tools\miniconda3\envs\geo\scripts\gdalmove.exe
    c:\tools\miniconda3\envs\geo\scripts\ogrmerge-script.py
    c:\tools\miniconda3\envs\geo\scripts\ogrmerge.exe
    c:\tools\miniconda3\envs\geo\scripts\pct2rgb-script.py
    c:\tools\miniconda3\envs\geo\scripts\pct2rgb.exe
    c:\tools\miniconda3\envs\geo\scripts\rgb2pct-script.py
    c:\tools\miniconda3\envs\geo\scripts\rgb2pct.exe
Proceed (Y/n)?

@rouault
Copy link
Member

rouault commented Feb 10, 2022

It seems this PR is on top of #5222 ? which I've just merged. Could you git rebase this pull request on top of latest master (something like "git fetch upstream master; git rebase upstream/master", replace upstream by origin or whatever remote name you've chosen for the OSGeo/gdal repository) so that this PR only shows the specific changes.

'gdal_merge = scripts.gdal_merge',
... ]
"""
xx = []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe console_scripts instead of xx ?

@rouault rouault changed the title Gdal utils mhw entry points Entry points for gdal-utils package Feb 10, 2022
@idanmiara
Copy link
Collaborator

Nice idea @maphew!
Thanks for looking into this issue 😀
I don't know entry_points, I'll try to test it without conda.

Please rebase on master and force push as this PR also includes your commits from #5222 that were merged already.

@maphew
Copy link
Contributor Author

maphew commented Feb 11, 2022

I'm not familiar with rebase. I've read over https://www.atlassian.com/git/tutorials/rewriting-history/git-rebase a couple times and understand the purpose but when I attempt to apply it I end up in a series of merge conflicts that I can't resolve (in the way that I'm familiar with in merging branches). I'm going to keep picking at it because clearly this an important process to learn, but I don't know how long that's going to take. :)

note: I use Atlassian docs because they're well written but the git gui I use is Git Extensions.

@rouault
Copy link
Member

rouault commented Feb 11, 2022

I'm not familiar with rebase. I've read over https://www.atlassian.com/git/tutorials/rewriting-history/git-rebase a couple times and understand the purpose but when I attempt to apply it I end up in a series of merge conflicts that I can't resolve (in the way that I'm familiar with in merging branches). I'm going to keep picking at it because clearly this an important process to learn, but I don't know how long that's going to take. :)

or just recreate a new branch and cherry-pick the commits of interest

@idanmiara
Copy link
Collaborator

idanmiara commented Feb 11, 2022 via email

@maphew maphew mentioned this pull request Feb 11, 2022
13 tasks
@rouault
Copy link
Member

rouault commented Feb 11, 2022

superseded per #5296

@rouault rouault closed this Feb 11, 2022
@maphew
Copy link
Contributor Author

maphew commented Feb 11, 2022

@idanmiara thanks for the Github Desktop recommendation. I'm exploring it now. The drag and drop reordering and squashing of commits is very nice, so intuitive! I'm perplexed why there's no graph visuals for branch merge history and relationships. Maybe that will come clear in time. In the meantime I can use it and Git Extensions side by side for that purpose.

I took @rouault's suggestion and cherry picked the relevant commit, into a new PR. I just can't wrap my head around rebase today.

@maphew maphew deleted the gdal-utils-mhw-entry-points branch February 11, 2022 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants