Skip to content

Rename modules and notebooks#149

Merged
rouille merged 3 commits intodevelopfrom
ben/rename
Aug 26, 2020
Merged

Rename modules and notebooks#149
rouille merged 3 commits intodevelopfrom
ben/rename

Conversation

@rouille
Copy link
Copy Markdown
Collaborator

@rouille rouille commented Aug 25, 2020

Purpose

Rename modules and notebooks for consistency with other modules

What is the code doing

There is no code

Where to look

I have updated the import statements accordingly and rerun the notebooks.

Time estimate

5 min

@rouille rouille added this to the WTT90s milestone Aug 25, 2020
Copy link
Copy Markdown
Contributor

@danielolsen danielolsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am unable to re-run the mapping demo notebooks, since I've been unable to get pyproj 1.9.6 installed into the Python 3.8 environment on my new Windows machine. The changes seem to just be naming though, and I trust you.

@rouille
Copy link
Copy Markdown
Collaborator Author

rouille commented Aug 26, 2020

I am unable to re-run the mapping demo notebooks, since I've been unable to get pyproj 1.9.6 installed into the Python 3.8 environment on my new Windows machine. The changes seem to just be naming though, and I trust you.

Did you use the workflow developed by @jon-hagg (pipenv) for installing all the dependencies or you follow another path? You are, I believe, the only developer using Windows in the group so we don't have a lot of feedback regarding the overall installation on this platform.

@danielolsen
Copy link
Copy Markdown
Contributor

danielolsen commented Aug 26, 2020

Did you use the workflow developed by @jon-hagg (pipenv) for installing all the dependencies or you follow another path? You are, I believe, the only developer using Windows in the group so we don't have a lot of feedback regarding the overall installation on this platform.

I did not. I don't believe the problem is a mismatch between python dependencies. When I attempt to install pyproj==1.9.6 via pip, I get an error that setup.py fails because of some problems with Microsoft Visual C++ 14.0 tools. There is a site that maintains Windows binaries of many popular packages (https://www.lfd.uci.edu/~gohlke/pythonlibs/), and this site has wheels for pyproj 1.9.6 up to python 3.7, but not python 3.8 (It has binaries for 3.8+ for newer version of pyproj, but the way we use pyproj is orders of magnitude slower in pyproj 2.x). pyproj 2.x installs fine, but throws many warnings and takes forever.

Is this something that you think pipenv would solve?

@rouille
Copy link
Copy Markdown
Collaborator Author

rouille commented Aug 26, 2020

Did you use the workflow developed by @jon-hagg (pipenv) for installing all the dependencies or you follow another path? You are, I believe, the only developer using Windows in the group so we don't have a lot of feedback regarding the overall installation on this platform.

I did not. I don't believe the problem is a mismatch between python dependencies. When I attempt to install pyproj==1.9.6 via pip, I get an error that setup.py fails because of some problems with Microsoft Visual C++ 14.0 tools. There is a site that maintains Windows binaries of many popular packages (https://www.lfd.uci.edu/~gohlke/pythonlibs/), and this site has wheels for pyproj 1.9.6 up to python 3.7, but not python 3.8 (It has binaries for 3.8+ for newer version of pyproj, but the way we use pyproj is orders of magnitude slower in pyproj 2.x). pyproj 2.x installs fine, but throws many warnings and takes forever.

Is this something that you think pipenv would solve?

No, it seems to be Windows specific. I think we should test the newest version of pyproj to see if it is still slower than the versions < 2. Do we have a test in PostREISE that does the calculation you mention?

@danielolsen
Copy link
Copy Markdown
Contributor

danielolsen commented Aug 26, 2020

I did not. I don't believe the problem is a mismatch between python dependencies. When I attempt to install pyproj==1.9.6 via pip, I get an error that setup.py fails because of some problems with Microsoft Visual C++ 14.0 tools. There is a site that maintains Windows binaries of many popular packages (https://www.lfd.uci.edu/~gohlke/pythonlibs/), and this site has wheels for pyproj 1.9.6 up to python 3.7, but not python 3.8 (It has binaries for 3.8+ for newer version of pyproj, but the way we use pyproj is orders of magnitude slower in pyproj 2.x). pyproj 2.x installs fine, but throws many warnings and takes forever.
Is this something that you think pipenv would solve?

No, it seems to be Windows specific. I think we should test the newest version of pyproj to see if it is still slower than the versions < 2. Do we have a test in PostREISE that does the calculation you mention?

I don't think we test this, because it's plotting/visualization. I installed the latest version of pyproj so that I could run pytest on PostREISE (I need to be able to import pyproj when importing PostREISE, even if the tests never use it), so I was able to start running the notebooks but I got lots of warnings and no output after quite some time. I remember from before it was very quick with 1.9.6 using Python 3.7 on my last Windows machine.

A few months back I looked at refactoring the pyproj calls to be compatible with 2.x, but it was more trouble than it seemed worth at the time so I gave up and just installed 1.9.6. The main difference seems to be that a function/method which is fast in 1.9.6 is slow in 2.x because there is something that only needs to be done once, but ends up being done for each data point (or something along those lines). There seems to be new syntax in 2.x to do the do-it-once part separately and use it many times for a collection of points.

If someone on a Mac can test that the new notebooks run as expected, then I think this PR is ready to go, and we can create an issue to update our code for pyproj 2.x as a follow-up task that should make the plotting compatible with Python 3.8+ on Windows.

@rouille
Copy link
Copy Markdown
Collaborator Author

rouille commented Aug 26, 2020

I did not. I don't believe the problem is a mismatch between python dependencies. When I attempt to install pyproj==1.9.6 via pip, I get an error that setup.py fails because of some problems with Microsoft Visual C++ 14.0 tools. There is a site that maintains Windows binaries of many popular packages (https://www.lfd.uci.edu/~gohlke/pythonlibs/), and this site has wheels for pyproj 1.9.6 up to python 3.7, but not python 3.8 (It has binaries for 3.8+ for newer version of pyproj, but the way we use pyproj is orders of magnitude slower in pyproj 2.x). pyproj 2.x installs fine, but throws many warnings and takes forever.
Is this something that you think pipenv would solve?

No, it seems to be Windows specific. I think we should test the newest version of pyproj to see if it is still slower than the versions < 2. Do we have a test in PostREISE that does the calculation you mention?

I don't think we test this, because it's plotting/visualization. I installed the latest version of pyproj so that I could run pytest on PostREISE (I need to be able to import pyproj when importing PostREISE, even if the tests never use it), so I was able to start running the notebooks but I got lots of warnings and no output after quite some time. I remember from before it was very quick with 1.9.6 using Python 3.7 on my last Windows machine.

A few months back I looked at refactoring the pyproj calls to be compatible with 2.x, but it was more trouble than it seemed worth at the time so I gave up and just installed 1.9.6. The main difference seems to be that a function/method which is fast in 1.9.6 is slow in 2.x because there is something that only needs to be done once, but ends up being done for each data point (or something along those lines). There seems to be new syntax in 2.x to do the do-it-once part separately and use it many times for a collection of points.

If someone on a Mac can test that the new notebooks run as expected, then I think this PR is ready to go, and we can create an issue to update our code for pyproj 2.x as a follow-up task that should make the plotting compatible with Python 3.8+ on Windows.

I believe, we have to refactor postreise.plot.projection_helpers following https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrade-transformer

@danielolsen
Copy link
Copy Markdown
Contributor

I believe, we have to refactor postreise.plot.projection_helpers following https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrade-transformer

Yes, I'm almost positive that's what I attempted to do. I remember it being less simple to refactor our code than the example shows, but perhaps I was just frustrated/lazy and it will not be so bad.

@rouille rouille merged commit 157292d into develop Aug 26, 2020
@rouille rouille deleted the ben/rename branch August 26, 2020 18:10
@ahurli ahurli mentioned this pull request Mar 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants