Got rid of `imp` rewriting of lib, plugins, and gui dirs #2020

cculianu · 2020-10-18T01:43:10Z

Module imp is deprecated and will be removed in a future Python
version. So, rather than do the imp rewriting of the lib, gui, and
plugins directories, we renamed these directories to electroncash,
electroncash_gui, and electroncash_plugins to match the Python module
names.

All setup and packaging files have been updated such that Windows,
Linux AppImage, OSX, iOS and srcdist now build correctly using these new
directories.

Android has not been properly migrated over yet, but that will be coming
soon.

Also in this commit: updated Dockerfiles for Wine and Linux AppImage
builds since they grew stale and some package versions needed updating.

We also bumped the app version to 4.2.0 since this filesystem renaming
change can potentially be considedered a significant enough change, as
per semantic versioning.

cculianu · 2020-10-18T01:46:53Z

@mhsmith or perhaps @jonas-lundqvist or @mpatc -- I wasn't able to correctly update the gradle file(s) -- and I think the Android version no longer builds correctly. Would you mind taking this branch out for a spin and getting the Android version to build correctly and then letting me know what I need to change?

@EchterAgo -- If you also have time and don't mind looking over this PR (I know it's large but most of it is renames) -- just to see if I missed anything. I built AppImage, Windows, OSX, and srcdist and they all work and are correct.

EchterAgo · 2020-10-18T02:30:45Z

A few more changes:

EchterAgo@156e95e

EchterAgo · 2020-10-18T04:46:00Z

Windows & Linux builds: https://ec.loping.net/4.1.1-32-ge4f688765/

cculianu · 2020-10-18T08:41:31Z

A few more changes:

EchterAgo@156e95e

Argh sorry I should have done a pull off your branch not a cherry-pick. Can you accept my apology? It was late...

cculianu · 2020-10-18T09:20:30Z

@EchterAgo Ok so my AppImage has a differeent sha256sum than yours. The Windows EXE's do match, however.

Here is my sha256sum:

fb44981d2b6ee294baea51cf1f9f9cc767c5ed74e940140989d869e96a688023  Electron-Cash-4.1.1-32-ge4f688765-x86_64.AppImage

I noticed that my build log differs from yours in that I got some warnings (looks like something went awry with the CLI args on one of the commands??):

Successfully installed Electron-Cash-4.2.0
You are using pip version 18.1, however version 20.2.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Uninstalling Cython-0.29.7:
  Successfully uninstalled Cython-0.29.7
💬 INFO:  Copying desktop integration
💬 INFO:  Adding launcher
💬 INFO:  Finalizing AppDir
ldd: ./usr/lib/python3.6/site-packages/electroncash_plugins/fusion/Cash: No such file or directory
ldd: ./Fusion: No such file or directory
ldd: ./Logo: No such file or directory
ldd: unrecognized option `-'
Try `ldd --help' for more information.
ldd: ./No: No such file or directory
ldd: ./Text: No such file or directory
ldd: ./Gray.svg: No such file or directory
ldd: ./usr/lib/python3.6/site-packages/electroncash_plugins/fusion/Cash: No such file or directory
ldd: ./Fusion: No such file or directory
ldd: ./Logo: No such file or directory
ldd: unrecognized option `-'
Try `ldd --help' for more information.
ldd: ./No: No such file or directory
ldd: ./Text.svg: No such file or directory
ldd: ./usr/lib/python3.6/site-packages/setuptools/command/launcher: No such file or directory
ldd: ./manifest.xml: No such file or directory
ldd: ./usr/lib/python3.6/site-packages/setuptools/script: No such file or directory
ldd: ./(dev).tmpl: No such file or directory
--- errors end above.. everything below proceeds as normal ---
/lib -> ./lib
/lib/x86_64-linux-gnu -> ./lib/x86_64-linux-gnu
'/lib/x86_64-linux-gnu/libc.so.6' -> './lib/x86_64-linux-gnu/libc.so.6'
--- everything below this line is normal ---

Any thoughts? Note I am building on a macOS host using Docker.

EchterAgo · 2020-10-18T09:20:39Z

Doesn't matter, I thought you were going to squash this anyway.

mhsmith

I assume the Android build problem was this one?

Execution failed for task ':app:mergeMainNetDebugPythonSources'.
> Encountered duplicate path "electroncash_gui/__init__.py" during copy operation configured with DuplicatesStrategy.FAIL

I'll have a look and push a fix to this branch.

Module `imp` is deprecated and will be removed in a future Python version. So, rather than do the `imp` rewriting of the lib, gui, and plugins directories, we renamed these directories to electroncash, electroncash_gui, and electroncash_plugins to match the Python module names. All setup and packaging files have been updated such that Windows, Linux AppImage, OSX, iOS and srcdist now build correctly using these new directories. Android has not been properly migrated over yet, but that will be coming soon. Also in this commit: updated Dockerfiles for Wine and Linux AppImage builds since they grew stale and some package versions needed updating. We also bumped the app version to 4.2.0 since this filesystem renaming change can potentially be considedered a significant enough change, as per semantic versioning.

Updated `MANIFEST.in` to include the new package name. Fixed some comments referring to the old package names. Removed `package_dir` from `setup.py`, it is not necessary anymore.

cculianu · 2020-10-18T12:33:26Z

Rebased after migrating the Dockerfile fixes over to master.

cculianu · 2020-10-18T12:35:00Z

I assume the Android build problem was this one?
Execution failed for task ':app:mergeMainNetDebugPythonSources'.
> Encountered duplicate path "electroncash_gui/__init__.py" during copy operation configured with DuplicatesStrategy.FAIL
I'll have a look and push a fix to this branch.

@mhsmith

Yes -- and I think it's because now there are 2 copies of electroncash_plugins and electroncash_gui.. so maybe rename one or .. something? No idea. I guess you copy them over so that generate-strings works...?

Thanks man! (Note I just rebased and force-pushed in order to make the Dockerfile fixes for Wine and Linux live in master.. sorry about that!)

cculianu · 2020-10-18T12:44:54Z

Regarding the non-reproducibility for AppImage -- @EchterAgo and I traced it down to a bug or quirk in MacOS docker vs Linux docker (I run macOS) -- it is not related to this PR -- it happens off master too. That can be investigated separetely.

mhsmith · 2020-10-18T12:51:04Z

The Android build works fine now.

cculianu · 2020-10-18T12:54:48Z

The Android build works fine now.

Awesome. I'm a little confused -- does this mean it ends up copying all of that code from electroncash_gui and electroncash_plugins into the generated app? Or no?

mhsmith · 2020-10-18T13:27:24Z

No, because I've changed the include lines to be more specific. It'll now only copy the __init__.py files so the directory is recognized as a package.

cculianu · 2020-10-18T21:31:14Z

Ah, ok great @mhsmith -- thanks a lot for your rapid help here. Awesome. I'm merging this...

This script got left behind and was still referring to the old lib/ folder. Fixed. Follow-up to #2020

…follow-up) This script got left behind and was still referring to the old lib/ folder. Fixed. Follow-up to Electron-Cash#2020

cculianu requested a review from mhsmith October 18, 2020 01:43

cculianu added packaging Issues related to building/packaging and not the app itself. WantWantWant Stuff we really want to do but there blocking issues and/or it will take time labels Oct 18, 2020

mhsmith reviewed Oct 18, 2020

View reviewed changes

cculianu and others added 2 commits October 18, 2020 15:32

Some more package rename changes

1c29d9a

Updated `MANIFEST.in` to include the new package name. Fixed some comments referring to the old package names. Removed `package_dir` from `setup.py`, it is not necessary anymore.

cculianu force-pushed the fix_dirnames branch from e4f6887 to 1c29d9a Compare October 18, 2020 12:33

Fix "duplicate path" error on Android

caa05b4

This was linked to issues Oct 18, 2020

Consider moving all code into a top-level package #807

Closed

DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses #1517

Closed

cculianu merged commit cab4e37 into master Oct 18, 2020

cculianu deleted the fix_dirnames branch October 18, 2020 21:31

cculianu mentioned this pull request Oct 18, 2020

Consider moving all code into a top-level package #807

Closed

cculianu added a commit that referenced this pull request Oct 18, 2020

ios: Updated copy_back_changes.sh to not use lib/ (#2020 follow-up)

30e96f4

This script got left behind and was still referring to the old lib/ folder. Fixed. Follow-up to #2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Got rid of `imp` rewriting of lib, plugins, and gui dirs #2020

Got rid of `imp` rewriting of lib, plugins, and gui dirs #2020

cculianu commented Oct 18, 2020 •

edited

cculianu commented Oct 18, 2020

EchterAgo commented Oct 18, 2020

EchterAgo commented Oct 18, 2020

cculianu commented Oct 18, 2020

cculianu commented Oct 18, 2020

EchterAgo commented Oct 18, 2020

mhsmith left a comment •

edited

cculianu commented Oct 18, 2020

cculianu commented Oct 18, 2020 •

edited

cculianu commented Oct 18, 2020 •

edited

mhsmith commented Oct 18, 2020

cculianu commented Oct 18, 2020 •

edited

mhsmith commented Oct 18, 2020 •

edited

cculianu commented Oct 18, 2020

Got rid of imp rewriting of lib, plugins, and gui dirs #2020

Got rid of imp rewriting of lib, plugins, and gui dirs #2020

Conversation

cculianu commented Oct 18, 2020 • edited

cculianu commented Oct 18, 2020

EchterAgo commented Oct 18, 2020

EchterAgo commented Oct 18, 2020

cculianu commented Oct 18, 2020

cculianu commented Oct 18, 2020

EchterAgo commented Oct 18, 2020

mhsmith left a comment • edited

Choose a reason for hiding this comment

cculianu commented Oct 18, 2020

cculianu commented Oct 18, 2020 • edited

cculianu commented Oct 18, 2020 • edited

mhsmith commented Oct 18, 2020

cculianu commented Oct 18, 2020 • edited

mhsmith commented Oct 18, 2020 • edited

cculianu commented Oct 18, 2020

Got rid of `imp` rewriting of lib, plugins, and gui dirs #2020

Got rid of `imp` rewriting of lib, plugins, and gui dirs #2020

cculianu commented Oct 18, 2020 •

edited

mhsmith left a comment •

edited

cculianu commented Oct 18, 2020 •

edited

cculianu commented Oct 18, 2020 •

edited

cculianu commented Oct 18, 2020 •

edited

mhsmith commented Oct 18, 2020 •

edited