New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hook for 'phonenumbers.data.' #3381

Closed
gitdjc opened this Issue Mar 8, 2018 · 3 comments

Comments

Projects
None yet
4 participants
@gitdjc

gitdjc commented Mar 8, 2018

This program:

import phonenumbers
Number = "+17034820623"
Parsed_Number = phonenumbers.parse(Number)
print("The parsed number is <%s>" % (Parsed_Number))

works when invoked directly using Python 3.6.4:

$ PF.py
The parsed number is <Country Code: 1 National Number: 7034820623>

but when packaged with pyinstaller 3.4.dev0:

$ pyinstaller --debug --onedir PF.py

fails, ultimately with:

ModuleNotFoundError: No module named 'phonenumbers.data.region_US'

which the phonenumbers package appears to try to load dynamically. It's in:

D:\Python36\Lib\site-packages\phonenumbers\data\region_US.py

That directory contains an "init.py" along with more than 300 other (small) modules which "phonenumbers" loads as needed.

Is there some magic to make this work with "--onedir", but ideally with "--onefile"? To give the script another place to look for those modules? To incorporate them into the .EXE? Since I'm fairly new to Python and totally new to pyinstaller, I don't doubt I'm overlooking something. The recipe for pytz seems to hold some promise, but I can't be sure and I'm not yet knowledgeable enough to work with it.

When I try to attach pyinstaller's run log or the executable's, I'm being told "we can't process that file", so I'm pasting them here. First the executable's run log:

$ dist\PF\PF.exe
[4300] PyInstaller Bootloader 3.x
[4300] LOADER: executable is f:\PFD\dist\PF\PF.exe
[4300] LOADER: homepath is f:\PFD\dist\PF
[4300] LOADER: _MEIPASS2 is NULL
[4300] LOADER: archivename is f:\PFD\dist\PF\PF.exe
[4300] LOADER: No need to extract files to run; setting extractionpath to homepath
[4300] LOADER: SetDllDirectory(f:\PFD\dist\PF)
[4300] LOADER: Already in the child - running user's code.
[4300] LOADER: Python library: f:\PFD\dist\PF\python36.dll
[4300] LOADER: Loaded functions from Python library.
[4300] LOADER: Manipulating environment (sys.path, sys.prefix)
[4300] LOADER: Pre-init sys.path is f:\PFD\dist\PF\base_library.zip;f:\PFD\dist\PF
[4300] LOADER: sys.prefix is f:\PFD\dist\PF
[4300] LOADER: Setting runtime options
[4300] LOADER: Initializing python
[4300] LOADER: Overriding Python's sys.path
[4300] LOADER: Post-init sys.path is f:\PFD\dist\PF\base_library.zip;f:\PFD\dist\PF
[4300] LOADER: Setting sys.argv
[4300] LOADER: setting sys._MEIPASS
[4300] LOADER: importing modules from CArchive
[4300] LOADER: extracted struct
[4300] LOADER: callfunction returned...
[4300] LOADER: extracted pyimod01_os_path
[4300] LOADER: callfunction returned...
[4300] LOADER: extracted pyimod02_archive
[4300] LOADER: callfunction returned...
[4300] LOADER: extracted pyimod03_importers
[4300] LOADER: callfunction returned...
[4300] LOADER: Installing PYZ archive with Python modules.
[4300] LOADER: PYZ archive: out00-PYZ.pyz
[4300] LOADER: Running pyiboot01_bootstrap.py
[4300] LOADER: Running PF.py
[4300] Failed to execute script PF
[4300] LOADER: OK.
[4300] LOADER: Cleaning up Python interpreter.
Traceback (most recent call last):
File "PF.py", line 4, in <module>
Parsed_Number = phonenumbers.parse(Number)
File "site-packages\phonenumbers\phonenumberutil.py", line 2835, in parse
File "site-packages\phonenumbers\phonemetadata.py", line 294, in metadata_for_region_or_calling_code
File "site-packages\phonenumbers\phonemetadata.py", line 267, in metadata_for_region
File "site-packages\phonenumbers\data\__init__.py", line 23, in _load_region
ModuleNotFoundError: No module named 'phonenumbers.data.region_US'

And pyinstaller's run log:

$ pyinstaller --debug --onedir PF.py
50 INFO: PyInstaller: 3.4.dev0
50 INFO: Python: 3.6.4
50 INFO: Platform: Windows-7-6.1.7601-SP1
50 INFO: wrote f:\PFD\PF.spec
80 INFO: UPX is available.
80 INFO: Extending PYTHONPATH with paths
['f:\\PFD', 'f:\\PFD']
80 INFO: checking Analysis
80 INFO: Building Analysis because out00-Analysis.toc is non existent
80 INFO: Initializing module dependency graph...
80 INFO: Initializing module graph hooks...
80 INFO: Analyzing base_library.zip ...
2320 INFO: running Analysis out00-Analysis.toc
2330 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable
required by d:\python36\python.exe
2790 INFO: Caching module hooks...
2790 INFO: Analyzing f:\PFD\PF.py
2960 INFO: Loading module hooks...
2960 INFO: Loading module hook "hook-encodings.py"...
3010 INFO: Loading module hook "hook-pydoc.py"...
3010 INFO: Loading module hook "hook-xml.py"...
3190 INFO: Looking for ctypes DLLs
3190 INFO: Analyzing run-time hooks ...
3190 INFO: Looking for dynamic libraries
3250 INFO: Looking for eggs
3250 INFO: Using Python library d:\python36\python36.dll
3250 INFO: Found binding redirects:
[]
3250 INFO: Warnings written to f:\PFD\build\PF\warnPF.txt
3280 INFO: Graph cross-reference written to f:\PFD\build\PF\xref-PF.html
3290 INFO: checking PYZ
3290 INFO: Building PYZ because out00-PYZ.toc is non existent
3290 INFO: Building PYZ (ZlibArchive) f:\PFD\build\PF\out00-PYZ.pyz
3660 INFO: Building PYZ (ZlibArchive) f:\PFD\build\PF\out00-PYZ.pyz completed successfully.
3660 INFO: checking PKG
3660 INFO: Building PKG because out00-PKG.toc is non existent
3660 INFO: Building PKG (CArchive) out00-PKG.pkg
3670 INFO: Building PKG (CArchive) out00-PKG.pkg completed successfully.
3670 INFO: Bootloader d:\python36\lib\site-packages\PyInstaller\bootloader\Windows-64bit\run_d.exe
3670 INFO: checking EXE
3670 INFO: Building EXE because out00-EXE.toc is non existent
3670 INFO: Building EXE from out00-EXE.toc
3670 INFO: Appending archive to EXE f:\PFD\build\PF\PF.exe
3670 INFO: Building EXE from out00-EXE.toc completed successfully.
3670 INFO: checking COLLECT
3670 INFO: Building COLLECT because out00-COLLECT.toc is non existent
3670 INFO: Building COLLECT out00-COLLECT.toc
3740 INFO: Building COLLECT out00-COLLECT.toc completed successfully.

@htgoebel htgoebel added the hooks label Mar 10, 2018

@htgoebel

This comment has been minimized.

Member

htgoebel commented Mar 10, 2018

I assume for full functionality you need to add all these 300 hidden modules. For this create a hook using collect_submodules() (see https://pyinstaller.readthedocs.io/en/stable/hooks.html#useful-items-in-pyinstaller-utils-hooks). See this section of the manual for more information about how to implement hooks.

We would appreciate if you'd submit a pull-request for the hook for this then :-) See our Development Guide on how to create pull-requests for PyInstaller. Thanks.

@htgoebel htgoebel changed the title from Packaged script fails on dynamic load; minimal example to Hook for 'phonenumbers.data.' Mar 13, 2018

arthurspa added a commit to arthurspa/pyinstaller that referenced this issue Jun 6, 2018

@arthurspa

This comment has been minimized.

Contributor

arthurspa commented Jun 6, 2018

Hi @htgoebel!

I just submitted PR #3558 which implements this hook.

Could you please check it?

Thanks!

arthurspa added a commit to arthurspa/pyinstaller that referenced this issue Jun 6, 2018

arthurspa added a commit to arthurspa/pyinstaller that referenced this issue Jun 6, 2018

Hooks: Add hook for phonenumbers.
Fixes pyinstaller#3381
Hooks: Add hook for phonenumbers.

Fixes pyinstaller#3381

arthurspa added a commit to arthurspa/pyinstaller that referenced this issue Jun 6, 2018

htgoebel added a commit that referenced this issue Jun 11, 2018

@htgoebel htgoebel added this to the PyInstaller 3.4 milestone Sep 2, 2018

@hykavitha

This comment has been minimized.

hykavitha commented Sep 18, 2018

context : I'm using snap-plugin-lib-py, my goal is to get rand.py workable in non python environment. So I'm trying pyinstaller.
I problem i see is:

cd snap-plugin-lib-py/examples/collector/dist$

./rand
Traceback (most recent call last):
File "rand.py", line 24, in
ImportError: No module named 'snap_plugin'
[10380] Failed to execute script rand

Why is this import error, shouldn't it import all the required packages?
I read the pyinstaller doc saying this, but I don't see this happening: PyInstaller reads a Python script written by you. It analyzes your code to discover every other module and library your script needs in order to execute. Then it collects copies of all those files – including the active Python interpreter!
– and puts them with your script in a single folder, or optionally in a single executable file.

Here is the pyinstaller run log.

pyinstaller --onefile rand.py
38 INFO: PyInstaller: 3.4
38 INFO: Python: 3.5.2
38 INFO: Platform: Linux-4.4.0-1066-aws-x86_64-with-Ubuntu-16.04-xenial
39 INFO: wrote /home/ubuntu/snap-plugin-lib-py/examples/collector/rand.spec
40 INFO: UPX is not available.
41 INFO: Extending PYTHONPATH with paths
['/home/ubuntu/snap-plugin-lib-py/examples/collector',
'/home/ubuntu/snap-plugin-lib-py/examples/collector']
41 INFO: checking Analysis
41 INFO: Building Analysis because Analysis-00.toc is non existent
41 INFO: Initializing module dependency graph...
43 INFO: Initializing module graph hooks...
44 INFO: Analyzing base_library.zip ...
3206 INFO: running Analysis Analysis-00.toc
3226 INFO: Caching module hooks...
3229 INFO: Analyzing /home/ubuntu/snap-plugin-lib-py/examples/collector/rand.py
3237 INFO: Loading module hooks...
3237 INFO: Loading module hook "hook-pydoc.py"...
3238 INFO: Loading module hook "hook-xml.py"...
3489 INFO: Loading module hook "hook-encodings.py"...
3552 INFO: Looking for ctypes DLLs
3566 INFO: Analyzing run-time hooks ...
3572 INFO: Looking for dynamic libraries
3715 INFO: Looking for eggs
3715 INFO: Python library not in binary dependencies. Doing additional searching...
3762 INFO: Using Python library /usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0
3765 INFO: Warnings written to /home/ubuntu/snap-plugin-lib-py/examples/collector/build/rand/warn-rand.txt
3788 INFO: Graph cross-reference written to /home/ubuntu/snap-plugin-lib-py/examples/collector/build/rand/xref-rand.html
3794 INFO: checking PYZ
3794 INFO: Building PYZ because PYZ-00.toc is non existent
3794 INFO: Building PYZ (ZlibArchive) /home/ubuntu/snap-plugin-lib-py/examples/collector/build/rand/PYZ-00.pyz
4102 INFO: Building PYZ (ZlibArchive) /home/ubuntu/snap-plugin-lib-py/examples/collector/build/rand/PYZ-00.pyz completed successfully.
4104 INFO: checking PKG
4104 INFO: Building PKG because PKG-00.toc is non existent
4104 INFO: Building PKG (CArchive) PKG-00.pkg
6230 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
6232 INFO: Bootloader /home/ubuntu/.local/lib/python3.5/site-packages/PyInstaller/bootloader/Linux-64bit/run
6232 INFO: checking EXE
6232 INFO: Building EXE because EXE-00.toc is non existent
6232 INFO: Building EXE from EXE-00.toc
6233 INFO: Appending archive to ELF section in EXE /home/ubuntu/snap-plugin-lib-py/examples/collector/dist/rand
6251 INFO: Building EXE from EXE-00.toc completed successfully.

cowo78 pushed a commit to cowo78/pyinstaller that referenced this issue Dec 7, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment