Unable to packaging tensorflow.estimator

[CzsGit]

ImportError: No module named tensorflow.estimator

• pyinstaller version: 3.4
• Version of Python: 2.7.5
• Platform: CentOS7

mesh-tensorflow 0.0.5
tensorboard 1.13.1
tensorflow 1.13.1
tensorflow-estimator 1.13.0
tensorflow-gpu 1.13.1
tensorflow-probability 0.6.0

test code

import tensorflow.estimator
print('success')

PS:
Adding tensorflow.estimator to hiddenimports has no effect.
The source code executes normally, but an error is reported when run packaged.

Thanks

[Legorooj]

PyInstaller 3.4 is no longer supported. Please let us know if the issue persists in 3.6 or the development version.

[CzsGit]

PyInstaller 3.4 is no longer supported. Please let us know if the issue persists in 3.6 or the development version.

I just retested with pyinstaller3.6, and the problem is the same as with pyinstaller3.4 @Legorooj

[Legorooj]

Can you link to the tensorflow.estimator docs?

[CzsGit]

Can you link to the tensorflow.estimator docs?

link can be referred to https://www.tensorflow.org/api_docs/python/tf/estimator/ModeKeys
Estimator is an outer package,If you use a specific API, you can use the following code

import tensorflow as tf
t=tf.estimator.ModeKeys.TRAIN
print(t)

There is a new error if you use tensorflow directly without importing estimator

error message:
import tensorflow as tf
File "/tmp/pip-unpacked-wheel-ektVhF/PyInstaller/loader/pyimod03_importers.py", line 391, in load_module
File "tensorflow/init.py", line 24, in
File "/tmp/pip-unpacked-wheel-ektVhF/PyInstaller/loader/pyimod03_importers.py", line 391, in load_module
File "tensorflow/python/init.py", line 122, in
File "/tmp/pip-unpacked-wheel-ektVhF/PyInstaller/loader/pyimod03_importers.py", line 391, in load_module
File "tensorflow/python/platform/test.py", line 47, in
File "/tmp/pip-unpacked-wheel-ektVhF/PyInstaller/loader/pyimod03_importers.py", line 391, in load_module
File "mock/init.py", line 2, in
File "/tmp/pip-unpacked-wheel-ektVhF/PyInstaller/loader/pyimod03_importers.py", line 391, in load_module
File "mock/mock.py", line 71, in
File "pbr/version.py", line 461, in semantic_version
File "pbr/version.py", line 448, in _get_version_from_pkg_resources
File "pbr/packaging.py", line 755, in get_version
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name mock was given, but was not able to be found.

[CzsGit]

Hidden import ERROR

166951 INFO: Analyzing hidden import 'tensorflow.estimator.ModeKeys.TRAIN'
166951 ERROR: Hidden import 'tensorflow.estimator.ModeKeys.TRAIN' not found

[Legorooj]

@CzsGit interesting. Which PyInstaller command raised that error? And can you upload the spec file?

[CzsGit]

@CzsGit interesting. Which PyInstaller command raised that error? And can you upload the spec file?

code

import tensorflow as tf
t=tf.estimator.ModeKeys.TRAIN
print(t)

spec file

-- mode: python ; coding: utf-8 --

block_cipher = None

a = Analysis(['test.py'],
pathex=['/home/caozs/pyinstaller_test'],
binaries=[],
datas=[],
hiddenimports=['tensorflow','tensorflow.python'],
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
[],
exclude_binaries=True,
name='test',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
console=True )
coll = COLLECT(exe,
a.binaries,
a.zipfiles,
a.datas,
strip=False,
upx=True,
upx_exclude=[],
name='test')

What else need to be added to the spec file?

[CzsGit]

@CzsGit interesting. Which PyInstaller command raised that error? And can you upload the spec file?

Any Suggestions?

[Legorooj]

@CzsGit the file should work. I'll get back in a day or two when I can replicate the issue.

[hdf]

I would do 2 things:
1: Make hiddenimports empty, as it is definitely visible in the script, so it should not be hidden. (https://pyinstaller.readthedocs.io/en/stable/usage.html#what-to-bundle-where-to-search)
2: Set upx to False (in both places). As I found, that upx and CUDA do not mix. And I assume you are not running TF on the CPU...

The example throws a different error for me, the infamous #4406 one. I'm on Windows 7, Python 3.7.6 x64, and using Anaconda 4.8.2...

[Legorooj]

@hdf really? Hmm. As I said, I'll get back in a day or two - I've a PR I need to finish first.

[CzsGit]

@CzsGit the file should work. I'll get back in a day or two when I can replicate the issue.

Looking forward to your reply

[CzsGit]

I would do 2 things:
1: Make hiddenimports empty, as it is definitely visible in the script, so it should not be hidden. (https://pyinstaller.readthedocs.io/en/stable/usage.html#what-to-bundle-where-to-search)
2: Set upx to False (in both places). As I found, that upx and CUDA do not mix. And I assume you are not running TF on the CPU...

The example throws a different error for me, the infamous #4406 one. I'm on Windows 7, Python 3.7.6 x64, and using Anaconda 4.8.2...

I tried your method and the result is the same as before
#4697 (comment)

[Legorooj]

https://stackoverflow.com/questions/60384288/pyinstaller-modulenotfounderror

[Legorooj]

I also tested with main.py as the code you mentioned above.

[CzsGit]

I also tested with main.py as the code you mentioned above.

I tried this method, but it was the same error
Create the hook folder in the same directory as test.py, and write hook_tensorflow.py in it:

from PyInstaller.utils.hooks import collect_all

def hook(hook_api):
    packages = [
        'tensorflow',
        'tensorflow_estimator'
    ]
    for package in packages:
        datas, binaries, hiddenimports = collect_all(package)
        hook_api.add_datas(datas)
        hook_api.add_binaries(binaries)
        hook_api.add_imports(*hiddenimports)

test.py

import tensorflow as tf

t=tf.estimator.ModeKeys.TRAIN
print(t)

Both versions of pyinstaller 3.4 3.5 3.6 have been tried, and tensorflow has tried both versions of 1.9 1.13

[CzsGit]

error log

Traceback (most recent call last):
File "test.py", line 1, in
import tensorflow as tf
File "/tmp/pip-unpacked-wheel-r8Cr7R/PyInstaller/loader/pyimod03_importers.py", line 395, in load_module
File "tensorflow/init.py", line 24, in
File "/tmp/pip-unpacked-wheel-r8Cr7R/PyInstaller/loader/pyimod03_importers.py", line 395, in load_module
File "tensorflow/python/init.py", line 122, in
File "/tmp/pip-unpacked-wheel-r8Cr7R/PyInstaller/loader/pyimod03_importers.py", line 395, in load_module
File "tensorflow/python/platform/test.py", line 47, in
File "/tmp/pip-unpacked-wheel-r8Cr7R/PyInstaller/loader/pyimod03_importers.py", line 395, in load_module
File "mock/init.py", line 2, in
File "/tmp/pip-unpacked-wheel-r8Cr7R/PyInstaller/loader/pyimod03_importers.py", line 395, in load_module
File "mock/mock.py", line 71, in
File "pbr/version.py", line 461, in semantic_version
File "pbr/version.py", line 448, in _get_version_from_pkg_resources
File "pbr/packaging.py", line 755, in get_version
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name mock was given, but was not able to be found.

[Legorooj]

Use tensorflow 2.1. that's what I tested with. Also add artor to the packages list.

[CzsGit]

Use tensorflow 2.1. that's what I tested with. Also add artor to the packages list.

Can you test it with version 1.13.1 or 1.9 of tensorflow?Because I can only use this version in my project now。now I still can't work with this method

[Legorooj]

@CzsGit I don't have the time. The linked SO question/answer should provide you with all you need, when combined with the docs.