data file not found when using --add-data with --onefile

[sharuzzaman]

+++ ONLY TEXT +++ DO NOT POST IMAGES +++

Description of the issue

I'm trying to contribute to this project: https://github.com/akmalhakimi1991/mousemover

As the first step, I'm trying to rebuild the project, which was successfully built with PyInstaller previously. After cloning the source to c:\myprojects\mousemover using Sourcetree, I created the virtual environment using the command

pip -m venv venv

Next, I install the dependency with command

pip install -r requirements.txt

Then cd into mousemover folder, and type

python mousemover.py

And the program runs correctly. Now building the exe by using the command

pyinstaller --onefile --add-data "config.yml;." mousemover.py

The executable mousemover.exe is created in folder dist. If I type the command

dist\mousemover.exe

it works because the exe is reading the config.yml inside the source folder. But if I cd into dist, then type mousemover.exe, the error message is:

(venv) C:\myprojects\mousemover\mousemover\dist>mousemover.exe
Traceback (most recent call last):
  File "mousemover\mousemover.py", line 27, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'config.yml'
[11200] Failed to execute script mousemover

The file mousemover.spec have the entry

datas=[('config.yml', '.')],

under the Analysis class.

I have searched the Internet and looks at several answers in StackOverflow, but none have worked so far. Maybe I'm still not familiar with the whole PyInstaller feature

Context information (for bug reports)

• Output of pyinstaller --version:
  (venv) C:\myprojects\mousemover>pyinstaller --version
  3.6

• Version of Python: e.g. 3.7
  (venv) C:\myprojects\mousemover>python --version
  Python 3.7.7

• Platform: e.g GNU/Linux (distribution), Windows (language settings), OS X, conda, FreeBSD
  (venv) C:\myprojects\mousemover>systeminfo

OS Name: Microsoft Windows 10 Home Single Language
OS Version: 10.0.18363 N/A Build 18363

• Did you also try this on another platform? Does it work there?
  No other platform to test

• try the latest development version, using the following command:

pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip

same error message:
(venv) C:\myprojects\mousemover\mousemover\dist>mousemover.exe
Traceback (most recent call last):
File "mousemover.py", line 27, in
FileNotFoundError: [Errno 2] No such file or directory: 'config.yml'
[12076] Failed to execute script mousemover

(venv) C:\myprojects\mousemover\mousemover\dist>pyinstaller --version
4.0.dev0

• follow all the instructions in our "If Things Go Wrong" Guide
  (https://github.com/pyinstaller/pyinstaller/wiki/If-Things-Go-Wrong) and

Make sure everything is packaged correctly

• start with clean installation

• use the latest development version

• Run your frozen program from a command window (shell) — instead of double-clicking on it

• Package your program in --onedir mode

• Package without UPX, say: use the option --noupx or set upx=False in your .spec-file

• Repackage you application in verbose/debug mode. For this, pass the option --debug to pyi-makespec or pyinstaller or use EXE(..., debug=1, ...) in your .spec file.

• Using --onedir worked because the file config.yml is inside the directory

• Using the command pyi-archive_viewer PKG-00.pkg can see the file in the archive

• Trying to add --debug all but it failed. Command:
  pyinstaller --debug all --clean --onefile --add-data "config.yml;." mousemover.py

Error message:
--snip earlier lines--
5585 INFO: Including run-time hook 'c:\myprojects\mousemover\venv\lib\site-packages\PyInstaller\hooks\rthooks\pyi_rth_pyqt5.py'
5591 INFO: Looking for dynamic libraries
7124 INFO: Looking for eggs
7124 INFO: Using Python library C:\Users\sharuzzaman\AppData\Local\Programs\Python\Python37\python37.dll
7126 INFO: Found binding redirects:
[]
Traceback (most recent call last):
File "C:\Users\sharuzzaman\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 193, in run_module_as_main
"main", mod_spec)
File "C:\Users\sharuzzaman\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\myprojects\mousemover\venv\Scripts\pyinstaller.exe_main.py", line 9, in
File "c:\myprojects\mousemover\venv\lib\site-packages\PyInstaller_main.py", line 114, in run
run_build(pyi_config, spec_file, **vars(args))
File "c:\myprojects\mousemover\venv\lib\site-packages\PyInstaller_main_.py", line 65, in run_build
PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
File "c:\myprojects\mousemover\venv\lib\site-packages\PyInstaller\building\build_main.py", line 716, in main
build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
File "c:\myprojects\mousemover\venv\lib\site-packages\PyInstaller\building\build_main.py", line 663, in build
exec(code, spec_namespace)
File "C:\myprojects\mousemover\mousemover\mousemover.spec", line 17, in
noarchive=True)
File "c:\myprojects\mousemover\venv\lib\site-packages\PyInstaller\building\build_main.py", line 241, in init
self.postinit()
File "c:\myprojects\mousemover\venv\lib\site-packages\PyInstaller\building\datastruct.py", line 160, in postinit
self.assemble()
File "c:\myprojects\mousemover\venv\lib\site-packages\PyInstaller\building\build_main.py", line 496, in assemble
for name, path, typecode in compile_py_files(new_toc, CONF['workpath']):
File "c:\myprojects\mousemover\venv\lib\site-packages\PyInstaller\utils\misc.py", line 152, in compile_py_files
with open(obj_fnm, 'rb') as fh:
FileNotFoundError: [Errno 2] No such file or directory: '-'

A minimal example program which shows the error

(paste text here)
“Minimal“ means: remove everything from your code which is not relevant for this bug,
esp. don't use external programs, remote requests, etc.
A very good example is https://gist.github.com/ronen/024cdae9ff2d50488438. This one helped
us reproducing and fixing a quite complex problem within approx 1 hour.

Stacktrace / full error message

(venv) C:\myprojects\mousemover\mousemover\dist>mousemover.exe
Traceback (most recent call last):
File "mousemover.py", line 27, in
FileNotFoundError: [Errno 2] No such file or directory: 'config.yml'
[12076] Failed to execute script mousemover

Please also see https://github.com/pyinstaller/pyinstaller/wiki/How-to-Report-Bugs
for more about what would use to solve the issue.

[sharuzzaman]

tree.txt

Tree structure and files

[sharuzzaman]

I created a very basic Python file

test.py:

f = open("demofile.txt", "r")
print(f.read())

demofile.txt:

Hello! Welcome to demofile.txt
This file is for testing purposes.
Good Luck!

running directly:
(venv) C:\myprojects\test>python test.py
Hello! Welcome to demofile.txt
This file is for testing purposes.
Good Luck!

compile successfully with command:
(venv) C:\myprojects\test>pyinstaller --clean --onefile --add-data "demofile.txt;." test.py

test.spec:

-- mode: python ; coding: utf-8 --

block_cipher = None

a = Analysis(['test.py'],
pathex=['C:\myprojects\test'],
binaries=[],
datas=[('demofile.txt', '.')],
hiddenimports=[],
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
[],
name='test',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
upx_exclude=[],
runtime_tmpdir=None,
console=True )

executing exe in dist folder:
(venv) C:\myprojects\test>cd dist

(venv) C:\myprojects\test\dist>test.exe
Traceback (most recent call last):
File "test.py", line 1, in
FileNotFoundError: [Errno 2] No such file or directory: 'demofile.txt'
[11684] Failed to execute script test

(venv) C:\myprojects\test>tree /a /f
Folder PATH listing for volume Windows
Volume serial number is D0AB-B4EC
C:.
|   demofile.txt
|   test.py
|   test.spec
|
+---build
|   \---test
|           Analysis-00.toc
|           base_library.zip
|           EXE-00.toc
|           PKG-00.pkg
|           PKG-00.toc
|           PYZ-00.pyz
|           PYZ-00.toc
|           test.exe.manifest
|           warn-test.txt
|           xref-test.html
|
+---dist
|       test.exe

[Legorooj]

You're using invalid paths. You need to use something like:

import os
import sys
bundle_dir = getattr(sys, '_MEIPASS', path.abspath(os.path.dirname(__file__)))
path_to_yml = os.path.abspath(os.path.join(bundle_dir, 'config.yml'))

[emrecpp]

You're using invalid paths. You need to use something like:

import os
import sys
bundle_dir = getattr(sys, '_MEIPASS', path.abspath(os.path.dirname(__file__)))
path_to_yml = os.path.abspath(os.path.join(bundle_dir, 'config.yml'))

Fix: path.abspath -> os.path.abspath