Conflict between module and app name (macos) #7314

kk7ds · 2022-12-09T15:33:05Z

Description of the issue

Our project is trying to move our data files into our module and pyinstaller fails to do handle this on macos because our --name is the same as our one and only module. When I run pyinstaller, I pass --name Foo --add-data=foo/datadir:foo and pyinstaller fails because it has already created Foo in the dist bundle:

Pyinstaller needs to make a directory at '/Users/me/Documents/foo.git/dist/Foo/foo', but there already exists a file at that path!

(If MacOS wasn't case-sensitive, this likely would work because of the capitalization of the app name, but alas)

I've also tried --add-data=foo:foo to add the whole module, but the same problem exists, although the error is slightly different (see below in the stacktrace section).

We are trying to "be good" and use importlib-resources in our app to find the data files by module:

with importlib_resources.as_file(
            importlib_resources.files('foo.datadir')
            .joinpath('datafile')
        ) as d:

I can get data files into the bundle just fine by doing:

--add-data=foo/datadir:datadir

...but importlib won't find them there because of the foo. module scoping.

It seems to me (not knowing much about MacOS app requirements) that these files should be able to go into Resources and importlib should be taught to find them there so that we don't have this conflict. If I mangle the --name I pass to pyinstaller, then it works because there is no conflict, but then my app's "name" in the title bar, app menu, etc is wrong, which is very unfortunate.

If there's something else I can do to work around this that doesn't involve mangling the app name or restructuring out project substantially that would be great. At this point, it seems like I will have to do "importlib.resources or find relative to _ _ file _ _" in our code.

Context information (for bug reports)

Output of pyinstaller --version: 5.7.0 (also tried from develop.zip per template)
Version of Python: 3.10.8 (but also repro'd on 3.8
Platform: MacOS (Monterey and Big Sur)
How you installed Python: Both platform-provided and brew, same difference
Did you also try this on another platform? Does it work there? MacOS-specific

Make sure everything is packaged correctly

[ X ] start with clean installation
[ X ] use the latest development version
[ X ] Run your frozen program from a command window (shell) — instead of double-clicking on it
[ X ] Package your program in --onedir mode
[ N/A ] Package without UPX, say: use the option --noupx or set upx=False in your .spec-file
[ N/A ] Repackage you application in verbose/debug mode. For this, pass the option --debug to pyi-makespec or pyinstaller or use EXE(..., debug=1, ...) in your .spec file.

A minimal example program which shows the error

It would need to be a multi-file tree to reproduce this. I could do it, if necessary but I think it's probably clear. Our directory structure looks like this:

project/
  foo.py
  foo/
    somecode1.py
    somecode2.py
    drivers/
      morecode.py
    datadir/
        datafile

And I'm calling pyinstaller like:

pyinstaller --name Foo --add-data=foo/datadir:foo foo.py

Stacktrace / full error message

If I do --add-data=foo/datadir:foo I get this:

Pyinstaller needs to make a directory at '/Users/me/Documents/foo.git/dist/Foo/foo', but there already exists a file at that path!

If I do --add-data=foo:foo I get this:

Traceback (most recent call last):
  File "/opt/homebrew/bin/pyinstaller", line 8, in <module>
    sys.exit(_console_script_run())
  File "/opt/homebrew/lib/python3.10/site-packages/PyInstaller/__main__.py", line 194, in _console_script_run
    run()
  File "/opt/homebrew/lib/python3.10/site-packages/PyInstaller/__main__.py", line 180, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "/opt/homebrew/lib/python3.10/site-packages/PyInstaller/__main__.py", line 61, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "/opt/homebrew/lib/python3.10/site-packages/PyInstaller/building/build_main.py", line 971, in main
    build(specfile, distpath, workpath, clean_build)
  File "/opt/homebrew/lib/python3.10/site-packages/PyInstaller/building/build_main.py", line 893, in build
    exec(code, spec_namespace)
  File "/Users/me/Documents/foo.git/Foo.spec", line 41, in <module>
    coll = COLLECT(
  File "/opt/homebrew/lib/python3.10/site-packages/PyInstaller/building/api.py", line 892, in __init__
    self.__postinit__()
  File "/opt/homebrew/lib/python3.10/site-packages/PyInstaller/building/datastruct.py", line 173, in __postinit__
    self.assemble()
  File "/opt/homebrew/lib/python3.10/site-packages/PyInstaller/building/api.py", line 926, in assemble
    os.makedirs(todir)
  File "/opt/homebrew/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/os.py", line 215, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/opt/homebrew/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/os.py", line 225, in makedirs
    mkdir(name, mode)
NotADirectoryError: [Errno 20] Not a directory: '/Users/me/Documents/foo.git/dist/Foo/foo/drivers'

The text was updated successfully, but these errors were encountered:

rokm · 2022-12-09T15:52:01Z

macOS is not case-sensitive by default, so you cannot have a foo directory for your top-level package and Foo executable at the same time.

Also, having foo.py and foo package directory in your source tree is likely going to confuse the PyInstaller's modulegraph analysis as well, as foo.py might end up treated as a top-level module and shadow the foopackage...

As far as I'm concerned, this issue is purely self-inflicted; if you want to use PyInstaller, rename one or the other...

It seems to me (not knowing much about MacOS app requirements) that these files should be able to go into Resources and importlib should be taught to find them there so that we don't have this conflict.

True, but our .app bundle code actually uses the standard POSIX application codepath as a middle step (the intermediate COLLECT step), so this problem is unavoidable in the current design. Plus, even in app bundles, we don't "teach importlib to search in Resources", but rather symlink the relocated resources back to the original location, to make things transparent to all different resource loading approaches.

That said, you can probably "rename" your application via app bundle's Info.plist, so there's really no reason for your executable to be called Foo.

kk7ds · 2022-12-09T16:03:14Z

macOS is not case-sensitive by default, so you cannot have a foo directory for your top-level package and Foo executable at the same time.

Yep, as noted, I get this :)

Also, having foo.py and foo package directory in your source tree is likely going to confuse the PyInstaller's modulegraph analysis as well, as foo.py might end up treated as a top-level module and shadow the foopackage...

Well, I lied a little in the washing of the example. Ours is actually not foo.py and foo, but the conflict is between the executable name and the module, regardless of the script name.

As far as I'm concerned, this issue is purely self-inflicted; if you want to use PyInstaller, rename one or the other...

I guess I'm surprised I'm the first person to have a module name that is the same as the app name. So I guess it is self-inflicted but, it seems like a reasonable thing to me :)

That said, you can probably "rename" your application via app bundle's Info.plist, so there's really no reason for your executable to be called Foo.

I tried this and got almost all the way to success. The app menu gets the proper name, but the menu items still say "Quit mangledname" instead of "Quit name". Is there something else in Info.plist I can set/change to fix these?

rokm · 2022-12-09T16:16:32Z

As far as I'm concerned, this issue is purely self-inflicted; if you want to use PyInstaller, rename one or the other...

I guess I'm surprised I'm the first person to have a module name that is the same as the app name. So I guess it is self-inflicted but, it seems like a reasonable thing to me :)

I suppose it becomes a problem only on macOS, because on Windows, the executable has .exe suffix so there's no name overlap, and on linux, you could cheat by capitalizing the executable's name, like you attempted to.

That said, you can probably "rename" your application via app bundle's Info.plist, so there's really no reason for your executable to be called Foo.

I tried this and got almost all the way to success. The app menu gets the proper name, but the menu items still say "Quit mangledname" instead of "Quit name". Is there something else in Info.plist I can set/change to fix these?

What UI framework are you using?

kk7ds · 2022-12-09T16:18:23Z

I suppose it becomes a problem only on macOS, because on Windows, the executable has .exe suffix so there's no name overlap, and on linux, you could cheat by capitalizing the executable's name, like you attempted to.

Yep, exactly. I'm all for cheating :)

What UI framework are you using?

wxPython

I surely appreciate the help, and other than this I love pyinstaller.

rokm · 2022-12-09T16:44:55Z

wxPython

I surely appreciate the help, and other than this I love pyinstaller.

Hmm, for a Qt/PySide2-based application, the following seems to suffice:

rename the bundle (name argument of BUNDLE)
set CFBundleDisplayName and CFBundleName in Info.plist

program_qt.spec

# -*- mode: python ; coding: utf-8 -*-


block_cipher = None


a = Analysis(
    ['program_qt.py'],
    pathex=[],
    binaries=[],
    datas=[],
    hiddenimports=[],
    hookspath=[],
    hooksconfig={},
    runtime_hooks=[],
    excludes=[],
    win_no_prefer_redirects=False,
    win_private_assemblies=False,
    cipher=block_cipher,
    noarchive=False,
)
pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher)

exe = EXE(
    pyz,
    a.scripts,
    [],
    exclude_binaries=True,
    name='program_qt',
    debug=False,
    bootloader_ignore_signals=False,
    strip=False,
    upx=True,
    console=False,
    disable_windowed_traceback=False,
    argv_emulation=False,
    target_arch=None,
    codesign_identity=None,
    entitlements_file=None,
)
coll = COLLECT(
    exe,
    a.binaries,
    a.zipfiles,
    a.datas,
    strip=False,
    upx=True,
    upx_exclude=[],
    name='program_qt',
)
app = BUNDLE(
    coll,
    name='Test App B.app',
    icon=None,
    bundle_identifier="Test App",
    info_plist={
      'CFBundleDisplayName': "Test App B",
      'CFBundleName': "Test App B",
    },
)

program_qt.py

from PySide2 import QtCore, QtWidgets
import sys
import signal

app = QtWidgets.QApplication(sys.argv)
app.setApplicationName("Test App")
signal.signal(signal.SIGINT, signal.SIG_DFL)

window = QtWidgets.QWidget()
window.setWindowTitle("Hello world!")
window.show()

app.exec_()

This gives me dist/program_qt (the intermediate POSIX onedir application) and dist/Test App B.app (the app bundle we want). The executable in app bundle is dist/Test App B.app/Contents/MacOS/program_qt, but the top menu shows "Test App B", and the entries under it are "Hide Test App B" and "Quit Test App B".

I'll check if wxPython is any different...

rokm · 2022-12-09T17:11:02Z

Looks like with wxPython, the executable name is used regardless of Info.plist entries for the top menu, unless application name is set via SetAppName:

program_wx.spec

# -*- mode: python ; coding: utf-8 -*-

a = Analysis(
    ['program_wx.py'],
)
pyz = PYZ(a.pure, a.zipped_data)

exe = EXE(
    pyz,
    a.scripts,
    exclude_binaries=True,
    name='program_wx',
    debug=False,
    bootloader_ignore_signals=False,
    strip=False,
    upx=True,
    console=False,
)
coll = COLLECT(
    exe,
    a.binaries,
    a.zipfiles,
    a.datas,
    strip=False,
    upx=True,
    upx_exclude=[],
    name='program_wx',
)
app = BUNDLE(
    coll,
    name='My App.app',
    icon=None,
    bundle_identifier="My App",
    info_plist={
      'CFBundleDisplayName': "My App",
      'CFBundleName': "My App",
    },
)

program_wx.py

import wx

app = wx.App()
app.SetAppName('My App')

frame = wx.Frame(None, title='Hello World!')
frame.Show()

menubar = wx.MenuBar()
wx.MenuBar.MacSetCommonMenuBar(menubar)

app.MainLoop()

kk7ds · 2022-12-09T17:39:24Z

Looks like with wxPython, the executable name is used regardless of Info.plist entries for the top menu, unless application name is set via SetAppName:

Oooh. I think SetAppName is the sweet sweet goodness I was looking for! That allows me to control the name in the app menu, and I guess I can fix the app menu itself with the CFBundleName stuff, so that works.

I'm not currently signing the app (nor do I really plan to) but is changing the Info.plist stuff after the build violating any "rules" about app creation that might cause trouble? I love how pyinstaller seems to make the app package better than I ever did manually (no surprise) so I don't want to break it by doing this.

I wonder if it would be worth a macos-specific flag to pyinstaller to set the CFBundleName and CFBundleDisplayName to something else? And/or some docs to explain the difference between the executable name and the display?

Anyway, thanks so much, I think I can piece together a working solution from this!

rokm · 2022-12-09T18:12:24Z

I'm not currently signing the app (nor do I really plan to) but is changing the Info.plist stuff after the build violating any "rules" about app creation that might cause trouble? I love how pyinstaller seems to make the app package better than I ever did manually (no surprise) so I don't want to break it by doing this.

If you are not explicitly providing codesign identity, PyInstaller signs (or at least, attempts to sign) the bundle with ad-hoc identity. So if you modify Info.plist post-hoc, you will invalidate that signature (which might or might not be a problem). But you could manually re-sign the bundle using ad-hoc identity again: codesign -s - --force --all-architectures --timestamp --deep /path/to/program.app (which is equivalent to what this call is doing).

But if you override the values in the spec file, like in the examples I gave above (they are collapsed by default due to use of <details> tag), you don't have to edit the Info.plist yourself, and can avoid having to re-sign.

If the question was about the values themselves - I don't know; PyInstaller by default writes executable name in there, but it seems the fields are intended for actual display/application name.

I wonder if it would be worth a macos-specific flag to pyinstaller to set the CFBundleName and CFBundleDisplayName to something else?

You mean in the CLI? I think not; these are advanced options, and if you need to override them, you should be using .spec file with info_plist dict passed to the BUNDLE.

And/or some docs to explain the difference between the executable name and the display?

The contents of Info.plist and its meaning are not specific to PyInstaller, so I think it falls outside the scope of our documentation. Plus, the display name is also tied to the UI framework that's used by the application.

kk7ds · 2022-12-09T18:28:20Z

If you are not explicitly providing codesign identity, PyInstaller signs (or at least, attempts to sign) the bundle with ad-hoc identity. So if you modify Info.plist post-hoc, you will invalidate that signature (which might or might not be a problem). But you could manually re-sign the bundle using ad-hoc identity again: codesign -s - --force --all-architectures --timestamp --deep /path/to/program.app (which is equivalent to what this call is doing).

But if you override the values in the spec file, like in the examples I gave above (they are collapsed by default due to use of <details> tag), you don't have to edit the Info.plist yourself, and can avoid having to re-sign.

Ah, I missed that they were in the spec you provided, that's perfect, thanks.

I wonder if it would be worth a macos-specific flag to pyinstaller to set the CFBundleName and CFBundleDisplayName to something else?

You mean in the CLI? I think not; these are advanced options, and if you need to override them, you should be using .spec file with info_plist dict passed to the BUNDLE.

And/or some docs to explain the difference between the executable name and the display?

The contents of Info.plist and its meaning are not specific to PyInstaller, so I think it falls outside the scope of our documentation. Plus, the display name is also tied to the UI framework that's used by the application.

Yeah, cool that they're set-able in the spec. By docs I just meant to help ward off this question in the future ("What if I get this error because my app and module name are the same, need to change the executable and then override the plist values"). Again, I'm shocked I'm the first to hit this (or the first to report) but maybe a blog post about the solution here would be good enough.

Anyway, thanks!

bwoodsend · 2022-12-10T10:03:03Z

I guess one way I've dodged this problem without realising is that my package is typically named something sluggified like foo_bar but the application is named Foo Bar -- that space instead of an underscore being all that makes the difference here.

jamesbcd · 2022-12-27T13:31:26Z

Not sure if I've understood the issue correctly, but this has worked for me for avoiding the name clash:

exe = EXE(...
          name='appname_', # Extra underscore to avoid name clash. User never sees this binary within the app bundle
          ...
          )

coll = COLLECT(exe,
          ...
          )

app = BUNDLE(coll,
        name='Appname.app',
        ...
        )

kk7ds · 2022-12-27T16:00:30Z

Yep, this works and is what I'm doing now. It wasn't obvious (to me) that I could do that, which is why it seems like it's worth mentioning as a recipe somewhere. But I'm happy at least :)

bwoodsend · 2022-12-27T17:27:40Z

We could probably augment the offending error message.

pyinstaller/PyInstaller/building/api.py

Lines 971 to 975 in d92bf8d

    
           elif not os.path.isdir(dest_dir): 
        
               raise SystemExit( 
        
                   f"Pyinstaller needs to create a directory at {dest_dir!r}, " 
        
                   "but there already exists a file at that path!" 
        
               )

Trying to put stuff this specific into documentation raises the question of where can we put it that's easy to find when you need it and not a distraction if you don't.

kk7ds · 2022-12-27T17:53:42Z

Yeah, understand the concern about where to put it in the docs. FWIW, the error message was pretty clear to me in terms of what was going on and why. And on windows, it's not a problem because the executable has an extension. I'm just surprised I'm the first person to report a conflict with the app and python module being the same - I would expect that to be quite common. So, it would be unfortunate if people are hitting that without a google-able solution.

But, maybe people will find this issue, so perhaps that's good enough :)

bwoodsend · 2023-09-28T17:35:24Z

Fixed by #7713

kk7ds added the triage Please triage and relabel this issue label Dec 9, 2022

bwoodsend removed the triage Please triage and relabel this issue label Jun 18, 2023

bwoodsend mentioned this issue Jun 18, 2023

Let there be symbolic links #7619

Merged

rokm mentioned this issue Sep 28, 2023

Make it possible to put files outside of contents_diretory #7951

Closed

bwoodsend closed this as completed Sep 28, 2023

github-actions bot locked as resolved and limited conversation to collaborators Nov 28, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Conflict between module and app name (macos) #7314

Conflict between module and app name (macos) #7314

kk7ds commented Dec 9, 2022

rokm commented Dec 9, 2022

kk7ds commented Dec 9, 2022

rokm commented Dec 9, 2022

kk7ds commented Dec 9, 2022

rokm commented Dec 9, 2022

rokm commented Dec 9, 2022

kk7ds commented Dec 9, 2022

rokm commented Dec 9, 2022

kk7ds commented Dec 9, 2022

bwoodsend commented Dec 10, 2022

jamesbcd commented Dec 27, 2022

kk7ds commented Dec 27, 2022

bwoodsend commented Dec 27, 2022

kk7ds commented Dec 27, 2022

bwoodsend commented Sep 28, 2023

Conflict between module and app name (macos) #7314

Conflict between module and app name (macos) #7314

Comments

kk7ds commented Dec 9, 2022

Description of the issue

Context information (for bug reports)

Make sure everything is packaged correctly

A minimal example program which shows the error

Stacktrace / full error message

rokm commented Dec 9, 2022

kk7ds commented Dec 9, 2022

rokm commented Dec 9, 2022

kk7ds commented Dec 9, 2022

rokm commented Dec 9, 2022

rokm commented Dec 9, 2022

kk7ds commented Dec 9, 2022

rokm commented Dec 9, 2022

kk7ds commented Dec 9, 2022

bwoodsend commented Dec 10, 2022

jamesbcd commented Dec 27, 2022

kk7ds commented Dec 27, 2022

bwoodsend commented Dec 27, 2022

kk7ds commented Dec 27, 2022

bwoodsend commented Sep 28, 2023