bug - macdist.py does not handle dylibs with relative paths - was: builds on linux and windows but not mac - cx_freeze 5.0.1 python 3.6.1

[elguavas]

retitled this to indicate the bug discovered as a result of the original post.

edited to add the following note:

the thread below resulted in the discovery of a bug / shortcoming in the setRelativeReferencePaths function of macdist.py.

setRelativeReferencePaths does not handle dylibs with relative paths, which creates a problem not present when cx-freezing the same codebase on linux and windows. this problem occurs if a library with dylibs with relative paths is present in the code to be frozen (noteably in this case: pillow / PIL).

the discussion below leads to danielk605 uncovering the issue causing the problem. in danielk605's case the problem was triggered by matplotlib.

build and build_exe work fine, however, when i run with bdist_mac i get the following:
build
... runs fine
build_exe
... runs fine
then i get output as follow:
create_plist()
setRelativeReferencePaths()
@loader_path/.dylibs/liblcms2.2.dylib

setRelativeReferencePaths()
then a whole series of errors like this (one for each library file):

error: /Library/Developer/CommandLineTools/usr/bin/install_name_tool: input file: build/ImageCat.app/Contents/MacOS/lib/python3.6/collections/init.pyc is not a Mach-O file
error: /Library/Developer/CommandLineTools/usr/bin/install_name_tool: input file: build/ImageCat.app/Contents/MacOS/lib/python3.6/collections/abc.pyc is not a Mach-O file
[...]
then:
error: can't copy '@loader_path/.dylibs/liblcms2.2.dylib': doesn't exist or not a regular file

after all this there is no dist folder created (or nothing in it if already there).

in the build folder there is a bundled .app created, however if i try to run this it just flashes up in the dock for a second and then aborts.

however, if i navigate inside the created app bundle to the MacOS folder and run the executable there it runs as expected!!

so, my questions are:
how can i get the bdist_mac to complete?
why does the bundled app instantly abort while the executable runs properly?
how can i debug what's happening when the bundled app fails to run properly?

tia for any help.

[elguavas]

bump, rude i know, but i'm really at a loss here.

cx_freeze works as expected on linux and windows for this same codebase, but on osx i get the issues i describe briefly above. how can i get some meaningful errors form the non-building app so i can procede?

thanks again.

[anthony-tuininga]

I do not know much about macOS. You may wish to send a message to the cx_Freeze mailing list referencing this issue. I'm not sure everyone has transitioned away from the mailing list yet!

[elguavas]

ok anthony, i'll give it one more try on the mailing list. pretty much despairing the state of cross platform bundling tools for python though, for desktop applications something like cxfreeze really needs to be a more central part of python imho, there are a number of solutions/tools and they all have strengths and weaknesses. fwiw cx-freeze always used to be my favourite though, thanks for all your work on it. ;)

[danielk605]

I've encountered this problem when trying to build an app with a module that references PIL(pillow); I think it was matplotlib. If you don't need PIL, then adding PIL to the excludes option should solve your problem. I believe the error occurs because the directory containing the dylibs for PIL is a hidden directory, but I didn't look further into it.

[elguavas]

thank you sooo much for the reply, i'll start investigating this immediately. ;) oh, and fwiw where i said "andrew suggested" in the op i meant "anthony suggested" of course, apologies.

…

On 23/06/2017 11:16 AM, danielk605 wrote: I've encountered this problem when trying to build an app with a module that references PIL(pillow); I think it was matplotlib. If you don't need PIL, then adding PIL to the excludes option should solve your problem. I believe the error occurs because the directory containing the dylibs for PIL is a hidden directory, but I didn't look further into it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#268 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFhDtmHzgTUk_YZdyNIxW2Hm_L5zTKZlks5sGxH0gaJpZM4NXYwr>.

[elguavas]

...and yes, my program does use pillow (PIL tng) so this seems useful. i'll report back here if i can solve the problem with pillow/PIL. cheers.

…

On 23/06/2017 11:16 AM, danielk605 wrote: I've encountered this problem when trying to build an app with a module that references PIL(pillow); I think it was matplotlib. If you don't need PIL, then adding PIL to the excludes option should solve your problem. I believe the error occurs because the directory containing the dylibs for PIL is a hidden directory, but I didn't look further into it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#268 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFhDtmHzgTUk_YZdyNIxW2Hm_L5zTKZlks5sGxH0gaJpZM4NXYwr>.

[danielk605]

I took a closer look, and from what I can tell, the dylib directory in PIL is being copied. However, it looks as though those dylibs have relative paths (@loader_path/...) in them, and the setRelativeReferencePaths function in macdist.py doesn't handle such paths.
A quick fix that works for me is to add a conditional statement that skips a file if it doesn't exist to the setRelativeReferencePaths function.

[elguavas]

ok. i'll look into this on monay when i'm at the office where this codebase is. thanks. should this be reported as a bug in macdist.py? is your quick fix a suitable patch for this?

…

On 23/06/2017 4:46 PM, danielk605 wrote: I took a closer look, and from what I can tell, the dylib directory in PIL is being copied. However, it looks as though those dylibs have relative paths ***@***.***_path/...) in them, and the setRelativeReferencePaths function in macdist.py doesn't handle such paths. A quick fix that works for me is to add a conditional statement that skips a file if it doesn't exist to the setRelativeReferencePaths function. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#268 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFhDtoK3K9-5MRRy7zJnvp-ZqQkab3KZks5sG19VgaJpZM4NXYwr>.

[danielk605]

Yes, it should probably be reported as a bug. I don't believe my fix is a suitable patch since all it does is to skip a file if the referenced path doesn't exist, which potentially could cause many more errors.