Skip to content

Feature: Splash screen based on Tcl/Tk - #4887

Merged
bwoodsend merged 19 commits into
pyinstaller:developfrom
Chrisg2000:splash-feature-v2
May 21, 2021
Merged

bwoodsend merged 19 commits into
pyinstaller:developfrom
Chrisg2000:splash-feature-v2

Conversation

@Chrisg2000

@Chrisg2000 Chrisg2000 commented May 20, 2020 •

Copy link
Copy Markdown
Contributor

Edit: Note

This pull request does not include a compiled version of the bootloader. If you want to test this PR, you have to recompile the bootloader. Please follow the instructions in the official PyInstaller documentation here. Please make sure that you build for your architecture (64- or 32bit).
To test the recompiled bootloader and this feature, please run the automatic test, instructions can be found here. The test is called test_pyi_splash

If you are running a splashscreen-enable bootloader and have bundled resources for it (i.e. you have used a Splash build target), you should see a message similar to SPLASH: Found splash screen resources. in the console of the bootloader. If you do not see such a message, please confirm that you have compiled the correct bootloader and used the Splash build target accordingly.
Details on how to use this feature can be found in the documentation, which you also need to recompile. To compile the documentation, please follow these instructions.


Original post

So, following on #4581 now a new implementation of the splash screen based on Tcl/Tk (the same library that tkinter uses). The idea of the splash screen came from #4354.

This change implements a splash screen based on Tcl/Tk, which is bundled by PyInstaller at compile time. The splash screen is now (unlike #4581) able to show the progress when unpacking a one-file archive.

The bootloader extracts the necessary files in onefile mode and starts a minimal Tcl/Tk environment in which a script is executed that starts the splash screen. Although this feature is platform independent, I have only tested it on Windows.

For the communication between Python and the splash screen the pyi_splash Python module is (again) bundled as fake modules. This has not changed much compared to #4581.

Of course I will add documentation/changelog in the next days and squash/reword the commits.

@Legorooj Legorooj left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! This is quite a lot of work, so thanks for doing this!

Travis is failing to compile the bootloader, so that'll need fixing. I think it's a linux problem.

Of course I will add documentation/changelog in the next days and squash/reword the commits

Understood. We don't nitpick on those until PRs are marked as ready to review.

Other than that, I just have one thing to ask about...

Comment thread tests/functional/test_interactive.py Resolved
@Legorooj
Legorooj requested a review from htgoebel May 21, 2020 02:37
@Legorooj Legorooj added this to the PyInstaller 4.x milestone May 21, 2020
@Legorooj Legorooj added area:bootloader Caused be or effecting the bootloader area:hooks Caused by or effecting some hook feature Feature request state:needs more work This PR needs more work labels May 21, 2020

@Legorooj Legorooj left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more question...

Comment thread bootloader/src/pyi_splash.c Resolved

@Legorooj Legorooj left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I've got another couple of things I'd like to ask about. The commit messages are, as you said they'd be, nice and neat. I've got a quick-to-fix changelog entry nitpick, which I'll push myself, but otherwise this looks good! I'd like a review from @htgoebel or @bjones1, as my C skills aren't nearly as good as theirs or yours (IE I can understand the C code you've written, but I wouldn't be able to write something that complex myself - I only learned C a few months ago).

Thanks for such a major feature, with clean and easy to understand code!

Comment thread bootloader/src/pyi_global.h Resolved
Comment on lines +27 to +40
try:
# Test if a module named 'pyi_splash' is locally installed.
# This prevents that a potentially required dependency is not
# packed
import pyi_splash # noqa: F401
except ImportError:
module_dir = os.path.join(PACKAGEPATH, 'fake-modules')

api.search_dirs = [module_dir]
logger.info('Adding pyi_splash module to application dependencies.')
else:
logger.info('A local module named "pyi_splash" is installed. '
'Use the installed one instead.')
return

@Legorooj Legorooj May 26, 2020 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we sure we want the user to be able to override this module? We need @htgoebel and @bjones1 to look at this and give their opinion.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My idea behind this check was, that I wanted to prevent issues in the future, where maybe someone installed a package named like pyi_splash (not necessarily a module to control the splash screen, but maybe a name collision with another project), which would be overwritten by our module. If such a module is installed I assumed the user explicitly installed and wanted to use it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could change this to something like PyInstaller.pyi_splash? Update the hooks and rewrite the module a bit so that it just does nothing when unfrozen - so you can call it in unfrozen code. @bjones1 do you have an opinion?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I worked on the pyi_splash module I had the same idea (naming it like PyInstaller.pyi_splash), but I discarded that idea, because I thought that would make the PyInstaller Python API somewhat inconsistent, since the "general" API is for building and that one module is for a user application. But the more I keep thinking of renaming the module the more I like the idea.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the core question is: where, conceptually, does the pyi_splash module belong? From what I understand, it is a module that will only be available in a frozen application and enables the frozen application to control the splash screen. The possibilities:

  1. It is a fake module -- it replaces an existing system modules, such as sites. But this isn't correct, since it's not (or shouldn't be) overriding a system module.
  2. It is a module that should be included as part of the modulegraph search process. However, it can't operate outside the frozen executable, so this isn't it.
  3. It's a part of the loader, used to get the frozen Python environment bootstrapped. While another part of this PR does reside in the bootloader, this module isn't required for bootstrapping.
  4. It's a part of the PyInstaller build process. No, because it runs when the frozen executable runs.
  5. It's a run-time hook. These help set up the frozen environment for specific modules, and belong only in that context. While it's not required for splash screen operation (or is it -- will the splash screen close on its own?), this seems the closest description of what it does.

So, I think the name should be pyi_rth_splash.py or something similar, and it listed as a run-time hook.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about putting the module into the runtime-hook too, but I had some problems implementing it as one:

  1. the runtime hook would need to be executed before the users script and after all other boot script, so it would need to be inserted into Analysis.scripts at a specific position after Analysis has finished. I think that would break very easily.

  2. To still provide the module with its current name the script would need to modify sys.modules and do some weird loading. I think that would break very easily too.

I somewhat explained my choice here in the previous PR.
Maybe you have a better way to implement this on the python side, but I didn't find any maintainable/satisfying way.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy to leave it as a non-runtime hook if that fits more nicely. Ispyi_splash a module that the user imports to control the splash screen, which can only be imported/used in a frozen executable, and should therefore be included in the frozen executable?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pyi_splash module is to control the splash screen from within a frozen application. If someone tries to import it from a non frozen python interpreter the module cannot be found. So it only serves a purpose if the application is in a frozen state. If if it is not included in the frozen application and the bootloader starts a splash screen, there will be no way to close it.

@Chrisg2000
Chrisg2000 marked this pull request as ready for review May 26, 2020 12:57
@Legorooj

Copy link
Copy Markdown
Member

@Chrisg2000 I've come across a slight problem testing it out locally:

6221 WARNING: The local Tcl/Tk installation is missing the file tk\images\pwrdLogo175.gif. The behavior of the splash screen is therefore undefined and may be unsupported.
6228 WARNING: The local Tcl/Tk installation is missing the file tk\images\pwrdLogo.eps. The behavior of the splash screen is therefore undefined and may be unsupported.
6230 WARNING: The local Tcl/Tk installation is missing the file tk\ttk\clamTheme.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
6231 WARNING: The local Tcl/Tk installation is missing the file tk\images\pwrdLogo200.gif. The behavior of the splash screen is therefore undefined and may be unsupported.
6231 WARNING: The local Tcl/Tk installation is missing the file tk\ttk\classicTheme.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
6235 WARNING: The local Tcl/Tk installation is missing the file tk\images\pwrdLogo100.gif. The behavior of the splash screen is therefore undefined and may be unsupported.
6243 WARNING: The local Tcl/Tk installation is missing the file tk\ttk\altTheme.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
6244 WARNING: The local Tcl/Tk installation is missing the file tk\images\pwrdLogo150.gif. The behavior of the splash screen is therefore undefined and may be unsupported.
6245 WARNING: The local Tcl/Tk installation is missing the file tk\ttk\xpTheme.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
6252 WARNING: The local Tcl/Tk installation is missing the file tk\images\logoMed.gif. The behavior of the splash screen is therefore undefined and may be unsupported.
6254 WARNING: The local Tcl/Tk installation is missing the file tk\ttk\aquaTheme.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
6257 WARNING: The local Tcl/Tk installation is missing the file tk\ttk\winTheme.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
6261 WARNING: The local Tcl/Tk installation is missing the file tk\images\logoLarge.gif. The behavior of the splash screen is therefore undefined and may be unsupported.
6265 WARNING: The local Tcl/Tk installation is missing the file tk\images\README. The behavior of the splash screen is therefore undefined and may be unsupported.
6267 WARNING: The local Tcl/Tk installation is missing the file tk\optMenu.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
6272 WARNING: The local Tcl/Tk installation is missing the file tk\ttk\vistaTheme.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
6275 WARNING: The local Tcl/Tk installation is missing the file tk\pkgIndex.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
6277 WARNING: The local Tcl/Tk installation is missing the file tk\images\pwrdLogo75.gif. The behavior of the splash screen is therefore undefined and may be unsupported.
6281 WARNING: The local Tcl/Tk installation is missing the file tk\tclIndex. The behavior of the splash screen is therefore undefined and may be unsupported.

As the warnings suggest, the splash screen doesn't work.

However, I inserted the following diff:

@@ -215,11 +215,13 @@ class Splash(Target):
             self.binaries.extend(toc for toc in tk_tree
                                  if toc[0] in self.splash_requirements)
 
         # Check if all requirements were found
         def _filter(_item):
-            if _item not in (datas + self.binaries).filenames:
+            fnames = (datas + self.binaries).filenames
+            if _item not in fnames:
+                print(_item.lower() in (x.lower() for x in fnames))
                 # Item is not bundled, so warn the user about it.
                 # This actually may happen on some tkinter installations
                 # on which there is no license.terms file
                 logger.warning("The local Tcl/Tk installation is missing"
                                " the file %s. The behavior of the splash"

Which gave me the following output:

True
6955 WARNING: The local Tcl/Tk installation is missing the file tk\images\pwrdLogo175.gif. The behavior of the splash screen is therefore undefined and may be unsupported.
True
6959 WARNING: The local Tcl/Tk installation is missing the file tk\ttk\vistaTheme.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
True
6962 WARNING: The local Tcl/Tk installation is missing the file tk\optMenu.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
True
6966 WARNING: The local Tcl/Tk installation is missing the file tk\images\pwrdLogo.eps. The behavior of the splash screen is therefore undefined and may be unsupported.
True
6972 WARNING: The local Tcl/Tk installation is missing the file tk\pkgIndex.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
True
6976 WARNING: The local Tcl/Tk installation is missing the file tk\tclIndex. The behavior of the splash screen is therefore undefined and may be unsupported.
True
6984 WARNING: The local Tcl/Tk installation is missing the file tk\images\pwrdLogo150.gif. The behavior of the splash screen is therefore undefined and may be unsupported.
True
6987 WARNING: The local Tcl/Tk installation is missing the file tk\images\pwrdLogo100.gif. The behavior of the splash screen is therefore undefined and may be unsupported.
True
6992 WARNING: The local Tcl/Tk installation is missing the file tk\ttk\altTheme.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
True
6994 WARNING: The local Tcl/Tk installation is missing the file tk\images\logoMed.gif. The behavior of the splash screen is therefore undefined and may be unsupported.
True
7003 WARNING: The local Tcl/Tk installation is missing the file tk\ttk\clamTheme.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
True
7008 WARNING: The local Tcl/Tk installation is missing the file tk\ttk\xpTheme.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
True
7016 WARNING: The local Tcl/Tk installation is missing the file tk\images\pwrdLogo200.gif. The behavior of the splash screen is therefore undefined and may be unsupported.
True
7020 WARNING: The local Tcl/Tk installation is missing the file tk\images\README. The behavior of the splash screen is therefore undefined and may be unsupported.
True
7025 WARNING: The local Tcl/Tk installation is missing the file tk\images\pwrdLogo75.gif. The behavior of the splash screen is therefore undefined and may be unsupported.
True
7029 WARNING: The local Tcl/Tk installation is missing the file tk\images\logoLarge.gif. The behavior of the splash screen is therefore undefined and may be unsupported.
True
7037 WARNING: The local Tcl/Tk installation is missing the file tk\ttk\winTheme.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
True
7045 WARNING: The local Tcl/Tk installation is missing the file tk\ttk\aquaTheme.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.
True
7049 WARNING: The local Tcl/Tk installation is missing the file tk\ttk\classicTheme.tcl. The behavior of the splash screen is therefore undefined and may be unsupported.

I then changed the splash source again:

@@ -215,11 +215,12 @@ class Splash(Target):
             self.binaries.extend(toc for toc in tk_tree
                                  if toc[0] in self.splash_requirements)
 
         # Check if all requirements were found
         def _filter(_item):
-            if _item not in (datas + self.binaries).filenames:
+            fnames = (datas + self.binaries).filenames
+            if _item.lower() not in (x.lower() for x in fnames) and _item not in fnames:
                 # Item is not bundled, so warn the user about it.
                 # This actually may happen on some tkinter installations
                 # on which there is no license.terms file
                 logger.warning("The local Tcl/Tk installation is missing"
                                " the file %s. The behavior of the splash"

Hey presto, warnings gone.

But the splash still doesn't work:

[6060] PyInstaller Bootloader 3.x
[6060] LOADER: executable is D:\PyInstaller\dist\splash-test.exe
[6060] LOADER: homepath is D:\PyInstaller\dist
[6060] LOADER: _MEIPASS2 is NULL
[6060] LOADER: archivename is D:\PyInstaller\dist\splash-test.exe
[6060] LOADER: Extracting binaries
[6060] LOADER: Executing self as child
[6060] LOADER: set _MEIPASS2 to C:\Users\legor\AppData\Local\Temp\_MEI60602
[6060] LOADER: Setting up to run child
[6060] LOADER: Creating child process
[6060] LOADER: Waiting for child process to finish...
[8580] PyInstaller Bootloader 3.x
[8580] LOADER: executable is D:\PyInstaller\dist\splash-test.exe
[8580] LOADER: homepath is D:\PyInstaller\dist
[8580] LOADER: _MEIPASS2 is C:\Users\legor\AppData\Local\Temp\_MEI60602
[8580] LOADER: archivename is D:\PyInstaller\dist\splash-test.exe
[8580] LOADER: SetDllDirectory(C:\Users\legor\AppData\Local\Temp\_MEI60602)
[8580] LOADER: Already in the child - running user's code.
[8580] LOADER: manifestpath: C:\Users\legor\AppData\Local\Temp\_MEI60602\splash-test.exe.manifest
[8580] LOADER: Activation context created
[8580] LOADER: Activation context activated
[8580] LOADER: Python library: C:\Users\legor\AppData\Local\Temp\_MEI60602\python37.dll
[8580] LOADER: Loaded functions from Python library.
[8580] LOADER: Manipulating environment (sys.path, sys.prefix)
[8580] LOADER: sys.prefix is C:\Users\legor\AppData\Local\Temp\_MEI60602
[8580] LOADER: Pre-init sys.path is C:\Users\legor\AppData\Local\Temp\_MEI60602\base_library.zip;C:\Users\legor\AppData\Local\Temp\_MEI60602
[8580] LOADER: Setting runtime options
[8580] LOADER: Bootloader option: pyi-windows-manifest-filename splash-test.exe.manifest
[8580] LOADER: Initializing python
[8580] LOADER: Overriding Python's sys.path
[8580] LOADER: Post-init sys.path is C:\Users\legor\AppData\Local\Temp\_MEI60602\base_library.zip;C:\Users\legor\AppData\Local\Temp\_MEI60602
[8580] LOADER: Setting sys.argv
[8580] LOADER: setting sys._MEIPASS
[8580] LOADER: importing modules from CArchive
[8580] LOADER: extracted struct
[8580] LOADER: callfunction returned...
[8580] LOADER: extracted pyimod01_os_path
[8580] LOADER: callfunction returned...
[8580] LOADER: extracted pyimod02_archive
[8580] LOADER: callfunction returned...
[8580] LOADER: extracted pyimod03_importers
[8580] LOADER: callfunction returned...
[8580] LOADER: Installing PYZ archive with Python modules.
[8580] LOADER: PYZ archive: PYZ-00.pyz
[8580] LOADER: Running pyiboot01_bootstrap.py
[8580] LOADER: Running test.py
The environment does not allow connecting to the splash screen. Are the splash resources attached to the bootloader or did an error occur?
Traceback (most recent call last):
  File "d:\pyinstaller\PyInstaller\fake-modules\pyi_splash.py", line 100, in <module>
    _ipc_port = int(os.environ['_PYIBoot_SPLASH'])
  File "c:\users\legor\appdata\local\programs\python\python37-32\lib\os.py", line 678, in __getitem__
    raise KeyError(key) from None
KeyError: '_PYIBoot_SPLASH'
Traceback (most recent call last):
  File "test.py", line 5, in <module>
    time.sleep(0.01)
KeyboardInterrupt
[8580] Failed to execute script test
[8580] LOADER: OK.
[8580] LOADER: Cleaning up Python interpreter.
[6060] LOADER: Back to parent (RC: -1)
[6060] LOADER: Doing cleanup
[6060] LOADER: Freeing archive status for D:\PyInstaller\dist\splash-test.exe

Any ideas? I've tried everything I can think of. Oh and the splash section of the spec is:

splash = Splash('tests\\functional\\data\\splash\\image.png',
                binaries=a.binaries,
                datas=a.datas,
                text_pos=(100,100),
                text_size=12,
                minify_script=False,
                full_tk=True,
                text_default='Splash test')

I tweaked it a bit of course.

@Legorooj
Legorooj marked this pull request as draft May 27, 2020 02:40
@Chrisg2000

Chrisg2000 commented May 27, 2020 •

Copy link
Copy Markdown
Contributor Author

@Legorooj Good catch there! I found the reason why those files were excluded: The TOC class only puts os.path.normcase'ed filenames into the TOC.filenames set:

def unique_name(entry):
"""
Return the filename used to enforce uniqueness for the given TOC entry
Parameters
----------
entry : tuple
Returns
-------
unique_name: str
"""
name, path, typecode = entry
if typecode in ('BINARY', 'DATA'):
name = os.path.normcase(name)
return name

So my solution is then to not use the TOC.filenames field, but instead create a filenames list from the TOC entries:

@@ -216,8 +216,10 @@ class Splash(Target):
                                  if toc[0] in self.splash_requirements)

         # Check if all requirements were found
+        fnames = [toc[0] for toc in (datas + self.binaries)]
+
         def _filter(_item):
-            if _item not in (datas + self.binaries).filenames:
+            if _item not in fnames:
                 # Item is not bundled, so warn the user about it.
                 # This actually may happen on some tkinter installations
                 # on which there is no license.terms file

This resolves the problem of the excluded files, but I am not sure what causes the bootloader to not find the resources which are appended, since you should at least see a message with the prefix SPLASH in the bootloaders log. Did you compile the new bootloader? Is the automatic test test_interactive.py::test_pyi_splash passing or failing?

@Legorooj

Copy link
Copy Markdown
Member

The tests fail. I've also rebuilt the bootloaders - with MSVC 2019. The pytest output of the tests is here. I'm attempting to debug now...

@Chrisg2000

Copy link
Copy Markdown
Contributor Author

I can't help myself, but I still have the feeling that it has to do with the compilation of the bootloader, since the pytest log should contain messages with the prefix SPLASH or TCL, even if the setup fails1.
I've noticed that your PyInstaller installation chooses the 32bit bootloader, did you compile that version too? On my systems waf chooses automatically the 64bit compiler, leaving the 32bit files untouched.

An idea to debug the problem would be to modify the pyi_splash_find function in pyi_splash.c to print every item it searched and work from thereon forward.

1At least the message SPLASH: Found splash screen resources. should appear.

@Legorooj Legorooj left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You were right. I hadn't built the bootloaders properly - I'd only built the 64bit ones for some reason. It all works - well works ish. There's 2 bugs. I've left a comment for one below - and a fix. It's quite trivial.

The other is I tried using the splash with a very high definition photo - 3k. The result was that the splash worked fine - it just splashed across the entire screen, covering everything, and I could only see a quarter of the image cause most if was off the screen. This needs fixing with a maximum size for the image.

I also got the opposite result when using the test image; it was tiny and I could barely see it.

Comment thread PyInstaller/building/templates.py Resolved
@Chrisg2000

Copy link
Copy Markdown
Contributor Author

As a quick (personal) note: I will be off the grid for the next two weeks and won't work/comment on this PR.

@Chrisg2000

Copy link
Copy Markdown
Contributor Author

The thing about the huge/small images: I had the concern, that such images would be used, but there is no real way to deal with that, except scaling the image. In a test version I implemented a bilinear image scaling function in the bootloader, which scaled the image to a specific portion of the screens width at runtime (e.g. scaling the image up to be 40% of the screens width). My implementation unfortunately produced some image artifact, so I discarded the idea.

Another suggestion I could think about is to add an option to the Splash build target, which scales the image (if PIL/Pillow is installed) to the specified size at build time. Would that be a solution?

@Legorooj

Legorooj commented Jun 2, 2020 •

Copy link
Copy Markdown
Member

@Chrisg2000 maybe. I'll look into it more. And take whatever time you need.

@Chrisg2000

Chrisg2000 commented Jun 21, 2020 •

Copy link
Copy Markdown
Contributor Author

So, back online I changed some minor things:

  • Update tk file filter
  • Improve pyi_splash test to actually test if the splash screen was started
  • Change makespec template to repr
  • Bootloader: Check buffers before copying data in pyi_splash_setup
  • Reformat bootloaders code according to uncrustify
  • Add max_img_size option to Splash build target
  • Update some minor outdated documentation

The max_img_size option now resizes the image according to its aspect ratio if PIL/Pillow is installed, otherwise a warning is printed. Although huge images will be resized, small images are not. I think the user has to choose an image which has a suitable size.

Comment thread PyInstaller/building/splash.py Resolved
@Legorooj

Copy link
Copy Markdown
Member

Thanks for the changes. I'll check them out in the next day or 2. Maybe we can get this merged in for the 4.0 release; you never know :-).

@Chrisg2000
Chrisg2000 marked this pull request as ready for review July 6, 2020 12:27

@Legorooj Legorooj left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't see anything wrong with this and it's working locally! Really awesome work on this!

Just one nitpick: Please add:

A huge thanks to `@Chrisg2000 <https://github.com/Chrisg2000>`_for programming this feature.

Into the changelog entries.

Other than that, @htgoebel and @bjones1 please review this amazing peice of work.

@AokiAhishatsu

Copy link
Copy Markdown
  File "C:\Code\LeM_Test\SplashTest\splash-test.spec", line 20
    splash = Splash(''test-img.png'',
                      ^
SyntaxError: invalid syntax

What am I doing wrong?

@Czarified

Copy link
Copy Markdown
    splash = Splash(''test-img.png'',
                      ^
SyntaxError: invalid syntax

What am I doing wrong?

Looks like you have 2 single quotes. Python takes a single quote as the start of a string. So you're giving it an empty string '', then calling a variable with improper syntax.

Try splash = Splash('test-img.png') instead, or splash = Splash("test-img.png") instead.

@AokiAhishatsu

Copy link
Copy Markdown

Sorry, I think I was a bit unclear/out of context.
I use: pyinstaller.exe --splash test-img.png --onefile --name splash-test main.py
And the spec file gets generated with 2 single quotes.

rokm added 2 commits May 21, 2021 13:14
If Pillow is missing and image needs to be resized, raise a
ValueError instead of None, to avoid a more confusing error
further down the road.

Similarly, if lack of Pillow precludes image conversion, mention
that in the error message to make it clear what needs to be done
to resolve the error.
It turns out that Tcl/Tk on macOS (or rather, the underlying UI
toolkit) disallows UI operations from any thread other than
the main one. This makes the current splash screen implementation,
which operates in a secondary thread, inherently incompatible with
macOS.

The frozen application with splash screen enabled crashes when
calling Tk_Init() in the splash screen thread:
```
2021-05-13 19:37:02.116 program[3394:49760] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'NSWindow drag regions should only be invalidated on the Main Thread!'
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff205f45bf __exceptionPreprocess + 242
    1   libobjc.A.dylib                     0x00007fff2032cbbb objc_exception_throw + 48
    2   CoreFoundation                      0x00007fff2061cdae -[NSException raise] + 9
    3   AppKit                              0x00007fff22cc9872 -[NSWindow(NSWindow_Theme) _postWindowNeedsToResetDragMarginsUnlessPostingDisabled] + 321
    4   AppKit                              0x00007fff22cb444a -[NSWindow _initContent:styleMask:backing:defer:contentView:] + 1296
    5   AppKit                              0x00007fff22cb3f33 -[NSWindow initWithContentRect:styleMask:backing:defer:] + 42
    6   libtk8.6.dylib                      0x0000000107ff3e61 TkMacOSXMakeRealWindowExist + 757
    7   libtk8.6.dylib                      0x0000000107ff3a3e TkWmMapWindow + 63
    8   libtk8.6.dylib                      0x0000000107f587ec Tk_MapWindow + 158
    9   libtk8.6.dylib                      0x0000000107f611f7 MapFrame + 59
    10  libtcl8.6.dylib                     0x0000000107e5a6d8 TclServiceIdle + 75
    11  libtcl8.6.dylib                     0x0000000107e3efd0 Tcl_DoOneEvent + 340
    12  libtk8.6.dylib                      0x0000000107fe6944 TkpInit + 766
    13  libtk8.6.dylib                      0x0000000107f59a37 Initialize + 2411
    14  program                             0x0000000106f87f31 program + 20273
    15  libsystem_pthread.dylib             0x00007fff20481954 _pthread_start + 224
    16  libsystem_pthread.dylib             0x00007fff2047d4a7 thread_start + 15
)
libc++abi: terminating with uncaught exception of type NSException
Abort trap: 6
```

Therefore, explicitly disable building with splash screen on macOS
by raising SystemExit in Splash constructor, following the same
behavior as if tkinter is missing. Also add/adjust notes about
lack of macOS support throughout the docs.
@rokm

rokm commented May 21, 2021

Copy link
Copy Markdown
Member

Alright, this PR is now rebased back on top of develop.

I've also fixed the race condition at exit that I mentioned before, and improved the error messages when Pillow is missing.

Unfortunately, the current design that has splash screen running in a secondary thread does not work on macOS at all, because the UI framework there disallows UI-related operations from any thread but the main one. So the frozen application crashes when bootloader tries to initialize Tk. Therefore, trying to enable splash screen in macOS will now raise SystemExit error, and the lack of macOS support is documented in the affected parts of documentation.

I suppose in follow-up work, we could try pushing archive unpacking into separate thread, and have splash screen displayed in the main thread, then tear it down once we're ready to start the embedded python interpreter. This will limit the functionality compared to the current one (right now, it is possible to interact with splash screen from the frozen python code), but should be compatible across all OSes.

But I'd rather see this done afterwards, and have this merged now as-is to prevent it from falling out of sync with develop again.

@bwoodsend

Copy link
Copy Markdown
Member

Alright! Here goes...

@bwoodsend
bwoodsend merged commit 4756847 into pyinstaller:develop May 21, 2021
@llamafilm

Copy link
Copy Markdown

I'm confused by this message about macOS. I've been using the splash screen for months now on Mac with no problem. Did this break recently?

@rokm

rokm commented May 21, 2021

Copy link
Copy Markdown
Member

What macOS version and what python version/distribution were/are you using?

@llamafilm

Copy link
Copy Markdown

Right now I'm on Catalina 10.15.7 with Python 3.7.9. I believe this was working on older versions too. I've been installing from this git pull request, but haven't updated in a few weeks.

@rokm

rokm commented May 21, 2021

Copy link
Copy Markdown
Member

Is that python.org python, or homebrew?

@llamafilm

Copy link
Copy Markdown

Sorry I don't remember where I installed from. Probably python.org judging by the path:

>>> sys.executable
'/Library/Frameworks/Python.framework/Versions/3.7/bin/python3'
>>> sys.implementation
namespace(_multiarch='darwin', cache_tag='cpython-37', hexversion=50792944, name='cpython', version=sys.version_info(major=3, minor=7, micro=9, releaselevel='final', serial=0))
>>> sys._git
('CPython', 'v3.7.9', '13c94747c7')

@bwoodsend

Copy link
Copy Markdown
Member

Can you check brew leaves or brew list to see if Python is in there?

@rokm

rokm commented May 21, 2021 •

Copy link
Copy Markdown
Member

Can you try installing the develop branch (with reverted last commit) and see if it works for you, then?

And for the reference, here the backup of this branch before I pushed the rebased version. But I get the same behavior with both the original and the rebased version (although it could be a problem with my system...).

@llamafilm

Copy link
Copy Markdown

I do have python@3.8 and python@3.9 installed with homebrew but they are in the Cellar. In this example I used a venv pointing to Python3.7 which is not homebrew (I can tell from the sys.executable location).

@llamafilm

Copy link
Copy Markdown

Okay I tried the penultimate commit of the develop branch and it's not working. Looks like the same kind of errors as mentioned in that last commit. I also get this error:

291 WARNING: The local Tcl/Tk installation is missing the file tk/license.terms. The behavior of the splash screen is therefore undefined and may be unsupported.

2021-05-21 14:50:07.404 test[9615:269571] WARNING: NSWindow drag regions should only be invalidated on the Main Thread! This will throw an exception in the future. Called from (
	0   AppKit                              0x00007fff3181e629 -[NSWindow(NSWindow_Theme) _postWindowNeedsToResetDragMarginsUnlessPostingDisabled] + 371
	1   AppKit                              0x00007fff31806052 -[NSWindow _initContent:styleMask:backing:defer:contentView:] + 1416
	2   AppKit                              0x00007fff31805ac3 -[NSWindow initWithContentRect:styleMask:backing:defer:] + 42
	3   libtk8.6.dylib                      0x000000010bbcba55 TkMacOSXMakeRealWindowExist + 729
	4   libtk8.6.dylib                      0x000000010bbcc979 Tk_WmObjCmd + 1066
	5   libtcl8.6.dylib                     0x000000010b9646dc TclNRRunCallbacks + 80
	6   libtcl8.6.dylib                     0x000000010b96593a TclEvalEx + 1953
	7   libtcl8.6.dylib                     0x000000010b965193 Tcl_EvalEx + 26
	8   test                                0x000000010aa07f74 test + 24436
	9   libsystem_pthread.dylib             0x00007fff6e852109 _pthread_start + 148
	10  libsystem_pthread.dylib             0x00007fff6e84db8b thread_start + 15
)

I tried going back a few commits but I cannot get it to work now from the develop branch. This was definitely working in the past, and it still does work in a different venv. I'm trying to figure out what's different.

@rokm

rokm commented May 21, 2021

Copy link
Copy Markdown
Member

Can you get it running with the original version? As I mentioned, I made a backup: https://github.com/rokm/pyinstaller/tree/splash-feature-v2-backup

But searching around, I found mentions of this warning: WARNING: NSWindow drag regions should only be invalidated on the Main Thread! This will throw an exception in the future. Did you notice anything like that when the splash screen was working on the macOS? Because I suspect that what we're witnessing now is Apple making good on that threat friendly reminder.

@rokm

rokm commented May 21, 2021

Copy link
Copy Markdown
Member

Ah, I see your version of "NSWindow drag regions should only be invalidated on the Main Thread!" is technically still a warning, whereas in my case, it is uncaught exception.

@rokm

rokm commented May 21, 2021

Copy link
Copy Markdown
Member

Also, if you have a venv where things still work, could you make a simple hello world example so I can try running it on my test system?

@llamafilm

Copy link
Copy Markdown

Shoot, I accidentally deleted that venv so now I'm stuck. In fact now that I think about it, I may have been using the system installation of PyInstaller before, not the venv. I've tried installing from your backup branch, and from this PR, and from ChrisG2000's repo, but I can't get anything to work. I also tried upgrading to 3.9.0 from python.org and that made no difference. Using your backup branch I get this error:

bash-3.2$ pyinstaller test.py --splash splash.png 
37 INFO: PyInstaller: 4.1.dev0
37 INFO: Python: 3.9.0
43 INFO: Platform: macOS-10.15.7-x86_64-i386-64bit
45 INFO: wrote /Users/elliott/splashtest/test.spec
48 INFO: UPX is available.
49 INFO: Extending PYTHONPATH with paths
['/Users/elliott/splashtest', '/Users/elliott/splashtest']
52 INFO: checking Analysis
52 INFO: Building Analysis because Analysis-00.toc is non existent
52 INFO: Initializing module dependency graph...
53 INFO: Caching module graph hooks...
58 INFO: Analyzing base_library.zip ...
2083 INFO: Processing pre-find module path hook distutils from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/hooks/pre_find_module_path/hook-distutils.py'.
2084 INFO: distutils: retargeting to non-venv dir '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9'
3669 INFO: Caching module dependency graph...
3751 INFO: running Analysis Analysis-00.toc
3755 INFO: Analyzing /Users/elliott/splashtest/test.py
3892 INFO: Processing pre-safe import module hook six.moves from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-six.moves.py'.
4139 INFO: Processing pre-safe import module hook urllib3.packages.six.moves from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-urllib3.packages.six.moves.py'.
4952 INFO: Processing module hooks...
4953 INFO: Loading module hook 'hook-boto3.py' from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
5296 INFO: Loading module hook 'hook-botocore.py' from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
5723 INFO: Loading module hook 'hook-pkg_resources.py' from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/hooks'...
6038 INFO: Processing pre-safe import module hook win32com from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/_pyinstaller_hooks_contrib/hooks/pre_safe_import_module/hook-win32com.py'.
6039 WARNING: Hidden import "pkg_resources.py2_warn" not found!
6040 WARNING: Hidden import "pkg_resources.markers" not found!
6040 INFO: Excluding import '__main__'
6041 INFO:   Removing import of __main__ from module pkg_resources
6041 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/hooks'...
6041 INFO: Loading module hook 'hook-lib2to3.py' from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/hooks'...
6065 INFO: Loading module hook 'hook-_tkinter.py' from '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/hooks'...
6067 ERROR: Tcl/Tk improperly installed on this system.
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/bin/pyinstaller", line 8, in <module>
    sys.exit(run())
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/__main__.py", line 114, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/__main__.py", line 65, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/building/build_main.py", line 722, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/building/build_main.py", line 669, in build
    exec(code, spec_namespace)
  File "/Users/elliott/splashtest/test.spec", line 6, in <module>
    a = Analysis(['test.py'],
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/building/build_main.py", line 243, in __init__
    self.__postinit__()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/building/datastruct.py", line 160, in __postinit__
    self.assemble()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/building/build_main.py", line 420, in assemble
    self.graph.process_post_graph_hooks()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/depend/analysis.py", line 365, in process_post_graph_hooks
    module_hook.post_graph()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/depend/imphook.py", line 451, in post_graph
    self._process_hook_func()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/depend/imphook.py", line 471, in _process_hook_func
    self._hook_module.hook(hook_api)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/PyInstaller/hooks/hook-_tkinter.py", line 31, in hook
    tcl_tree, tk_tree = collect_tcl_tk_files(hook_api.__file__)
ValueError: not enough values to unpack (expected 2, got 0)

I spent about 2 hours debugging this, without success.

@rokm

rokm commented May 22, 2021

Copy link
Copy Markdown
Member

Just to clarify, chrisg2000 repo branch that was used to open this PR is the same as develop now, because I had to force-push the rebased version into it to have the PR updated. So that and develop are bound to behave the same.

My backup branch is the same as what this PR was before the rebase was pushed. And the above error is quite expected, because at the time when the PR was open, the state of Tcl/Tk support on macOS in PyInstaller was rather messy. So if you got it working, it must have been a very specific combination that managed to get you past the errors in Tcl/Tk collection itself. (But I think that's a separate problem from main-thread enforcement from UI toolkit, which probably also came in with one of system updates).

@rokm

rokm commented May 22, 2021

Copy link
Copy Markdown
Member

I did some more testing, and it looks like that UI-in-main-thread enforcement came as part of graphics system overhaul in macOS 10.14 Mojave.

But whether it is just a warning or an actual exception depends on the version of macOS SDK that the program (in our case, PyInstaller's bootloader) is built against (or, rather, the version that the program claims to be built against). If the declared macOS SDK version is old enough (e.g., 10.9), the offending program gets just a warning. If it is 10.14 or newer, it gets an exception.

We recently introduced the reported SDK version adjustment to develop (e5d6932). Which means that with develop branch, freezing with python.org Intel installer (3.7, 3.8, 3.9) and splash screen enabled produces an executable that generates "just" a warning (because it gets its reported SDK version adjusted to 10.9, which is what those pythons are built against). The splash screen itself does not seem to work, though, and empty Tk window (or rather, just title bar) is displayed. I suspect this might be related to the Tcl/Tk 8.6.8 that's shipped with these Python.org builds.

Python.org 3.9 universal2/macos11 variant comes with Tcl/Tk 8.6.10 instead, but is also built against SDK 11.0, so with bootloaders rebuilt with XCode tools 12.x, the frozen binary reports 11.0 SDK, and terminates with uncaught exception. But, if I manually adjust the reported SDK version to 10.9, it produces just a warning and displays the splash screen. Dropping reported SDK version below 11.x opts us out of Big Sur specific features, though (like dynamic library cache searching for discovery of system libraries, version being reported as 11.x instead of 10.16).

The python-tk in homebrew comes with Tcl/Tk 8.6.11. But in homebrew, the native SDK version is used for all its programs (10.14 on mojave, 10.15 on catalina, 11.0 on bigsur), so we need to manually adjust the SDK version on frozen application again, and splash screen works there as well.

So my guess would be that if you managed to get the original branch working, you were working with recent Tcl/Tk (which rules out python.org builds with their 8.6.8). And when you were rebuilding bootloaders, you were using toolchain (XCode version) with SDK version lower than 10.14. (Or alternatively, the UI-in-main-thread enforcement came with one of system updates, and you were testing before that happened).

Considering all the above, I think I stand by my decision to disable splash screen on macOS for now. At least until it is reworked to operate in the main thread... and even then we will probably keep running afoul of that Tcl/Tk 8.6.8 that comes with python.org builds.

@eric15342335

eric15342335 commented Jun 4, 2021 •

Copy link
Copy Markdown
Contributor

my pyi_splash.update_text() doesn't work

the splash part of my .spec file:

splash = Splash('splash.png',
                binaries=a.binaries,
                datas=a.datas,
                text_pos=None,
                text_size=12,
                minify_script=True)

compared with the test .spec file (test_pyi_splash.py works)
tests/functional/specs/spec_with_splash.spec
i don't have these two lines:
text_pos=(10, 50),
text_color='red'

i guess pyinstaller did not add those lines to my spec file

@rokm

rokm commented Jun 4, 2021

Copy link
Copy Markdown
Member

my pyi_splash.update_text() doesn't work

From the docs: By default, the option to display the optional text is disabled (text_pos=None).

@B16f00t

B16f00t commented Sep 1, 2021 •

Copy link
Copy Markdown

Is it possible that the splash window on loading is not on the top the system window?
I mean sometimes the program takes a long time to load and the windows splash stays on for a long time so that the user can't do anything else on his computer without the windows splash being over it. E.g. open windows explorer or use a browser and the windows splash is not on top of it.

@xo-HADES-xo

Copy link
Copy Markdown

Splash screen is great for slow loading apps, however, static image is misleading as it does not convey progress. Allowing a GIF or APNG support for splash screen images would be better. Tkinter seems to support GIF by cycling through frames, e.g. https://stackoverflow.com/questions/28518072/play-animations-in-gif-with-tkinter.

@Legorooj

Copy link
Copy Markdown
Member

If you have a feature request related to splash screens, please feel free to open a new issue.

@pyinstaller pyinstaller locked as resolved and limited conversation to collaborators Apr 25, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area:bootloader Caused be or effecting the bootloader area:hooks Caused by or effecting some hook feature Feature request state:needs more work This PR needs more work

Projects

None yet

Development

Successfully merging this pull request may close these issues.