Is This Repository Still Maintained?

[mrclary]

It seems that this repository is no longer maintained.
I'd like to build a standalone Spyder app, but my initial attempt was unsuccessful.
Before pursuing this further, are there any hiccups I should be aware of?

[ccordoba12]

It seems that this repository is no longer maintained.

It's not maintained anymore, but it can be updated.

I'd like to build a standalone Spyder app, but my initial attempt was unsuccessful.

You should use pip packages for this because they are self-contained (specially PyQt5), whereas conda ones are not.

Before pursuing this further, are there any hiccups I should be aware of?

It's very hard to make things work with py2app. I mean, you need a lot of trial and error before generating the app.

But perhaps you just need a minimal app bundle that makes Spyder integrate better with macOS, instead of full app that embeds Spyder along with Numpy, Scipy, Pandas, etc (which is what we do here). @impact27 has worked on that and maybe he can give you some pointers about it.

[CAM-Gerlach]

You could also try PyInstaller; it seems to be the most modern and well-supported standalone package these days, is cross platform and includes builtin support for PyQt5, and the Numpy/Scipy/Matplotlib/Pandas stack.

[mrclary]

You should use pip packages for this because they are self-contained (specially PyQt5), whereas conda ones are not.

When I install packages using pip, it will install into my anaconda environment; is this okay?

[ccordoba12]

I'd recommend you to use a different Python installation and install everything there (no virtualenvs), instead of using Anaconda. That's because conda envs use hard links to try to save disk space, so that could give you problems too.

[impact27]

For reference: spyder-ide/spyder#10054

[impact27]

The question is: Do you want a standalone app, or do you want to access whichever python version you installed?

I am not using a standalone app because this means that I have a different python version in the terminal and in Spyder. Futhermore, the installed packages are differnet as well.

This is usually fine with a standalone app that uses a couple of packages that shouldn't change over time, but Spyder is an IDE, so the packages it need are potentially ALL the packages.

Actually I am not sure about this, can you access conda installed packages from the standalone app?

[impact27]

For a wrapper you just need an icon, a info.plist file, and a script that says:

#!/usr/bin/python3

from spyder.app import start
start.main()

[CAM-Gerlach]

This is usually fine with a standalone app that uses a couple of packages that shouldn't change over time, but Spyder is an IDE, so the packages it need are potentially ALL the packages.

@impact27 Maybe I'm misunderstanding this, but since the introduction of spyder-kernels the approach @ccordoba12 and I always recommended to users both on Github and in the official guide we wrote on the topic was to treat Spyder like any other IDE and simply use a separate working environment(s) for their code as their Spyder environment, which offers a number of advantages and is generally a best practice in most cases. This neatly works around the problem of needing to bundle any more packages than at most Spyder's required and optional dependencies in the Spyder application itself, and is how essentially every other IDE works (since Spyder is one of the only Python IDEs out there actually written in Python).

So long as the optional but recommended packages are bundled in the Spyder environment that Spyder is explicitly integrated with (Pandas, Scipy, Numpy, etc) the only downside to this is that the Object Explorer of Spyder's Variable Explorer cannot generally introspect custom object classes from user packages (that are already hidden by default) that do not exist in the base Spyder environment currently, though its possible this could be fixed eventually.

[impact27]

Ok thanks for the clarification, this makes more sense.

[mrclary]

@impact27, Ideally what I would like is a standalone Spyder application with standard packages that I can distribute to colleagues and have it behave as a native Mac OS application and, optionally, choose a python interpreter for the console, as implemented in Spyder preferences.

What I mean by native Mac OS application is:

• An application, Spyder.app, that resides in /Applications or ~/Applications
• Single icon in the Dock that opens Spyder and indicates that Spyder is running
• Drag files to the Dock icon and/or set python files to open in Spyder
• Menu bar says "Spyder" instead of "Python"

Currently, I have a dedicated conda environment setup for Spyder and made a "wrapper" of sorts using Automator. This launches spyder from the dedicated environment and permits me to use any conda environment for the console by setting the python interpreter in Spyder's preferences.

However, this creates an additional application icon in the Dock: the launcher application, which I click to open Spyder, and the Python.app that Spyder is running in, which is why "Python" is in the menu bar. These are what I wanted to remedy by creating a standalone application.

[CAM-Gerlach]

@mrclary Currently, all of this is possible on Windows with the Anaconda installed-Spyder and @ccordoba12 came up with a proposal for and I wrote a grant to achieve essentially what you are requesting on macOS built right in to the standard Anaconda install, but it didn't end up getting funded. The distribution to colleagues would be by simply installing Anaconda, which would have all this built in plus all the standard packages and also allows creation of other environments.

If this sounds like what you are interested in, you could ask @ccordoba12 if he's willing to give you direction on how to do that so instead of building a whole bespoke application for just yourself and your colleagues, your changes could potentially benefit the entire Spyder community on macOS. It also may be less work than doing the whole bundling thing yourself just to get a few OS-specific integrations. But I'm not sure if things have changed since we wrote up that proposal a year ago so you'd have to ask @ccordoba12 if that's what you're interested in.

[goanpeca]

What I mean by native Mac OS application i

You do not need to do that, you can just install anaconda and copy a mac bundle that contains a script that launches spyder in the activated environment, that's it.

I can send you an example application later today. Cheers

[goanpeca]

@mrclary with a normal anaconda or miniconda installation you should be able to get Spyder running by placing the unzipped version of Spyder3.app.zip on the ~/Applications/ folder

Also in order to see the Spyder application name, the script that runs Spyder needs a modification (We could ask Anaconda for this @ccordoba12?)

The original script read:

/Users/gpena-castellanos/anaconda2/bin/spyder

#!/bin/bash /Users/gpena-castellanos/anaconda2/bin/pythonw

# -*- coding: utf-8 -*-
import re
import sys

from spyder.app.start import main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

I had to change it to

#!/Users/gpena-castellanos/anaconda2/bin/python

# -*- coding: utf-8 -*-
import re
import sys

from spyder.app.start import main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

There you go, a properly conda activated Spyder application

[mrclary]

@goanpeca, your Spyder3.app seems to do all that I need, thanks!
I made a few modifications for my implementation...
run.sh:

#!/bin/bash

if [[ -e /etc/profile ]]; then
    source /etc/profile
fi
if [[ -e ~/.bash_profile ]]; then
    source ~/.bash_profile
fi

source ~/anaconda3/bin/activate spyder
~/anaconda3/envs/spyder/bin/spyder $@

I incorporate user profile shell environment variables (useful for accessing external applications such as LaTeX) and I assume a dedicated spyder environment.
I also made your recommended modification to the spyder script by removing /bin/bash from the shabang line and setting the executable to .../python instead of .../pythonw.

@ccordoba12 @CAM-Gerlach, is this similar to your proposal? Would @goanpeca's solution be difficult to incorporate into the Anaconda install of Spyder?

[CAM-Gerlach]

source ~/anaconda3/bin/activate spyder

Would it be better to do ~/anaconda3/bin/conda activate spyder? I know that's strongly recommended for interactive use in a terminal but I'm not sure about programmatically. Also, as a best practice I would avoid naming environments the same as packages to avoid any potential for confusion or ambiguity, but that's a personal preference.

@ccordoba12 @CAM-Gerlach, is this similar to your proposal? Would @goanpeca's solution be difficult to incorporate into the Anaconda install of Spyder?

I would presume @ccordoba12 and @goanpeca have already discussed and considered that. I haven't been a Spyder core dev for a number of months now so I'm out of the loop regarding the latest developments on that front and am not an authoritative source on any of this. That may in fact be the purpose of @goanpeca 's changes in the first place and glancing at the changelog it looks like some improvements in that regard were integrated into Spyder 4 recently.

[impact27]

If you want to be able to open files with this app, the info.plist needs to have CFBundleDocumentTypes.

[impact27]

Spyder.app.zip
The info.plist was adapted from atom (https://github.com/atom/atom, under MIT)

[impact27]

The first line of Contents/MacOS/Spyder says #!/usr/bin/python3. Change it to where your python is.

[impact27]

Or maybe #!/usr/bin/env python3 is better

[impact27]

Or execute '#!' + sys.executable (Works with python2 as well)

[impact27]

Looking at https://github.com/ContinuumIO/menuinst/blob/master/menuinst/darwin.py, I don't think their approch of creating a plist file is the best way. I would probably just copy a given plist as they can be quite long. Mine is >800 lines long.

[CAM-Gerlach]

@impact27 I'm sure we all appreciate your input, but to be a little more considerate to the four other people subscribed to this thread, you might want to consider using Github's edit feature or organizing your thoughts into one coherent comment before hitting submit, rather than posting six successive ~one liners within13 minutes of one another. Thanks.

[impact27]

@impact27 I'm sure we all appreciate your input, but to be a little more considerate to the four other people subscribed to this thread, you might want to consider using Github's edit feature or organizing your thoughts into one coherent comment before hitting submit, rather than posting six successive ~one liners within13 minutes of one another. Thanks.

Thanks for letting me know, I'll be more careful in the future.

[mrclary]

@impact27, thanks for the update to the plist file. I had not checked opening files, but that works now.

I could not get your Spyder.app to work; I believe because Contents/MacOS/Spyder doesn't set path variables that permit python to find the spyder module. Nevertheless, I kept the run.sh used before since that allows me to add user bash profile environment variables and it correctly sets the anaconda environment.

@CAM-Gerlach,

Would it be better to do ~/anaconda3/bin/conda activate spyder?

I don't know, but I accept your suggestion since you probably know more about it than I do.

@ccordoba12 @goanpeca ,
I think the combined solutions presented here would be very valuable to many Mac OS users and would be worth incorporating into Anaconda. I don't have any experience working on the development side of Anaconda, unfortunately.
The only downside I see is that this solution requires an Anaconda installation of Spyder, and so is not a standalone application. While this does not adversely affect me (I and my colleagues require Anaconda anyway), I can imagine that the broader community may have an interest in a true standalone application.

[CAM-Gerlach]

I don't know, but I accept your suggestion since you probably know more about it than I do.

Not really; as I mentioned I'm no longer formally with Spyder and @ccordoba12 and @goanpeca know lightyears more than I do, especially about conda infra.

[goanpeca]

The only downside I see is that this solution requires an Anaconda installation of Spyder, and so is not a standalone application. While this does not adversely affect me (I and my colleagues require Anaconda anyway), I can imagine that the broader community may have an interest in a true standalone application.

I would really prefer to use https://github.com/conda/constructor and have a Spyder installation which under the hood really is a slimmed down Anaconda installation with a Spyder dedicated env and a mac bundle for this Spyder in the dedicated env.

I would also point the Spyder default python prefix to an environment different from where Spyder lives.

The only issue to solve would be to have a mechanism to update Spyder, which would preferably be an executable (so that it does not depend on python or pyqt etc... but that would call conda and do the update).

This would allow Spyder users to have a more application like experience to download from the website. It would be extra work on our side to maintain but luckily most of the process could be automated to generate installers every couple of months.

[goanpeca]

@CAM-Gerlach

~/anaconda3/bin/conda activate spyder

This was not working, so programmatically I do not think we can use it like that (or something needs to be done but I could not get it to work on my limited testing)

[mrclary]

@CAM-Gerlach

~/anaconda3/bin/conda activate spyder

This was not working, so programmatically I do not think we can use it like that (or something needs to be done but I could not get it to work on my limited testing)

@goanpeca, can you provide a little more information?

[CAM-Gerlach]

@goanpeca Perhaps because with conda 4.6+ conda activate needs to initialize the shell being used with conda init first? In that case it seems calling the activate script directly is the safer option, as I thought it might be (which is why I posed it as a question rather than a recommendation and advised @mrclary to defer to your advice on the matter).

[mrclary]

@goanpeca,
Interestingly, the application implementation for Spyder 3.3.6 does not seem to open files correctly: opening from Finder or dragging to Dock correctly launches the application but does not show up in the editor. This works fine for Spyder 4.0.0b7. I use identical bundle contents and info.plist files for both versions and the only difference is which spyder environment and version of spyder is executed in the run.sh script.

The following correctly opens a file in the editor for 3.3.6, whether Spyder is already running or not:

>> spyder file.py

The following works in 4.0.0b7, whether spyder is already running or not:

>> open file.py

or

>> spyder file.py

In either case, I've set the appropriate default application to open the file in its Info window.

Is there a difference between 3.3.6 and 4.0.0b7 that would cause this behavior?

[CAM-Gerlach]

@mrclary That would be Python Language Server, which in turn uses pylint and pycodestyle, along with Mccabe, Pydocstyle and others. If you enable Spyder's debug mode with the command line arg or environment variable, between the client, server and Spyder logs (check the ~/.spyder-py3/lsp_logs dir for the first two) you should see an indication of what's going on.

[ccordoba12]

@CAM-Gerlach is right.

[mrclary]

@CAM-Gerlach & @ccordoba12
I've attached log files from launching Spyder.app/Contents/MacOS/Spyder.
I didn't see anything that indicated any issues with the language server, but maybe your expert and discerning eyes will see something that I don't.

I've also attached two screen shots: one showing bootstrap (works) and one from Spyder.app.
spyder.log
client_python_10293.log
server_python_10293.log

[CAM-Gerlach]

@mrclary I see you are using that proprietary non-free third-party Kite addon. Its possible it could be interfering or not working correctly...other than that I looked through all your logs and didn't see anything out of the ordinary though @ccordoba12 knows far more than I. Given LSP appears to be returning responses I would naively wonder whether the issue may be with them not being drawn properly to the editor due to the different Qt versions and installation mechanisms between the two builds though again that's merely a guess.

[mrclary]

Hmm... I'll try disabling Kite. However, I should have also clarified that it works properly with both bootstrap and build/install into local environment.

[ccordoba12]

other than that I looked through all your logs and didn't see anything out of the ordinary

They look fine to me too, although you can see the server is returning an empty list for linting and hovers. But I don't know why that happens.

[mrclary]

If server is not supposed to return empty lists, then that at least gives me a place to start...

[mrclary]

Okay, I have successfully solved the app bundle portability issue!
The issue was that I used anaconda python to create the virtual environment.

The Problem

DO NOT DO...

(conda) >> python -m venv --clear --copies env
(conda) >> source env/bin/activate
(env) (conda) >> pip install -r requirements/pip.txt
(env) (conda) >> python create_app.py py2app

I incorrectly believed that creating a virtual environment with '--copies' should make the python environment completely independent of the original python command used to create it (isn't that what a virtual environment is supposed to do?). Well, technically, environments really only manage packages, not the Python version, so there are issues with dylib linking with Anaconda's Python version management. At least that's what I glean from my experience.

The Solution

After learning much more about virtual environments than I had intended, I was able to produce a successful solution. The bottom line is don't use anaconda anywhere for a standalone build. So, my final solution was to create a virtual environment from a Homebrew install of python. The virtual environment can be of any flavor (except conda!), such as venv or pyenv + pyvirtualenv. I've settle on the latter. Note that with pyenv, when installing a python version, it is required to temporarily set the environment variable PYTHON_CONFIGURE_OPTS="--enable-framework".

So here is the final process.

1. Obtain a recent version of Python (e.g. 3.8.2) and (optionally) pyenv and pyvirtualenv. I imagine there are a myriad of ways to do this (e.g. Homebrew) as long as it's not Anaconda.
2. Setup Python version and virtual environment (If you have a conda base environment activated, then deactivate it first). In my case, it looks like this:

>> pyenv install 3.8.2
>> pyenv virtualenv 3.8.2 env-spyder-dev
>> pyenv local env-spyder-dev
(env-spyder-dev) >> pip install --upgrade pip
(env-spyder-dev) >> pip install -r requirements/pip.txt

3. Build the app

(env-spyder-dev) >> cp tools/create_app.py ./; python create_app.py py2app; rm create_app.py

4. Use the instructions in this repo to build the dmg file.

Now What?

Where do you guys see this going? I personally would like to see the MacOS standalone application maintained and distributed in some fashion without users having to build it themselves. Since the standalone build process must be done in the main Spyder repo, and since it only requires a few extra files from this repo (for building the dmg), I propose archiving this repository, and moving the required files to the main Spyder repo. What do you think?

There are some outstanding issues that need to be resolved.

1. There are issues with using external conda environments from the standalone app. Fortunately, I believe I've fixed those issues and I will create a separate PR soon. The principle issue lies with environment variables when launching the ipython kernel (spyder-kernels) from outside a conda-aware environment (i.e. inside a standalone app).
2. I've removed extraneous packages from the standalone app in order to test the build process. We should decide what packages should be included for a true standalone IDE (numpy, scipy, matplotlib, ...). Of course, these will have to be tested as well within the standalone app (I'm looking at YOU matplotlib!)
3. The linting/code-folding issue mentioned earlier. While Spyder still appears otherwise fully functional, this will need to be addressed
4. In a further stage of development, it would be nice to also accommodate any type of virtual environment (conda, venv, pyvirtualenv, etc.).

I look forward to your feedback.

[ccordoba12]

I personally would like to see the MacOS standalone application maintained and distributed in some fashion without users having to build it themselves

Agreed.

Since the standalone build process must be done in the main Spyder repo, and since it only requires a few extra files from this repo (for building the dmg), I propose archiving this repository, and moving the required files to the main Spyder repo. What do you think?

Unfortunately we don't have the resources nor the time to maintain this endeavor, so it depends on you or other members of the community to do it. That's why I don't agree with moving this into the main repo (you can move create_app.py here if you want).

Instead, I think we need to setup Github actions in this repo and build the app bundle with them (we can easily grant you the permissions for you to start doing that).

[CAM-Gerlach]

There are issues with using external conda environments from the standalone app. Fortunately, I believe I've fixed those issues and I will create a separate PR soon. The principle issue lies with environment variables when launching the ipython kernel (spyder-kernels) from outside a conda-aware environment (i.e. inside a standalone app).

I know we were having a number of issues doing this properly within Spyder/Spyder-Kernels as well when external binaries (e.g. Windows DLLs) were involved even between two different Anaconda environments. that was very tricky to resolve and was only just done very recently. Not sure if you've looked at that or if it might be helpful to you here.

I've removed extraneous packages from the standalone app in order to test the build process. We should decide what packages should be included for a true standalone IDE (numpy, scipy, matplotlib, ...). Of course, these will have to be tested as well within the standalone app (I'm looking at YOU matplotlib!)

My suggestion would be to just keep the core packages that directly integrate with Spyder in some way (i.e. are listed in the optional deps), i.e. Numpy, Scipy, Pandas, Matplotlib, Cython, Sympy, maybe Pillow. Other than object introspection (which may be able to be worked around in the future, as this is an issue for Spyder in general) the others only really need to be in the working environment for user code, not Spyder's env.

I've removed extraneous packages from the standalone app in order to test the build process. We should decide what packages should be included for a true standalone IDE (numpy, scipy, matplotlib, ...). Of course, these will have to be tested as well within the standalone app (I'm looking at YOU matplotlib!)

While this may be a little disappointing at first, its actually not uncommon for a number of other major software projects to have many of their builds for specific platforms maintained separately from the main project, and given all your hard work on this it you could be given access rights to this repo to do whatever you need to and proper credit for your central role in re-vamping and maintaining the builds.

[goanpeca]

@mrclary I suggest you take a look at https://regro.github.io/rever-docs/ to automate the process first locally before moving to a CI.

[goanpeca]

There are issues with using external conda environments from the standalone app. Fortunately, I believe I've fixed those issues and I will create a separate PR soon. The principle issue lies with environment variables when launching the ipython kernel (spyder-kernels) from outside a conda-aware environment (i.e. inside a standalone app).

I can help you with this one ;-)

[CAM-Gerlach]

Yeah, it was @goanpeca 's hard work resolving spyder-ide/spyder#9077 in spyder-ide/spyder#11327 was what I was referring to above. It took several rounds of testing before he got it working as it was much trickier than expected.

[mrclary]

Okay, I think we're ready to go. #3 and spyder-ide/spyder#12011

[mrclary]

@ccordoba12

Instead, I think we need to setup Github actions in this repo and build the app bundle with them (we can easily grant you the permissions for you to start doing that).

I'm okay with that. I'll look into rever per @goanpeca's suggestion.

[mrclary]

For those interested, I've created a standalone Mac application release on my fork of this repository. Hopefully a few brave souls will take the opportunity to test it and provide feedback :-).
https://github.com/mrclary/mac-application/releases

I expect that we will have a release on this repository as well in the near future.

[vancromy]

For those interested, I've created a standalone Mac application release on my fork of this repository. Hopefully a few brave souls will take the opportunity to test it and provide feedback :-).
https://github.com/mrclary/mac-application/releases

I expect that we will have a release on this repository as well in the near future.

I will happily give them a go and let you know how I get on! :-)

[ccordoba12]

For those interested, I've created a standalone Mac application release on my fork of this repository.

Thanks a lot @mrclary!

I have one comment about your work. In your Releases page you say:

The "lite" version does not contain numpy, matplotlib, scipy, pandas, cython, or sympy, internally. This means that using the "Same as Spyder" python interpreter will be limited. The advantage is that it saves space (~260 MB) in the application. If you use external (conda) environments, this will not affect you unless you try to edit variables from the Variable Explorer.

What do you think about better handling the error message we show to users when Numpy and Pandas are not available in the lite version? Since users can't install other libraries in the app, I think the current message is going to create a lot of confusion.

[mrclary]

What do you think about better handling the error message we show to users when Numpy and Pandas are not available in the lite version? Since users can't install other libraries in the app, I think the current message is going to create a lot of confusion.

Do you mean in addition to what was already done for spyder-ide/spyder#12563?

We could modify it for the standalone application to say something about the Lite version and recommend the full version.

[ccordoba12]

We could modify it for the standalone application to say something about the Lite version and recommend the full version.

Exactly, that was my idea! Could you open a PR about it?

[mrclary]

We could modify it for the standalone application to say something about the Lite version and recommend the full version.

Exactly, that was my idea! Could you open a PR about it?

Since it was directly related to the Mac application, I just added it to spyder-ide/spyder#12011

[ccordoba12]

Thanks!

[vancromy]

For those interested, I've created a standalone Mac application release on my fork of this repository. Hopefully a few brave souls will take the opportunity to test it and provide feedback :-).
https://github.com/mrclary/mac-application/releases
I expect that we will have a release on this repository as well in the near future.

I will happily give them a go and let you know how I get on! :-)

Just to provide an update on this, I've tested the following in the past week:

• Running a normal script in open console, dedicated console and external console

• Opening an existing project, changing to another existing project with a project already open

• Interact with the several panes available (move, customise and hide/show them)

• Toggle on/off and test code style warnings as well as code folding

• Use the variable explorer and open/navigate through variables

• Change the python interpreter to a virtual condo env

All of the above work perfectly fine! There's only one thing that is not working for me which is the option to toggle on and off indent guides. Doing so does nothing and I no longer see the familiar lines in my code that I'm used to from previous versions of Spyder. Could be that I've forgotten to do something on my end though.

I also got 1 failed attempt at opening an existing project which resulted in the automatic GitHub issue prompt to show up. But I've not been able to reproduce this since so could have been a one-off.

PS: Sorry forgot to add. This is all on the normal version (not the Lite version)

[mrclary]

@vancromy, thanks for testing!

I've been using the Lite version for some time in my day job and have not seen any issues with indent guides. Is it possible that you were looking at a file that was over 2000 lines long? I know there is a warning message that code folding and indent guides will be disabled for such large files for performance reasons. If you selected to not show the warning message again, you may not see it.

I will check the full version to see if there is problem.

[vancromy]

@mrclary I can't recall whether it was more than 2000 lines long but I doubt it. Good news though is that I did notice the indent guides after a while! Could be that it took a while for them to buffer/show or that in the meantime I closed the module and reopened it. Regardless of what I did though, I've now got indent lines so not a problem at all.

It's been pretty much smooth sailing for me so thanks for all the hard work you've put into making this possible for Mac users :).

[mrclary]

@ccordoba12, closing this issue since we now have macOS installer in spyder-ide/spyder since v4.2.0.