Episode 10 struggles on some operating systems

[profgiuseppe]

Episode 10 starts with the assumption that Python 3 is available in the user's shell.
However, at least o Windows, that may not be true:

• Git Bash may not be able to find Pyhon 3 (either because it is not in the path, or because there are multiple installation of Python, e.g. Python 2.7)
• Anaconda Prompt does not support head command

Proper way to proceed should be stated at the beginning. If there is a flag in Anaconda installation that helps, this is quite too far in the material for learners to have time to fix it. Also, after successfull using Jupyter for quite a while, it is a setback and they may feel isolated from the others (e.g. people with Linux)

[annefou]

When the lesson is taught as part of a Software Carpentry workshop, git bash is installed and this is what we would use for Episode 10. Each lesson has a very short setup, mostly related to data needed for the lesson. We usually have a workshop setup with all the necessary packages (git bash, python or R, etc.) to be installed. I don't really know what is the policy here and if some information about git bash should be added in the setup.

[ldko]

This was an issue we encountered at our last workshop, and it was kind of sloppy to get past because we weren't prepared (the instructors and helpers were using Linux and Mac). Windows learners couldn't run the programs in Git Bash with python because it wasn't on the path, so we had them open up Anaconda Prompt to run the code, but then they couldn't use nano or commands like head and ls in Anaconda Prompt. As @profgiuseppe indicates, this was a frustrating way to end the Python lesson on day 2.

In the setup for the workshop, the instructions to install Anaconda on Windows only specify to "Make Anaconda the Default Python". The instructions don't say to select to "Add Anaconda to My PATH environment variable", and the installer actually now says this is "Not recommended", however in the Video Tutorial linked on the setup page, it shows that setting checked, as I believe it used to be enabled by default. I think this is why we had a much bigger problem with this episode 10 issue at our recent workshop than we saw with our first one--this time Windows users did not have Python on their PATH.

I am wondering if the instructions should add that users should add Anaconda to their path during installation. Otherwise or in addition, perhaps in the python-novice-inflammation lesson setup we should add instructions on adding Anaconda to the path in case people trying to run python in Git Bash get a command not found sort of error. Directions could be to first determine the executable location by opening Anaconda Prompt and typing where python then adding that location with SETX PATH such as with:

 SETX PATH "%PATH%;C:\Users\Username\AppData\Local\Continuum\anaconda3\"

I am not typically a Windows user, so if someone has a better instruction to handle this, it would be good to hear!

[rgaiacs]

We are planning a revision of the installation instructions. Please see carpentries/workshop-template#459.

[petebachant]

I believe the current best practice on Windows is to add this to ~/.bashrc:

. $HOME/AppData/Local/Continuum/anaconda3/Scripts/activate

[ldko]

@petebachant that seems much better than having people set their path with SETX which could have bad results. I removed Anaconda Python from my path and tried your suggested way by doing:
echo ". \$HOME/AppData/Local/Continuum/anaconda3/Scripts/activate" >> ~/.bashrc
and that worked. Thanks!

[msarahan]

@petebachant has the best answer. I work for Anaconda, and we have tried to make sure that the activate shell scripts work in bash on windows. If you have issues, please file them on the conda issue tracker at https://github.com/conda/conda

@ldko I'm glad you removed Anaconda from PATH - that's cleaner and less likely to interfere with random stuff. We advise that by default in new installations, but the old default was to add it to PATH.

One other way to do all of this is to use Windows 10's Windows Subsystem for Linux. That's interesting in that it allows you to use the Linux anaconda/miniconda installers.

Feel free to ping me if you have further questions.

[msarahan]

There was some discussion at unt-carpentries/2018-03-14-unt-swc#14 (comment) about when/why adding entries to PATH is something you either want to do (for convenience) or want to avoid (for avoiding problems).

[maxim-belkin]

So, did you guys agree on using

echo '. $HOME/AppData/Local/Continuum/anaconda3/Scripts/activate' >> ~/.bashrc

?

[ldko]

At UNT we have since switched to doing something else that @somexpert determined to be better in https://github.com/unt-carpentries/2018-03-14-unt-swc/pull/15/files, though not very concise instructions. I think the idea with the change was that the installation may not always be in $HOME and that using .profile was preferable to .bashrc. I don't know what others are doing.

[DamienIrving]

@ldko, @somexpert - Thanks for sharing those instructions - they're great.

I'm teaching a slightly more advanced workshop next month and I want people to be able to create a conda environment, activate that environment and then execute a python script from within that environment at the command line. i.e.

$ conda create -n pyaos-lesson jupyter iris cmocean
$ conda activate pyaos-lesson
(pyaos-lesson) $ python plot_precipitation_climatology.py -h

After executing echo "source [Anaconda path]" >> ~/.profile && source ~/.profile, will conda be available at the git bash command line?

[annefou]

We have started to create conda environment for each workshop using yml file. And it works well with anaconda navigator on Linux and windows (we did not have Mac OS X) for those who are not familiar with command lines. We add it to .bashrc not .profile.

[ldko]

@DamienIrving yes, conda is available at the git bash command line after executing that. We tried out the commands you listed and noticed before running them we needed to do:

$ conda config --append channels conda-forge

because otherwise iris and cmocean were not available.

Also, we ran the create command as:

$ conda create -n pyaos-lesson python=3.6 jupyter iris cmocean

because the Windows machine we ran this on also had an Anaconda for Python 2.7 installed and that is initially what got used when we didn't specify python=3.6.

@annefou do you have a link you could share with instructions on how you all set up the conda environment for each workshop with the yml file? We are interested to look into that setup option. Thanks!

[DamienIrving]

Thanks, @ldko!

[ocefpaf]

I had a nice experience in my last workshop by installing everything, including git, with conda on all platforms. On Windows we can start the shell by calling bash at the Anaconda prompt, it is part of the git package. No path issues, no .bashrc changes, no extra packages to install (git-bash). The smoother install experience I ever had in all workshops I participated.

[DamienIrving]

@ocefpaf Whoa! So just conda install git and that's it?! Windows users can have a complete bash experience by just typing bash at the Anaconda Prompt? That makes things SO much easier.

[annefou]

@ldko here is an example on how we create new conda environment. https://annefou.github.io/jupyter_dashboards/setup.html
(using anaconda-navigator or the command line).

[ocefpaf]

@ocefpaf Whoa! So just conda install git and that's it?!

Yep. conda-forge git package for Windows is a re-package of git-bash, so you don't have to manage that installation and path issues yourself.

@ldko here is an example on how we create new conda environment. annefou.github.io/jupyter_dashboards/setup.html

@annefou I would recommend miniconda instead of the full anaconda distribution to avoid a big download at first. Also, when using 3rd party channels like in the env in that env file, you'll replace most of the packages from anaconda anyway,

channels:
  - tim_shawver
  - conda-forge
  - defaults

BTW, I'm pretty sure only qgrid 1.0.0 is coming from tim_shawver, I guess you an remove that channel and use only conda-forge and defaults there.

[annefou]

@ocefpaf you are right: miniconda is prefered and we use it most of the time now. We still use the "full" anaconda installation when many python packages are required but I admit that it is not a good practice.
And you are also right for the usage of channels. We had problems when we did this workshop (between versions for jupyter_dashboards, ipywidgets, qgrid and beakerx but I am pretty sure it is not necessary anymore. We will revise it for the next workshop. Thanks for your inputs.

[ocefpaf]

@annefou I'm planning on creating a repo that "runs" the SWC lessons on CIs for Linux, OS X and Windows using the install instructions in a similar way as this repo. That would ensure the lesson works and we can always just fire up the CIs before a workshop to double check it.

[annefou]

@ocefpaf That's definitely a great approach. Thanks for the link.

[ldko]

Thank you for the documentation link @annefou !

@ocefpaf @DamienIrving using bash in Anaconda Prompt sounds like it could be easier for a workshop. I tried what was mentioned above running conda install git in the Anaconda Prompt (it installs, and I can run git), but when I type bash at the Anaconda Prompt, I get the message "'bash' is not recognized as an internal or external command, operable program or batch file".

Am I missing something? I am having trouble finding documentation on running bash in the Anaconda Prompt to help me troubleshoot. I am attempting this on Windows 10 with conda 4.5.5. Thanks for any guidance you can provide!

[ocefpaf]

I get the message "'bash' is not recognized as an internal or external command, operable program or batch file".

I'll take a closer look soon, maybe the last update broke it but the executable should be in the Library/bin/ directory for that environment.

[ocefpaf]

@ldko try using the posix package instead. The git package from defaults and conda-forge are different and the former does not package bash. Using posix you should get a more stable and reliable bash for sure.

[ldko]

Thanks @ocefpaf! When I do conda install -c conda-forge git then the git package installed does have bash and it works. :) Perhaps we will try this method at our next workshop to avoid worrying about getting Anaconda Python on the path.

[ocefpaf]

Perhaps we will try this method at our next workshop to avoid worrying about getting Anaconda Python on the path.

I do recommend it. It reduces the amount of steps and instructions we need to get a working environment for the workshop.

When I do conda install -c conda-forge git then the git package installed does have bash and it works.

Good to know but the posix package is actually better b/c it won't interfere with the git package from defaults, in case people install that one, and I believe it has more functionalities.

[DamienIrving]

@ocefpaf So that's just conda install posix?

[ocefpaf]

@ocefpaf So that's just conda install posix?

Yep, then just call bash and it should work.

[ldko]

ah thanks @ocefpaf, I misunderstood when you first mentioned the posix package. I have now tried that one, and it worked for me too. One thing I am wondering about, in Episode 10, at our past workshops, we have had learners use Nano from Bash for editing the code. Is that something you are able to install in conda, or how do you prefer to handle editing the .py files used in the episode? Also, are you usually having the learners do the Unix shell lesson by launching bash at Anaconda Prompt as well?

[ocefpaf]

Is that something you are able to install in conda, or how do you prefer to handle editing the .py files used in the episode?

We, conda-forge, only have nano for Linux and OS X at the moment. We are waiting for a Windows expert to help us with the build on that platform 😬

However, I usually recommend VSCode b/c it is easy to install, has a nice GUI interface that Windows users like, and it is available on all platforms.

Also, are you usually having the learners do the Unix shell lesson by launching bash at Anaconda Prompt as well?

Yes, but one can write a menu install that creates an icon that would drop the users directly into that (ping @msarahan who suggested that to me yesterday 😉).

[ldko]

Thanks for the input @ocefpaf !

[maxim-belkin]

anyone willing to submit a PR?

[ldko]

I personally am hesitant to open a PR at this point not having yet done the suggestions provided in an actual workshop and nano not being available to use with the conda method described. I have not used VSCode, so don't know what learning overhead that adds to the workshop and how it might affect the Unix Shell lesson for which we usually use nano.

[ocefpaf]

I personally am hesitant to open a PR at this point not having yet done the suggestions provided in an actual workshop and nano not being available to use with the conda method described. I have not used VSCode, so don't know what learning overhead that adds to the workshop and how it might affect the Unix Shell lesson for which we usually use nano.

I agree. VSCode is my personal option, and ideally conda-forge should add nano on Windows before we recommend conda as an alternative.

As people try the conda approach and report back their experience we can think about opening a PR and "officially" recommend it.