Overhaul installation instructions

[peterjc]

Currently we have brief install instructions in the main README.rst file,

• https://github.com/biopython/biopython/blob/master/README.rst

and the downloads page,

• http://biopython.org/wiki/Download
• https://github.com/biopython/biopython.github.io/blob/master/wiki/Download.md

Those are relatively easy to update, but we also produce a long HTML or PDF file which is included in the source code releases:

• http://biopython.org/DIST/docs/install/Installation.html
• http://biopython.org/DIST/docs/install/Installation.pdf
• https://github.com/biopython/biopython/blob/master/Doc/install/Installation.tex

There is a barrier to contribution because the source is written in LaTeX and also requires hevea be installed to make the HTML output.

It would be good to review this file as much of the content is many years old - and perhaps take this chance to rewrite it as reStructuredText which is rendered directly on GitHub and easily edited from the GitHub website too.

We can also include validating the markup of the new INSTALL.rst file (or whatever we call it) directly in the continuous integration checks.

[peterjc]

We should also remove all references to python setup.py install once Biopython 1.70 is out, in favour of pip install biopython or pip install . as appropriate.

See also #1302 which will likely be in Biopython 1.70 and automate NumPy installation via pip.

[peterjc]

Regarding the Mac, on our download page we currently have this (you don't need all of XCode anymore):

For Mac OS X, or as it is now branded, macOS, we recommend installing
Biopython from source as above. You will need to have installed Apple's
command line tools, which can de done with the terminal command:

xcode-select --install

This will offer to install Apple's XCode development suite - you can, but
it is not needed and takes a lot of disk space.

[peterjc]

I'm hoping to have wheels out before too long, which will further change things - it ought to be possible to install the latest Biopython release under Linux/Windows/Mac without needing a compiler 👍

#1296

That would mean we could refocus the compiler detail heavy information in Installation.tex into a developer centric document?

[peterjc]

Wheels are released - still an open question as to if we still need to release Windows Installers or not.

[mdehoon]

The installation documentation in Doc/install/Installation.tex says that we strongly recommend NumPy, while README.rst and the downloads page says it's required. It doesn't matter much for the end user, but for developers it's easier if they can assume that NumPy is present.

[peterjc]

I'd like to heavily cull Doc/install/Installation.tex or at least repurpose it for contributors/developers only. In that context, it is useful to be able to work without NumPy and/or the C modules, but that is still quite niche. Is this distinction worth making?

The other historical reason to work without NumPy was PyPy (not an issue anymore) and Jython (which we deprecated support for).

[mdehoon]

To reduce the burden on the developers, I think it is acceptable to simply require NumPy and the C modules.

[peterjc]

I am OK with the current situation where we deliberately have a TravisCI target without NumPy to ensure this corner case can be used. This was recently helpful for testing on Python 3.8 betas, since there is not (or was not) any easy way to install NumPy short of self-compiling. See #2146 when on TravisCI it was useful to be able to test a developement release of Python without NumPy (where compiling NumPy is not possible due to the time it takes).

Requiring the C modules isn't a big issue for Linux or macOS based people - but does it make the barrier to entry harder for Windows contributors? CC @MarkusPiotrowski as IIRC you were in that situation?

[MarkusPiotrowski]

Yes, setting up an environment where you can install Biopython from source or even setting up a C compiler to compile single C files into extensions is not trivial on Windows. It's not rocket science but definitely raises the bar for Windows contributors. (The minimal working Visual C++ set-up is several gigabytes large)

BTW, we have already a de facto requirement for one C module: _aligners:
Before Biopython ver. 1.72 it was quite possible to work without compiled C modules; running the test suite gave only a handful of errors due to missing C modules.
Now, the _aligners C module is a top-level import in Bio.Align and many modules are importing Bio.Align. Without the _aligners C module nearly half(!) of the tests are failing.

[peterjc]

I hadn't realised that the _aligners code had such an impact. With hindsight perhaps putting it directly into Bio.Align was not ideal. If we can have a gracefull failure here, that to me would be better... but that does as Michiel says add costs of a different kind.

[mdehoon]

@MarkusPiotrowski In my experience, with Cygwin it's trivial on windows.

[mdehoon]

@peterjc Bio.Align could be modified to allow a graceful failure if _aligners is not compiled, even with _aligners being imported directly into Bio.Align. But in the near future, we will have Seq objects implemented in C (*), so anyway sooner or later developers will need to be able to compile C extensions.

(*) The current Python code in Bio.SeqIO and Bio.Seq is way too slow for genome-sized data.

[peterjc]

We digress, but I don't think C is the only answer to speeding up that code. Dropping the heavy alphabet objects will help a lot, that will leave the Seq objects much more string like - and make your long held wish to just subclass string much more attainable.

Anyway, it seems we agree that for end users, Numpy is expected, C compilers are not (they will get things pre-compiled as wheels or other packages). Therefore, only minimal installation instructions are needed (as in the current README).

For developers, it is useful to be able to run without NumPy, and even without a C compiler.

So, should the goal be to make it clear that Doc/install/Installation.tex is developer focused?

[MarkusPiotrowski]

@mdehoon

In my experience, with Cygwin it's trivial on windows.

Do you have sort of tutorial or step-by-step instructions for this? Do you use this during installation of Biopython or for single C files? I would like to check this out and possibly add something to our homepage.

[mdehoon]

@MarkusPiotrowski See this page: https://www.cygwin.com/
You would run Cygwin's Python and Cygwin's gcc C compiler.

[mdehoon]

@peterjc

For developers, it is useful to be able to run without NumPy, and even without a C compiler.

Such developers wouldn't be able to assess how their changes impact the C modules or the numpy-dependent modules (and may find out only from the continuous integration tests on github).

[peterjc]

@mdehoon Yes, exactly. This covers a lot of useful contributions which need not touch the C code (e.g. docstring work, Python style issues). Perhaps especially true of first time contributors, where expecting them to install the C compilers is a big ask. We want to make their life easier, not harder.

(Even on macOS, installing the command line tools is a substantial download)

[peterjc]

I think that the website and tutorial references to the following have been removed or updated now:

http://biopython.org/DIST/docs/install/Installation.html
http://biopython.org/DIST/docs/install/Installation.pdf

That I think just leaves removing the stub install/Installation.tex file...

[peterjc]

Belatedly closing this with the removal of the Installation.* files.