Download releases and non-OMERO artifacts

[manics]

Adds --release as an alias for --branch, and adds support for parsing the downloads release pages. Also adds all artifacts matching 'OMERO\.(\w+).*\.zip$' in addition to the hard-coded ones.

Follow a latest redirect, then parse the downloads page:

• omego download matlab -v --branch 5.1
• omego download matlab -v --release 5.1
• omego download matlab -v --release 5
• omego download matlab -v --release 4
• omego download matlab -v --release latest

Request a specific version:

• omego download matlab -v --release 4.4.8
• omego download matlab -v --release 5.1.3-m3

Request an artifact which is not explicitly supported in omego:

• omego download java

Originally I was just downloading the zips with explicit redirects (e.g. .../omero/server.zip, see 0c6a111) but after implementing a downloads page html parser I realised they're no longer necessary since it's enough to follow the redirect to the page.

Note release artifacts are only supported for the latest ice at present.

Updated:
More artifacts can be downloaded, see https://github.com/manics/omego/blob/17bb9e0c961f2f3aef13550a06c740cd952bcd05/omego/artifacts.py#L98 for the specification.

The following should all be equivalent:

• omego download --release 5 --skipunzip python
• omego download --release 5 --skipunzip py
• omego download --release 5 --skipunzip OMERO.py

Partial initial matches are supported in some cases, e.g.:

• omego download --release 5 --skipunzip se will currently download the server since there are no other matching strings.

Omit the artifact name to list all available:

• omego download --release 5

Non-OMERO jobs are supported:

• omego download --branch BIOFORMATS-5.1-merge-build (list artifacts)
• omego download --branch BIOFORMATS-5.1-merge-build bftools
• omego download --branch BIOFORMATS-5.1-merge-build bio (partial match, downloads the source)

Jars can be downloaded:

• omego download --branch BIOFORMATS-5.1-merge-build ij should download ij.jar

As a last resort you can also specify the full filename in the event that the shorter names conflict, or you want to download a non-zip non-jar, e.g.

• omego download --branch OMERO-5.1-latest GIT_INFO

Example output listing all zips and jars, with the shortnames which can be truncated

$ omego download
Artifacts available for download. Initial partial matching is supported for all except named-components). Alternatively a full filename can be specified to download any artifact, including those not listed.
named-components:
  linux
  mac
  mac6
  matlab
  python
  server
  win
omerozips:
  docs-5.1.3-ice35-b52
  insight-5.1.3-ice35-b52-linux
  insight-5.1.3-ice35-b52-mac_Java6
  insight-5.1.3-ice35-b52-mac_Java7+
  insight-5.1.3-ice35-b52-win
  insight-ij-5.1.3-ice35-b52
  java-5.1.3-ice35-b52
  matlab-5.1.3-ice35-b52
  py-5.1.3-ice35-b52  
  server-5.1.3-ice35-b52
zips:
  OMERO.docs-5.1.3-ice35-b52
  OMERO.insight-5.1.3-ice35-b52-linux
  OMERO.insight-5.1.3-ice35-b52-mac_Java6
  OMERO.insight-5.1.3-ice35-b52-mac_Java7+
  OMERO.insight-5.1.3-ice35-b52-win
  OMERO.insight-ij-5.1.3-ice35-b52
  OMERO.java-5.1.3-ice35-b52
  OMERO.matlab-5.1.3-ice35-b52
  OMERO.py-5.1.3-ice35-b52
  OMERO.server-5.1.3-ice35-b52

[joshmoore]

Pre-testing, a big 👍 for the general scheme! Default behavior of --release latest?

[manics]

--release latest is now the default
I've switched to downloading OMERO.py (which is significantly larger) since OMERO.cpp no longer exists.

[pwalczysko]

~/Work/omero-setup/omego/main.py download insight -v --release 4.4.5 works.
~/Work/omero-setup/omego/main.py download insight-ij -v --release 4.4.5 does not. What is the list of the aliases here ? (matlab, insight... anything else ?)

From help"

Download an OMERO artifact from a CI server.

positional arguments:
  artifact              The artifact to download e.g. {win,mac,mac6,linux,matl
                        ab,server,python,source,cpp,...}

mac6 ?

[manics]

In general XXX in OMERO.<XXX>.zip, with the addition of some custom names e.g. mac/linux/win to specify the different clients. mac6 is for Java 6 in Mac.

[pwalczysko]

In general all works. @manics will combine this with another PR in order to have more goodies (like insight-ij download) available for testing.

[manics]

Description updated

[pwalczysko]

Works as expected.
Except the usual - help, explanations, documenting the complex behaviour.

[pwalczysko]

@sbesson : I thought it should be possible for @manics to reformat the output lines I commented on here ?

[sbesson]

@pwalczysko: sorry for the confusion, I created a Trello card to address your comments in a follow-up PR.

[pwalczysko]

@sbesson Thanks.