nightly builds / releases

[RussTedrake]

Moving the last remaining discussion from #1526 (and closing that issue).

everyone agrees we want nightly releases, and we need build servers running tests on those releases (ideally before it was posted, but github has a webhook for release postings that could make it easy to do things serially).
Specifically:

• [] nightly build should replace a single "nightly" release on the github release page.
• [] we can tag a good release (and increment the version num)
• [] we carry a branch "stable" which gets moved up to the latest release, and the default install instructions tell people to clone stable.

[jamiesnape]

Jenkins can post nightly builds on success. Ideally, I would post these to an S3 bucket with a public link. You could place a link to the latest nightly on the wiki. I would reserve the GitHub releases for the stable versions.

[RussTedrake]

Great. Let me know the link.
On Tue, Mar 1, 2016 at 10:52 AM, Jamie Snape notifications@github.com wrote:
Jenkins can post nightly builds on success. Ideally, I would post these to an S3
bucket with a public link. You could place a link the latest nightly on the
wiki. I would reserve the GitHub releases for the stable versions.

—
Reply to this email directly or view it on GitHub
[https://github.com//issues/1766#issuecomment-190779847] .[https://github.com/notifications/beacon/AGJNNPeC8H7x8xD4xib87yYwCl3vY7PUks5ppGClgaJpZM4Hkeks.gif]

[jamiesnape]

Will do. To get this fully functional, we first need to resolve #1751 and #1781.

[billhoffman]

Work on a partial one for linux that uses the shell scripts.

[RussTedrake]

note: have to be very careful to not include proprietary source. (e.g. snopt). we have some “release_filelist” logic in the make files which help with this. i can also share our existing packaging scripts.

[jamiesnape]

Are they the scripts in the admin repo?

[RussTedrake]

Yes. In the www subdir

[RussTedrake]

Make that packaging subdir :-/

[RussTedrake]

the other thing that we will want to be conscious of (eventually) is to have the minimum required versions for all of the libraries on the servers that build the official releases. (see e.g. #1428)

[david-german-tri]

Downgraded to medium because this is blocked on having code that's stable enough to release.

[jwnimmer-tri]

So here's a good question. Should the releases be the C++ and Python code only, or also include the MATLAB support? It seems like a big delta on effort to add MATLAB.

[RussTedrake]

I’ve been leaning to not including the matlab, just to get these up. We’ll still need to supply some files from the source directory (urdf + meshes), but the list should be pretty reasonable. Will also have to address path relocation issues.

[jamiesnape]

I will start with C++ and take it from there. Dealing with the pure CMake projects should be relatively easy, since the functionality for creating packages is built-in. It is mostly a case of ensuring that everything that needs to be in the installer has an install command in front of it. If projects (especially Drake) are not using CMake life gets more difficult.

[jamiesnape]

The other question is which externals would you like to include (#4045). Given the variety of licenses, that impacts packaging and distribution. Obviously everything marked with ! in that ticket is not going to be included.

[RussTedrake]

Looks like it might be worth cleaning up a little more on #4045, like setting the eigen flag and resolving the thirdParty issues in drake. (plus resolving, or explaining, the remaining ???s)

It looks like we can get pretty far using drake + the BSD-3 and Zlib (bullet is worth including) licensed externals. How restrictive is Zlib?

[RussTedrake]

sorry, i forgot. We have permission fromthe author to redistribute code linked (statically) against snopt, but not to distribute the snopt libraries themselves. We should definitely handle that case here (snopt is the solver of choice)

[jamiesnape]

Cool. That makes life easier.

[jamiesnape]

(I am assuming no SNOPT headers may be distributed, which is not an issue at present.)

[jamiesnape]

@RussTedrake Would you like to distribute binary executables of the examples, or just libraries?

[RussTedrake]

definitely examples, too. thanks!

[david-german-tri]

This is actively being worked, bumping the priority.

[david-german-tri]

Note from f2f: We will need to make sure that build tree RPATHs do not leak into the installed package.

[patmarion]

btw, regarding RPATH and redistributable packages, Director has packaging for linux and mac that fix up RPATH and I am quite happy with how it works!

https://github.com/RobotLocomotion/director/tree/master/distro/package

[jamiesnape]

Cool. Looks useful.

[patmarion]

fyi, I just tested Director's packaging script on the drake superbuild. It successfully makes redistributable packages for director that also include all of drake. If you want nightly packages/releases, I'd recommend turning this on as a starting point.

[stonier]

Jenkins now has a set of jobs under Packaging and binaries up on AWS, tagged latest or by yyyymmdd:

• https://s3.amazonaws.com/drake-packages/drake/nightly/drake-latest-xenial.tar.gz
• https://s3.amazonaws.com/drake-packages/drake/nightly/drake-20170719-xenial.tar.gz

[EricCousineau-TRI]

BTW Should we consider this closed for now?
While explicit tagging is not being done, we can just "manually" tag the releases, and possibly mirror the release to a different URL if so desired?

[jamiesnape]

This issue about nightly releases, so tags and the whole stable release process would be different issue anyway.

[EricCousineau-TRI]

Sounds good! Closing for now.