Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve packaging and distribution system #2951

Open
pditommaso opened this issue Jun 10, 2022 · 0 comments
Open

Improve packaging and distribution system #2951

pditommaso opened this issue Jun 10, 2022 · 0 comments
Milestone

Comments

@pditommaso
Copy link
Member

pditommaso commented Jun 10, 2022

Summary

Nextflow application is a packaged as a set of JAR files (Java archives) that are uploaded to Maven central repository with the exception of the launcher app that's stored in http://nextflow.io/releases/

The nextflow "binary" is a mere Bash script that checks if the launcher app is stored locally. If it cannot be found it download it from the http://nextflow.io/releases URL and run it.

Then the launcher app, uses the Capsule package manager to download the main application JARs from Maven along with all required dependencies files.

This system worked quite nicely for many years, however there are some serious open issues:

  • The Capsule system has been is not maintained anymore by the original author. Considering the new fast release cadence of the JVM this can because soon a problem (see below)
  • Recent Java release has introduced a slightly different version scheme that the Capsule system fail to recognise. This creates problems with some users that need to change Java distribution to use Nextflow. See here and here and here
  • The Capsule system breaks when using _JAVA_OPTIONS variable. See here
  • The Capsule system fails to download and unpack the deps in some systems because it requires to create file system lock, which not all file system supports. See here
  • The new nextflow plugins system makes the capsule system not needed anymore.

Goals

  • Remove the dependency on Capsule package system
  • Create instead a standalone nextflow distribution for the core runtime. The Nextflow plugin system will continue to download the required plugins separately
  • Keep the compatibility with the existing nextflow launcher wrapper

Non-goals

  • the support for JReleaser or other Linux package managers such as RPM, Debian, etc. is out of the scope of this issue, however, it's desirable that the overall design to be compatible with the packaging those system, that could be adopted in the future
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants