Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

installation: upper boundaries of dependencies #80

Closed
tiborsimko opened this issue Aug 28, 2018 · 3 comments
Closed

installation: upper boundaries of dependencies #80

tiborsimko opened this issue Aug 28, 2018 · 3 comments
Assignees
Projects
Milestone

Comments

@tiborsimko
Copy link
Member

Now that reana-commons and reana-db are released on PyPI, let us put an upper boundary in our setup.py dependencies so that REANA v0.3.0 will be re-buildable and re-installable even in several months when we start working on and releasing REANA 0.4.0 or later versions.

(1) We should set upper boundaries for our own shared packages, for example:

 'reana-commons>=0.3.0,<0.4.0',

This needs to be done in many repositories:

$ grep 0.3.0 r*/setup.py | grep reana-
reana-client/setup.py:    'reana-commons>=0.3.0',
reana-cluster/setup.py:    'reana-commons>=0.3.0',
reana-job-controller/setup.py:        'sphinxcontrib-openapi>=0.3.0'
reana-job-controller/setup.py:    'reana-commons>=0.3.0',
reana-job-controller/setup.py:    'reana-db>=0.3.0',
reana-server/setup.py:        'sphinxcontrib-openapi>=0.3.0'
reana-server/setup.py:    'reana-commons>=0.3.0',
reana-server/setup.py:    'reana-db>=0.3.0',
reana-workflow-controller/setup.py:        'sphinxcontrib-openapi>=0.3.0',
reana-workflow-controller/setup.py:    'reana-commons>=0.3.0',
reana-workflow-controller/setup.py:    'reana-db>=0.3.0',
reana-workflow-engine-cwl/setup.py:        'sphinxcontrib-openapi>=0.3.0'
reana-workflow-engine-cwl/setup.py:    'reana-commons>=0.3.0',
reana-workflow-engine-serial/setup.py:    'reana-commons>=0.3.0'
reana-workflow-engine-yadage/setup.py:    'reana-commons>=0.3.0',

(2) We should consider pinning all important dependencies, for example here is how we depend on Celery:

$ grep -i celery r*/setup.py
reana-workflow-controller/setup.py:    'celery==3.1.17',
reana-workflow-engine-cwl/setup.py:    'celery==4.1.0',
reana-workflow-engine-serial/setup.py:    'celery>=4.1.0',
reana-workflow-engine-yadage/setup.py:    'celery>=4.2.1',
reana-workflow-monitor/setup.py:    'celery==3.1.17',

If Celery 4.3 is released in a few months and if the way how we use it is changed, we use it, then REANA v0.3.0 may not be installable then.

We may want to pin fully (celery==4.1.0) or we may rather pin for a reasonable interval (celery>=4.1.0,<4.2.0). The best pinning philosophy will depend on each concrete dependency.

(3) Sometimes we don't even indicate any version, e.g. only zmq. We should also add versions that are known to work at this v0.3.0 release time.

@tiborsimko tiborsimko added this to the v0.3.0 milestone Aug 28, 2018
tiborsimko added a commit to tiborsimko/reana-server that referenced this issue Sep 4, 2018
* Uses only `setup.py` to install REANA-Commons and REANA-DB dependencies and
  introduced upper boundary for their versions. (addresses reanahub/reana#80)

* Removes installation of their `master` versions from `Dockerfile`.
  (closes reanahub#85)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
@tiborsimko tiborsimko self-assigned this Sep 4, 2018
tiborsimko added a commit to tiborsimko/reana-workflow-engine-serial that referenced this issue Sep 4, 2018
* Uses only `setup.py` to install REANA-Commons dependency and introduces upper
  boundary for its version. (addresses reanahub/reana#80)

* Removes installation of its `master` versions from `Dockerfile`.
  (closes reanahub#35)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to reanahub/reana-server that referenced this issue Sep 4, 2018
* Uses only `setup.py` to install REANA-Commons and REANA-DB dependencies and
  introduces upper boundary for their versions. (addresses reanahub/reana#80)

* Removes installation of their `master` versions from `Dockerfile`.
  (closes #85)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-job-controller that referenced this issue Sep 4, 2018
* Introduces upper boundaries for REANA-Commons and REANA-DB dependency
  versions. (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-job-controller that referenced this issue Sep 4, 2018
* Introduces upper boundaries for REANA-Commons and REANA-DB dependency
  versions. (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-workflow-controller that referenced this issue Sep 4, 2018
* Introduces upper boundaries for REANA-Commons and REANA-DB dependency
  versions. (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-workflow-engine-cwl that referenced this issue Sep 4, 2018
* Introduces upper boundary for REANA-Commons dependency.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-workflow-engine-yadage that referenced this issue Sep 4, 2018
* Introduces upper boundary for REANA-Commons dependency.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-cluster that referenced this issue Sep 4, 2018
* Introduces upper boundary for REANA-Commons dependency.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-client that referenced this issue Sep 4, 2018
* Introduces upper boundary for REANA-Commons dependency.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
@tiborsimko
Copy link
Member Author

Internal REANA dependencies done.

We may still want to limit some external dependencies, especially some "critical" ones that may break things, such as Celery or Bravado.

diegodelemos pushed a commit to diegodelemos/reana-server that referenced this issue Sep 5, 2018
* Introduces upper boundary for REANA-Commons and REANA-DB dependencies.
  (addresses reanahub/reana#80)
diegodelemos pushed a commit to diegodelemos/reana-workflow-controller that referenced this issue Sep 5, 2018
* Introduces upper boundary for REANA-Commons and REANA-DB dependencies.
  (addresses reanahub/reana#80)
diegodelemos pushed a commit to diegodelemos/reana-workflow-engine-serial that referenced this issue Sep 5, 2018
* Removes installation of REANA-Commons from git master (closes reanahub#35).

* Introduces upper boundary for REANA-Commons dependency
  (addresses reanahub/reana#80).
diegodelemos pushed a commit to diegodelemos/reana-server that referenced this issue Sep 5, 2018
* Introduces upper boundary for REANA-Commons and REANA-DB dependencies.
  (addresses reanahub/reana#80)

* Removes installation of REANA-Commons from git master (closes reanahub#85).
tiborsimko added a commit to tiborsimko/reana-client that referenced this issue Sep 5, 2018
* Introduces more upper boundary limits for dependencies to make sure all the
  client components are installable from PyPI in the future.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-cluster that referenced this issue Sep 5, 2018
* Introduces more upper boundary limits for dependencies to make sure all the
  client components are installable from PyPI in the future.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-client that referenced this issue Sep 5, 2018
* Introduces more upper boundary limits for dependencies to make sure all the
  client components are installable from PyPI in the future.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-server that referenced this issue Sep 5, 2018
* Introduces more upper boundary limits for selected important
  dependencies (Bravado, Celery) for the cluster components.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-workflow-engine-serial that referenced this issue Sep 5, 2018
* Introduces more upper boundary limits for selected important
  dependencies (Bravado, Celery) for the cluster components.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-workflow-engine-yadage that referenced this issue Sep 5, 2018
* Introduces more upper boundary limits for selected important
  dependencies (Bravado, Celery) for the cluster components.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-client that referenced this issue Sep 5, 2018
* Introduces more upper boundary limits for dependencies to make sure all the
  client components are installable from PyPI in the future.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
@tiborsimko
Copy link
Member Author

All done WRT Bravado and Celery. Did not touch Flask or SQLAlchemy.

tiborsimko added a commit to tiborsimko/reana-client that referenced this issue Sep 6, 2018
* Introduces more upper boundary limits for dependencies to make sure all the
  client components are installable from PyPI in the future.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-client that referenced this issue Sep 6, 2018
* Introduces more upper boundary limits for dependencies to make sure all the
  client components are installable from PyPI in the future.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
tiborsimko added a commit to tiborsimko/reana-client that referenced this issue Sep 7, 2018
* Introduces more upper boundary limits for dependencies to make sure all the
  client components are installable from PyPI in the future.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
@tiborsimko
Copy link
Member Author

Everything done in respective repositories.

dinosk pushed a commit to dinosk/reana-cluster that referenced this issue Sep 20, 2018
* Introduces upper boundary for REANA-Commons dependency.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
dinosk pushed a commit to dinosk/reana-cluster that referenced this issue Sep 20, 2018
* Introduces more upper boundary limits for dependencies to make sure all the
  client components are installable from PyPI in the future.
  (addresses reanahub/reana#80)

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
@tiborsimko tiborsimko added this to Done in v0.3.0 Oct 7, 2019
mdonadoni added a commit to mdonadoni/reana that referenced this issue Mar 5, 2024
chore(reana-ui/master): release 0.9.4

build(reana-ui/package): update yarn.lock (reanahub#399)

build(reana-ui/package): require jsroot<7.6.0 (reanahub#399)

ci(reana-ui/commitlint): allow release commit style (reanahub#400)

docs(reana-ui/authors): complete list of contributors (reanahub#396)

ci(reana-ui/shellcheck): exclude node_modules from the analyzed paths (reanahub#387)

fix(reana-ui/progress): update failed workflows duration using finish time (reanahub#387)

feat(reana-ui/footer): link privacy notice to configured URL (reanahub#393)

refactor(reana-ui/docs): move from reST to Markdown (reanahub#391)

ci(reana-ui/commitlint): check for the presence of concrete PR number (reanahub#390)

ci(reana-ui/shellcheck): fix exit code propagation (reanahub#390)

fix(reana-ui/launcher): remove dollar sign in generated Markdown (reanahub#389)

ci(reana-ui/release-please): update version in package.json and Dockerfile (reanahub#385)

ci(reana-ui/release-please): switch to `simple` release strategy (reanahub#383)

fix(reana-ui/router): show 404 page for invalid URLs (reanahub#382)

ci(reana-ui/release-please): initial configuration (reanahub#380)

ci(reana-ui/commitlint): addition of commit message linter (reanahub#380)

chore(reana-message-broker/master): release 0.9.3

ci(reana-message-broker/commitlint): allow release commit style (reanahub#67)

docs(reana-message-broker/authors): complete list of contributors (reanahub#66)

refactor(reana-message-broker/docs): move from reST to Markdown (reanahub#65)

ci(reana-message-broker/commitlint): check for the presence of concrete PR number (reanahub#64)

ci(reana-message-broker/shellcheck): fix exit code propagation (reanahub#64)

ci(reana-message-broker/release-please): update version in Dockerfile (reanahub#63)

fix(reana-message-broker/startup): handle signals for graceful shutdown (reanahub#59)

ci(reana-message-broker/release-please): initial configuration (reanahub#60)

ci(reana-message-broker/commitlint): addition of commit message linter (reanahub#60)

chore(reana-server/master): release 0.9.3

build(reana-server/python): bump all required packages as of 2024-03-04 (reanahub#674)

build(reana-server/python): bump shared REANA packages as of 2024-03-04 (reanahub#674)

build(reana-server/python): bump shared modules (reanahub#676)

ci(reana-server/commitlint): allow release commit style (reanahub#675)

docs(reana-server/authors): complete list of contributors (reanahub#673)

ci(reana-server/pytest): move to PostgreSQL 14.10 (reanahub#672)

refactor(reana-server/docs): move from reST to Markdown (reanahub#671)

style(reana-server/black): format with black v24 (reanahub#670)

ci(reana-server/commitlint): check for the presence of concrete PR number (reanahub#669)

ci(reana-server/shellcheck): fix exit code propagation (reanahub#669)

ci(reana-server/release-please): update version in Dockerfile/OpenAPI specs (reanahub#668)

build(reana-server/docker): non-editable submodules in "latest" mode (reanahub#656)

build(reana-server/deps): pin invenio-userprofiles to 1.2.4 (reanahub#665)

ci(reana-server/release-please): initial configuration (reanahub#665)

ci(reana-server/commitlint): addition of commit message linter (reanahub#665)

chore(reana-workflow-controller/master): release 0.9.3

build(reana-workflow-controller/python): bump all required packages as of 2024-03-04 (reanahub#574)

build(reana-workflow-controller/python): bump shared REANA packages as of 2024-03-04 (reanahub#574)

feat(reana-workflow-controller/manager): increase termination period of run-batch pods (reanahub#572)

ci(reana-workflow-controller/commitlint): allow release commit style (reanahub#575)

feat(reana-workflow-controller/manager): pass custom env variables to job controller (reanahub#571)

feat(reana-workflow-controller/manager): pass custom env variables to workflow engines (reanahub#571)

docs(reana-workflow-controller/authors): complete list of contributors (reanahub#570)

ci(reana-workflow-controller/pytest): move to PostgreSQL 14.10 (reanahub#568)

fix(reana-workflow-controller/manager): use valid group name when calling `groupadd` (reanahub#566)

refactor(reana-workflow-controller/docs): move from reST to Markdown (reanahub#567)

fix(reana-workflow-controller/stop): store engine logs of stopped workflow (reanahub#563)

fix(reana-workflow-controller/manager): graceful shutdown of job-controller (reanahub#559)

feat(reana-workflow-controller/manager): call shutdown endpoint before workflow stop (reanahub#559)

refactor(reana-workflow-controller/consumer): do not update status of jobs (reanahub#559)

style(reana-workflow-controller/black): format with black v24 (reanahub#564)

ci(reana-workflow-controller/commitlint): check for the presence of concrete PR number (reanahub#562)

ci(reana-workflow-controller/shellcheck): fix exit code propagation (reanahub#562)

ci(reana-workflow-controller/release-please): update version in Dockerfile/OpenAPI specs (reanahub#558)

build(reana-workflow-controller/docker): non-editable submodules in "latest" mode (reanahub#551)

ci(reana-workflow-controller/release-please): initial configuration (reanahub#555)

ci(reana-workflow-controller/commitlint): addition of commit message linter (reanahub#555)

chore(reana-job-controller/master): release 0.9.3

build(reana-job-controller/python): bump all required packages as of 2024-03-04 (reanahub#442)

build(reana-job-controller/python): bump shared REANA packages as of 2024-03-04 (reanahub#442)

ci(reana-job-controller/commitlint): allow release commit style (reanahub#443)

build(reana-job-controller/certificates): update expired CERN Grid CA certificate (reanahub#440)

fix(reana-job-controller/database): limit the number of open database connections (reanahub#437)

docs(reana-job-controller/authors): complete list of contributors (reanahub#434)

perf(reana-job-controller/cache): avoid caching jobs when the cache is disabled (reanahub#435)

ci(reana-job-controller/pytest): move to PostgreSQL 14.10 (reanahub#429)

refactor(reana-job-controller/docs): move from reST to Markdown (reanahub#428)

ci(reana-job-controller/commitlint): check for the presence of concrete PR number (reanahub#425)

ci(reana-job-controller/shellcheck): fix exit code propagation (reanahub#425)

feat(reana-job-controller/shutdown): stop all running jobs before stopping workflow (reanahub#423)

refactor(reana-job-controller/monitor): move fetching of logs to job-manager (reanahub#423)

refactor(reana-job-controller/db): set job status also in the main database (reanahub#423)

refactor(reana-job-controller/monitor): centralise logs and status updates (reanahub#423)

style(reana-job-controller/black): format with black v24 (reanahub#426)

ci(reana-job-controller/release-please): update version in Dockerfile/OpenAPI specs (reanahub#421)

build(reana-job-controller/docker): non-editable submodules in "latest" mode (reanahub#416)

ci(reana-job-controller/release-please): initial configuration (reanahub#417)

ci(reana-job-controller/commitlint): addition of commit message linter (reanahub#417)

chore(reana-workflow-engine-cwl/master): release 0.9.3

build(reana-workflow-engine-cwl/python): bump all required packages as of 2024-03-04 (reanahub#267)

build(reana-workflow-engine-cwl/python): bump shared REANA packages as of 2024-03-04 (reanahub#267)

docs(reana-workflow-engine-cwl/conformance-tests): update CWL conformance test badges (reanahub#264)

ci(reana-workflow-engine-cwl/commitlint): allow release commit style (reanahub#268)

docs(reana-workflow-engine-cwl/authors): complete list of contributors (reanahub#266)

refactor(reana-workflow-engine-cwl/docs): move from reST to Markdown (reanahub#263)

fix(reana-workflow-engine-cwl/progress): handle stopped jobs (reanahub#260)

ci(reana-workflow-engine-cwl/commitlint): check for the presence of concrete PR number (reanahub#262)

ci(reana-workflow-engine-cwl/shellcheck): fix exit code propagation (reanahub#262)

build(reana-workflow-engine-cwl/docker): install correct extras of reana-commons submodule (reanahub#261)

ci(reana-workflow-engine-cwl/release-please): update version in Dockerfile (reanahub#259)

build(reana-workflow-engine-cwl/docker): non-editable submodules in "latest" mode (reanahub#255)

ci(reana-workflow-engine-cwl/release-please): initial configuration (reanahub#256)

ci(reana-workflow-engine-cwl/commitlint): addition of commit message linter (reanahub#256)

chore(reana-workflow-engine-serial/master): release 0.9.3

build(reana-workflow-engine-serial/python): bump all required packages as of 2024-03-04 (reanahub#200)

build(reana-workflow-engine-serial/python): bump shared REANA packages as of 2024-03-04 (reanahub#200)

ci(reana-workflow-engine-serial/commitlint): allow release commit style (reanahub#201)

docs(reana-workflow-engine-serial/authors): complete list of contributors (reanahub#199)

refactor(reana-workflow-engine-serial/docs): move from reST to Markdown (reanahub#198)

fix(reana-workflow-engine-serial/progress): handle stopped jobs (reanahub#195)

ci(reana-workflow-engine-serial/commitlint): check for the presence of concrete PR number (reanahub#197)

ci(reana-workflow-engine-serial/shellcheck): fix exit code propagation (reanahub#197)

build(reana-workflow-engine-serial/docker): install correct extras of reana-commons submodule (reanahub#196)

ci(reana-workflow-engine-serial/release-please): update version in Dockerfile (reanahub#194)

build(reana-workflow-engine-serial/docker): non-editable submodules in "latest" mode (reanahub#190)

ci(reana-workflow-engine-serial/release-please): initial configuration (reanahub#191)

ci(reana-workflow-engine-serial/commitlint): addition of commit message linter (reanahub#191)

chore(reana-workflow-engine-yadage/master): release 0.9.4

build(reana-workflow-engine-yadage/python): bump all required packages as of 2024-03-04 (reanahub#261)

build(reana-workflow-engine-yadage/python): bump shared REANA packages as of 2024-03-04 (reanahub#261)

ci(reana-workflow-engine-yadage/commitlint): allow release commit style (reanahub#262)

docs(reana-workflow-engine-yadage/authors): complete list of contributors (reanahub#260)

refactor(reana-workflow-engine-yadage/docs): move from reST to Markdown (reanahub#259)

fix(reana-workflow-engine-yadage/progress): correctly handle running and stopped jobs (reanahub#258)

ci(reana-workflow-engine-yadage/commitlint): check for the presence of concrete PR number (reanahub#257)

ci(reana-workflow-engine-yadage/shellcheck): fix exit code propagation (reanahub#257)

build(reana-workflow-engine-yadage/docker): install correct extras of reana-commons submodule (reanahub#256)

ci(reana-workflow-engine-yadage/release-please): update version in Dockerfile (reanahub#254)

build(reana-workflow-engine-yadage/docker): non-editable submodules in "latest" mode (reanahub#249)

ci(reana-workflow-engine-yadage/release-please): initial configuration (reanahub#251)

ci(reana-workflow-engine-yadage/commitlint): addition of commit message linter (reanahub#251)

chore(reana-workflow-engine-snakemake/master): release 0.9.3

build(reana-workflow-engine-snakemake/python): bump all required packages as of 2024-03-04 (reanahub#85)

build(reana-workflow-engine-snakemake/python): bump shared REANA packages as of 2024-03-04 (reanahub#85)

ci(reana-workflow-engine-snakemake/commitlint): allow release commit style (reanahub#86)

feat(reana-workflow-engine-snakemake/config): get max number of parallel jobs from env vars (reanahub#84)

feat(reana-workflow-engine-snakemake/executor): upgrade to Snakemake v7.32.4 (reanahub#81)

docs(reana-workflow-engine-snakemake/authors): complete list of contributors (reanahub#83)

refactor(reana-workflow-engine-snakemake/docs): move from reST to Markdown (reanahub#82)

fix(reana-workflow-engine-snakemake/progress): handle stopped jobs (reanahub#78)

ci(reana-workflow-engine-snakemake/commitlint): check for the presence of concrete PR number (reanahub#80)

ci(reana-workflow-engine-snakemake/shellcheck): fix exit code propagation (reanahub#80)

build(reana-workflow-engine-snakemake/docker): install correct extras of reana-commons submodule (reanahub#79)

ci(reana-workflow-engine-snakemake/release-please): update version in Dockerfile (reanahub#77)

build(reana-workflow-engine-snakemake/docker): non-editable submodules in "latest" mode (reanahub#73)

ci(reana-workflow-engine-snakemake/release-please): initial configuration (reanahub#74)

ci(reana-workflow-engine-snakemake/commitlint): addition of commit message linter (reanahub#74)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
v0.3.0
  
Done
Development

No branches or pull requests

2 participants