Update dependencies #2258

jburel · 2022-06-30T06:33:56Z

Update dependencies as part of the daily run of GHA
Update model object

The proposed changes can be tested by running bash omero/dependencies.sh (jq and omego need to be installed)
This will avoid the manual upgrade of the dependencies
I have renamed 3 variables to make the parsing easier and avoid unnecessary if statement

snoopycrimecop · 2022-07-01T08:31:18Z

Conflicting PR. Removed from build OMERO-docs#1273. See the console output for more details.
Possible conflicts:

Upstream changes
- omero/autogen_db_version.py

~~--conflicts~~ Conflict resolved in build OMERO-docs#1274. See the console output for more details.

jburel · 2022-07-01T08:35:19Z

I have been thinking about it a bit more, I need to find a better way to handle the links of openmicroscopy dependencies stack. the chosen approach is making some assumptions that might lead to error

sbesson

As part of the review, I found omero/autogen_docs and the sub-scripts extremely hard to read and understand. The top-level script already has a lot of logic to pull and combine data from different source and this PR is adding extra steps on top of that. Given this experience, this raises the question of its maintainability as the reviewing burden is substantial (having not even executed the script yet).

One outstanding issue is about the version of the server which should be downloaded/built consistently across the script.

Trying to think how to address the larger question, I wondered whether splitting the uber-script into separate update scripts/workflows per source e.g. omero-install, omeroweb-install, openmicroscopy would help the review and the execution.

sbesson · 2022-07-06T08:52:48Z

.github/workflows/update.yml

@@ -31,7 +31,7 @@ jobs:
      - name: Update documentation
        run: |
          python -m venv $space/.venv3
-          $space/.venv3/bin/pip install -U jinja2==3.0.1
+          $space/.venv3/bin/pip install -U jinja2==3.0.1 omego


Why this requirement? (edit: spotted a typo in the dependencies.sh script)

I was using omego to download but this will be removed when using head

sbesson · 2022-07-06T11:39:50Z

omero/dependencies.sh

+    dirs=("OMERO.server/lib/server/omero-blitz.jar" "OMERO.server/lib/server/omero-server.jar" "OMERO.server/lib/server/omero-gateway.jar"
+      "OMERO.server/lib/server/omero-romio.jar" "OMERO.server/lib/server/omero-renderer.jar" "OMERO.server/lib/server/omero-common.jar"
+      "OMERO.server/lib/server/omero-model.jar" "OMERO.server/lib/server/formats-gpl.jar")
+    omero download --release 5


sbesson · 2022-07-06T11:41:32Z

omero/dependencies.sh

+        version=`unzip -p $dir META-INF/MANIFEST.MF | grep "Implementation-Version:" | sed 's/^.*[^0-9]\([0-9]*\.[0-9]*\.[0-9]*\).*$/\1/'`
+        echo $v
+        echo $version
+        sed -i -e "s/version_${v} = .*/version_${v} = \"${version}\"/" omero/onf_autogen.py


I assume it's omero/conf_autogen.py

sbesson · 2022-07-06T11:45:40Z

omero/autogen_docs

@@ -14,6 +14,11 @@ WORKSPACE=${WORKSPACE:-$(pwd)}
 WORKSPACE=${WORKSPACE%/}  # Remove trailing slashes
 USER=${USER:-$(whoami)}

+echo "Retrieve model object"
+URL="https://latest-ci.openmicroscopy.org/jenkins/job/OMERO-server/lastSuccessfulBuild/artifact/EveryObject.rst"


I am lost on which OMERO.server version sources are being used:

the GitHub action checks out and builds the HEAD of ome/openmicroscopy

this step downloads an artifact generated by latest-ci (i.e. assuming HEAD)

omero dependencies downloads a released server and updates the versions

Shouldn't all of this use the same version?

I could use head. I was testing independently and could now reconcile the 2

sbesson · 2022-07-06T11:47:35Z

omero/autogen_docs

@@ -86,6 +91,10 @@ done
 echo "Getting db properties"
 omero/autogen_db_version.py $WORKSPACE/OMERO.server > omero/conf_autogen.py

+echo "Update dependencies"
+bash omero/dependencies.sh


The purpose of this script seems to be largely identical to omero/autogen_db_version.py above except it's in Bash vs Python and the new script downloads the latest release server vs using OMERO.server.
What was the rationale for not amending the existing logic?

It is all a bit messy since we have bash script invoking python script. I opted for a bash script since the source scrip is a bash one

sbesson · 2022-07-06T11:48:25Z

omero/dependencies.sh

+        value=${values[${#values[@]}-1]}
+        v=${value#"$PREFIX"}
+        v=${v//"-"/"_"}
+        echo $v


Are these debugging statements?

Helpful to have to follow the process

jburel · 2022-07-06T13:44:47Z

I will split the original script (I am not the author) and make it more readable and easier to review

jburel requested a review from sbesson June 30, 2022 06:33

jburel added 6 commits July 1, 2022 11:01

add script to update dependencies as part of daily run

ee72685

rename component to ease parsing

0eb310d

extract dependencies version bump

fdebda9

remove duplicate line

8da6d25

update dependencies if a new version of openmicroscopy is available

088af8b

remove variable not used

4cfea05

jburel changed the title ~~Update dependencies~~ WIP: Update dependencies Jul 1, 2022

jburel mentioned this pull request Jul 2, 2022

Artifacts ome/omego#132

Open

get version from jar

d7ffef6

jburel changed the title ~~WIP: Update dependencies~~ Update dependencies Jul 4, 2022

jburel mentioned this pull request Jul 4, 2022

Release process review ome/openmicroscopy#6330

Open

jburel added 2 commits July 4, 2022 10:53

use omego to download the artifacts

c075f21

install omego

5afc192

jburel mentioned this pull request Jul 4, 2022

Simplify components versions files duplication #2254

Open

jburel added 2 commits July 4, 2022 21:56

fix path

ca4f52c

update EveryObject

c756aa7

sbesson requested changes Jul 6, 2022

View reviewed changes

jburel mentioned this pull request Jul 7, 2022

Auto update #2261

Merged

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update dependencies #2258

Update dependencies #2258

jburel commented Jun 30, 2022 •

edited

Loading

snoopycrimecop commented Jul 1, 2022 •

edited

Loading

jburel commented Jul 1, 2022 •

edited

Loading

sbesson left a comment

sbesson Jul 6, 2022 •

edited

Loading

jburel Jul 6, 2022

sbesson Jul 6, 2022

sbesson Jul 6, 2022 •

edited

Loading

jburel Jul 6, 2022

sbesson Jul 6, 2022 •

edited

Loading

jburel Jul 6, 2022

sbesson Jul 6, 2022 •

edited

Loading

jburel Jul 6, 2022

sbesson Jul 6, 2022

jburel Jul 6, 2022

jburel commented Jul 6, 2022

Update dependencies #2258

Update dependencies #2258

Conversation

jburel commented Jun 30, 2022 • edited Loading

snoopycrimecop commented Jul 1, 2022 • edited Loading

jburel commented Jul 1, 2022 • edited Loading

sbesson left a comment

Choose a reason for hiding this comment

sbesson Jul 6, 2022 • edited Loading

Choose a reason for hiding this comment

jburel Jul 6, 2022

Choose a reason for hiding this comment

sbesson Jul 6, 2022

Choose a reason for hiding this comment

sbesson Jul 6, 2022 • edited Loading

Choose a reason for hiding this comment

jburel Jul 6, 2022

Choose a reason for hiding this comment

sbesson Jul 6, 2022 • edited Loading

Choose a reason for hiding this comment

jburel Jul 6, 2022

Choose a reason for hiding this comment

sbesson Jul 6, 2022 • edited Loading

Choose a reason for hiding this comment

jburel Jul 6, 2022

Choose a reason for hiding this comment

sbesson Jul 6, 2022

Choose a reason for hiding this comment

jburel Jul 6, 2022

Choose a reason for hiding this comment

jburel commented Jul 6, 2022

jburel commented Jun 30, 2022 •

edited

Loading

snoopycrimecop commented Jul 1, 2022 •

edited

Loading

jburel commented Jul 1, 2022 •

edited

Loading

sbesson Jul 6, 2022 •

edited

Loading

sbesson Jul 6, 2022 •

edited

Loading

sbesson Jul 6, 2022 •

edited

Loading

sbesson Jul 6, 2022 •

edited

Loading