Fix disapearing Thredds docker image 4.6.18 #313

mishaschwartz · 2023-04-11T16:44:29Z

Overview

The current docker image version for thredds (4.6.18) is no longer hosted in the Unidata docker repository.

Pushed the same image from Ouranos production to PAVICS DockerHub, restoring
the missing Thredds image.

Discovered that Unidata is also not keeping their tag immutable, like Kartoza Geoserver image.

So Ouranos tag has the approximate month of Unidata re-release of 4.6.18.

On our production server:

$ docker images |grep thredds|grep 4.6.18
unidata/thredds-docker              4.6.18              25997a1b2893   15 months ago   5.63GB

On our staging server:

$ docker images |grep thredds | grep 4.6.18
unidata/thredds-docker              4.6.18              09103737360a   16 months ago   5.62GB

Changes

Non-breaking changes

Restore the same Thredds docker image as the one on Ouranos production

Related Issue / Discussion

Additional Information

crim-jenkins-bot · 2023-04-11T16:44:33Z

Can one of the admins verify this patch?

crim-jenkins-bot · 2023-04-11T17:05:49Z

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1437/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : update-thredds
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-118.rdext.crim.ca

Infrastructure deployment failed. Instance has not been destroyed. @matprov

mishaschwartz · 2023-04-11T17:19:19Z

@matprov

The current docker version running on jenkins does not support the updated tomcat image used by this version of thredds.

Current version is 18.09.2 (see image below) and I believe this breaks prior to version 20 (docker-library/tomcat#269).

Can we update the Jenkins build? If this is something that I can do can you please remind me where to do so?

tlvu · 2023-04-11T18:30:40Z

@mishaschwartz Quick fix, since we do not have access to the Jenkins host, is to push the current thredds 4.6.18 to pavics dockerhub, so we basically "restore" the same binary.

Safer too since I do not have to test upgrade Thredds. We have quite a large quantity of .nc and .ncml files so I'd rather not take a chance.

Will push to this branch, unless you prefer me to PR on top of this PR?

mishaschwartz · 2023-04-11T18:32:47Z

@tlvu please feel free to push to this branch. Thanks!

tlvu

My own change looks good to me LOL :D

I also updated the PR description to match the new CHANGES.md.

tlvu · 2023-04-11T19:13:11Z

@mishaschwartz Wait for a working Jenkins before merging this.

Given I discovered Thredds image tag are not immutable, I think we should also cache the next upgrade to pavics DockerHub to ensure reproductibility.

mishaschwartz · 2023-04-11T19:18:15Z

@tlvu

Given I discovered Thredds image tag are not immutable, I think we should also cache the next upgrade to pavics DockerHub to ensure reproductibility.

Oh good catch! Ok I'll wait for the tests and then I'll merge

crim-jenkins-bot · 2023-04-11T19:24:40Z

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1438/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : update-thredds
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-36.rdext.crim.ca

Infrastructure deployment failed. Instance has not been destroyed. @matprov

mishaschwartz · 2023-04-11T19:51:24Z

run tests

crim-jenkins-bot · 2023-04-11T20:11:59Z

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1439/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : update-thredds
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-216.rdext.crim.ca

Infrastructure deployment failed. Instance has not been destroyed. @matprov

tlvu · 2023-04-11T20:17:47Z

This is very weird, let me do a manual check. Maybe fat finger I tagged the wrong image ...

tlvu · 2023-04-11T20:23:10Z

The PR starts fine on my test VM. I think you can merge this.

tlvu · 2023-04-11T20:33:09Z

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1439/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : update-thredds
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-216.rdext.crim.ca

Infrastructure deployment failed. Instance has not been destroyed. @matprov

@matprov FYI, it seems the instance did not start.

[CHECK] Waiting for canarie-api [https://host-140-216.rdext.crim.ca/canarie/node/service/status]...
....................................................................................................................................................................................................................................
[WARNING] Timeout of [1200] seconds reached. Exiting, but leaving the instance running.

$ curl  --include https://host-140-216.rdext.crim.ca/
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to host-140-216.rdext.crim.ca:443 

$ curl  --include http://host-140-216.rdext.crim.ca/
curl: (56) Recv failure: Connection reset by peer

# just to have the IP
$ ping host-140-216.rdext.crim.ca
PING host-140-216.rdext.crim.ca (132.217.140.216) 56(84) bytes of data.
# ping no response as well

fmigneault · 2023-04-11T21:59:37Z

Can 4.6.20 be used instead?
Still a good idea to cache them on our side though to avoid disappearing tags.

tlvu · 2023-04-11T22:08:54Z

Can 4.6.20 be used instead?

I think no, see #313 (comment)

fmigneault · 2023-04-12T17:01:21Z

@mishaschwartz @tlvu

Unfortunately, @matprov is on vacation until April 25.
I'm not too aware of the Jenkins configuration and setup, but I think the docker engine is predefined and static on the E2E instance. I think updating it would require more manual steps to make sure any relevant plugins will still work with it.

According to 4.6.20 build: ENV TOMCAT_VERSION=8.5.87

I checked on an instance that still has THREDDS 4.6.18 running, and the tomcat instance is running version 8.5.73.0.
It seems close enough to not require moving to 5.x versions in the meantime.

tlvu · 2023-04-12T17:23:29Z

I think the docker engine is predefined and static on the E2E instance. I think updating it would require more manual steps to make sure any relevant plugins will still work with it.

@fmigneault
You're right! Let's not feature creep this PR with upgrading the docker engine which might need Jenkins and all its plugins upgraded as well (upgrade for the docker plugin as least).

As for Thredds 4.6.20, from what @mishaschwartz found out, it might need a new docker engine. I find this pretty weird but I don't have time to investigate.

This disappering Thredds image tag is breaking all the pipeline now so I think we should put off the fire first and schedule an upgrade later when @matprov is back.

mishaschwartz · 2023-04-12T22:40:06Z

@tlvu just confirming based on the discussion above that the current docker image the one we want to stick with for now? (pavics/thredds-docker:4.6.18-unidata-2022-01)

If so I can merge this in.

tlvu · 2023-04-12T22:46:49Z

@tlvu just confirming based on the discussion above that the current docker image the one we want to stick with for now? (pavics/thredds-docker:4.6.18-unidata-2022-01)

If so I can merge this in.

Exact, merge party for you today :D

crim-jenkins-bot · 2023-04-12T23:12:02Z

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/1448/
Result : failure

BIRDHOUSE_DEPLOY_BRANCH : update-thredds
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-69.rdext.crim.ca

Infrastructure deployment failed. Instance has not been destroyed. @matprov

update thredds

deec868

mishaschwartz mentioned this pull request Apr 11, 2023

fix CanarieAPI config to support Python 3.x #284

Merged

thredds: restore disappearing 4.6.18 from Ouranos production

6060a82

tlvu changed the title ~~Update Thredds docker image to 4.6.20~~ Fix disapearing Thredds docker image 4.6.18 Apr 11, 2023

tlvu approved these changes Apr 11, 2023

View reviewed changes

mishaschwartz added 2 commits April 12, 2023 18:49

Merge branch 'master' into update-thredds

5b4894a

Bump version: 1.25.4 → 1.25.5

2d28dc2

github-actions bot added ci/operations Continuous Integration components documentation Improvements or additions to documentation labels Apr 12, 2023

mishaschwartz merged commit b094e16 into master Apr 12, 2023

mishaschwartz deleted the update-thredds branch April 12, 2023 22:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix disapearing Thredds docker image 4.6.18 #313

Fix disapearing Thredds docker image 4.6.18 #313

mishaschwartz commented Apr 11, 2023 •

edited by tlvu

Loading

crim-jenkins-bot commented Apr 11, 2023

crim-jenkins-bot commented Apr 11, 2023

mishaschwartz commented Apr 11, 2023

tlvu commented Apr 11, 2023

mishaschwartz commented Apr 11, 2023

tlvu left a comment

tlvu commented Apr 11, 2023

mishaschwartz commented Apr 11, 2023

crim-jenkins-bot commented Apr 11, 2023

mishaschwartz commented Apr 11, 2023

crim-jenkins-bot commented Apr 11, 2023

tlvu commented Apr 11, 2023

tlvu commented Apr 11, 2023

tlvu commented Apr 11, 2023

E2E Test Results

DACCS-iac Pipeline Results

fmigneault commented Apr 11, 2023

tlvu commented Apr 11, 2023

fmigneault commented Apr 12, 2023

tlvu commented Apr 12, 2023

mishaschwartz commented Apr 12, 2023

tlvu commented Apr 12, 2023

crim-jenkins-bot commented Apr 12, 2023

Fix disapearing Thredds docker image 4.6.18 #313

Fix disapearing Thredds docker image 4.6.18 #313

Conversation

mishaschwartz commented Apr 11, 2023 • edited by tlvu Loading

Overview

Changes

Related Issue / Discussion

Additional Information

crim-jenkins-bot commented Apr 11, 2023

crim-jenkins-bot commented Apr 11, 2023

E2E Test Results

DACCS-iac Pipeline Results

mishaschwartz commented Apr 11, 2023

tlvu commented Apr 11, 2023

mishaschwartz commented Apr 11, 2023

tlvu left a comment

Choose a reason for hiding this comment

tlvu commented Apr 11, 2023

mishaschwartz commented Apr 11, 2023

crim-jenkins-bot commented Apr 11, 2023

E2E Test Results

DACCS-iac Pipeline Results

mishaschwartz commented Apr 11, 2023

crim-jenkins-bot commented Apr 11, 2023

E2E Test Results

DACCS-iac Pipeline Results

tlvu commented Apr 11, 2023

tlvu commented Apr 11, 2023

tlvu commented Apr 11, 2023

E2E Test Results

DACCS-iac Pipeline Results

fmigneault commented Apr 11, 2023

tlvu commented Apr 11, 2023

fmigneault commented Apr 12, 2023

tlvu commented Apr 12, 2023

mishaschwartz commented Apr 12, 2023

tlvu commented Apr 12, 2023

crim-jenkins-bot commented Apr 12, 2023

E2E Test Results

DACCS-iac Pipeline Results

mishaschwartz commented Apr 11, 2023 •

edited by tlvu

Loading