SRE-3104 ci: Use DAOS specific HTTPS_PROXY variable #470

grom72 · 2025-06-11T08:42:24Z

The HTTPS_PROXY variable set in Jenkins may force any shell script to use a proxy server.
This is not expected, as we want to avoid using a proxy unless it is explicitly needed by the DAOS tests or the build process.

The HTTPS_PROXY variable set in Jenkins may force any shell script to use a proxy server. This is not expected, as we want to avoid using a proxy unless it is explicitly needed by the DAOS tests or the build process. Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

daosbuild3 · 2025-06-11T08:44:54Z

Test stage provisionNodes on EL 9 with master Repo completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/1/display/redirect

daosbuild3 · 2025-06-11T08:44:55Z

Test stage provisionNodes on EL 8 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/1/display/redirect

daosbuild3 · 2025-06-11T08:44:56Z

Test stage provisionNodes on Leap 15 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/1/display/redirect

daosbuild3 · 2025-06-11T08:47:11Z

Test stage provisionNodes on EL 8 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/2/display/redirect

daosbuild3 · 2025-06-11T08:47:27Z

Test stage provisionNodes on EL 9 with master Repo completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/2/display/redirect

daosbuild3 · 2025-06-11T08:48:02Z

Test stage provisionNodes on Leap 15 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/2/display/redirect

Pass HTTPS_PROXY to docekr build command Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

daosbuild3 · 2025-06-11T09:34:30Z

Test stage provisionNodes on EL 9 with master Repo completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/3/display/redirect

daosbuild3 · 2025-06-11T09:34:37Z

Test stage provisionNodes on EL 8 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/3/display/redirect

daosbuild3 · 2025-06-11T09:35:09Z

Test stage provisionNodes on Leap 15 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/3/display/redirect

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

daosbuild3 · 2025-06-11T09:45:59Z

Test stage provisionNodes on EL 9 with master Repo completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/4/display/redirect

daosbuild3 · 2025-06-11T09:46:08Z

Test stage provisionNodes on EL 8 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/4/display/redirect

daosbuild3 · 2025-06-11T09:46:45Z

Test stage provisionNodes on Leap 15 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/4/display/redirect

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

daosbuild3 · 2025-06-11T11:48:17Z

Test stage provisionNodes on EL 8 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/5/display/redirect

daosbuild3 · 2025-06-11T11:48:18Z

Test stage provisionNodes on EL 9 with master Repo completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/5/display/redirect

daosbuild3 · 2025-06-11T11:48:20Z

Test stage provisionNodes on Leap 15 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/5/display/redirect

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

daosbuild3 · 2025-06-11T14:01:35Z

Test stage provisionNodes on EL 9 with master Repo completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/6/display/redirect

daosbuild3 · 2025-06-11T14:01:43Z

Test stage provisionNodes on EL 8 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/6/display/redirect

daosbuild3 · 2025-06-11T14:02:24Z

Test stage provisionNodes on Leap 15 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/6/display/redirect

JohnMalmberg

I think this is good enough, but at some time we will want to take a second pass and cleanup a few nits.

JohnMalmberg · 2025-06-11T14:07:43Z

vars/buildRpm.groovy

+/*
+ * Copyright (C) 2020-2024 Intel Corporation
+ * Copyright (C) 2025 Hewlett Packard Enterprise Development LP
+ */


The (C) actually should not be in Copyright messages. It has no legal meaning, and we should only have it in places where we have to match a known boilerplate that has it.

I have been removing '(C)' when I update copyright lines in code modules.

JohnMalmberg · 2025-06-11T14:08:14Z

vars/dockerBuildArgs.groovy

 // vars/dockerBuildArgs.groovy
+/*
+ * Copyright (C) 2020-2024 Intel Corporation
+ * Copyright (C) 2025 Hewlett Packard Enterprise Development LP


JohnMalmberg · 2025-06-11T14:08:27Z

vars/functionalTest.groovy

+ * Copyright (C) 2020-2024 Intel Corporation
+ * Copyright (C) 2025 Hewlett Packard Enterprise Development LP


JohnMalmberg · 2025-06-11T14:12:51Z

vars/functionalTestPostV2.groovy

+    if (env.DAOS_HTTPS_PROXY) {
+        script += ' --proxy "' + env.DAOS_HTTPS_PROXY + '"'
+    }
+    script += ' --user --upgrade launchable~=1.0'


The pip program searches a number of locations for a config file name pip.conf or something similar.
When we setup a docker container or a system to use pip, we should be updating that file instead of adding --proxy to the pip command.

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

daosbuild3 · 2025-06-11T18:40:44Z

Test stage provisionNodes on EL 8 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/9/display/redirect

daosbuild3 · 2025-06-11T18:40:45Z

Test stage provisionNodes on EL 9 with master Repo completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/9/display/redirect

daosbuild3 · 2025-06-11T18:41:18Z

Test stage provisionNodes on Leap 15 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/9/display/redirect

daosbuild3 · 2025-06-11T18:54:14Z

Test stage provisionNodes on EL 8 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/10/display/redirect

daosbuild3 · 2025-06-11T18:54:15Z

Test stage provisionNodes on EL 9 with master Repo completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/10/display/redirect

daosbuild3 · 2025-06-11T18:54:41Z

Test stage provisionNodes on Leap 15 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/10/display/redirect

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

daosbuild3 · 2025-06-12T06:26:39Z

Test stage provisionNodes on EL 9 with master Repo completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/11/display/redirect

daosbuild3 · 2025-06-12T06:26:49Z

Test stage provisionNodes on EL 8 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/11/display/redirect

daosbuild3 · 2025-06-12T06:27:44Z

Test stage provisionNodes on Leap 15 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/11/display/redirect

tanabarr

It would be nice to create a groovy helper to perform the proxy checks rather than duplicating but otherwise LGTM.

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

daosbuild3 · 2025-06-13T06:39:20Z

Test stage provisionNodes on EL 9 with master Repo completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/12/display/redirect

daosbuild3 · 2025-06-13T06:39:27Z

Test stage provisionNodes on EL 8 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/12/display/redirect

daosbuild3 · 2025-06-13T06:40:07Z

Test stage provisionNodes on Leap 15 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/12/display/redirect

daosbuild3 · 2025-06-13T07:30:51Z

Test stage provisionNodes on EL 9 with master Repo completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/13/display/redirect

daosbuild3 · 2025-06-13T07:30:58Z

Test stage provisionNodes on EL 8 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/13/display/redirect

daosbuild3 · 2025-06-13T07:31:37Z

Test stage provisionNodes on Leap 15 with slurm completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/pipeline-lib/job/PR-470/13/display/redirect

grom72 added 3 commits June 11, 2025 10:36

Temporary add debug info

51c056c

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

Pass HTTPS_PROXY explicitly to the build script.

a4b2129

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

grom72 requested a review from JohnMalmberg June 11, 2025 08:42

Custom library for testing

cb15027

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

Pass HTTPS_PROXY to post_provision_config.sh

e1fd125

Pass HTTPS_PROXY to docekr build command Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

pip3 install requires proxy to access internet

f4e2615

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

pip3 install requires explicity pass proxy parameter

9a9ca51

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

pip3 install requires explicity pass proxy parameter 2nd

453b7b1

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

JohnMalmberg previously approved these changes Jun 11, 2025

View reviewed changes

Remove (C) from copyright text

4650205

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

grom72 dismissed JohnMalmberg’s stale review via 4650205 June 11, 2025 14:23

grom72 requested a review from JohnMalmberg June 11, 2025 14:37

grom72 marked this pull request as ready for review June 11, 2025 14:37

grom72 added the forced-landing The PR has known failures or intentionally reduced testing, but should still be landed label Jun 11, 2025

grom72 requested a review from ryon-jensen June 11, 2025 18:06

grom72 added 2 commits June 11, 2025 20:29

Remove debug code

2173fad

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

Unify the solution and maintain backward compatibility

6df7f78

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

Fix - remove unnecessary

b37fd6f

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

tanabarr previously approved these changes Jun 12, 2025

View reviewed changes

Remove test library

722fa21

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>

grom72 dismissed tanabarr’s stale review via 722fa21 June 13, 2025 06:36

grom72 requested a review from tanabarr June 13, 2025 06:37

tanabarr approved these changes Jun 13, 2025

View reviewed changes

grom72 merged commit 4df343a into master Jun 13, 2025
4 of 10 checks passed

grom72 deleted the grom72/sre-3104 branch June 13, 2025 11:24

grom72 removed the forced-landing The PR has known failures or intentionally reduced testing, but should still be landed label Jul 4, 2025

		* Copyright (C) 2020-2024 Intel Corporation
		* Copyright (C) 2025 Hewlett Packard Enterprise Development LP

SRE-3104 ci: Use DAOS specific HTTPS_PROXY variable #470

SRE-3104 ci: Use DAOS specific HTTPS_PROXY variable #470

Conversation

grom72 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

JohnMalmberg left a comment

Choose a reason for hiding this comment

Uh oh!

JohnMalmberg Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

grom72 Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

JohnMalmberg Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

grom72 Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

JohnMalmberg Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

grom72 Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

JohnMalmberg Jun 11, 2025

Choose a reason for hiding this comment

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 11, 2025

Uh oh!

daosbuild3 commented Jun 12, 2025

Uh oh!

daosbuild3 commented Jun 12, 2025