OCPBUGS-18776: [sig-instrumentation] tests fail due to JSON parsing error. #28320

raptorsun · 2023-10-11T15:40:07Z

Replace shell call of curl with net/http implementation in Prometheus helper functions.
So that timeout requests will not get truncated and reassembled responses.

openshift-ci-robot · 2023-10-11T15:40:15Z

@raptorsun: This pull request references Jira Issue OCPBUGS-18776, which is invalid:

expected the bug to target the "4.15.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Replace shell call of curl with net/http implementation in Prometheus helper functions.
So that timeout requests will not get truncated and reassembled responses.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

raptorsun · 2023-10-11T15:51:39Z

/jira refresh

openshift-ci-robot · 2023-10-11T15:51:47Z

@raptorsun: This pull request references Jira Issue OCPBUGS-18776, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.15.0) matches configured target version for branch (4.15.0)
bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @gpei

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

raptorsun · 2023-10-12T07:17:00Z

/retest

jan--f · 2023-10-12T07:30:54Z

/lgtm

test/extended/util/prometheus/helpers.go

raptorsun · 2023-10-12T19:14:28Z

/retest-required

raptorsun · 2023-10-13T06:52:30Z

/retest

simonpasquier · 2023-10-13T12:20:00Z

test/extended/util/prometheus/helpers.go

+
+	const maxRetries = 2
+	var resp *http.Response = nil
+	for retries := 0; retries < maxRetries; retries++ {


I'd recommend using wait.PollUntilContextTimeout() from the k8s.io/apimachinery/pkg/util/wait package.

The retry loop is replaced by the functionwait.PollUntilContextTimeout(). Meanwhile the timeout limit is increased to 60 seconds with retry interval of 10 seconds.

raptorsun · 2023-10-14T08:33:33Z

/retest

raptorsun · 2023-10-14T11:34:47Z

/retest-required

raptorsun · 2023-10-14T16:11:31Z

/retest

raptorsun · 2023-10-14T19:33:59Z

/retest

test/extended/util/prometheus/helpers.go

raptorsun · 2023-10-31T16:22:05Z

Thanks for the review :D @simonpasquier
The PR has been updated accordingly. The deprecated function wait.PollImmediate is replaced with wait.PollUntilContextTimeout, too.

raptorsun · 2023-11-02T07:08:55Z

/retest-required Le mar. 31 oct. 2023, 19:48, openshift-ci[bot] ***@***.***> a écrit :

…

simonpasquier · 2023-11-02T11:12:28Z

https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/28320/pull-ci-openshift-origin-master-e2e-aws-ovn-fips/1719974869644349440 fails with an error that looks to be triggered by this change:

{  fail [github.com/openshift/origin/test/extended/prometheus/prometheus.go:405]: Unexpected error:
    <*fmt.wrapErrors | 0xc00307e360>: 
    Get "https:/prometheus-k8s-openshift-monitoring.apps.ci-op-wl3xfyw5-3182b.aws-2.ci.openshift.org/api/v1/targets": http: no Host in request URL: %!w(<nil>)
    {
        msg: "Get \"https:/prometheus-k8s-openshift-monitoring.apps.ci-op-wl3xfyw5-3182b.aws-2.ci.openshift.org/api/v1/targets\": http: no Host in request URL: %!w(<nil>)",
        errs: [
            <*url.Error | 0xc00307e330>{
                Op: "Get",
                URL: "https:/prometheus-k8s-openshift-monitoring.apps.ci-op-wl3xfyw5-3182b.aws-2.ci.openshift.org/api/v1/targets",
                Err: <*errors.errorString | 0xc001556190>{
                    s: "http: no Host in request URL",
                },
            },
        ],
    }
occurred
Ginkgo exit error 1: exit with code 1}

simonpasquier · 2023-11-02T11:13:38Z

I suspect that you'll have to create the http.Request in the condition function.

raptorsun · 2023-11-02T16:52:37Z

Thanks for the quick review :D @simonpasquier
The problem comes from the path.Join function, it cut the https://xxx to https:/xxx by eliminating a slash, causing the HTTP request fails.

simonpasquier · 2023-11-03T13:38:50Z

test/extended/util/prometheus/helpers.go

+
+// url.JoinPath but with no error checks
+func JoinUrl(base string, paths ...string) string {
+	path, _ := url.JoinPath(base, paths...)


For safety, it would better to panic if an error is returned (which should never happen)

Suggested change

path, _ := url.JoinPath(base, paths...)

path, err := url.JoinPath(base, paths...)

if err != nil {

panic(err)

}

Right, like a throw() :D

raptorsun · 2023-11-03T18:32:40Z

/retest-required

raptorsun · 2023-11-03T21:29:00Z

/retest

raptorsun · 2023-11-05T20:43:27Z

/retest

raptorsun · 2023-11-08T08:01:41Z

/unhold

test/extended/util/prometheus/helpers.go

… helper functions.

raptorsun · 2023-11-08T11:41:47Z

Thank you for the timely review. @simonpasquier
PR updated accordingly :)

simonpasquier · 2023-11-08T13:36:54Z

/lgtm

openshift-ci · 2023-11-08T13:44:46Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jan--f, raptorsun, simonpasquier

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~test/extended/prometheus/OWNERS~~ [jan--f,simonpasquier]
~~test/extended/util/prometheus/OWNERS~~ [jan--f,simonpasquier]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2023-11-08T13:55:46Z

/retest-required

Remaining retests: 0 against base HEAD 01db63c and 2 for PR HEAD 3b61545 in total

openshift-ci · 2023-11-08T15:48:26Z

@raptorsun: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-metal-ipi-ovn-ipv6	`3b61545`	link	false	`/test e2e-metal-ipi-ovn-ipv6`
ci/prow/e2e-gcp-csi	`3b61545`	link	false	`/test e2e-gcp-csi`
ci/prow/e2e-openstack-ovn	`3b61545`	link	false	`/test e2e-openstack-ovn`
ci/prow/e2e-metal-ipi-sdn	`3b61545`	link	false	`/test e2e-metal-ipi-sdn`
ci/prow/e2e-aws-ovn-single-node-upgrade	`3b61545`	link	false	`/test e2e-aws-ovn-single-node-upgrade`
ci/prow/e2e-aws-ovn-single-node-serial	`3b61545`	link	false	`/test e2e-aws-ovn-single-node-serial`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-ci-robot · 2023-11-08T15:50:28Z

@raptorsun: Jira Issue OCPBUGS-18776: All pull requests linked via external trackers have merged:

openshift/origin#28320

Jira Issue OCPBUGS-18776 has been moved to the MODIFIED state.

In response to this:

Replace shell call of curl with net/http implementation in Prometheus helper functions.
So that timeout requests will not get truncated and reassembled responses.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

simonpasquier · 2023-11-09T13:24:55Z

test/extended/util/prometheus/helpers.go

+	condition := func(ctx context.Context) (bool, error) {
+		resp, err := client.Do(req)
+		if err != nil {
+			return false, err


we missed here that the error should be captured in lastErr and the function should return false, nil.

This reverts commit 29ad128, reversing changes made to 01db63c.

openshift-ci bot requested review from jan--f and simonpasquier October 11, 2023 15:46

openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Oct 11, 2023

openshift-ci bot requested a review from gpei October 11, 2023 15:51

openshift-ci bot assigned jan--f Oct 12, 2023

openshift-ci bot added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Oct 12, 2023

simonpasquier reviewed Oct 12, 2023

View reviewed changes

test/extended/util/prometheus/helpers.go Show resolved Hide resolved

openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 12, 2023

simonpasquier reviewed Oct 12, 2023

View reviewed changes

test/extended/util/prometheus/helpers.go Show resolved Hide resolved

raptorsun force-pushed the OCPBUGS-18776 branch from 07d2f93 to d3336f3 Compare October 12, 2023 14:38

openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Oct 12, 2023

simonpasquier reviewed Oct 13, 2023

View reviewed changes

raptorsun force-pushed the OCPBUGS-18776 branch from d3336f3 to a19b9f2 Compare October 13, 2023 15:31

jan--f reviewed Oct 25, 2023

View reviewed changes

test/extended/util/prometheus/helpers.go Show resolved Hide resolved

raptorsun force-pushed the OCPBUGS-18776 branch from 69854d7 to 3f28d2f Compare November 2, 2023 16:50

simonpasquier reviewed Nov 3, 2023

View reviewed changes

raptorsun force-pushed the OCPBUGS-18776 branch from 3f28d2f to a0a6b0d Compare November 3, 2023 14:54

openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 8, 2023

simonpasquier reviewed Nov 8, 2023

View reviewed changes

test/extended/util/prometheus/helpers.go Outdated Show resolved Hide resolved

Replace shell call of curl with net/http implementation in Prometheus…

3b61545

… helper functions.

raptorsun force-pushed the OCPBUGS-18776 branch from a0a6b0d to 3b61545 Compare November 8, 2023 11:41

openshift-ci bot assigned simonpasquier Nov 8, 2023

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Nov 8, 2023

openshift-merge-bot bot merged commit 29ad128 into openshift:master Nov 8, 2023
17 of 23 checks passed

simonpasquier mentioned this pull request Nov 9, 2023

OCPBUGS-23102: Revert "[sig-instrumentation] tests fail due to JSON parsing error." #28389

Merged

simonpasquier reviewed Nov 9, 2023

View reviewed changes

raptorsun added a commit to raptorsun/origin that referenced this pull request Nov 9, 2023

Revert "Merge pull request openshift#28320 from raptorsun/OCPBUGS-18776"

7124c87

This reverts commit 29ad128, reversing changes made to 01db63c.

raptorsun mentioned this pull request Nov 9, 2023

OCPBUGS-18776: [sig-instrumentation] tests fail due to JSON parsing error. #28390

Closed

openshift-ci-robot mentioned this pull request Jan 26, 2024

OCPBUGS-18776: replace curl calls by net/http #28546

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OCPBUGS-18776: [sig-instrumentation] tests fail due to JSON parsing error. #28320

OCPBUGS-18776: [sig-instrumentation] tests fail due to JSON parsing error. #28320

raptorsun commented Oct 11, 2023

openshift-ci-robot commented Oct 11, 2023

raptorsun commented Oct 11, 2023

openshift-ci-robot commented Oct 11, 2023

raptorsun commented Oct 12, 2023

jan--f commented Oct 12, 2023

raptorsun commented Oct 12, 2023

raptorsun commented Oct 13, 2023

simonpasquier Oct 13, 2023

raptorsun Oct 13, 2023

raptorsun commented Oct 14, 2023

raptorsun commented Oct 14, 2023

raptorsun commented Oct 14, 2023

raptorsun commented Oct 14, 2023

raptorsun commented Oct 31, 2023

raptorsun commented Nov 2, 2023 via email

simonpasquier commented Nov 2, 2023

simonpasquier commented Nov 2, 2023

raptorsun commented Nov 2, 2023

simonpasquier Nov 3, 2023

raptorsun Nov 3, 2023

raptorsun commented Nov 3, 2023

raptorsun commented Nov 3, 2023

raptorsun commented Nov 5, 2023

raptorsun commented Nov 8, 2023

raptorsun commented Nov 8, 2023

simonpasquier commented Nov 8, 2023

openshift-ci bot commented Nov 8, 2023

openshift-ci-robot commented Nov 8, 2023

openshift-ci bot commented Nov 8, 2023

openshift-ci-robot commented Nov 8, 2023

simonpasquier Nov 9, 2023

OCPBUGS-18776: [sig-instrumentation] tests fail due to JSON parsing error. #28320

OCPBUGS-18776: [sig-instrumentation] tests fail due to JSON parsing error. #28320

Conversation

raptorsun commented Oct 11, 2023

openshift-ci-robot commented Oct 11, 2023

raptorsun commented Oct 11, 2023

openshift-ci-robot commented Oct 11, 2023

raptorsun commented Oct 12, 2023

jan--f commented Oct 12, 2023

raptorsun commented Oct 12, 2023

raptorsun commented Oct 13, 2023

simonpasquier Oct 13, 2023

Choose a reason for hiding this comment

raptorsun Oct 13, 2023

Choose a reason for hiding this comment

raptorsun commented Oct 14, 2023

raptorsun commented Oct 14, 2023

raptorsun commented Oct 14, 2023

raptorsun commented Oct 14, 2023

raptorsun commented Oct 31, 2023

raptorsun commented Nov 2, 2023 via email

simonpasquier commented Nov 2, 2023

simonpasquier commented Nov 2, 2023

raptorsun commented Nov 2, 2023

simonpasquier Nov 3, 2023

Choose a reason for hiding this comment

raptorsun Nov 3, 2023

Choose a reason for hiding this comment

raptorsun commented Nov 3, 2023

raptorsun commented Nov 3, 2023

raptorsun commented Nov 5, 2023

raptorsun commented Nov 8, 2023

raptorsun commented Nov 8, 2023

simonpasquier commented Nov 8, 2023

openshift-ci bot commented Nov 8, 2023

openshift-ci-robot commented Nov 8, 2023

openshift-ci bot commented Nov 8, 2023

openshift-ci-robot commented Nov 8, 2023

simonpasquier Nov 9, 2023

Choose a reason for hiding this comment