Skip to content

Add library version detection for parametric tests#1442

Merged
Kyle-Verhoog merged 8 commits intomainfrom
kylev/parametric-versions
Jul 31, 2023
Merged

Add library version detection for parametric tests#1442
Kyle-Verhoog merged 8 commits intomainfrom
kylev/parametric-versions

Conversation

@Kyle-Verhoog
Copy link
Copy Markdown
Member

@Kyle-Verhoog Kyle-Verhoog commented Jul 26, 2023

Description

As per title add version detection. This enables parametric tests to reuse the @released decorator used by other system tests.

It works by retrieving the real tracer version from the library being tested by reading the http headers of requests made to the test agent.

This enables tests to be merged prior to the release of a feature.

A more accurate approach is proposed in #1445

Reviewer checklist

  • Check what scenarios are modified. If needed, add the relevant label (run-parametric-scenario, run-profiling-scenario...). If this PR modifies any system-tests internal, then add the run-all-scenarios label (more info).
  • CI is green
    • If not, failing jobs are not related to this change (and you are 100% sure about this statement)
  • if any of build-some-image label is present
    1. is the image labl have been updated ?
    2. just before merging, locally build and push the image to hub.docker.com

@Kyle-Verhoog Kyle-Verhoog requested a review from a team as a code owner July 26, 2023 19:37
@Kyle-Verhoog Kyle-Verhoog force-pushed the kylev/parametric-versions branch 2 times, most recently from c6fad41 to 9abaca3 Compare July 26, 2023 19:49
@Kyle-Verhoog Kyle-Verhoog force-pushed the kylev/parametric-versions branch from 9a8c5f6 to 1614658 Compare July 26, 2023 20:24
stats test cases may not result in trace requests being sent
@Kyle-Verhoog
Copy link
Copy Markdown
Member Author

Darn, there seems to be some kind of incompatibility with the Java library.

An alternative approach to getting the tracer version might be to add version checks in the Test Agent interface rather than trying to do a trace request prior to the test case starting. This approach would be less invasive to the test cases. I'm going to give that a try next.

This saves having to make separate requests to the agent which slows
down the test suite.
Kyle-Verhoog added a commit that referenced this pull request Jul 27, 2023
@Kyle-Verhoog
Copy link
Copy Markdown
Member Author

Kyle-Verhoog commented Jul 27, 2023

Switching to checking the version in the test agent interface seems to have worked. It does carry the caveat that the test is only skipped at the point that data is requested from the test agent. If there are any operations performed in the test case that the library doesn't support then the test might crash/fail before it can be skipped.

In the future we could have the factories of the test servers declare the version of the library that they install and use that version. The challenge there is detecting the version for dev/custom installations of libraries that run in CI/locally.

@pawelchcki
Copy link
Copy Markdown
Collaborator

LGTM - thanks Kyle! 🙇

@Kyle-Verhoog Kyle-Verhoog merged commit 83b29a4 into main Jul 31, 2023
@Kyle-Verhoog Kyle-Verhoog deleted the kylev/parametric-versions branch July 31, 2023 16:44
@cbeauchesne
Copy link
Copy Markdown
Collaborator

thank's a lot @Kyle-Verhoog

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants