Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Networking stress tests infrastructure improvements. #108325

Merged
merged 17 commits into from
Oct 4, 2024

Conversation

rzikm
Copy link
Member

@rzikm rzikm commented Sep 27, 2024

Key changes:

  • Moved common parts of the infra to Common/tests/System/Net/StressTests
  • Added support for collecting testhost binaries in case there are crash dumps (enables crash dump investigation)
  • Current .NET version is not hardcoded anymore (does not have to be updated every year), is being read from eng/Versions.props

@rzikm
Copy link
Member Author

rzikm commented Sep 27, 2024

/azp run runtime-libraries stress-http

Copy link
Contributor

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@rzikm
Copy link
Member Author

rzikm commented Sep 30, 2024

/azp run runtime-libraries stress-http

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@rzikm
Copy link
Member Author

rzikm commented Sep 30, 2024

/azp run runtime-libraries stress-http

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@rzikm
Copy link
Member Author

rzikm commented Sep 30, 2024

/azp run runtime-libraries stress-http

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@rzikm
Copy link
Member Author

rzikm commented Sep 30, 2024

/azp run runtime-libraries stress-http

@rzikm
Copy link
Member Author

rzikm commented Sep 30, 2024

/azp run runtime-libraries stress-ssl

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

1 similar comment
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@rzikm
Copy link
Member Author

rzikm commented Sep 30, 2024

/azp run runtime-libraries stress-http

@rzikm
Copy link
Member Author

rzikm commented Sep 30, 2024

/azp run runtime-libraries stress-http

@rzikm
Copy link
Member Author

rzikm commented Sep 30, 2024

/azp run runtime-libraries stress-ssl

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

1 similar comment
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@rzikm
Copy link
Member Author

rzikm commented Oct 1, 2024

/azp run runtime-libraries stress-ssl

@rzikm
Copy link
Member Author

rzikm commented Oct 1, 2024

/azp run runtime-libraries stress-http

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

1 similar comment
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@rzikm rzikm marked this pull request as ready for review October 1, 2024 10:20
@rzikm
Copy link
Member Author

rzikm commented Oct 1, 2024

Stress is running, H/3 stress failure was present before as well.

@@ -7,7 +7,7 @@ RUN apt-get update -y && \
apt-get upgrade -y && \
apt-get install -y cmake clang ruby-dev gem lttng-tools libssl-dev && \
gem install fpm
RUN git clone --depth 1 --single-branch --branch v2.3.5 --recursive https://github.com/microsoft/msquic
RUN git clone --depth 1 --single-branch --branch v2.4.4 --recursive https://github.com/microsoft/msquic
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to build msquic? It feels like we should just consume it from packages.microsoft.com ... and simply get latest e.g. no need to vary about particular varian?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's build in Debug (to catch asserts) and with the sanitizer on - which helped us in the past to catch some errors in stress. So that's why we build it inside the image instead of just using a binary package.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note may be useful. It was not obvious to me. I'mwondering if libsmquic-dbg package would be useful to anybody besides us...

@@ -1,6 +1,6 @@
# Builds and copies library artifacts into target dotnet sdk image
ARG BUILD_BASE_IMAGE=mcr.microsoft.com/dotnet-buildtools/prereqs:centos-stream9
ARG SDK_BASE_IMAGE=mcr.microsoft.com/dotnet/nightly/sdk:8.0
ARG SDK_BASE_IMAGE=mcr.microsoft.com/dotnet/nightly/sdk:latest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to worry about cases when "latest" != VERSION

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not really, we download the nightly SDK anyway for build purposes (there is no docker image for that). "latest" should guarantee we have all necessary prerequisites (I didn't want to use runtime-deps image because it does not have wget)

Copy link
Member

@wfurt wfurt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I left few comments.

Copy link
Member

@antonfirsov antonfirsov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SslStress build-local.ps1 has been left unchanged, we need to update it.

Otherwise looks good, great work!

@rzikm
Copy link
Member Author

rzikm commented Oct 2, 2024

/azp run runtime-libraries stress-http

@rzikm
Copy link
Member Author

rzikm commented Oct 2, 2024

/azp run runtime-libraries stress-ssl

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

1 similar comment
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@rzikm rzikm requested a review from antonfirsov October 2, 2024 10:27
@rzikm
Copy link
Member Author

rzikm commented Oct 4, 2024

Test failures are unrelated (this PR does not change anything other than http and ssl stress pipelines

@rzikm rzikm merged commit 2c5b609 into dotnet:main Oct 4, 2024
150 of 156 checks passed
sirntar pushed a commit to sirntar/runtime that referenced this pull request Oct 8, 2024
* Update HttpStress

* !fixup Uncomment

* Fix linux build-docker-sdk script

* Fix linux scripts

* Create artifact folders on windows

* Fixes

* fix permissions on bash scripts

* Apply same changes to SslStress

* revert some changes

* Improve build-local scripts

* Update SslStress

* Update sslstress yml files

* !fixup

* mkdir -p in yaml

* Revert cli arg change

* Reference original envvars

* code review changes
@github-actions github-actions bot locked and limited conversation to collaborators Nov 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants