Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ssl: Avoid partial connection trees #6270

Merged

Conversation

IngelaAndin
Copy link
Contributor

If the "User" process, the process starting the TLS connection, gets
killed in the middle of spawning the dynamic connection tree make sure
we do not leave any processes behind.

Close #6244

@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2022

CT Test Results

No tests were run for this PR. This is either because the build failed, or the PR is based on a branch without GH actions tests configured.

Results for commit 3456747

To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass.

See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally.

Artifacts

// Erlang/OTP Github Action Bot

@IngelaAndin IngelaAndin added the team:PS Assigned to OTP team PS label Sep 2, 2022
@sneako
Copy link
Contributor

sneako commented Sep 2, 2022

Thanks @IngelaAndin ! I can test this out on Monday and let you know if it resolves the issue we were seeing. I am also curious if this new process that starts the connection tree will add any noticeable overhead. I will report back soon.

@IngelaAndin IngelaAndin force-pushed the ingela/ssl/dy-sup-leak/GH-6244/OTP-18233 branch from bc90049 to daf3c21 Compare September 2, 2022 08:51
@IngelaAndin IngelaAndin added the testing currently being tested, tag is used by OTP internal CI label Sep 2, 2022
lib/ssl/src/tls_gen_connection.erl Outdated Show resolved Hide resolved
@IngelaAndin IngelaAndin force-pushed the ingela/ssl/dy-sup-leak/GH-6244/OTP-18233 branch from daf3c21 to 8abcbdd Compare September 2, 2022 10:31
lib/ssl/src/tls_gen_connection.erl Outdated Show resolved Hide resolved
@IngelaAndin IngelaAndin added this to the OTP-25.1 milestone Sep 5, 2022
@IngelaAndin
Copy link
Contributor Author

@sneako How is the testing going?

@sneako
Copy link
Contributor

sneako commented Sep 6, 2022

I was waiting for the requested changes to be addressed so that I could be sure to test the final version. If you think this is ready to go I can start testing this afternoon!

@IngelaAndin IngelaAndin force-pushed the ingela/ssl/dy-sup-leak/GH-6244/OTP-18233 branch from 8abcbdd to 1fdf101 Compare September 6, 2022 12:49
@IngelaAndin
Copy link
Contributor Author

@sneako well the changes were mainly cosmetic, but I made a new version now.

If the "User" process, the process starting the TLS connection, gets
killed in the middle of spawning the dynamic connection tree make sure
we do not leave any processes behind.

Close erlang#6244
@IngelaAndin IngelaAndin force-pushed the ingela/ssl/dy-sup-leak/GH-6244/OTP-18233 branch 2 times, most recently from e882332 to 72bafaf Compare September 7, 2022 07:02
@IngelaAndin
Copy link
Contributor Author

@sneako I added a test case cuddle, but I would not worry about it for the purpose of your testing, the test case is sensitive to timing and testing timeout functionality is tricky, I do not think this is actualy related to my change.

@sneako
Copy link
Contributor

sneako commented Sep 7, 2022

Sounds good! I am going to deploy the test shortly. Will have some results for you by the end of the day.

@sneako
Copy link
Contributor

sneako commented Sep 7, 2022

This patch has been running for almost 4 hours now on a subset of our prod nodes and we have not seen a single instance of a tls_dyn_connection_sup having less than 2 children during this period. We have observed the other nodes running OTP 25.0.2 slowly accumulate tls_dyn_connection_sup with 0, or 1 children. Performance is very similar across the two OTP versions as well.

As far as I can tell, this PR seems to have completely resolved the reported issue, and to be extra sure I will let this test continue running for the rest of the day, thanks @IngelaAndin !

@IngelaAndin IngelaAndin self-assigned this Sep 7, 2022
lib/ssl/test/ssl_basic_SUITE.erl Outdated Show resolved Hide resolved
lib/ssl/test/ssl_basic_SUITE.erl Outdated Show resolved Hide resolved
lib/ssl/test/ssl_basic_SUITE.erl Outdated Show resolved Hide resolved
lib/ssl/test/ssl_basic_SUITE.erl Outdated Show resolved Hide resolved
lib/ssl/src/tls_gen_connection.erl Show resolved Hide resolved
lib/ssl/test/ssl_basic_SUITE.erl Outdated Show resolved Hide resolved
Wait "long enough" in ssl_api_SUITE. Cosmetic changes to ssl_api_SUITE.
@IngelaAndin IngelaAndin force-pushed the ingela/ssl/dy-sup-leak/GH-6244/OTP-18233 branch from 72bafaf to 3456747 Compare September 8, 2022 07:02
@sneako
Copy link
Contributor

sneako commented Sep 8, 2022

I ended up leaving this test running overnight, almost 24 hours now, and things still look great 😀

@IngelaAndin
Copy link
Contributor Author

@sneako Super :) I will be merging once the test enhancements (suggested by @u3s) have passed the test step.

@IngelaAndin IngelaAndin merged commit 9bd57f7 into erlang:maint Sep 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
team:PS Assigned to OTP team PS testing currently being tested, tag is used by OTP internal CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants