Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pmproxy / libpcp_web multithreading, http and https/tls fixes, QA #1321

Merged

Conversation

goodwinos
Copy link
Contributor

3f5ba221842e6 libpcp_web: add mutex to struct webgroup protecting the context dict
2bad6aef10339 pmproxy: add mutex for client req lists, fix https/tls support, QA

Resolves BZ 1947989 - pmproxy hangs and consume 100% cpu if the redis datasource is configured with TLS
Resolves #1311
Related #1203, qa/297 and qa/1290
Tested with updates to qa/1457

Add a mutex to the local webgroups structure in libpcp_web and
use it to protect multithreaded parallel updates (dictAdd,
dictDelete) to the groups->contexts dict and the dict traversal
in the timer driven garbage collector.

Tested by qa/297 and related tests and also an updated version
of qa/1457 (which now stress tests parallel http and https/tls
pmproxy RESTAPI calls .. in a later commit).

Related: RHBZ#1947989
Resolves: performancecopilot#1311
Add a new mutext to struct proxy and use it to protect parallel
multithreaded updates to the proxy->first client list.

Also use the same mutext to protect updates to the pending_writes
client list and avoid the doubly linked list corruption that was
causing parallel https/tls requests to get stuck spinning in
flush_secure_module(), as reported in BZ#1947989.

qa/1457 is extensively updated to test parallel http, https/tls
(and combinations of http and https/tls) RESTAPI calls. Previously
it only tested a single https/tls call.

With these changes, parallel https/tls RESTAPI requests from the
grafana-pcp datasource to pmproxy now work correctly whereas previously
pmproxy would hang/spin.

Resolves: RHBZ#1947989 - pmproxy hangs and consume 100% cpu if the
redis datasource is configured with TLS.

Related: performancecopilot#1311
@goodwinos goodwinos merged commit f8ce232 into performancecopilot:main Jun 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

parallel http stress test exposes multiple client related pmproxy/libpcp_web race conditions
1 participant