Skip to content

Autoclean gentler#7805

Merged
vincenzopalazzo merged 2 commits intoElementsProject:masterfrom
rustyrussell:autoclean-gentler
Nov 12, 2024
Merged

Autoclean gentler#7805
vincenzopalazzo merged 2 commits intoElementsProject:masterfrom
rustyrussell:autoclean-gentler

Conversation

@rustyrussell
Copy link
Contributor

Fixes: #7753

On a large node, especially with postgres, this causes every other command
to take 30 seconds plus.  The first, obvious, step is to reduce how many
commands we will do at once.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means we don't consume *all* the CPU.

Changelog-Fixed: Plugins: `autoclean` is now gentler on the node when doing giant cleans.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
@rustyrussell rustyrussell added this to the v24.11 milestone Nov 12, 2024
@wtogami
Copy link
Contributor

wtogami commented Nov 12, 2024

Can you auto clean during runtime without disrupting routing?

@vincenzopalazzo
Copy link
Collaborator

Can you auto clean during runtime without disrupting routing?

Should not be a problem, why it should disrupting routing? The action of a middle node is just a "witness step", no big operation involved

Copy link
Collaborator

@vincenzopalazzo vincenzopalazzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 7d95004

@vincenzopalazzo
Copy link
Collaborator

@rustyrussell probably a good stacktrace for your timeout debugging

test_autoclean passed 1 out of the required 1 times. Success!
test_getroutes_auto_sourcefree passed 1 out of the required 1 times. Success!
test_real_biases failed (1 runs remaining out of 2).
	<class 'Failed'>
	Timeout >1200.0s
	[<TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/_pytest/runner.py:341>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/flaky/flaky_pytest_plugin.py:146>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/pluggy/_hooks.py:513>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/pluggy/_manager.py:120>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/pluggy/_callers.py:182>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/pluggy/_result.py:100>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/pluggy/_callers.py:103>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/_pytest/runner.py:169>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/_pytest/python.py:1792>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/pluggy/_hooks.py:513>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/pluggy/_manager.py:120>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/pluggy/_callers.py:182>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/pluggy/_result.py:100>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/pluggy/_callers.py:103>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/_pytest/python.py:194>, <TracebackEntry /home/runner/work/lightning/lightning/tests/test_askrene.py:1188>, <TracebackEntry /home/runner/work/lightning/lightning/contrib/pyln-client/pyln/client/lightning.py:332>, <TracebackEntry /home/runner/work/lightning/lightning/contrib/pyln-testing/pyln/testing/utils.py:735>, <TracebackEntry /home/runner/work/lightning/lightning/contrib/pyln-client/pyln/client/lightning.py:392>, <TracebackEntry /home/runner/work/lightning/lightning/contrib/pyln-client/pyln/client/lightning.py:309>, <TracebackEntry /home/runner/work/lightning/lightning/contrib/pyln-client/pyln/client/lightning.py:278>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/pytest_timeout.py:313>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/pytest_timeout.py:498>, <TracebackEntry /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/_pytest/outcomes.py:198>]
test_real_biases passed 1 out of the required 1 times. Success!
test_plugin_start passed 1 out of the required 1 times. Success!
test_rust_plugin_subscribe_wildcard passed 1 out of the required 1 times. Success!

@vincenzopalazzo vincenzopalazzo merged commit ccd9b21 into ElementsProject:master Nov 12, 2024
@rustyrussell
Copy link
Contributor Author

Definitely! And you can test it with autoclean-once...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

autoclean can DoS the node

3 participants