Fix memory leak

[deepthidevaki]

Description

When we install services for a role, it installs several listeners. Most of them were not removed when these services are closed. References to StreamProcessor and LogStream are kept indirectly via these listeners. Thus they are not garbage collected. This lead to OOM in direct memory and eventually OOM in heap.

This PR fixes it by unregistering listeners when closing services.
After this fix, StreamProcessor is garbage collected. Following is from a heapdump after several role transitions in a broker

Related issues

closes #7744

Definition of Done

Not all items need to be done depending on the issue and the pull request.

Code changes:

• The changes are backwards compatibility with previous versions
• If it fixes a bug then PRs are created to backport the fix to the last two minor versions. You can trigger a backport by assigning labels (e.g. backport stable/0.25) to the PR, in case that fails you need to create backports manually.

Testing:

• There are unit/integration tests that verify all acceptance criterias of the issue
• New tests are written to ensure backwards compatibility with further versions
• The behavior is tested manually
• The change has been verified by a QA run
• The impact of the changes is verified by a benchmark

Documentation:

• The documentation is updated (e.g. BPMN reference, configuration, examples, get-started guides, etc.)
• New content is added to the release announcement

[deepthidevaki]

A cluster is running with this fix under namespace "dd-7744-mem-leak".

[saig0]

Looks good 👍 Thanks 🥞

[deepthidevaki]

I will leave the cluster running to check if this fully fixes the issue. But will merge this PR anyway.

bors r+

[ghost]

Build succeeded:

• continuous-integration/jenkins/branch

[github-actions]

Backport failed for stable/1.0, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally.

git fetch origin stable/1.0
git worktree add -d .worktree/backport-7762-to-stable/1.0 origin/stable/1.0
cd .worktree/backport-7762-to-stable/1.0
git checkout -b backport-7762-to-stable/1.0
ancref=$(git merge-base aef2ecf940c0cb2ec807b0d972e448337328d036 4459ea19d146a8cb32d108fa93c0b09b1b661f7d)
git cherry-pick -x $ancref..4459ea19d146a8cb32d108fa93c0b09b1b661f7d

[github-actions]

Backport failed for stable/1.1, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally.

git fetch origin stable/1.1
git worktree add -d .worktree/backport-7762-to-stable/1.1 origin/stable/1.1
cd .worktree/backport-7762-to-stable/1.1
git checkout -b backport-7762-to-stable/1.1
ancref=$(git merge-base aef2ecf940c0cb2ec807b0d972e448337328d036 4459ea19d146a8cb32d108fa93c0b09b1b661f7d)
git cherry-pick -x $ancref..4459ea19d146a8cb32d108fa93c0b09b1b661f7d