Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make MockLogAppender threadsafe #108206

Merged
merged 6 commits into from
May 8, 2024

Conversation

rjernst
Copy link
Member

@rjernst rjernst commented May 2, 2024

Adding and removing appenders in Log4j is not threadsafe. Yet some tests rely on capturing logging by adding an in memory appender, MockLogAppender. This commit makes the mock logging threadsafe by creating a new, singular appender for mock logging that delegates, in a threadsafe way, to the existing appenders created. Confusingly MockLogAppender is no longer really an appender, but I'm leaving clarifying that for a followup so as to limit the scope of this PR.

closes #106425

Adding and removing appenders in Log4j is not threadsafe. Yet some tests
rely on capturing logging by adding an in memory appender,
MockLogAppender. This commit makes the mock logging threadsafe by
creating a new, singular appender for mock logging that delegates, in a
threadsafe way, to the existing appenders created. Confusingly
MockLogAppender is no longer really an appender, but I'm leaving
clarifying that for a followup so as to limit the scope of this PR.

closes elastic#106425
@rjernst rjernst added >test Issues or PRs that are addressing/adding tests :Core/Infra/Logging Log management and logging utilities labels May 2, 2024
@rjernst rjernst requested review from DaveCTurner and a team May 2, 2024 16:06
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@elasticsearchmachine elasticsearchmachine added Team:Core/Infra Meta label for core/infra team v8.15.0 labels May 2, 2024
Copy link
Contributor

@ldematte ldematte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very nice, I have left a couple of questions

@@ -31,12 +33,35 @@
/**
* Test appender that can be used to verify that certain events were logged correctly
*/
public class MockLogAppender extends AbstractAppender {
public class MockLogAppender {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: since this is no longer an appender, could this break things in serverless? (if tests there use something like Loggers.addAppender(mockAppender))

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will "break" things, but I have the necessary changes ready.

mockAppenders.compute(logger, (k, v) -> {
assert v != null;
v.remove(this);
return v.isEmpty() ? null : v;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL: compute remappingFunction returning null removes the mapping. Nice!


private static final Logger logger = LogManager.getLogger(MockLogAppender.class);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why do we need this. What I'm missing?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leftover from debugging, will remove

@@ -259,6 +259,7 @@ public static void resetPortCounter() {
// TODO: consolidate logging initialization for tests so it all occurs in logconfigurator
LogConfigurator.loadLog4jPlugins();
LogConfigurator.configureESLogging();
MockLogAppender.init();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be called every time we (re) configure log4j in tests, right?
So besides the special case above, ESTestCase (should) cover all, correct?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, any tests subclassing LuceneTestCase directly will need to call it if they use MockLogAppender.

private final List<WrappedLoggingExpectation> expectations;
private final AtomicBoolean isAlive = new AtomicBoolean(true);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be an AtomicBoolean? Couldnt it just be a volatile boolean?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same in WrappedLoggingExpectation

Copy link
Contributor

@pgomulka pgomulka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rjernst
Copy link
Member Author

rjernst commented May 7, 2024

@ldematte @thecoop I believe I've addressed your comments.

Copy link
Contributor

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM2 but I haven't looked at this very deeply so I'll leave the final approval to the other reviewers.

Copy link
Contributor

@ldematte ldematte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 2! Nice change!

@ldematte
Copy link
Contributor

ldematte commented May 8, 2024

(btw, CI test failure looks related but should be fixable)

@rjernst rjernst merged commit 31abf3e into elastic:main May 8, 2024
15 checks passed
@rjernst rjernst deleted the test/mock_appender_threadsafe branch May 8, 2024 22:38
markjhoy pushed a commit to markjhoy/elasticsearch that referenced this pull request May 9, 2024
Adding and removing appenders in Log4j is not threadsafe. Yet some tests
rely on capturing logging by adding an in memory appender,
MockLogAppender. This commit makes the mock logging threadsafe by
creating a new, singular appender for mock logging that delegates, in a
threadsafe way, to the existing appenders created. Confusingly
MockLogAppender is no longer really an appender, but I'm leaving
clarifying that for a followup so as to limit the scope of this PR.

closes elastic#106425
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Core/Infra/Logging Log management and logging utilities Team:Core/Infra Meta label for core/infra team >test Issues or PRs that are addressing/adding tests v8.15.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CI] SingleNodeDiscoveryIT testCannotJoinNodeWithSingleNodeDiscovery failing
6 participants