Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use copy-on-write list in InMemoryAppender #8808

Merged
merged 3 commits into from Nov 7, 2019

Conversation

zhenxiao
Copy link
Contributor

@zhenxiao zhenxiao commented Nov 1, 2019

@@ -94,7 +94,7 @@ public void append(LogEvent logEvent)

List<LogEvent> getLogEvents()
{
return Collections.unmodifiableList(logEvents);
return Collections.synchronizedList(logEvents);
Copy link
Member

@leventov leventov Nov 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. It's not copy-on-write (as the title of your PR says), it's synchronizedList, but this doesn't solve the problem: see https://github.com/code-review-checklists/java-concurrency#synchronized-collection-iter. However, if you actually use CopyOnWriteArrayList instead of ArrayList as logEvents, that would help. Don't forget to change the type of the field from List<LogEvent> to CopyOnWriteArrayList<LogEvent> and to add a comment explaining the need for thread safety (in the form of using CopyOnWriteArrayList), along the lines of https://github.com/code-review-checklists/java-concurrency#justify-document.

@zhenxiao zhenxiao changed the title use copy-on-write synchronized list in InMemoryAppender use copy-on-write list in InMemoryAppender Nov 2, 2019
@zhenxiao
Copy link
Contributor Author

zhenxiao commented Nov 2, 2019

thank you, @leventov
comments addressed

@@ -78,12 +77,13 @@ public void clearLogEvents()
{
static final String NAME = InMemoryAppender.class.getName();

private final List<LogEvent> logEvents;
// need copy-on-write collection for thread safety of iteration and modification concurrently outside of a critical section
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment doesn't achieve its goal: to help somebody to see why the enclosing class needs to be thread-safe. With the current comment, in order to do that, the reader (developer) should invoke "Find usages" IDE action and analyze these usages carefully. This is time-consuming. In essence, there should be concurrent access documentation, as described in https://github.com/code-review-checklists/java-concurrency#justify-document.

This comment is also misleading because there are no critical sections in this code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about:
logEvents has concurrent iteration and modification in CuratorModuleTest::exitsJvmWhenMaxRetriesExceeded(), needs to be thread safe

@LoriLori
Copy link

LoriLori commented Nov 5, 2019

I am learning from better developers and was thinking about contract of logger and how would it could be solved should it be inmemorylogger be used somewhere else some day.
Isn't copy on write an overhead and a bit teaching inappropriate use of copy on write?
Isn't logEvents.toArray() atomic appropriate?
return Collections.unmodifiableList(Arrays.asList((LogEvent[]) logEvents.toArray()));
Code readability is lower and would require a comment maybe.

@leventov
Copy link
Member

leventov commented Nov 6, 2019

@LoriLori logEvents.toArray() doesn't solve the race condition - it may still race with an addition to logEvents and throw some exception, unless you make both append() and getLogEvents() synchronized.

Since this class resides in tests, not production, the performance doesn't matter and we shouldn't waste time comparing synchronized and CopyOnWriteArrayList. While CopyOnWriteArrayList solution is a bit simpler.

@leventov
Copy link
Member

leventov commented Nov 6, 2019

@zhenxiao please merge the master branch into your PR branch to make the CI pass.

@zhenxiao
Copy link
Contributor Author

zhenxiao commented Nov 6, 2019

master merged into this PR. failure seems not related

@leventov
Copy link
Member

leventov commented Nov 7, 2019

@zhenxiao don't rebase your PRs. Your should never do this in Druid. Once you open a PR, you only merge other branches into your PR branch, never rebase it.

@zhenxiao
Copy link
Contributor Author

zhenxiao commented Nov 7, 2019

oh, sure. seems tests passed

@leventov leventov merged commit fca23d0 into apache:master Nov 7, 2019
@jon-wei jon-wei added this to the 0.17.0 milestone Dec 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix a concurrency bug in InMemoryAppender
4 participants