8355726: LinkedBlockingDeque fixes and improvements#24925
Conversation
…ty has been reached
…edException in put/take
|
👋 Welcome back kabutz! A progress list of the required criteria for merging this PR into |
|
@kabutz This change now passes all automated pre-integration checks. ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details. After integration, the commit message for the final commit will be: You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed. At the time when this comment was updated there had been 783 new commits pushed to the
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details. As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@DougLea, @viktorklang-ora) but any other Committer may sponsor as well. ➡️ To flag this PR as ready for integration with the above commit message, type |
Webrevs
|
|
/reviewers 2 |
|
@viktorklang-ora |
|
@kabutz Thanks for opening this PR—just confirming that it's on my to-review queue. |
DougLea
left a comment
DougLea
left a comment
There was a problem hiding this comment.
I agree that these changes make behavior more consistent with LinkedBlockingQueue, which is worth doing. Thanks for finding straightforward ways to do these that don't impact anything else.
Changing count field to volatile and sometimes read outside of lock introduces more potential non-sequential-consistency (for example checking size while adding) but doesn't impact observable guarantees. It may have small performance impact in either direction that could vary across platforms, but I have not detected any.
There are possible (not likely, and still legal according to spec) observable consequences of using lockInterruptibly, but better to have them now not surprisingly different than other blocking queues.
|
Thanks @DougLea ! |
|
@viktorklang-ora any idea whom else we can ask to approve this PR? |
|
@kabutz I think @AlanBateman might be able to have a look as well. As for timing, it seems to me most reasonable if this PR (if it is to be integrated) to go in after JDK25 has been forked, to give enough time for JDK26 early access feedback and giving the changes time to harden. I hope that makes sense. |
That sound reasonable. When would this be? |
| @@ -858,7 +860,7 @@ public boolean addAll(Collection<? extends E> c) { | |||
|
|
|||
There was a problem hiding this comment.
We could likely check if there's any remaining capacity up front, and immediately return false?
There was a problem hiding this comment.
We could likely check if there's any remaining capacity up front, and immediately return false?
We could if you like. I wanted to make as few changes as possible, to not introduce unexpected changes. This particular bug was to stop a size overflow.
There was a problem hiding this comment.
@DougLea What do you think about checking if the target collection is likely going to fit into the queue early?
There was a problem hiding this comment.
Because of races, prechecks can't be exactly accurate, but I think Heinz's current approach is a good choice, and is similar to how it's done elsewhere.
| q.clear(); | ||
| q.add(four); | ||
| q.add(five); | ||
| q.add(six); |
There was a problem hiding this comment.
Out of curiosity, how does it.remove() work under these conditions?
There was a problem hiding this comment.
Out of curiosity, how does
it.remove()work under these conditions?
If we call it.remove() on the first element, it delegates to unlinkFirst() (if we are using an ascending iterator), and unlinkLast (if we are using a descending iterator). Similarly, if we call it.remove() on the last element it will call unlinkLast() or unlinkFirst(). With unlinkFirst(), it will make f.next = f (thus linking back to itself) and with unlinkLast(), it will make l.prev = l.
If we call it.remove() on a middle element, then we simply link the p.next = n; n.prev = p; and does not do self-linking. Thus if we have an LBD with 1,2,3,4,5 with two iterators pointing onto 3, if one of them removes it, then the other will continue with 3 (cached), 4, 5, and it won't go back to the beginning and see duplicate elements.
There was a problem hiding this comment.
I've added a unit test for this.
| int n = 0; | ||
| long n = 0; | ||
| for (E e : c) { | ||
| Objects.requireNonNull(e); |
There was a problem hiding this comment.
This makes me wonder: Does it make sense to create new nodes if we don't track if they will still fit into the capacity?
There was a problem hiding this comment.
We could if you like, but that would subtly change the current behaviour. I tried to make as few changes as possible.
There was a problem hiding this comment.
On the other hand, there could be intermediate operations modifying the underlying collection before the lock is obtained, so checking wouldn't help.
About a month or so. |
Co-authored-by: Viktor Klang <viktor.klang@oracle.com>
Co-authored-by: Viktor Klang <viktor.klang@oracle.com>
viktorklang-ora
left a comment
viktorklang-ora
left a comment
There was a problem hiding this comment.
Just made some final suggestions, then I think this is good to go.
Great job on this, @kabutz—I really appreciate you taking this on!
I'll have a look once you've commented/addressed the comments, and I'll move this forward after that.
…gDeque.java Co-authored-by: Viktor Klang <viktor.klang@oracle.com>
…gDeque.java Co-authored-by: Viktor Klang <viktor.klang@oracle.com>
viktorklang-ora
left a comment
viktorklang-ora
left a comment
There was a problem hiding this comment.
Great work, @kabutz — this is now ready to integrate.
|
/integrate |
|
@kabutz This pull request has not yet been marked as ready for integration. |
What are the next steps for me to do ? |
|
@kabutz Apologies, I'll let you know as soon as you can /integrate this. |
|
Changing to 1 reviewer after discussing with @AlanBateman |
|
/reviewers 1 |
|
@viktorklang-ora |
|
@kabutz This is now ready for integration. After you've commented with "/integrate", I'll sponsor the integration. |
|
/integrate |
|
/sponsor |
|
Going to push as commit 91fdd72.
Your commit was automatically rebased without conflicts. |
openjdk
Bot
removed
ready
|
Thanks @kabutz! |
|
@viktorklang-ora @kabutz Pushed as commit 91fdd72. 💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored. |
3 participants
We logged several bugs on the LinkedBlockingDeque. This aggregates them into a single bug report and PR.
The LinkedBlockingDeque does not behave consistently with other concurrency components. If we call putFirst(), putLast(), takeFirst(), or takeLast() with a thread that is interrupted, it does not immediately throw an InterruptedException, the way that ArrayBlockingQueue and LInkedBlockingQueue does, because instead of lockInterruptibly(), we call lock(). It will only throw an InterruptedException if the queue is full (on put) or empty (on take). Since interruptions are frequently used as a shutdown mechanism, this might prevent code from ever shutting down.
LinkedBlockingDeque.clear() should preserve weakly-consistent iterators by linking f.prev and f.next back to f, allowing the iterators to continue from the first or last respectively. This would be consistent with how the other node-based weakly consistent queues LinkedBlockingQueue LinkedTransferQueue, ConcurrentLinkedQueue/Deque work.
The LBD already supports self-linking, since that is done by the unlinkFirst() and unlinkLast() methods, and the iterators and spliterator thus all support self-linking.
This can be fixed very easily by linking both f.prev and f.next back to f.
In the JavaDoc of LinkedBlockingDeque, it states: "Linked nodes are dynamically created upon each insertion unless this would bring the deque above capacity." However, in the current implementation, nodes are always created, even if the deque is full. This is because count is non-volatile, and we only check inside the linkFirst/Last() methods whether the queue is full. At this point we have already locked and have created the Node. Instead, the count could be volatile, and we could check before locking.
In the current version, calling offer() on a full LinkedBlockingDeque creates unnecessary objects and contention. Similarly for poll() and peek(), we could exit prior to locking by checking the count field.
In LinkedBlockingDeque.addAll() we first build up the chain of nodes and then add that chain in bulk to the existing nodes. We count the nodes in "int n" and then whilst holding the lock, we check that we haven't exceeded the capacity with "if (count + n <= capacity)". However, if we pass in a collection that has more than Integer.MAX_VALUE items in it, then we can overflow n, making it negative. Since "count + n" is also negative, we can add the chain to our last item, and thus we end up with a LinkedBlockingDeque with more than Integer.MAX_VALUE of items and a negative size(). stream().count() gives the correct number of items.
This happens both via the bulk add constructor LinkedBlockingDeque(Collection) and when we call addAll(Collection) directly.
Progress
Issue
Reviewers
Reviewing
Using
gitCheckout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/24925/head:pull/24925$ git checkout pull/24925Update a local copy of the PR:
$ git checkout pull/24925$ git pull https://git.openjdk.org/jdk.git pull/24925/headUsing Skara CLI tools
Checkout this PR locally:
$ git pr checkout 24925View PR using the GUI difftool:
$ git pr show -t 24925Using diff file
Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/24925.diff
Using Webrev
Link to Webrev Comment