Additional improvements to memory pooling and management. #2960
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kozlovic
reviewed
Mar 28, 2022
server/filestore.go
Outdated
| @@ -2047,6 +2069,7 @@ func (fs *fileStore) removeMsg(seq uint64, secure, needFSLock bool) (bool, error | |||
| // we don't lose track of the first sequence. | |||
| if firstSeqNeedsUpdate { | |||
| fs.selectNextFirst() | |||
| // FIXME(dlc)? | |||
There was a problem hiding this comment.
Thanks for reminder! I don't think that statement is correct, will look more closely now.
| } | ||
| } | ||
| } | ||
| // Copy here to not reference underlying buffer. |
There was a problem hiding this comment.
I don't understand this... subjString() is invoked with a string that is already a string (either from caller stack or made a string out of the underlying buffer), so I don't see how "key" is referencing the underlying buffer.
There was a problem hiding this comment.
I was seeing higher memory usage until this change, if it was referencing the underlying buffer I believe runtime could reference it vs copy on the cast.
…fix for firstMatching that did unnecessary work when matching all. During contention to the head write blk, the system could perform worse memory wise compared to simple go runtime. Also had some references for the subject of messages bloating memory. Signed-off-by: Derek Collison <derek@nats.io>
This may prevent memory copies when not necessary. Also fixed a bug there that would check twice if there was only 1 subject and that subject did not match (say configured subject is foo.* and key is foo.bar). Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
…. This would possibly cause the load code to thrash since it would not be big enough for a full block and we would need to recycle again and make a new one. Signed-off-by: Derek Collison <derek@nats.io>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
During contention to the head write block, the system could perform worse memory wise compared to simple go runtime and GC. Also had some references for the subject of messages being read back in to the underlying block which was bloating memory.
Additionally a fix for firstMatching() that did unnecessary work when matching all. This would cause excessive CPU and excessive load on the garbage collector.
Signed-off-by: Derek Collison derek@nats.io
/cc @nats-io/core