Skip to content

Conversation

@Yogu
Copy link
Contributor

@Yogu Yogu commented Nov 30, 2023

Fix "higher" and "lower" in explanation for commitIntervalMsec

Description

A higher value for the interval means the data is commited less frequently, so there will be more time where the index does not account for the changes. Also, memory consumption will grow because the data is not yet flushed to disk yet.

I'm quite confused by this part of the documentation, and I'd be a little bit less confused if you confirm that higher / lower are the wrong way around here.

I still don't quite understand why the documentation dinstinguishes between lots of inserts/updates and few inserts/updates in the way it does. Does the parameter really matter in the case of few inserts/updates, because the commit would not have anything to do anyway (and just be a no-op) when there are no changes?

My understanding would be

  • The parameter is more important when you have many insert/updates, and is less important when you don't have many inserts/updates.
  • If you configure a higher value, you reduce CPU and disk utilization during writes, but you increase memory usage and you increase the latency of the view
  • (vice-versa) If you configure a lower value, you increase CPU and disk utilization during writes, but you decrease memory usage and you decrease the latency of the views

If you confirm my understanding and you agree that the docs could be improved in this way, I'd be happy to extend my PR to reflect these changes (and also port it to 3.12 and 3.10). If you tell me I'm wrong, I'd also try to make the docs a little bit clearer.

Fix "higher" and "lower" in explanation for commitIntervalMsec

A higher value for the interval means the data is commited less frequently, so there will be more time where the index does not account for the changes. Also, memory consumption will grow because the data is not yet flushed to disk yet.
@cla-bot cla-bot bot added the cla-signed label Nov 30, 2023
@arangodb-docs-automation
Copy link
Contributor

Deploy Preview Available Via
https://deploy-preview-362--docs-hugo.netlify.app

@Simran-B Simran-B requested a review from MBkkt December 4, 2023 08:07
@nerpaula nerpaula requested a review from Dronplane December 12, 2023 11:58
Copy link

@Dronplane Dronplane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proposed change is LGTM.
We distinguish between "many changes/not many changes" only because if index is almost constant it is better to let other threads use CPU. Yes, it is quite fast to detect that there is nothing to do in commit, but it requires some addition actions like taking a sync lock.

@nerpaula nerpaula merged commit dd81cfa into arangodb:main Dec 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants