Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update diego sync to not fetch every bit of every object, only completely fetch objects that need syncing #3503

Merged
merged 7 commits into from Nov 6, 2023

Conversation

Benjamintf1
Copy link
Member

@Benjamintf1 Benjamintf1 commented Nov 2, 2023

Thanks for contributing to cloud_controller_ng. To speed up the process of reviewing your pull request please provide us with:

  • A short explanation of the proposed change:

Possible way to decrease cost of diego syncing? Have not investigated it fully to determine the effects of the change and whether it's actually more performant. Have not figured out how to remove the inner joins.

  • An explanation of the use cases your change solves

Large envirnments may fetch all the process details of all the processes of the system rather then the information needed to preform a sync

  • Links to any other associated PRs

  • I have reviewed the contributing guide

  • I have viewed, signed, and submitted the Contributor License Agreement

  • I have made this pull request to the main branch

  • I have run all the unit tests using bundle exec rake

  • I have run CF Acceptance Tests

Copy link
Member

@philippthun philippthun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @Benjamintf1!

Fetching all the related stuff (e.g. apps, spaces, routes, service bindings) for all the unchanged processes seems to be really useless. I've checked the runtime of these queries on a large foundation; the queries themselves seem to be rather fast (i.e. 20 to 40ms each), but in the end this sums up to quite some useless query/busy time.

The inner joins come from here. This ensures that only processes having an associated droplet are fetched.

lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
@philippthun philippthun self-assigned this Nov 6, 2023
lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
lib/cloud_controller/diego/processes_sync.rb Outdated Show resolved Hide resolved
@philippthun philippthun merged commit cb6e7c6 into cloudfoundry:main Nov 6, 2023
6 checks passed
@Benjamintf1 Benjamintf1 deleted the diego-process-sync branch November 6, 2023 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants