Skip to content

Issue 6183 - Slow ldif2db import on a newly created BDB backend #6208

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 12, 2024

Conversation

droideck
Copy link
Member

@droideck droideck commented Jun 8, 2024

Bug Description: After creating a new BDB backend, we autotune the cache only when restarting. So, an administrator will try to import an LDIF before that; she will have a very slow import.

Fix Description: Do the autotuning during the backend creation. Add a CI test for the scenario.

Fixes: #6183

Reviewed by: ?

@droideck droideck force-pushed the ldif2db_autotune branch from e41aaa2 to f24bca0 Compare June 8, 2024 03:00
Copy link
Contributor

@progier389 progier389 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but IMHO now that the CI tests are working again, you should better rebase and double check the CI test result.

Bug Description: After creating a new BDB backend, we autotune the cache only when restarting.
So, an administrator will try to import an LDIF before that; she will have a very slow import.

Fix Description: Do the autotuning during the backend creation.
Add a CI test for the scenario.

Fixes: 389ds#6183

Reviewed by: ?
Copy link
Contributor

@tbordaz tbordaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tbordaz
Copy link
Contributor

tbordaz commented Jun 11, 2024

@droideck , the patch looks good to me as well but something to clarify: the tuning of the cache is missing, what specific cache (db cache, entry cache, import cache,..) ?

If it is db cache, something that is not clear to me is that when the instance is created, it contains a default backend and I would expect db cache to be set.

@droideck
Copy link
Member Author

droideck commented Jun 12, 2024

@droideck , the patch looks good to me as well but something to clarify: the tuning of the cache is missing, what specific cache (db cache, entry cache, import cache,..) ?

I have suspicions it's the DN cache, but I am not exactly sure.
After I found that the issue was with autotuning, I focused on the solution. And as it's the correct thing to do — perform an autotuning after a backend creation (if autotuning is enabled), I just left with that as I had more stuff to do.

I can dive deeper into that if you want, though. It'll be interesting to dive into. :)

If it is db cache, something that is not clear to me is that when the instance is created, it contains a default backend and I would expect db cache to be set.

@droideck droideck merged commit d261ea2 into 389ds:main Jun 12, 2024
191 of 193 checks passed
droideck added a commit that referenced this pull request Jun 12, 2024
Bug Description: After creating a new BDB backend, we autotune the cache only when restarting.
So, an administrator will try to import an LDIF before that; she will have a very slow import.

Fix Description: Do the autotuning during the backend creation.
Add a CI test for the scenario.

Fixes: #6183

Reviewed by: @progier389, @tbordaz (Thanks!!)
droideck added a commit that referenced this pull request Jun 12, 2024
Bug Description: After creating a new BDB backend, we autotune the cache only when restarting.
So, an administrator will try to import an LDIF before that; she will have a very slow import.

Fix Description: Do the autotuning during the backend creation.
Add a CI test for the scenario.

Fixes: #6183

Reviewed by: @progier389, @tbordaz (Thanks!!)
droideck added a commit that referenced this pull request Jun 12, 2024
Bug Description: After creating a new BDB backend, we autotune the cache only when restarting.
So, an administrator will try to import an LDIF before that; she will have a very slow import.

Fix Description: Do the autotuning during the backend creation.
Add a CI test for the scenario.

Fixes: #6183

Reviewed by: @progier389, @tbordaz (Thanks!!)
droideck added a commit that referenced this pull request Jun 12, 2024
Bug Description: After creating a new BDB backend, we autotune the cache only when restarting.
So, an administrator will try to import an LDIF before that; she will have a very slow import.

Fix Description: Do the autotuning during the backend creation.
Add a CI test for the scenario.

Fixes: #6183

Reviewed by: @progier389, @tbordaz (Thanks!!)
droideck added a commit that referenced this pull request Jun 12, 2024
Bug Description: After creating a new BDB backend, we autotune the cache only when restarting.
So, an administrator will try to import an LDIF before that; she will have a very slow import.

Fix Description: Do the autotuning during the backend creation.
Add a CI test for the scenario.

Fixes: #6183

Reviewed by: @progier389, @tbordaz (Thanks!!)
droideck added a commit that referenced this pull request Jun 12, 2024
Bug Description: After creating a new BDB backend, we autotune the cache only when restarting.
So, an administrator will try to import an LDIF before that; she will have a very slow import.

Fix Description: Do the autotuning during the backend creation.
Add a CI test for the scenario.

Fixes: #6183

Reviewed by: @progier389, @tbordaz (Thanks!!)
droideck added a commit that referenced this pull request Jun 12, 2024
Bug Description: After creating a new BDB backend, we autotune the cache only when restarting.
So, an administrator will try to import an LDIF before that; she will have a very slow import.

Fix Description: Do the autotuning during the backend creation.
Add a CI test for the scenario.

Fixes: #6183

Reviewed by: @progier389, @tbordaz (Thanks!!)
droideck added a commit that referenced this pull request Jun 12, 2024
Bug Description: After creating a new BDB backend, we autotune the cache only when restarting.
So, an administrator will try to import an LDIF before that; she will have a very slow import.

Fix Description: Do the autotuning during the backend creation.
Add a CI test for the scenario.

Fixes: #6183

Reviewed by: @progier389, @tbordaz (Thanks!!)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Slow ldif2db import on a newly created BDB backend
3 participants