devel: Speed up leapp database writes by disabling synchronization

[vinzenz]

This patch introduces an optimization of the database access when the
LEAPP_DEVEL_DATABASE_SYNC_OFF environment variable is set.

This causes to write to the database up to 20 times faster than normal,
depending on the underlying hard drive and file system.

I managed to increase the speed leapp preupgrade from 9 minutes 7
seconds to 26 seconds.

However this feature comes at the cost of risking database corruption,
in case of powerloss or OS crashes. According to the SQLite
documentation, application crashes are safe.

Since this option poses a potential risk, it is deemed for now
a development option only and will help to speed up our CI systems.

[github-actions]

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergable.
If you want to re-run tests or request review, you can use following commands as a comment:

• leapp-ci build to run copr build and e2e tests in OAMG CI
• review please to notify leapp developers of review request
• /packit copr-build to submit a public copr build using packit

Please open ticket in case you experience technical problem with the CI. (RH internal only)

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please consider rerunning the CI by commenting leapp-ci build (might require several comments). If the problem persists, contact leapp-infra.

[fernflower]

Love it!

[fernflower]

/rerun

[github-actions]

Copr build succeeded: https://copr.fedorainfracloud.org/coprs/build/2831780

[github-actions]

Testing Farm request for tmt test was created. Once finished, results should be available here.
Full pipeline log.

[pirat89]

love it. and so far, it still works. as it is covered by a "LEAPP_DEVEL_*" envar, it's safe to merge it. We will document additional envars soon, so merging it even when it's still not documented.

[fernflower]

Tested on vms from 1MT - 40-50% speedup.
Default no env var case:

real	23m17.276s
user	7m13.111s
sys	1m57.130s

Env var set:

real	12m45.962s
user	6m59.196s
sys	1m49.907s

[xmontagut]

You saved my day, TY. Without this improvment, after 15 min, the leapp preupgrade was still at the PCI scan step...

[anandanramadoss]

in PPC64LE system leapp upgrade took around 2 hrs 30 mins

As per below logs the udevadm_info start time 14:11:21 and end time 16:07:09 time taken approximately 2 Hours which is too long.

$ cat var/log/leapp/leapp-upgrade.log | grep udevadm_info
2022-03-19 14:11:21.148 INFO PID: 162587 leapp.workflow.FactsCollection: Executing actor udevadm_info <---- Started
...
...
...
2022-03-19 16:07:09.198 DEBUG PID: 42064 leapp.workflow.FactsCollection.udevadm_info: External command has finished: ['udevadm', 'info', '-e'] <---Ended

$ cat var/log/leapp/leapp-upgrade.log | grep udevadm_info | wc -l
55717 (approx 56K lines)

[pirat89]

@anandanramadoss Hi, in case you still see the problem, report please new issue with all required information inside the template. But first, please ensure you have installed up-to-date versions of leapp & leapp-repository (leapp-upgrade-*) packages and that you have set the expected devel envars. As well, if you test with the current upstream builds (leapp 0.14.0; leapp-repository-0.16.0) the behaviour should be default for the phases until the first reboot.