Use write-ahead log for RepositoryContent.#943
Conversation
- Make sure we do not write full snapshots on every publication - Migrate old style RepositoryContent if present - Schedule updating full snapshots every 24 hours
|
Deployed to benchmarking VM with 5000 CAs with 10 ROAs each. The system is responsive and CPU load is low, except for when publication happens. With the additional log messages added we can see that this is due to updating the RRDP state and writing new RRDP files for every update. This should be solved when #693 is implemented. That will allow us to defer RRDP updates and include multiple publisher deltas in one. Furthermore, this will mean that the RRDP are not blocking publisher. The RRDP update process itself can also be optimised more. Currently it's creating the snapshot XML twice (and for a 280 MB XML file this is what's blocking the system). Once to get the hash to include in the notification, and once to write the actual file. We may want to optimise this, and perhaps use the RRDP support from rpki-rs rather than the older RRDP XML support in krill itself. |
|
This needs more work - it's currently saving the entire new snapshot in an RrdpUpdate change. This naive implementation ends up using way too much data. So, will change this and only then mark this for review. The CPU usage seems quite improved at least. |
…r the CA certificate. Add missing openssl.cnf file.
|
The deltas are now much smaller. Overall performance is still not good enough because creating new RRDP snapshots is still too expensive and happens too often. This will be fixed in a separate PR for issue #693. This should be good to merge now, unless other issues are raised during review of course. |
2 participants
[draft] reviewing allowed, but can wait for the following:
Will deploy to benchmark environment to test upgrade and effect.
Still need to update the developer docs.