Topic/repeatable #9
Conversation
The gz format puts a timestamp in the compressed file. Therefore repeated runs produce different gz files even though the input data is the same. Use a timestamp of 0 to ensure the compressed data is the same when the input data is the same. Neither the gzip executable nor the libarchive library provide a way to specify the timestamp explicitly, so we set the modification time of the input file. This is going to be removed anyway, so it doesn't matter what it's set to.
Parallelism gives rise to a non-deterministic ordering of the icons and hints. Removing parallelism makes no appreciable difference to the speed.
|
Ha, nice! Fixing nondeterminism was on my TODO list right after fonts support, so this is very welcome! |
|
Oh, btw, if you want to modify templates in a way that doesn't involve modifying upstream-shipped ones: You can supply your own templates, even partial ones, via the And: removing parallelism for the icon search doesn't make any difference on small repos, it has an impact on very large ones (once you have more than 1000 packages), like in Debian. |
|
Okay, the changes done in master in the past few days should catch all the use-cases those patches would have provided (but are a bit more holistic in their approach). |
These are some changes I've made for use with my own repo and workflow, but you may find some or all of them useful.
I run the generator on all suites one per day, from a cronjob. However, most days the repo hasn't changed, since this is a small, 3rd-party repo. I then copy the new metadata into the repo. However,
asgenproduces different output every time, even when there have been no changes to the repo, This results in a lot of churn when the master copy of the repo is pushed to the mirror that people actually have access to, which in turn produces unnecessary emails to my inbox.I found that the churn was occurring in three areas:
Each of these commits fixes one of those.
I copy the data from workspace to repo using
rsync -rlHc. That changes a destination file only if the source file has a different checksum. However, it doesn't copy across the modification time like-adoes, so a destination file with the same checksum doesn't have its modification date changed, but files with a different checksum have their modification time set to the time of the copy. I can then usefindto see if there are any files in thedep11subdirectory that are newer than theReleasefile, and if so the suite needs to be reexported (in my case, withreprepo export SUITE) and a mirror push initiated.This change is