cmd-generate-hashlist: rename to buildextend-hashlist-experimental #3248
Conversation
|
suggestion: Also, we don't have to backport this to any RHCOS branches because the goal is to not generate the hashlist there? |
We've made hashlist generation look more like just another buildextend command. This allows us to clean up our code related to this. See: coreos/coreos-assembler#3248
|
Will wait until coreos/fedora-coreos-pipeline#776 is stamped before merging this since I didn't bother making this switch ratchetable. Nothing else should be using it.
I'd like to keep some marker that this is experimental and shouldn't be used by anyone other than devs currently. Probably before "stabilizing" it, we should make it a more proper artifact by adding it to the schema under the
Right. We haven't been generating it so far in the old pipeline. Short-term, we can keep not generating it. Once we work out the ppc64le I/O issues, we could turn it on for 4.13 and forward (or wait for it to be explicitly requested). |
counter proposal: Again ^^ optional. |
That way we can call it using the same pattern we call other buildextend commands. It's not a bona-fide artifact (we don't add anything to `meta.json` yet) because it's still experimental. But doing it this way will allow us to easily skip running this on pipeliens where we don't need it. Motivated by issues we're currently hitting in RHCOS with generating the hashlist being excruciatingly slow. Current theory is that the disk is rotational and the random reads that the command triggers by checking out the full commit and hashing everything is extremely expensive on hard drives. I'd like to skip hashlists in RHCOS for now until we can either move parts of it to tmpfs or ideally move to a builder with better storage. But regardless, this will also help us clean up in the pipeline how this command is called.
jlebon
force-pushed
the
pr/hashlist-exp
branch
from
13794a6
to
d4ac721
Compare
jlebon
changed the title
|
|
We've made hashlist generation look more like just another buildextend command. This allows us to clean up our code related to this. See: coreos/coreos-assembler#3248
We've made hashlist generation look more like just another buildextend command. This allows us to clean up our code related to this. See: coreos/coreos-assembler#3248
That way we can call it using the same pattern we call other buildextend commands. It's not a bona-fide artifact (we don't add anything to
meta.jsonyet) because it's still experimental. But doing it this way will allow us to easily skip running this on pipeliens where we don't need it.Motivated by issues we're currently hitting in RHCOS with generating the hashlist being excruciatingly slow. Current theory is that the disk is rotational and the random reads that the command triggers by checking out the full commit and hashing everything is extremely expensive on hard drives. I'd like to skip hashlists in RHCOS for now until we can either move parts of it to tmpfs or ideally move to a builder with better storage.
But regardless, this will also help us clean up in the pipeline how this command is called.