build: move not copy files to BIN_DIR #10655
Conversation
Building images usally stores them in KDIR_TMP and then copies them over
to BIN_DIR. This is fine as rebuilding of images overwrites existing
images. When using the EXTRA_IMAGE_NAME variable frequently this fills
up the ImageBuilder KDIR_TMP folder since every built image is stored
forever.
This commit moves every image instead of copying it. I couldn't find any
reason which this shouldn't be done. While at it rename FILE_DIR to
BIN_DIR to lower the entropy inside the codebase.
Below an example how sysupgrade.openwrt.org filled up after the release
of 22.03.0 where every created image contains a hash of the package
selction in the filename:
aparcar@asu-01:~/asu/worker1/cache/22.03.0$ du -d 1 -h
400M ./kirkwood
260M ./gemini
2.0G ./ipq806x
1.7G ./ipq40xx
8.1G ./ramips
4.0K ./octeon
495M ./sunxi
728M ./lantiq
1.8G ./rockchip
3.7G ./mediatek
4.0K ./realtek
5.4G ./mvebu
8.9G ./ath79
3.0G ./bcm47xx
14G ./bcm27xx
11G ./x86
4.0K ./bcm63xx
312M ./mpc85xx
600M ./apm821xx
5.4G ./bcm53xx
66G .
Signed-off-by: Paul Spooren <mail@aparcar.org>
|
@ynezz please have a look, you implemented the artifacts stuff a long time ago. |
|
@nbd168 also you please, this is old old Makefile stuff |
|
Okay this is certainly WIP since it breaks when building multiple profiles at once. It would be a fix for the issue described but not for regular setups. |
| @@ -474,7 +474,7 @@ define Device/Build/initramfs | |||
| $(KDIR)/$$(KERNEL_INITRAMFS_NAME):: image_prepare | |||
| $(1)-images: $$(if $$(KERNEL_INITRAMFS),$(BIN_DIR)/$$(KERNEL_INITRAMFS_IMAGE)) | |||
| $(BIN_DIR)/$$(KERNEL_INITRAMFS_IMAGE): $(KDIR)/tmp/$$(KERNEL_INITRAMFS_IMAGE) | |||
| cp $$^ $$@ | |||
| mv $$^ $$@ | |||
There was a problem hiding this comment.
As the ImageBuilder doesn't itself create initramfs images, doesn't that then mean that the image file placed there by the buildroot will be moved away on the first build and unavailable for future builds for the same profile?
There was a problem hiding this comment.
Did you test if this issue actually happens? Looking at Imagebuilder archives I don't see the KDIR_TMP folder being shipped and everything that's moved comes. from KDIR_TMP.
There was a problem hiding this comment.
I just tested this and it works just fine.
|
An alternative implementation is below where a cleanup step is performed after each image create: |
|
Cleaning KDIR_TMP is the easier approach as it touches less things. Closing this |
mans0n
added
the
build/scripts/tools
Building images usally stores them in KDIR_TMP and then copies them over to BIN_DIR. This is fine as rebuilding of images overwrites existing images. When using the EXTRA_IMAGE_NAME variable frequently this fills up the ImageBuilder KDIR_TMP folder since every built image is stored forever.
This commit moves every image instead of copying it. I couldn't find any reason which this shouldn't be done. While at it rename FILE_DIR to BIN_DIR to lower the entropy inside the codebase.
Below an example how sysupgrade.openwrt.org filled up after the release of 22.03.0 where every created image contains a hash of the package selction in the filename:
Signed-off-by: Paul Spooren mail@aparcar.org