Fix README and sstate issues #1

liuming50 · 2018-12-07T11:45:14Z

No description provided.

agners

Hi Ming,

Thanks for your improvements! Some review though. Especially, can you split out the non-controversial part so we can get this merged soon?

Thanks!

agners · 2018-12-07T12:23:20Z

classes/image_type_tezi_run.bbclass

+python () {
+    if bb.utils.contains("IMAGE_FSTYPES", "tezirunimg", "1", "0", d) == "1":
+        bb.build.addtask('do_assemble_tezirunimg', 'do_image_complete', 'do_image_squashfs', d)
+}


This adds a new task which is before do_image_complete and after do_image_squashfs, right?

Isn't that what a image type would do with squashfs as dependency (e.g. what we had before with IMAGE_TYPEDEP_tezirunimg += "squashfs")? I don't quite get the difference here, it seems to me a bit of misuse of the image_type when we don't use IMAGE_CMD_... and friends...

Yes, usually it could be handled by IMAGE_TYPEDEP, but the tricky part here is, the do_image_tezirunimg task is a bundle of python tasks and shell tasks, but IMAGE_CMD only support shell tasks, so we can not make do_image_tezirunimg a IMAGE_CMD, and we also can not make it to be a empty IMAGE_CMD as well, since there are quite some processes in image.bbclass to add prefuncs and so on to IMAGE_CMD, they could not work with RM_WORK, for instance, the set_image_size task requires IMAGE_ROOTFS, but it does not always exist since RM_WORK is being enabled, which means the IMAGE_CMD might fail if its task hash changed(or its dependent tasks' hashes) while the do_rootfs task hash not changed. So we have to mask the processes in image.bbclass with IMAGE_TYPES_MASKED += "tezirunimg", so that's why we need add the task dependency do_assemble_tezirunimg -> do_image_squashfs in this way.

agners · 2018-12-07T12:24:07Z

classes/image_type_tezi_run.bbclass

 TEZI_DISTRO_BOOT_SCRIPTS ??= "boot-sdp.scr boot.scr"
 UBOOT_BINARY ??= "u-boot.${UBOOT_SUFFIX}"
 TEZI_UBOOT_BINARY_EMMC ??= "${UBOOT_BINARY}"
 TEZI_UBOOT_BINARY_RAWNAND ??= "${UBOOT_BINARY}"
 TEZI_UBOOT_BINARY_RECOVERY ??= "${UBOOT_BINARY}"
+TEZI_UBOOT_BINARIES ??= "${@' '.join(x for x in sorted(set([bb.utils.contains('TORADEX_FLASH_TYPE', 'emmc', d.getVar('TEZI_UBOOT_BINARY_EMMC', True), '', d), \
+                         bb.utils.contains('TORADEX_FLASH_TYPE', 'rawnand', d.getVar('TEZI_UBOOT_BINARY_RAWNAND', True), '', d), \
+                         d.getVar('TEZI_UBOOT_BINARY_RECOVERY', True)])))}"


Populating TEZI_UBOOT_BINARIES outside of python is nicer, can you split this change out into a separate commit? Than we can merge that on its own.

Yes, sure, will split it in V2.

agners · 2018-12-07T12:25:06Z

classes/image_type_tezi_run.bbclass

+	if [ -e ${IMGDEPLOYDIR}/${IMAGE_BASENAME}-${MACHINE}.squashfs ]; then
+		cp ${IMGDEPLOYDIR}/${IMAGE_BASENAME}-${MACHINE}.squashfs ${WORKDIR}/tmp-fitimage
+	else
+		cp ${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.squashfs ${WORKDIR}/tmp-fitimage


That looks a bit hacky :-)

Yes, it is. I can explain the idea here.

In the original implementation, why the artifact could not be covered by sstate is we were adding do_assemble_tezirunimg after do_image_complete, and why we were doing that is because do_assemble_tezirunimg requires kernel images, dtbs, and squashfs in the same directory with its file, to assemble the fitimage, so, to be able to make all the files in the same directories with its file, we copy them into ${WORKDIR}/tmp-fitimage, including the squashfs image, and that file could be untared from sstate (in DEPLOY_DIR_IMAGE) or generated by do_image_squashfs (in IMGDEPLOYDIR).

Yes, I agree, this part is really tricky.

I see what you are saying, but when we have a IMAGE_TYPEDEP_tezirunimg it is somehow guaranteed that the squashfs is in the right place...

Can we maybe somehow enable python with IMAGE_CMD?

agners · 2018-12-07T12:26:36Z

classes/image_type_tezi_run.bbclass

+do_assemble_tezirunimg[cleandirs] = "${WORKDIR}/tmp-fitimage ${WORKDIR}/tmp-tezirunimage"
+do_assemble_tezirunimg[depends] = "virtual/bootloader:do_deploy u-boot-distro-boot:do_deploy virtual/kernel:do_deploy \
+                                 tezi-run-metadata:do_deploy u-boot-mkimage-native:do_populate_sysroot \
+                                 p7zip-native:do_populate_sysroot zip-native:do_populate_sysroot \


I guess those dependency fixes are somewhat independent on the image type change. Can you also create a separate patch which fixes do_image_tezirunimg[depends]?

Sure, will do that in V2.

liuming50 · 2018-12-07T20:04:27Z

OK, will think about that. //Ming Liu Stefan Agner <notifications@github.com> 於 2018年12月7日週五下午4:19寫道：

…

***@***.**** commented on this pull request. ------------------------------ In classes/image_type_tezi_run.bbclass <#1 (comment)> : > - # zip does update if the file exist, explicitly delete before adding files to the archive - if [ -e ${IMAGE_NAME}.zip ]; then - rm ${IMAGE_NAME}.zip + # Copy kernel image + cp ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE} ${WORKDIR}/tmp-fitimage + + # Copy device tree + for dtb in ${KERNEL_DEVICETREE}; do + cp ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE}-$dtb ${WORKDIR}/tmp-fitimage + done + + # Copy rootfs image + if [ -e ${IMGDEPLOYDIR}/${IMAGE_BASENAME}-${MACHINE}.squashfs ]; then + cp ${IMGDEPLOYDIR}/${IMAGE_BASENAME}-${MACHINE}.squashfs ${WORKDIR}/tmp-fitimage + else + cp ${DEPLOY_DIR_IMAGE}/${IMAGE_BASENAME}-${MACHINE}.squashfs ${WORKDIR}/tmp-fitimage I see what you are saying, but when we have a IMAGE_TYPEDEP_tezirunimg it is somehow guaranteed that the squashfs is in the right place... Can we maybe somehow enable python with IMAGE_CMD? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHsYsj34Ou5k6Ngl63qlNBodQ1QshPinks5u2ob2gaJpZM4ZIS2a> .

liuming50 · 2018-12-10T16:45:54Z

Hi, Stefan:

I have split the commits and modified to use the IMAGE_TYPEDEP_tezirunimg as you suggested, please kindly help me review them.

agners · 2018-12-11T08:41:46Z

Thanks for the update! Patch 1-3 look good, I will apply them.

However, with the last one I am still not quite happy. Two questions:
Why do you think we need the change? You say that it is sstate tracked, but do we care? What is the advantage?

It now feels a bit hacky that we have to omit rm_work... Can we maybe do it entirly differently by creating the fit image as part of the kernel recipe, similar to how it is done in openembedded-core/meta/classes/kernel-fitimage.bbclass? This probably would need that the kernel depends on the image, which is a bit wired, but could work?

liuming50 · 2018-12-11T10:18:56Z

Hi, Stefan: I am on board the changes are really hacky, since we are doing a fitimage with the normal rootfs, and the paths of kernel/dtb/rootfs are being hard-coded in .its file, hence the assembling task would reply on the output of its dependent tasks to be present when it executes...and so on I am totally on board, the better way to do this is inheriting kernel-fitimage.bbclass //Ming Liu Stefan Agner <notifications@github.com> 於 2018年12月11日週二上午9:41寫道：

…

Thanks for the update! Patch 1-3 look good, I will apply them. However, with the last one I am still not quite happy. Two questions: Why do you think we need the change? You say that it is sstate tracked, but do we care? What is the advantage? It now feels a bit hacky that we have to omit rm_work... Can we maybe do it entirly differently by creating the fit image as part of the kernel recipe, similar to how it is done in openembedded-core/meta/classes/kernel-fitimage.bbclass? This probably would need that the kernel depends on the image, which is a bit wired, but could work? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHsYsrNseZDs-qWFuaNu0aT_eqUofOe9ks5u32_LgaJpZM4ZIS2a> .

It could be used to deploy a docker compose file to a rootfs, meanwhile create and enable a systemd service to start the containers in it. Two parameters are added to the original dcsg: --root-dir: install the docker-compose and systemd service to a specific root directory. --compose-dir: the users could install the docker compose file to this directory, it defaults to /opt/containers. a native dcsg is also supported, to have a try, please follow: ``` $ bitbake dcsg-native $ ./tmp-glibc/sysroots-components/x86_64/dcsg-native/usr/bin/dcsg --root-dir /tmp/dcsg install ./my-compose/docker-compose.yml $ tree /tmp/dcsg /tmp/dcsg/ ├── etc │ └── systemd │ └── system │ ├── multi-user.target.wants │ │ └── mycompose.service -> /etc/systemd/system/mycompose.service │ └── mycompose.service └── opt └── containers └── docker-compose.yml ``` Signed-off-by: Ming Liu <liu.ming50@gmail.com>

According to ba2ad0ee4f61("rdp: don't release the seat until it is safe v2"), releasing a seat is not safe because of design pitfalls and ill-conceived syncronization [1], which leads to vnc-backend crash with this error: wl_registry@2: error 0: invalid global wl_seat (13) weston_seat_release already calls weston_keyboard_destroy(seat->keyboardstate) but later wl_resource_destroy->destroy_resource->wl_list_remove accesses this memory. address sanitizer report : ==10695==ERROR: AddressSanitizer: heap-use-after-free on address 0x611000020d50 at pc 0x7f05e9f6c567 bp 0x7ffee886bf10 sp 0x7ffee886bf00 WRITE of size 8 at 0x611000020d50 thread T0 #0 0x7f05e9f6c566 in wl_list_remove /usr/src/debug/dev-libs/wayland-9999/wayland-9999/src/wayland-util.c:57 #1 0x7f05e9f5df7a in destroy_resource /usr/src/debug/dev-libs/wayland-9999/wayland-9999/src/wayland-server.c:571 #2 0x7f05e9f5f89e in wl_resource_destroy /usr/src/debug/dev-libs/wayland-9999/wayland-9999/src/wayland-server.c:584 #3 0x7f05e84cae2f in ffi_call_unix64 (/usr/lib64/libffi.so.6+0xce2f) #4 0x7f05e84c9a2d in ffi_call (/usr/lib64/libffi.so.6+0xba2d) #5 0x7f05e9f6af75 in wl_closure_invoke /usr/src/debug/dev-libs/wayland-9999/wayland-9999/src/connection.c:949 #6 0x7f05e9f603b5 in wl_client_connection_data /usr/src/debug/dev-libs/wayland-9999/wayland-9999/src/wayland-server.c:337 #7 0x7f05e9f650d1 in wl_event_loop_dispatch /usr/src/debug/dev-libs/wayland-9999/wayland-9999/src/event-loop.c:421 #8 0x7f05e9f611af in wl_display_run /usr/src/debug/dev-libs/wayland-9999/wayland-9999/src/wayland-server.c:1051 #9 0x40a333 in main src/main.c:859 #10 0x7f05e8ea459f in __libc_start_main (/lib64/libc.so.6+0x2059f) #11 0x40a8c8 in _start (/usr/bin/weston+0x40a8c8) 0x611000020d50 is located 16 bytes inside of 232-byte region [0x611000020d40,0x611000020e28) freed by thread T0 here: #0 0x7f05ea1d455f in __interceptor_free (/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/libasan.so.1+0x5755f) #1 0x42c92c in weston_seat_release src/input.c:2675 previously allocated by thread T0 here: #0 0x7f05ea1d4935 in calloc (/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.3/libasan.so.1+0x57935) #1 0x423e6f in zalloc shared/zalloc.h:38 #2 0x423e6f in weston_keyboard_create src/input.c:756 As the solution isn't found yet for RDP, lets use the same workaround till it's correctly fixed. [1] https://gitlab.freedesktop.org/wayland/weston/issues/72 [2] https://bugs.freedesktop.org/show_bug.cgi?id=94519 Relates-to: TEI-496 Signed-off-by: Igor Opaniuk <igor.opaniuk@toradex.com>

Update patch to remove fuzz, fixing the following bitbake warning: WARNING: curl-7.82.0-r0 do_patch: Fuzz detected: Applying patch 0001-tool_operate-resume-at-last-byte-when-retrying.patch patching file src/tool_operate.c Hunk #1 succeeded at 468 with fuzz 2 (offset -47 lines). Hunk #2 succeeded at 535 (offset -47 lines). Signed-off-by: Hiago De Franco <hiago.franco@toradex.com>

agners suggested changes Dec 7, 2018

View reviewed changes

liuming50 force-pushed the rocko branch from c0892c5 to 7fd7f8b Compare December 10, 2018 16:42

liuming50 force-pushed the rocko branch from 7fd7f8b to aac3d81 Compare December 16, 2018 14:18

hiagofranco closed this Apr 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix README and sstate issues #1

Fix README and sstate issues #1

liuming50 commented Dec 7, 2018

agners left a comment

agners Dec 7, 2018

liuming50 Dec 7, 2018

agners Dec 7, 2018

liuming50 Dec 7, 2018

agners Dec 7, 2018

liuming50 Dec 7, 2018

liuming50 Dec 7, 2018

agners Dec 7, 2018

agners Dec 7, 2018

liuming50 Dec 7, 2018

liuming50 commented Dec 7, 2018 via email

liuming50 commented Dec 10, 2018

agners commented Dec 11, 2018

liuming50 commented Dec 11, 2018 via email

Fix README and sstate issues #1

Fix README and sstate issues #1

Conversation

liuming50 commented Dec 7, 2018

agners left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liuming50 commented Dec 7, 2018 via email

liuming50 commented Dec 10, 2018

agners commented Dec 11, 2018

liuming50 commented Dec 11, 2018 via email