libc/vfs: tool/create_builddir on Genode creates invalid etc/build.conf #3991

alex-ab · 2021-01-19T12:53:57Z

The created x86_64/etc/build.conf by invoking tool/create_builddir x86_64 on Genode looks as follows (start of file):

#
# Drivers ported from the OpenBSD
#
REPOSITORIES += $(GENODE_DIR)/repos/dde_bsd

#
# Drivers ported from iPXE
#                                             
REPOSITORIES += $(GENODE_DIR)/repos/dde_ipxe

rio
##
REPOSITORIES += $(GENODE_DIR)/repos/base
REPOSITORIES += $(GENODE_DIR)/repos/os
REPOSITORIES += $(GENODE_DIR)/repos/demo

...

During instrumenting a bit tool/create_builddir witch some debugging echos lead to a good build.conf. So it seems, that a race exists between closing the file and re-opening by another component that start another append by using cat.

The text was updated successfully, but these errors were encountered:

chelmuth · 2021-01-19T13:14:48Z

Out of curiosity: The tool was not run with -j<num>? I'm asking because create_builddir does not explicitly guard against parallel runs like follows.

--- a/tool/create_builddir
+++ b/tool/create_builddir
@@ -211,3 +211,5 @@ message:
 	@echo "Please adjust $(BUILD_DIR)/etc/build.conf according to your needs."
 
 .PHONY: $(PLATFORM)
+
+.NOTPARALLEL: $(PLATFORM)

alex-ab · 2021-01-19T13:25:32Z

No, -j using on Genode I did not tried yet, this is for later. On issue_3991 the tool chain pkg with the tool chain is available and testable with depot_deploy.

nfeske · 2021-01-19T15:02:56Z

This reminds me of an issue with a missing flush operation of a written-to file that I observed (and fixed) with the genodians.org scenario on NOVA. The flushing happens when closing a file descriptors. It is important that the libc-internal close_file_descriptors_on_exit function is called when the program exits, regardless of how it exits. The function is registered as atexit handler.

Maybe there exists a code path that exits the program w/o doing the atexit processing?

The fds in O_APPEND state may have changed by the child. The local seek state needs to be valid/current for the next to be forked child, which seek state is set by File_descriptor_allocator::generate_info(). Issue genodelabs#3991

If fd is opened in append mode and just used (so never written) for a forked child, the seek state was not pointing to the end of the file. The wrong seek value then was used in File_descriptor_allocator::generate_info(). Issue genodelabs#3991

If a fd is opened in append mode and just is to be used (so never written by the parent component) for the to be forked child, the seek state was not pointing to the end of the file. The wrong seek value then was used in File_descriptor_allocator::generate_info(). Issue genodelabs#3991

alex-ab · 2021-01-20T09:15:59Z

It turned out that the seek state is inconsistent of file descriptors used in O_APPEND mode when used together with fork+wait. With the both commits 2 corner cases get addressed and fix for me the wrong content of the generated build.conf.

The fds in O_APPEND state may have changed by the child. The local seek state needs to be valid/current for the next to be forked child, which seek state is set by File_descriptor_allocator::generate_info(). Issue #3991

If a fd is opened in append mode and just is to be used (so never written by the parent component) for the to be forked child, the seek state was not pointing to the end of the file. The wrong seek value then was used in File_descriptor_allocator::generate_info(). Issue #3991

nfeske · 2021-01-20T12:46:36Z

Thank you for investigating and fixing the problem. I'm surprised that we have not encountered it before, e.g., in the genodians.org scenario.

I merged both commits to staging.

The fds in O_APPEND state may have changed by the child. The local seek state needs to be valid/current for the next to be forked child, which seek state is set by File_descriptor_allocator::generate_info(). Issue genodelabs#3991

If a fd is opened in append mode and just is to be used (so never written by the parent component) for the to be forked child, the seek state was not pointing to the end of the file. The wrong seek value then was used in File_descriptor_allocator::generate_info(). Issue genodelabs#3991

Issue genodelabs#3991 [init -> ...] Error: LD: jump slot relocation failed for symbol: 'XYZ' [init -> ...] Error: deadlock ahead, mutex=0x161860, return ip=0xeb870 turns into: [init -> ...] Error: LD: jump slot relocation failed for symbol: 'XYZ' [init -> ...] Error: Uncaught exception of type 'Linker::Not_found' [init -> ...] Warning: abort called - thread: ep

The fds in O_APPEND state may have changed by the child. The local seek state needs to be valid/current for the next to be forked child, which seek state is set by File_descriptor_allocator::generate_info(). Issue genodelabs#3991

If a fd is opened in append mode and just is to be used (so never written by the parent component) for the to be forked child, the seek state was not pointing to the end of the file. The wrong seek value then was used in File_descriptor_allocator::generate_info(). Issue genodelabs#3991

The fds in O_APPEND state may have changed by the child. The local seek state needs to be valid/current for the next to be forked child, which seek state is set by File_descriptor_allocator::generate_info(). Issue #3991

If a fd is opened in append mode and just is to be used (so never written by the parent component) for the to be forked child, the seek state was not pointing to the end of the file. The wrong seek value then was used in File_descriptor_allocator::generate_info(). Issue #3991

The fds in O_APPEND state may have changed by the child. The local seek state needs to be valid/current for the next to be forked child, which seek state is set by File_descriptor_allocator::generate_info(). Issue genodelabs#3991

If a fd is opened in append mode and just is to be used (so never written by the parent component) for the to be forked child, the seek state was not pointing to the end of the file. The wrong seek value then was used in File_descriptor_allocator::generate_info(). Issue genodelabs#3991

Issue genodelabs#3991 [init -> ...] Error: LD: jump slot relocation failed for symbol: 'XYZ' [init -> ...] Error: deadlock ahead, mutex=0x161860, return ip=0xeb870 turns into: [init -> ...] Error: LD: jump slot relocation failed for symbol: 'XYZ' [init -> ...] Error: Uncaught exception of type 'Linker::Not_found' [init -> ...] Warning: abort called - thread: ep Fixes genodelabs#4002

The fds in O_APPEND state may have changed by the child. The local seek state needs to be valid/current for the next to be forked child, which seek state is set by File_descriptor_allocator::generate_info(). Issue genodelabs#3991

If a fd is opened in append mode and just is to be used (so never written by the parent component) for the to be forked child, the seek state was not pointing to the end of the file. The wrong seek value then was used in File_descriptor_allocator::generate_info(). Issue genodelabs#3991

Issue #3991 [init -> ...] Error: LD: jump slot relocation failed for symbol: 'XYZ' [init -> ...] Error: deadlock ahead, mutex=0x161860, return ip=0xeb870 turns into: [init -> ...] Error: LD: jump slot relocation failed for symbol: 'XYZ' [init -> ...] Error: Uncaught exception of type 'Linker::Not_found' [init -> ...] Warning: abort called - thread: ep Fixes #4002

The fds in O_APPEND state may have changed by the child. The local seek state needs to be valid/current for the next to be forked child, which seek state is set by File_descriptor_allocator::generate_info(). Issue genodelabs#3991

If a fd is opened in append mode and just is to be used (so never written by the parent component) for the to be forked child, the seek state was not pointing to the end of the file. The wrong seek value then was used in File_descriptor_allocator::generate_info(). Issue genodelabs#3991

The fds in O_APPEND state may have changed by the child. The local seek state needs to be valid/current for the next to be forked child, which seek state is set by File_descriptor_allocator::generate_info(). Issue genodelabs#3991

If a fd is opened in append mode and just is to be used (so never written by the parent component) for the to be forked child, the seek state was not pointing to the end of the file. The wrong seek value then was used in File_descriptor_allocator::generate_info(). Issue genodelabs#3991

Issue #3991 [init -> ...] Error: LD: jump slot relocation failed for symbol: 'XYZ' [init -> ...] Error: deadlock ahead, mutex=0x161860, return ip=0xeb870 turns into: [init -> ...] Error: LD: jump slot relocation failed for symbol: 'XYZ' [init -> ...] Error: Uncaught exception of type 'Linker::Not_found' [init -> ...] Warning: abort called - thread: ep Fixes #4002

alex-ab added the bug label Jan 19, 2021

alex-ab added the fixed label Jan 20, 2021

alex-ab closed this as completed Jan 25, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

libc/vfs: tool/create_builddir on Genode creates invalid etc/build.conf #3991

libc/vfs: tool/create_builddir on Genode creates invalid etc/build.conf #3991

alex-ab commented Jan 19, 2021 •

edited

Loading

chelmuth commented Jan 19, 2021

alex-ab commented Jan 19, 2021

nfeske commented Jan 19, 2021

alex-ab commented Jan 20, 2021 •

edited

Loading

nfeske commented Jan 20, 2021

libc/vfs: tool/create_builddir on Genode creates invalid etc/build.conf #3991

libc/vfs: tool/create_builddir on Genode creates invalid etc/build.conf #3991

Comments

alex-ab commented Jan 19, 2021 • edited Loading

chelmuth commented Jan 19, 2021

alex-ab commented Jan 19, 2021

nfeske commented Jan 19, 2021

alex-ab commented Jan 20, 2021 • edited Loading

nfeske commented Jan 20, 2021

alex-ab commented Jan 19, 2021 •

edited

Loading

alex-ab commented Jan 20, 2021 •

edited

Loading