Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release fast doesn't work in complex project: ld.lld: error: Invalid value reference from metadata Global is external, but doesn't have external or weak linkage! #13349

Open
IntegratedQuantum opened this issue Oct 29, 2022 · 5 comments
Labels
bug Observed behavior contradicts documented or intended behavior
Milestone

Comments

@IntegratedQuantum
Copy link
Contributor

Zig Version

0.10.0-dev.4640+20925b2f5

Steps to Reproduce and Observed Behavior

I tried to reduce this error, but I was unable to do that. Whenever I remove too many lines of code the error transforms into #13348. I think the error requires a certain function(+inlined calls) size. Maybe llvm tries to make the function external if it gets too big?

Anyways because I wasn't able to reduce it, here is a link to the exact commit that I tested it with: PixelGuys/Cubyz@d4e9fa2

This is the output that I get:

LLD Link... error(link): /home/mint/Downloads/zig/zig terminated with stderr:
ld.lld: error: Invalid value reference from metadata
Global is external, but doesn't have external or weak linkage!
ptr @renderer.render
Global is external, but doesn't have external or weak linkage!
ptr @"log.scoped(.default).err__anon_20404"
Global is external, but doesn't have external or weak linkage!
ptr @Thread.Mutex.FutexImpl.lockSlow
Global is external, but doesn't have external or weak linkage!
ptr @network.Connection.sendUnimportant
Global is external, but doesn't have external or weak linkage!
ptr @main.Window.toggleFullscreen
Global is external, but doesn't have external or weak linkage!
ptr @fs.file.File.write
Global is external, but doesn't have external or weak linkage!
ptr @fmt.formatBuf__anon_26731
Global is external, but doesn't have external or weak linkage!
ptr @network.Connection.disconnect
Global is external, but doesn't have external or weak linkage!
ptr @Thread.Condition.FutexImpl.wait
Global is external, but doesn't have external or weak linkage!
ptr @"heap.general_purpose_allocator.GeneralPurposeAllocator(.{.stack_trace_frames = 0, .enable_memory_limit = false, .safety = false, .thread_safe = true, .MutexType = null, .never_unmap = false, .retain_metadata = false, .verbose_log = false}).freeLarge"
Global is external, but doesn't have external or weak linkage!
ptr @graphics.TextureArray.generate
Global is external, but doesn't have external or weak linkage!
ptr @"mem.SplitIterator(u8).first"
Global is external, but doesn't have external or weak linkage!
ptr @"hash_map.HashMap([]const u8,json.JsonElement,hash_map.StringContext,80).put"
Global is external, but doesn't have external or weak linkage!
ptr @json.JsonElement.free
Global is external, but doesn't have external or weak linkage!
ptr @mem.Allocator.reallocAdvancedWithRetAddr__anon_22386
Global is external, but doesn't have external or weak linkage!
ptr @json.JsonElement.recurseToString__anon_19352
Global is external, but doesn't have external or weak linkage!
ptr @network.Connection.sendImportant
Global is external, but doesn't have external or weak linkage!
ptr @Thread.PosixThreadImpl.spawn__anon_20992.Instance.entryFn
Global is external, but doesn't have external or weak linkage!
ptr @crypto.xoodoo.State.permute
Global is external, but doesn't have external or weak linkage!
ptr @network.Protocols.Protocols__struct_18322.receive
Global is external, but doesn't have external or weak linkage!
ptr @"log.scoped(.default).err__anon_20929"
Global is external, but doesn't have external or weak linkage!
ptr @network.Protocols.Protocols__struct_18332.receive
Global is external, but doesn't have external or weak linkage!
ptr @"log.scoped(.default).err__anon_20930"
Global is external, but doesn't have external or weak linkage!
ptr @network.Protocols.Protocols__struct_18336.receive
Global is external, but doesn't have external or weak linkage!
ptr @"log.scoped(.default).err__anon_20931"
Global is external, but doesn't have external or weak linkage!
ptr @network.Protocols.Protocols__struct_18340.receive
Global is external, but doesn't have external or weak linkage!
ptr @"log.scoped(.default).err__anon_20932"
Global is external, but doesn't have external or weak linkage!
ptr @network.Protocols.Protocols__struct_18345.receive
Global is external, but doesn't have external or weak linkage!
ptr @"log.scoped(.default).err__anon_20933"
Global is external, but doesn't have external or weak linkage!
ptr @network.Protocols.Protocols__struct_18349.receive
Global is external, but doesn't have external or weak linkage!
ptr @"log.scoped(.default).err__anon_20934"
Global is external, but doesn't have external or weak linkage!
ptr @network.Protocols.Protocols__struct_18355.receive
Global is external, but doesn't have external or weak linkage!
ptr @"log.scoped(.default).err__anon_20935"
Global is external, but doesn't have external or weak linkage!
ptr @network.Protocols.Protocols__struct_18359.receive
Global is external, but doesn't have external or weak linkage!
ptr @"log.scoped(.default).err__anon_20936"
Global is external, but doesn't have external or weak linkage!
ptr @network.Protocols.Protocols__struct_18363.receive
Global is external, but doesn't have external or weak linkage!
ptr @"log.scoped(.default).err__anon_20937"
Global is external, but doesn't have external or weak linkage!
ptr @renderer.buffers.updateBufferSize
Global is external, but doesn't have external or weak linkage!
ptr @"array_list.ArrayListAligned(u32,null).ensureTotalCapacityPrecise"
Global is external, but doesn't have external or weak linkage!
ptr @fs.Dir.openDirFlagsZ
Global is external, but doesn't have external or weak linkage!
ptr @fs.IterableDir.Iterator__struct_11199.next
Global is external, but doesn't have external or weak linkage!
ptr @"array_list.ArrayListAligned([]const u8,null).append"
Global is external, but doesn't have external or weak linkage!
ptr @assets.readAllJsonFilesInAddons
Global is external, but doesn't have external or weak linkage!
ptr @"array_list.ArrayListAligned([]const u8,null).deinit"
Global is external, but doesn't have external or weak linkage!
ptr @"array_list.ArrayListAligned(fs.Dir,null).deinit"
Global is external, but doesn't have external or weak linkage!
ptr @mem.Allocator.init.gen.allocImpl
Global is external, but doesn't have external or weak linkage!
ptr @mem.Allocator.init__anon_20694.gen.resizeImpl
Global is external, but doesn't have external or weak linkage!
ptr @mem.Allocator.init.gen.freeImpl
Global is external, but doesn't have external or weak linkage!
ptr @heap.PageAllocator.alloc
Global is external, but doesn't have external or weak linkage!
ptr @heap.PageAllocator.resize
Global is external, but doesn't have external or weak linkage!
ptr @heap.PageAllocator.free
Global is external, but doesn't have external or weak linkage!
ptr @graphics.Shader.addShader
Global is external, but doesn't have external or weak linkage!
ptr @"log.scoped(.default).err__anon_17538"
Global is external, but doesn't have external or weak linkage!
ptr @mem.Allocator.init__anon_20485.gen.freeImpl
Global is external, but doesn't have external or weak linkage!
ptr @"log.scoped(.default).err__anon_20631"
Global is external, but doesn't have external or weak linkage!
ptr @debug.panicExtra__anon_20484
Global is external, but doesn't have external or weak linkage!
ptr @unicode.utf8CountCodepoints
Global is external, but doesn't have external or weak linkage!
ptr @"io.writer.Writer(*io.fixed_buffer_stream.FixedBufferStream([]u8),error{NoSpaceLeft},(function 'write')).writeByteNTimes"
Global is external, but doesn't have external or weak linkage!
ptr @"hash_map.HashMapUnmanaged(usize,heap.general_purpose_allocator.GeneralPurposeAllocator(.{.stack_trace_frames = 0, .enable_memory_limit = false, .safety = false, .thread_safe = true, .MutexType = null, .never_unmap = false, .retain_metadata = false, .verbose_log = false}).LargeAlloc,hash_map.AutoContext(usize),80).grow"
Global is external, but doesn't have external or weak linkage!
ptr @"heap.general_purpose_allocator.GeneralPurposeAllocator(.{.stack_trace_frames = 0, .enable_memory_limit = false, .safety = false, .thread_safe = true, .MutexType = null, .never_unmap = false, .retain_metadata = false, .verbose_log = false}).createBucket"
Global is external, but doesn't have external or weak linkage!
ptr @mem.Allocator.init__anon_20485.gen.resizeImpl
Global is external, but doesn't have external or weak linkage!
ptr @os.openatZ
Global is external, but doesn't have external or weak linkage!
ptr @mem.Allocator.init__anon_20470.gen.allocImpl
Global is external, but doesn't have external or weak linkage!
ptr @mem.Allocator.init__anon_20470.gen.resizeImpl
Global is external, but doesn't have external or weak linkage!
ptr @mem.Allocator.init__anon_20470.gen.freeImpl
LLVM ERROR: Broken module found, compilation aborted!

error: LLDCrashed
error: Cubyzig...
error: The following command exited with error code 1:
/home/mint/Downloads/zig/zig build-exe /home/mint/Desktop/Cubyzig/src/main.zig -lc -cflags -g -std=c99 -D_GLFW_X11 -- /home/mint/Desktop/Cubyzig/lib/glfw/src/linux_joystick.c /home/mint/Desktop/Cubyzig/lib/glfw/src/x11_init.c /home/mint/Desktop/Cubyzig/lib/glfw/src/x11_monitor.c /home/mint/Desktop/Cubyzig/lib/glfw/src/x11_window.c /home/mint/Desktop/Cubyzig/lib/glfw/src/xkb_unicode.c /home/mint/Desktop/Cubyzig/lib/glfw/src/posix_time.c /home/mint/Desktop/Cubyzig/lib/glfw/src/posix_thread.c /home/mint/Desktop/Cubyzig/lib/glfw/src/glx_context.c /home/mint/Desktop/Cubyzig/lib/glfw/src/egl_context.c /home/mint/Desktop/Cubyzig/lib/glfw/src/osmesa_context.c /home/mint/Desktop/Cubyzig/lib/glfw/src/context.c /home/mint/Desktop/Cubyzig/lib/glfw/src/init.c /home/mint/Desktop/Cubyzig/lib/glfw/src/input.c /home/mint/Desktop/Cubyzig/lib/glfw/src/monitor.c /home/mint/Desktop/Cubyzig/lib/glfw/src/vulkan.c /home/mint/Desktop/Cubyzig/lib/glfw/src/window.c -lX11 -lGL -cflags -g -- /home/mint/Desktop/Cubyzig/lib/glad.c /home/mint/Desktop/Cubyzig/lib/stb_image.c /home/mint/Desktop/Cubyzig/lib/cross_platform_udp_socket.c -OReleaseFast --cache-dir /home/mint/Desktop/Cubyzig/zig-cache --global-cache-dir /home/mint/.cache/zig --name Cubyzig -I /home/mint/Desktop/Cubyzig/include --enable-cache 
error: the following build command failed with exit code 1:
/home/mint/Desktop/Cubyzig/zig-cache/o/8319b4f5701387a4e13ff51606d15bee/build /home/mint/Downloads/zig/zig /home/mint/Desktop/Cubyzig /home/mint/Desktop/Cubyzig/zig-cache /home/mint/.cache/zig run -Drelease-fast=true

Expected Behavior

It should compile without an error like in release-small and release-safe and debug.

@IntegratedQuantum IntegratedQuantum added the bug Observed behavior contradicts documented or intended behavior label Oct 29, 2022
@andrewrk
Copy link
Member

As a workaround, you might try disabling LTO.

@andrewrk andrewrk added this to the 0.11.0 milestone Oct 31, 2022
@IntegratedQuantum
Copy link
Contributor Author

IntegratedQuantum commented Oct 31, 2022

It compiles.
However it doesn't run correctly.
It's outputting wrong data to the network which in result leads to a crash.

I managed to find the relevant code. It appears to be the same piece of code as in #13348, so it's likely the same issue:

pub fn sendRequest(conn: *Connection, requests: []chunk.ChunkPosition) !void {
	if(requests.len == 0) return;
	var data = try main.threadAllocator.alloc(u8, 16*requests.len);
	defer main.threadAllocator.free(data);
	var remaining = data;
	for(requests) |req| {
		std.mem.writeIntBig(chunk.ChunkCoordinate, remaining[0..4], req.wx);
		std.mem.writeIntBig(chunk.ChunkCoordinate, remaining[4..8], req.wy);
		std.mem.writeIntBig(chunk.ChunkCoordinate, remaining[8..12], req.wz);
		std.mem.writeIntBig(chunk.ChunkCoordinate, remaining[12..16], req.voxelSize); // ← this line
		remaining = remaining[16..];
	}
	try conn.sendImportant(id, data);
}

req.voxelSize is a u31 and it only ever has the values {1, 2, 4, 8, 16, 32}
chunk.ChunkCoordinate is i32
Here a sample of the data it actually produced:

      wx, wy, wz are equivalent to a debug build       all of these should be 0, 0, 0, 1
                            /\                                   /\
/¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯\ /¯¯¯¯¯¯¯¯¯\
255, 255, 255,  96, 255, 255, 255, 224, 255, 255, 255, 224,  0, 0, 0, 1,
255, 255, 255,  96, 255, 255, 255, 224,   0,   0,   0,   0,  0, 0, 0, 2,
255, 255, 255,  96,   0,   0,   0,   0, 255, 255, 255, 224,  0, 0, 0, 4,
255, 255, 255,  96,   0,   0,   0,   0,   0,   0,   0,   0,  0, 0, 0, 8,
255, 255, 255, 128, 255, 255, 255, 160, 255, 255, 255, 192,  0, 0, 0, 16,
255, 255, 255, 128, 255, 255, 255, 160, 255, 255, 255, 224,  0, 0, 0, 32,
255, 255, 255, 128, 255, 255, 255, 160,   0,   0,   0,   0,  0, 0, 0, 64,
255, 255, 255, 128, 255, 255, 255, 160,   0,   0,   0,  32,  0, 0, 0, 128,
255, 255, 255, 128, 255, 255, 255, 224, 255, 255, 255, 224,  0, 0, 0, 1

This pattern continues, cycling through {1, 2, 4, 8, 16, 32, 64, 128}

If I try to print the req.voxelSize directly, it starts working. So I'm guessing this is some issue in the optimizer.

@IntegratedQuantum
Copy link
Contributor Author

I just updated to 0.11.0-dev.368+1829b6eab and the error seems to be resolved.

@Vexu
Copy link
Member

Vexu commented Dec 20, 2022

I'm able to reproduce this (or something similar) with Vexu/bun@94d54c3 and running make headers.

@Vexu Vexu reopened this Dec 20, 2022
@Vexu
Copy link
Member

Vexu commented Dec 22, 2022

Reduction:

fn FnPtrStage2Wrap(comptime T: anytype) @TypeOf(&struct {
    const FnPtrType = T;
    pub const fn_ptr: FnPtrType = undefined;
}.fn_ptr) {
    return &struct {
        const FnPtrType = T;
        pub const fn_ptr: FnPtrType = undefined;
    }.fn_ptr;
}
fn foo() u32 {
    return 0;
}
test {
    const a: @TypeOf(FnPtrStage2Wrap(@TypeOf(foo))) = foo;
    _ = a();
}

@andrewrk andrewrk modified the milestones: 0.11.0, 0.12.0 Jun 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Observed behavior contradicts documented or intended behavior
Projects
None yet
Development

No branches or pull requests

3 participants