setting global cache directory to relative path causes build failure of dependencies

[Jan200101]

Zig Version

0.13.0-dev.344+b2588de6c

Steps to Reproduce and Observed Behavior

Build a project with dependencies (what exactly is needed is unclear, but I managed to cut it down to a minimal example https://github.com/Jan200101/zig-bug-local-cache-dep) and a global cache directory being set to a local path (e.g. zig build --global-cache-dir zig-cache)

This will result in the following error:

error: unable to open source directory 'zig-cache/p/1220b81f6ecfb3fd222f76cf9106fecfa6554ab07ec7fdc4124b9bb063ae2adf969d/include': FileNotFound

Expected Behavior

Everything works as expected

[nektro]

I think this is known and you need to do --global-cache-dir $PWD/zig-cache

[Jan200101]

I think this is known and you need to do --global-cache-dir $PWD/zig-cache

figured, but I'd rather just fix it outright (god knows I already sunk enough time into figuring out solution to this)

Solutions that I see so

1. turn global cache path into absolute path when argument is parsed
   • this would make sure it works everywhere but I've heard that apparently relative paths are preferred for build system stuff?
2. never change the working directory
   • this would break projects that expect this behavior

personally I think 1 is better solution here

[castholm]

I believe this is be fixed by changing line 2048 of

zig/lib/std/Build.zig

Lines 2047 to 2055 in e54fcdb

	const build_root: std.Build.Cache.Directory = .{
	.path = build_root_string,
	.handle = fs.cwd().openDir(build_root_string, .{}) catch |err| {
	std.debug.print("unable to open '{s}': {s}\n", .{
	build_root_string, @errorName(err),
	});
	process.exit(1);
	},
	};

to .path = b.pathFromCwd(build_root_string),. #20132 might be redundant if this fix is used instead.

The source of the problem seems to be this:

In the generated dependencies.zig file, relative dependencies will have their build_root set to absolute paths, while remote dependencies will have them set to paths resolved from the provided global cache dir. When building with a relative global cache dir like --global-cache-dir my-cache, all remote dependencies will have build_root paths that are relative:

pub const packages = struct {
    pub const @"1220b81f6ecfb3fd222f76cf9106fecfa6554ab07ec7fdc4124b9bb063ae2adf969d" = struct {
        pub const build_root = "my-cache\\p\\1220b81f6ecfb3fd222f76cf9106fecfa6554ab07ec7fdc4124b9bb063ae2adf969d";
        pub const deps: []const struct { []const u8, []const u8 } = &.{};
    };
    pub const @"1220cc5411d0624dd44d1eb476fe4285fb1a45fa17e5dcf1812dfe387b74404b2887" = struct {
        pub const build_root = "C:\\tmp\\zig-bug-local-cache-dep\\pkg/freetype";
        pub const build_zig = @import("1220cc5411d0624dd44d1eb476fe4285fb1a45fa17e5dcf1812dfe387b74404b2887");
        pub const deps: []const struct { []const u8, []const u8 } = &.{
            .{ "freetype", "1220b81f6ecfb3fd222f76cf9106fecfa6554ab07ec7fdc4124b9bb063ae2adf969d" },
        };
    };
};

pub const root_deps: []const struct { []const u8, []const u8 } = &.{
    .{ "freetype", "1220cc5411d0624dd44d1eb476fe4285fb1a45fa17e5dcf1812dfe387b74404b2887" },
};

When instantiating a dependency, b.build_root.path is set to the dependency's build_root. If build_root is relative, it means that b.build_root.path will also be relative, which in turn means that b.pathFromRoot() and subsequently LazyPath.getPath2() will return relative paths.

This will cause problems in build steps like WriteFile and InstallDir since code like b.build_root.handle.openDir(full_src_dir_path, .{ .iterate = true }) needs full_src_dir_path to be absolute to work correctly.

[Jan200101]

That works too
Though I am concerned that this would try to resolve the path for every build_root.

Going down the chain its

• dependencies.zig gets its values set by Fetch.run
  

  zig/src/Package/Fetch.zig

  Lines 377 to 380 in 30a35a8

  	f.package_root = .{
  	.root_dir = cache_root,
  	.sub_path = try arena.dupe(u8, pkg_sub_path),
  	};

• Fetch.run gets the global cache from the job queue
  

  zig/src/Package/Fetch.zig

  Line 301 in 30a35a8

  const cache_root = f.job_queue.global_cache;

• the job queue is created by main
  

  zig/src/main.zig

  Line 5015 in 30a35a8

  .job_queue = &job_queue,

  
  

  zig/src/main.zig

  Lines 4975 to 4984 in 30a35a8

  	var job_queue: Package.Fetch.JobQueue = .{
  	.http_client = &http_client,
  	.thread_pool = &thread_pool,
  	.global_cache = global_cache_directory,
  	.read_only = false,
  	.recursive = true,
  	.debug_hash = false,
  	.work_around_btrfs_bug = work_around_btrfs_bug,
  	.unlazy_set = unlazy_set,
  	};

which just gets it from either a command line flag, env var or a default location

[Jan200101]

Was fixed by a966eee (unintentionally?)