fix: cached symlinks causing cache misses

[adamdottv]

In trying to use turborepo with a CDK app, I noticed that I never got remote cache hits for my cdk.out assets. After some digging I discovered that the issue was due to some symlink edge cases not being handled.

My changes boil down to:

• Creating any nested directories for symlinks (in the same way we handle normal files)
• Allowing for the overwriting of existing symlinks (by first removing any that already exist)
• Keeping track of any symlinks that are attempted before the destination file has been copied out of the archive; we then loop through the missing links at the end and create them once all the normal files have been copied.

Another note: currently all of these error cases result in a silent cache miss, due to cache.go:

func (mplex cacheMultiplexer) Fetch(target string, key string, files []string) (bool, []string, error) {
	// Retrieve from caches sequentially; if we did them simultaneously we could
	// easily write the same file from two goroutines at once.
	for i, cache := range mplex.caches {
		if ok, actualFiles, _ := cache.Fetch(target, key, files); ok {
			// Store this into other caches
			mplex.storeUntil(target, key, 0, actualFiles, i)
			return ok, actualFiles, nil
		}
	}
	return false, files, nil // <------- we always drop errors from the underlying caches
}

Not sure if that's desired, but I left it as is; I wasn't entirely sure how to handle it properly anyway.

[vercel]

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/vercel/turbo-site/3Q8zJoCaUfx5sH3x21hwHyQzX7MS
✅ Preview: https://turbo-site-git-fork-adamelmore-adamelmore-cached-symlinks.vercel.sh

[adamdottv]

Need anything from me on this one, @jaredpalmer?

[jaredpalmer]

Want to merge #544 first and then this and then cut a canary

[adamdottv]

Want to merge #544 first and then this and then cut a canary

This still planned for the next canary release?