HAMT Preloader does not return error even if blocks do not load

[aschmahmann]

The HAMT Preloader ADL is meant to load all the blocks in the HAMT without following any links in the map's values.

Unfortunately, the way it does the preloading is by calling

go-unixfsnode/hamt/shardeddir.go

Lines 68 to 71 in 364a549

	traverse := n.Length()
	if traverse == -1 {
	return n, fmt.Errorf("could not fully explore hamt during preload")
	}

While this would be fine if in fact -1 was returned during an error loading the HAMT.

However, it does not because the contract for that function for this function is that -1 is only returned if the node is not a list or a map and we know it is. https://github.com/ipld/go-ipld-prime/blob/65bfa53512f2328d19273e471ce4fd6d964055a2/datamodel/node.go#L131-L133

This means that when we have trouble loading a block we just continue on as if nothing happened which does not fulfill the contract of the preloader function

go-unixfsnode/hamt/shardeddir.go

Lines 280 to 283 in 364a549

	child, err := n.loadChild(pbLink)
	if err != nil {
	continue
	}

---

This seems most readily resolvable by just using a different code path than Length() for doing the enumeration. Two that come to mind are:

1. Copying/reusing the Length code but where we return errors
2. Going through a map iterator that doesn't load the map value nodes

[willscott]

This makes sense. either of the options you propose sound pretty plausible. I'll take a look at a PR