Storage client write requests should retry on "node not found" errors

[ptrus]

Storage write client already retries on some requests such as ErrPreviousVersionMismatch & ErrRootNotFound, but currently treats ErrNodeNotFound as permanent, which should actually also be retried:

oasis-core/go/storage/client/client.go

Lines 111 to 129 in 9f3b694

	var rerr error
	resp, rerr = fn(ctx, api.NewStorageClient(conn.ClientConn), conn.Node)
	switch {
	case status.Code(rerr) == codes.Unavailable:
	// Storage node may be temporarily unavailable.
	return rerr
	case status.Code(rerr) == codes.PermissionDenied:
	// Writes can fail around an epoch transition due to policy errors.
	return rerr
	case errors.Is(rerr, api.ErrPreviousVersionMismatch):
	// Storage node may not have yet processed the epoch transition.
	return rerr
	case errors.Is(rerr, api.ErrRootNotFound):
	// Storage node may not have yet processed the epoch transition.
	return rerr
	default:
	// All other errors are permanent.
	return backoff.Permanent(rerr)
	}

todo: Check other storage api errors, if any other should be treated as non-permanent as well.