Implement the ability to index a CAR using `io.Reader` alone for both v1 and v2 #146

masih · 2021-07-08T15:00:38Z

there should be a way to do this on just an io.Reader without ever needing to seek, but we would end up needing to use internal functions to jump to the internal (post-header) portion of index generation when we encounter a car v1 header.

Seeker is also used to skip over blocks during indexing. Nevertheless, i can imagine situations where you're streaming a car onto disk, and want to generate an index while you're at it. Being able to pipe through this with just an io.Reader is likely the way you'd hope to do that, so we should eventually relax the interface here to what we actually need.

Originally posted by @willscott in #144 (comment)

The text was updated successfully, but these errors were encountered:

Implement the ability to generate index from a CARv1 payload given only an `io.Reader`, where the previous implementation required `io.ReadSeeker`. The rationale is to be minimal in what we expect in the API, since index generation from a CARv1 payload never need to rewind the reader and only moves forward in the stream. Refactor utility IO functions that convert between types in one place. Implement constructor functions that only instantiate wrappers when the passed argument does not satisfy a required interface. Fixes: - #146 Relates to: - #145

masih · 2021-07-15T21:10:26Z

Fixed in #161

Implement the ability to generate index from a CARv1 payload given only an `io.Reader`, where the previous implementation required `io.ReadSeeker`. The rationale is to be minimal in what we expect in the API, since index generation from a CARv1 payload never need to rewind the reader and only moves forward in the stream. Refactor utility IO functions that convert between types in one place. Implement constructor functions that only instantiate wrappers when the passed argument does not satisfy a required interface. Fixes: - #146 Relates to: - #145

Implement the ability to generate index from a CARv1 payload given only an `io.Reader`, where the previous implementation required `io.ReadSeeker`. The rationale is to be minimal in what we expect in the API, since index generation from a CARv1 payload never need to rewind the reader and only moves forward in the stream. Refactor utility IO functions that convert between types in one place. Implement constructor functions that only instantiate wrappers when the passed argument does not satisfy a required interface. Fixes: - ipld/go-car#146 Relates to: - ipld/go-car#145 This commit was moved from ipld/go-car@6b085bc

masih added this to the CAR v2 milestone Jul 8, 2021

masih mentioned this issue Jul 8, 2021

Implement read or generate index from CAR accepting v1 or v2 #144

Merged

masih added the v2-before-release label Jul 9, 2021

masih self-assigned this Jul 13, 2021

masih mentioned this issue Jul 15, 2021

Implement index generation using only io.Reader #161

Merged

masih linked a pull request Jul 15, 2021 that will close this issue

Implement index generation using only io.Reader #161

Merged

masih closed this as completed Jul 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement the ability to index a CAR using `io.Reader` alone for both v1 and v2 #146

Implement the ability to index a CAR using `io.Reader` alone for both v1 and v2 #146

masih commented Jul 8, 2021

masih commented Jul 15, 2021

Implement the ability to index a CAR using io.Reader alone for both v1 and v2 #146

Implement the ability to index a CAR using io.Reader alone for both v1 and v2 #146

Comments

masih commented Jul 8, 2021

masih commented Jul 15, 2021

Implement the ability to index a CAR using `io.Reader` alone for both v1 and v2 #146

Implement the ability to index a CAR using `io.Reader` alone for both v1 and v2 #146