Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add car and car generator #102

Merged
merged 4 commits into from
Dec 1, 2023
Merged

feat: add car and car generator #102

merged 4 commits into from
Dec 1, 2023

Conversation

rvagg
Copy link
Member

@rvagg rvagg commented Nov 28, 2023

also updated deps, and published the current CAR to web3.storage so they're available on ipfs, e.g.: https://ipfs.io/ipfs/bafyreidhjbzws7yyooefukqt4xvbrctkz5pj5c7dnhdea6nepemymhkccm/

@lidel
Copy link

lidel commented Nov 28, 2023

@rvagg does import work for you? Kubo 0.24 gives me:

$ ipfs dag import --pin-roots=false tmp/fixtures.car
Error: import failed at block "bafyreieir43khjzemsmgahaozab2vjvtdxavszixhhurvdqg2xkhrwinyi": cbor: negative integer out of rage of int64 type

@rvagg
Copy link
Member Author

rvagg commented Nov 28, 2023

Ah yeah, I forgot about the Go special cases, they're listed here: https://github.com/ipld/codec-fixtures/blob/master/go/special_cases.go

Some extra details about the uint64 / int64 boundary in Go are here: ipld/go-ipld-prime#413 (also see notes in the tests in there), mostly large >int64 values should be parseable (aside from the faulty 0x3bffffffffffffffff case as noted in the tests there) but are not consumable with standard APIs because an n.AsInt() call will overflow, but an n.(datamodel.UintNode).AsUint() won't.

I'm not sure exactly which problem Kubo will be encountering here, it could simply be the 0x3bffffffffffffffff case.

Works in the JS stack though, and therefore web3.storage. Maybe this needs a caveat in the README.

@rvagg
Copy link
Member Author

rvagg commented Nov 28, 2023

ah, more details:

    --- FAIL: TestFixtures/int--11959030306112471732 (0.00s)
        codecs_test.go:39: failed to load fixture: cbor: negative integer out of rage of int64 type
    --- FAIL: TestFixtures/int-11959030306112471731 (0.00s)
        codecs_test.go:39: failed to load fixture: strconv.ParseInt: parsing "11959030306112471731": value out of range
    --- FAIL: TestFixtures/int-18446744073709551615 (0.00s)
        codecs_test.go:39: failed to load fixture: strconv.ParseInt: parsing "18446744073709551615": value out of range

First one is out, regardless, it's negative max uint64, which isn't a thing in Go. I did have an early proposal in ipld/go-ipld-prime#413 to return a negative indicator that could deal with this, but the complexity made it not worth it.

The other two failures are dag-json strconv failures. They could potentially be fixed, but are low priority, and I already have other json fixes in play that I can't get over the line (polydawn/refmt#58, not that I've pushed very hard on that).

@lidel
Copy link

lidel commented Nov 29, 2023

Thanks! A bit unfortunate edge cases, I've filled ipfs/kubo#10236 for tracking follow-up work in Kubo, but yeah, lower priority until this impacts real world use case.

@rvagg
Copy link
Member Author

rvagg commented Dec 1, 2023

this reminded me to ping about getting polydawn/refmt#58 merged and that's done now so I should update for that at least; I don't doubt that if we cared enough the json things are entirely fixable. Negative uint64 for Go is a bit more iffy though.

@rvagg rvagg merged commit e0c9d76 into master Dec 1, 2023
54 of 58 checks passed
@rvagg rvagg deleted the rvagg/car branch December 1, 2023 07:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants