Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(deps): upgrade to celestia-app v2.0.0-rc2 #3453

Draft
wants to merge 37 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
63bc288
fix: imports
rootulp May 31, 2024
926e5a3
fix: error message
rootulp May 31, 2024
8c75d6a
wip: adapt to v2
rootulp May 31, 2024
8c9fc08
fix: copy broadcastTx from v1.x celestia-app
rootulp Jun 6, 2024
c3e84e1
go mod tidy
rootulp Jun 6, 2024
fc8dac3
refactor: error msg
rootulp Jun 6, 2024
44e65e0
refactor: rename to namespace
rootulp Jun 6, 2024
e1286a4
test: validate extended header
rootulp Jun 6, 2024
866cd1e
test: extended header validate
rootulp Jun 6, 2024
69a05a9
Merge branch 'main' into rp/support-v2
rootulp Jun 6, 2024
99b0b29
fix: most lint errors
rootulp Jun 7, 2024
b0401f3
refactor: rename b to appBlob
rootulp Jun 7, 2024
54eadb7
fix: last linter warning about copying mutexes
rootulp Jun 7, 2024
bcde3f7
Merge branch 'main' into rp/support-v2
rootulp Jun 25, 2024
9c9ff4e
bump to celestia-app v2.0.0-rc2
rootulp Jun 25, 2024
a59a373
fix: split txClient from signer
rootulp Jun 25, 2024
ae72d69
fix: lint
rootulp Jun 25, 2024
b7633c9
chore(deps): upgrade to Go 1.22.4
rootulp Jun 25, 2024
dff037e
fix: lint
rootulp Jun 25, 2024
efb6c9d
try to fix lint
rootulp Jun 27, 2024
2bde950
Merge branch 'main' into rp/support-v2
rootulp Jun 27, 2024
5542aa7
Merge branch 'main' into rp/support-v2
rootulp Jul 2, 2024
f95d9c7
refactor: update core_access.go implementation
rootulp Jul 2, 2024
bccc2b4
test: remove min gas price assertion b/c TODO
rootulp Jul 2, 2024
f147fb6
fix: tests by converting squareblob to nodeblob
rootulp Jul 2, 2024
3e3fedd
refactor: stop using convert
rootulp Jul 4, 2024
8b5ef8f
test: attempt to fix TestCoreExchange_RequestHeaders
rootulp Jul 4, 2024
383490c
fix: da_test comma
rootulp Jul 4, 2024
4857abe
fix: TestBlob
rootulp Jul 4, 2024
a5650ac
HACKHACK: I think bug in celestia-app testnode that prevents custom c…
rootulp Jul 4, 2024
e2eb8e9
Revert "HACKHACK: I think bug in celestia-app testnode that prevents …
rootulp Jul 4, 2024
c4be6a8
resolve merge conflicts
rootulp Jul 9, 2024
9e0f501
test: attempt to fix integration tests
rootulp Jul 9, 2024
c120fd2
wip: fix some lint issues
rootulp Jul 9, 2024
4eddd43
fix: build
rootulp Jul 9, 2024
b8fbd4b
fix: manual re-review
rootulp Jul 9, 2024
daf1a81
attempt to fix import order
rootulp Jul 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion api/gateway/share.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (

"github.com/gorilla/mux"

"github.com/celestiaorg/celestia-app/pkg/shares"
"github.com/celestiaorg/go-square/shares"

"github.com/celestiaorg/celestia-node/share"
)
Expand Down
16 changes: 8 additions & 8 deletions api/gateway/share_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ import (
"testing"

"github.com/stretchr/testify/require"
coretypes "github.com/tendermint/tendermint/types"

"github.com/celestiaorg/celestia-app/pkg/appconsts"
"github.com/celestiaorg/celestia-app/pkg/shares"
"github.com/celestiaorg/celestia-app/v2/pkg/appconsts"
"github.com/celestiaorg/go-square/blob"
"github.com/celestiaorg/go-square/shares"

"github.com/celestiaorg/celestia-node/share/sharetest"
)
Expand All @@ -23,13 +23,13 @@ func Test_dataFromShares(t *testing.T) {
ns := sharetest.RandV0Namespace()
sss := shares.NewSparseShareSplitter()
for _, data := range testData {
b := coretypes.Blob{
b := blob.Blob{
Data: data,
NamespaceID: ns.ID(),
NamespaceVersion: ns.Version(),
ShareVersion: appconsts.ShareVersionZero,
NamespaceId: ns.ID(),
NamespaceVersion: uint32(ns.Version()),
ShareVersion: uint32(appconsts.ShareVersionZero),
}
err := sss.Write(b)
err := sss.Write(&b)
require.NoError(t, err)
}

Expand Down
35 changes: 19 additions & 16 deletions blob/blob.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,12 @@ import (
"errors"
"fmt"

tmproto "github.com/tendermint/tendermint/proto/tendermint/types"
"github.com/tendermint/tendermint/crypto/merkle"

"github.com/celestiaorg/celestia-app/pkg/appconsts"
"github.com/celestiaorg/celestia-app/x/blob/types"
"github.com/celestiaorg/celestia-app/v2/pkg/appconsts"
v1 "github.com/celestiaorg/celestia-app/v2/pkg/appconsts/v1"
"github.com/celestiaorg/go-square/blob"
"github.com/celestiaorg/go-square/inclusion"
"github.com/celestiaorg/nmt"

"github.com/celestiaorg/celestia-node/share"
Expand Down Expand Up @@ -69,7 +71,7 @@ func (p Proof) equal(input Proof) error {

// Blob represents any application-specific binary data that anyone can submit to Celestia.
type Blob struct {
types.Blob `json:"blob"`
*blob.Blob `json:"blob"`

Commitment Commitment `json:"commitment"`

Expand Down Expand Up @@ -97,18 +99,18 @@ func NewBlob(shareVersion uint8, namespace share.Namespace, data []byte) (*Blob,
return nil, err
}

blob := tmproto.Blob{
blob := blob.Blob{
NamespaceId: namespace.ID(),
Data: data,
ShareVersion: uint32(shareVersion),
NamespaceVersion: uint32(namespace.Version()),
}

com, err := types.CreateCommitment(&blob)
com, err := inclusion.CreateCommitment(&blob, merkle.HashFromByteSlices, appconsts.SubtreeRootThreshold(v1.Version))
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[for reviewers] Invoking with v1.Version behaves identically to invoking with v2.Version because the implementation of this function doesn't actually care about the version that is provided as an argument.

See https://github.com/celestiaorg/celestia-app/blob/c0904490d3e60a8206b43728378b4ae4c097edd1/pkg/appconsts/versioned_consts.go#L21-L23

if err != nil {
return nil, err
}
return &Blob{Blob: blob, Commitment: com, namespace: namespace, index: -1}, nil
return &Blob{Blob: &blob, Commitment: com, namespace: namespace, index: -1}, nil
}

// Namespace returns blob's namespace.
Expand Down Expand Up @@ -146,18 +148,19 @@ func (b *Blob) MarshalJSON() ([]byte, error) {
}

func (b *Blob) UnmarshalJSON(data []byte) error {
var blob jsonBlob
err := json.Unmarshal(data, &blob)
var jsonBlob jsonBlob
err := json.Unmarshal(data, &jsonBlob)
if err != nil {
return err
}

b.Blob.NamespaceVersion = uint32(blob.Namespace.Version())
b.Blob.NamespaceId = blob.Namespace.ID()
b.Blob.Data = blob.Data
b.Blob.ShareVersion = blob.ShareVersion
b.Commitment = blob.Commitment
b.namespace = blob.Namespace
b.index = blob.Index
b.Blob = &blob.Blob{}
b.Blob.NamespaceVersion = uint32(jsonBlob.Namespace.Version())
b.Blob.NamespaceId = jsonBlob.Namespace.ID()
b.Blob.Data = jsonBlob.Data
b.Blob.ShareVersion = jsonBlob.ShareVersion
b.Commitment = jsonBlob.Commitment
b.namespace = jsonBlob.Namespace
b.index = jsonBlob.Index
return nil
}
27 changes: 20 additions & 7 deletions blob/blob_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,19 @@ package blob

import (
"bytes"
"fmt"
"math"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/tendermint/tendermint/types"

apptypes "github.com/celestiaorg/celestia-app/x/blob/types"
"github.com/celestiaorg/celestia-app/v2/pkg/appconsts"
v1 "github.com/celestiaorg/celestia-app/v2/pkg/appconsts/v1"
apptypes "github.com/celestiaorg/celestia-app/v2/x/blob/types"
"github.com/celestiaorg/go-square/blob"
"github.com/celestiaorg/go-square/inclusion"
"github.com/celestiaorg/go-square/merkle"

"github.com/celestiaorg/celestia-node/blob/blobtest"
)
Expand All @@ -35,7 +41,11 @@ func TestBlob(t *testing.T) {
{
name: "compare commitments",
expectedRes: func(t *testing.T) {
comm, err := apptypes.CreateCommitment(&blob[0].Blob)
comm, err := inclusion.CreateCommitment(
blob[0].Blob,
merkle.HashFromByteSlices,
appconsts.SubtreeRootThreshold(v1.Version),
)
require.NoError(t, err)
assert.Equal(t, blob[0].Commitment, Commitment(comm))
},
Expand Down Expand Up @@ -69,7 +79,7 @@ func TestBlob(t *testing.T) {

newBlob := &Blob{}
require.NoError(t, newBlob.UnmarshalJSON(data))
require.True(t, bytes.Equal(blob[0].Blob.Data, newBlob.Data))
require.True(t, bytes.Equal(blob[0].Blob.GetData(), newBlob.Data))
require.True(t, bytes.Equal(blob[0].Commitment, newBlob.Commitment))
},
},
Expand All @@ -80,10 +90,13 @@ func TestBlob(t *testing.T) {
}
}

func convertBlobs(appBlobs ...types.Blob) ([]*Blob, error) {
func convertBlobs(appBlobs ...*blob.Blob) ([]*Blob, error) {
blobs := make([]*Blob, 0, len(appBlobs))
for _, b := range appBlobs {
blob, err := NewBlob(b.ShareVersion, append([]byte{b.NamespaceVersion}, b.NamespaceID...), b.Data)
for _, appBlob := range appBlobs {
if shareVersion := appBlob.GetShareVersion(); shareVersion > math.MaxUint8 {
return nil, fmt.Errorf("share version %d is greater than max share version %d", shareVersion, math.MaxUint8)
}
blob, err := NewBlob(uint8(appBlob.GetShareVersion()), appBlob.Namespace().Bytes(), appBlob.GetData())
if err != nil {
return nil, err
}
Expand Down
18 changes: 9 additions & 9 deletions blob/blobtest/testing.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,30 @@ package blobtest

import (
tmrand "github.com/tendermint/tendermint/libs/rand"
"github.com/tendermint/tendermint/types"

"github.com/celestiaorg/celestia-app/pkg/appconsts"
"github.com/celestiaorg/celestia-app/pkg/shares"
"github.com/celestiaorg/celestia-app/test/util/testfactory"
"github.com/celestiaorg/celestia-app/v2/pkg/appconsts"
"github.com/celestiaorg/celestia-app/v2/test/util/testfactory"
"github.com/celestiaorg/go-square/blob"
"github.com/celestiaorg/go-square/shares"

"github.com/celestiaorg/celestia-node/share"
)

// GenerateV0Blobs is a test utility producing v0 share formatted blobs with the
// requested size and random namespaces.
func GenerateV0Blobs(sizes []int, sameNamespace bool) ([]types.Blob, error) {
blobs := make([]types.Blob, 0, len(sizes))
func GenerateV0Blobs(sizes []int, sameNamespace bool) ([]*blob.Blob, error) {
blobs := make([]*blob.Blob, 0, len(sizes))

for _, size := range sizes {
size := rawBlobSize(appconsts.FirstSparseShareContentSize * size)
appBlob := testfactory.GenerateRandomBlob(size)
if !sameNamespace {
nid, err := share.NewBlobNamespaceV0(tmrand.Bytes(7))
namespace, err := share.NewBlobNamespaceV0(tmrand.Bytes(7))
if err != nil {
return nil, err
}
appBlob.NamespaceVersion = nid[0]
appBlob.NamespaceID = nid[1:]
appBlob.NamespaceVersion = uint32(namespace[0])
appBlob.NamespaceId = namespace[1:]
}

blobs = append(blobs, appBlob)
Expand Down
34 changes: 16 additions & 18 deletions blob/helper.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,27 @@ import (
"bytes"
"sort"

"github.com/tendermint/tendermint/types"

"github.com/celestiaorg/celestia-app/pkg/shares"
apptypes "github.com/celestiaorg/celestia-app/x/blob/types"
squareblob "github.com/celestiaorg/go-square/blob"
"github.com/celestiaorg/go-square/shares"

"github.com/celestiaorg/celestia-node/share"
)

// BlobsToShares accepts blobs and convert them to the Shares.
func BlobsToShares(blobs ...*Blob) ([]share.Share, error) {
b := make([]types.Blob, len(blobs))
for i, blob := range blobs {
namespace := blob.Namespace()
b[i] = types.Blob{
NamespaceVersion: namespace[0],
NamespaceID: namespace[1:],
Data: blob.Data,
ShareVersion: uint8(blob.ShareVersion),
func BlobsToShares(nodeBlobs ...*Blob) ([]share.Share, error) {
b := make([]*squareblob.Blob, len(nodeBlobs))
for i, nodeBlob := range nodeBlobs {
namespace := nodeBlob.Namespace()
b[i] = &squareblob.Blob{
NamespaceVersion: uint32(namespace[0]),
NamespaceId: namespace[1:],
Data: nodeBlob.Data,
ShareVersion: nodeBlob.ShareVersion,
}
}

sort.Slice(b, func(i, j int) bool {
val := bytes.Compare(b[i].NamespaceID, b[j].NamespaceID)
val := bytes.Compare(b[i].Namespace().Bytes(), b[j].Namespace().Bytes())
return val < 0
})

Expand All @@ -37,11 +35,11 @@ func BlobsToShares(blobs ...*Blob) ([]share.Share, error) {
return shares.ToBytes(rawShares), nil
}

// ToAppBlobs converts node's blob type to the blob type from celestia-app.
func ToAppBlobs(blobs ...*Blob) []*apptypes.Blob {
appBlobs := make([]*apptypes.Blob, 0, len(blobs))
// ToAppBlobs converts node's blob type to the blob type from go-square.
func ToAppBlobs(blobs ...*Blob) []*squareblob.Blob {
appBlobs := make([]*squareblob.Blob, 0, len(blobs))
for i := range blobs {
appBlobs[i] = &blobs[i].Blob
appBlobs[i] = blobs[i].Blob
}
return appBlobs
}
Expand Down
2 changes: 1 addition & 1 deletion blob/parser.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"errors"
"fmt"

"github.com/celestiaorg/celestia-app/pkg/shares"
"github.com/celestiaorg/go-square/shares"
)

// parser helps to collect shares and transform them into a blob.
Expand Down
8 changes: 4 additions & 4 deletions blob/service.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import (
"go.opentelemetry.io/otel/codes"
"go.opentelemetry.io/otel/trace"

"github.com/celestiaorg/celestia-app/pkg/shares"
"github.com/celestiaorg/go-square/shares"

"github.com/celestiaorg/celestia-node/header"
"github.com/celestiaorg/celestia-node/libs/utils"
Expand Down Expand Up @@ -69,15 +69,15 @@ func NewService(
func (s *Service) Submit(ctx context.Context, blobs []*Blob, txConfig *SubmitOptions) (uint64, error) {
log.Debugw("submitting blobs", "amount", len(blobs))

appblobs := make([]*state.Blob, len(blobs))
squareBlobs := make([]*state.Blob, len(blobs))
for i := range blobs {
if err := blobs[i].Namespace().ValidateForBlob(); err != nil {
return 0, err
}
appblobs[i] = &blobs[i].Blob
squareBlobs[i] = blobs[i].Blob
}

resp, err := s.blobSubmitter.SubmitPayForBlob(ctx, appblobs, txConfig)
resp, err := s.blobSubmitter.SubmitPayForBlob(ctx, squareBlobs, txConfig)
if err != nil {
return 0, err
}
Expand Down
12 changes: 6 additions & 6 deletions blob/service_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ import (
"github.com/stretchr/testify/require"
tmrand "github.com/tendermint/tendermint/libs/rand"

"github.com/celestiaorg/celestia-app/pkg/appconsts"
appns "github.com/celestiaorg/celestia-app/pkg/namespace"
"github.com/celestiaorg/celestia-app/pkg/shares"
"github.com/celestiaorg/celestia-app/v2/pkg/appconsts"
"github.com/celestiaorg/go-header/store"
squarens "github.com/celestiaorg/go-square/namespace"
"github.com/celestiaorg/go-square/shares"

"github.com/celestiaorg/celestia-node/blob/blobtest"
"github.com/celestiaorg/celestia-node/header"
Expand Down Expand Up @@ -343,7 +343,7 @@ func TestBlobService_Get(t *testing.T) {
{
name: "empty result and err when blobs were not found ",
doFn: func() (interface{}, error) {
nid, err := share.NewBlobNamespaceV0(tmrand.Bytes(appns.NamespaceVersionZeroIDSize))
nid, err := share.NewBlobNamespaceV0(tmrand.Bytes(squarens.NamespaceVersionZeroIDSize))
require.NoError(t, err)
return service.GetAll(ctx, 1, []share.Namespace{nid})
},
Expand Down Expand Up @@ -628,8 +628,8 @@ func TestSkipPaddingsAndRetrieveBlob(t *testing.T) {

appBlob, err := blobtest.GenerateV0Blobs([]int{6}, true)
require.NoError(t, err)
appBlob[0].NamespaceVersion = nid[0]
appBlob[0].NamespaceID = nid[1:]
appBlob[0].NamespaceVersion = uint32(nid[0])
appBlob[0].NamespaceId = nid[1:]

blobs, err := convertBlobs(appBlob...)
require.NoError(t, err)
Expand Down
4 changes: 2 additions & 2 deletions cmd/cel-key/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ import (
"github.com/cosmos/cosmos-sdk/x/auth/types"
"github.com/spf13/cobra"

"github.com/celestiaorg/celestia-app/app"
"github.com/celestiaorg/celestia-app/app/encoding"
"github.com/celestiaorg/celestia-app/v2/app"
"github.com/celestiaorg/celestia-app/v2/app/encoding"
)

var encodingConfig = encoding.MakeConfig(app.ModuleEncodingRegisters...)
Expand Down
4 changes: 2 additions & 2 deletions cmd/start.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ import (
"github.com/cosmos/cosmos-sdk/crypto/keyring"
"github.com/spf13/cobra"

"github.com/celestiaorg/celestia-app/app"
"github.com/celestiaorg/celestia-app/app/encoding"
"github.com/celestiaorg/celestia-app/v2/app"
"github.com/celestiaorg/celestia-app/v2/app/encoding"

"github.com/celestiaorg/celestia-node/nodebuilder"
)
Expand Down
16 changes: 10 additions & 6 deletions core/eds.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ import (
"github.com/filecoin-project/dagstore"
"github.com/tendermint/tendermint/types"

"github.com/celestiaorg/celestia-app/app"
"github.com/celestiaorg/celestia-app/pkg/appconsts"
"github.com/celestiaorg/celestia-app/pkg/shares"
"github.com/celestiaorg/celestia-app/pkg/square"
"github.com/celestiaorg/celestia-app/pkg/wrapper"
"github.com/celestiaorg/celestia-app/v2/app"
"github.com/celestiaorg/celestia-app/v2/pkg/appconsts"
"github.com/celestiaorg/celestia-app/v2/pkg/wrapper"
"github.com/celestiaorg/go-square/shares"
"github.com/celestiaorg/go-square/square"
"github.com/celestiaorg/nmt"
"github.com/celestiaorg/rsmt2d"

Expand All @@ -32,7 +32,11 @@ func extendBlock(data types.Data, appVersion uint64, options ...nmt.Option) (*rs
}

// Construct the data square from the block's transactions
dataSquare, err := square.Construct(data.Txs.ToSliceOfBytes(), appVersion, appconsts.SquareSizeUpperBound(appVersion))
dataSquare, err := square.Construct(
data.Txs.ToSliceOfBytes(),
appconsts.SquareSizeUpperBound(appVersion),
appconsts.SubtreeRootThreshold(appVersion),
)
if err != nil {
return nil, err
}
Expand Down
Loading
Loading