chore: In Amino, use ToLowerSnakeCase for Protobuf field names

[jefft0]

This PR resolves issue #1211 so that Amino uses snake case for Protobuf field names, as required by the official style guide. Otherwise the Protobuf linter in our CI workflow outputs many warnings which get in the way of seeing issues that need to be fixed.

We copy ToLowerSnakeCase from the buf repository which is the same code that the linter uses to produce the expected field name. If the Go struct field is PackagePath then the Protobuf field should be package_path. We simply update GenerateProto3MessagePartial to use ToLowerSnakeCase(field.Name) when outputting the Protobuf field name. For backwards compatibility with existing tools that generate JSON from the Protobuf file, we include [json_name = "OriginalCamelCase"] where needed.

You can see the difference by the changes in genproto_test.go. Before:

// messages
message StructSM {
	sint64 FieldA = 1;
	string FieldB = 2;
	submodule2.StructSM2 FieldC = 3;
}

After:

// messages
message StructSM {
	sint64 field_a = 1 [json_name = "FieldA"];
	string field_b = 2 [json_name = "FieldB"];
	submodule2.StructSM2 field_c = 3 [json_name = "FieldC"];
}

Here is the difference in the Go structs for Protobuf that buf creates. Before:

type StructSM struct {
	state         protoimpl.MessageState
	sizeCache     protoimpl.SizeCache
	unknownFields protoimpl.UnknownFields

	FieldA int64      `protobuf:"zigzag64,1,opt,name=FieldA,proto3" json:"FieldA,omitempty"`
	FieldB string     `protobuf:"bytes,2,opt,name=FieldB,proto3" json:"FieldB,omitempty"`
	FieldC *StructSM2 `protobuf:"bytes,3,opt,name=FieldC,proto3" json:"FieldC,omitempty"`
}

After:

type StructSM struct {
	state         protoimpl.MessageState
	sizeCache     protoimpl.SizeCache
	unknownFields protoimpl.UnknownFields

	FieldA int64      `protobuf:"zigzag64,1,opt,name=field_a,json=fieldA,proto3" json:"field_a,omitempty"`
	FieldB string     `protobuf:"bytes,2,opt,name=field_b,json=fieldB,proto3" json:"field_b,omitempty"`
	FieldC *StructSM2 `protobuf:"bytes,3,opt,name=field_c,json=fieldC,proto3" json:"field_c,omitempty"`
}

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (98f384e) 55.93% compared to head (4a4900a) 47.16%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1213      +/-   ##
==========================================
- Coverage   55.93%   47.16%   -8.77%     
==========================================
  Files         420      372      -48     
  Lines       65415    61715    -3700     
==========================================
- Hits        36592    29110    -7482     
- Misses      25966    30220    +4254     
+ Partials     2857     2385     -472     

Files	Coverage Δ
tm2/pkg/amino/genproto/bindings.go	82.80% <ø> (-0.04%)	⬇️
tm2/pkg/amino/genproto/genproto.go	25.89% <ø> (-0.15%)	⬇️
tm2/pkg/amino/genproto/types.go	54.71% <ø> (-0.33%)	⬇️

... and 157 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[moul]

LGTM

[moul]

Hey @jefft0, could you share a before/after .proto generated files.

[jefft0]

Hey @jefft0, could you share a before/after .proto generated files.

I updated the description with the changes to the test .proto output of genproto_test.go .

[moul]

Ok, @jefft0, I apologize for asking multiple small things separately.

Could you please provide a before/after example of a generated Go file using the tool you use (buf, gogo, or protoc-gen-go)? As far as I remember, the generated Go code should be mostly the same, as snake case is translated into Go-style variable naming.

However, we have a concern regarding other fields, such as JSON in Go tags (FieldA 'json:"field_a"...). This could potentially cause compatibility issues and should be taken into consideration.

In the main repository, I don't think it will affect anything since we directly use amino marshalling. However, for some clients, it might create incompatible JSON clients.

[jefft0]

Could you please provide a before/after example of a generated Go file

I updated the description with before and after generated Go structs.

[moul]

Based on my understanding, it appears that our server-side case will remain unaffected since we utilize amino instead of protobuf to expose our API. Therefore, when an RPC/HTTP/REST request is made using JSON as the format, the resulting output will remain unchanged (CamelCase).

However, clients who previously used protobuf-generated JS clients will now anticipate snake_case keys, whereas we currently provide CamelCase keys.

Would it be possible to consider incorporating protobuf field options to ensure that JSON keys are formatted in CamelCase?

[jefft0]

Would it be possible to consider incorporating protobuf field options to ensure that JSON keys are formatted in CamelCase?

We can update the Protobuf output to be:

message StructSM {
	sint64 field_a = 1 [json_name = "FieldA"];
	string field_b = 2 [json_name = "FieldB"];
	StructSM2 field_c = 3 [json_name = "FieldC"];
}

In this case, the Go struct that buf creates is:

type StructSM struct {
	state         protoimpl.MessageState
	sizeCache     protoimpl.SizeCache
	unknownFields protoimpl.UnknownFields

	FieldA int64      `protobuf:"zigzag64,1,opt,name=field_a,json=FieldA,proto3" json:"field_a,omitempty"`
	FieldB string     `protobuf:"bytes,2,opt,name=field_b,json=FieldB,proto3" json:"field_b,omitempty"`
	FieldC *StructSM2 `protobuf:"bytes,3,opt,name=field_c,json=FieldC,proto3" json:"field_c,omitempty"`
}

Will that work?

[piux2]

@jefft0 The change seems solid. However, we’d like to further understand the impact on existing transactions after replacing the current proto files. For instance, if we substitute vm.proto, would it be possible to use the new client to make calls to test3.gno.land? Also, will the node still be capable of unmarshalling the existing call messages and addpkg messages in the genesis transactions?

We might not necessarily require backward compatibility at this stage, but we aim to understand the current implications to make informed decisions later.

[jefft0]

@piux2 : I restarted my local gno.land node from genesis. I loads all the genesis transactions and I see the initial state of the "boards" realm. (This is not surprising. The changes in this PR don't affect reading existing messages.) I can try changing vm.proto. What is the correct command to regenerate it using Amino? Is it to do make in misc/genproto ?

[piux2]

@piux2 : I restarted my local gno.land node from genesis. I loads all the genesis transactions and I see the initial state of the "boards" realm. (This is not surprising. The changes in this PR don't affect reading existing messages.) I can try changing vm.proto. What is the correct command to regenerate it using Amino? Is it to do make in misc/genproto ?

@jefft0 that is correct command. it will generate vm.proto according to the package.go in the folder.

We want to verify

1. vm.proto does use the LowerSnakeCase for Protobuf field names
2. The newly built gno.land can start from genesis
3. The newly built gnokey can call and deploy contracts to test3.gno.land

[jefft0]

Hello @piux2 . Here is the generated vm.proto:

message m_call {
	string caller = 1;
	string send = 2;
	string pkg_path = 3;
	string func = 4;
	repeated string args = 5;
}

message m_addpkg {
	string creator = 1;
	std.MemPackage package = 2;
	string deposit = 3;
}

message InvalidPkgPathError {
}

message InvalidStmtError {
}

message InvalidExprError {
}

The newly-built gno.land does start from genesis. I'm able to use the command-line gnokey to query account balances (both local and on test3.gno.land), and the web interface works.

To test if the newly-built gnokey can call contracts on test3.gno.land, I ran the command:

./build/gnokey query vm/qeval -data "gno.land/r/demo/boards
GetBoardIDFromName(\"testboard\")" -remote test3.gno.land:36657

As expected it prints:

height: 0
data: (1 gno.land/r/demo/boards.BoardID)
(true bool)

Is this the right way to test calling a contract on test3.gno.land?

[piux2]

./build/gnokey query vm/qeval -data "gno.land/r/demo/boards
GetBoardIDFromName(\"testboard\")" -remote test3.gno.land:36657

As expected it prints:

height: 0
data: (1 gno.land/r/demo/boards.BoardID)
(true bool)

Is this the right way to test calling a contract on test3.gno.land?

@jefft0 thank you for running the tests.

We want to verify the gnokey call | send and gnokey addpkg -deposit against the test3
caller,send,func,creator and deposit are the fields involved in these calls.

[netlify]

✅ Deploy Preview for gno-docs2 canceled.

Name	Link
🔨 Latest commit	4a4900a
🔍 Latest deploy log	https://app.netlify.com/sites/gno-docs2/deploys/654e47554be999000804eae9

[jefft0]

As before, I start with a fresh local node with ./build/gnoland start. (The genesis_balances.txt has some coins for my jefft0 account.) I enter the following to test addpkg with "boards2":

./build/gnokey maketx addpkg jefft0 -pkgpath gno.land/r/demo/boards2 -pkgdir ../examples/gno.land/r/demo/boards -deposit 100000000ugnot -gas-fee 1000000ugnot -gas-wanted 10000000 -broadcast -chainid dev -remote localhost:26657
./build/gnokey maketx call -pkgpath gno.land/r/demo/users -func Register -args "" -args "jefft0" -args "Profile description" -gas-fee "10000000ugnot" -gas-wanted "2000000" -send "200000000ugnot" -broadcast -chainid dev -remote 127.0.0.1:26657 jefft0
./build/gnokey maketx call -pkgpath gno.land/r/demo/boards2 -func CreateBoard -args test_snake_case -gas-fee "1000000ugnot" -gas-wanted "10000000" -broadcast -chainid dev -remote localhost:26657 jefft0

As expected, the CreateBoard command prints "(1 gno.land/r/demo/boards2.BoardID)", and the local web page http://localhost:8888/r/demo/boards2 shows the created board "test_snake_case".

The above commands correctly use addpkg, call, send, func and deposit. I don't see creator in gnokey. How do I test creator?

[jefft0]

We want to verify the gnokey call | send and gnokey addpkg -deposit against the test3
caller,send,func,creator and deposit are the fields involved in these calls.

Hi @piux2 . Are you saying that you want me to use the code in this PR as the client to call the test3 node, to check backwards compatibility? I ask because when I run the client against a local node which is also compiled with the code in this PR, then the functions mentioned in my previous post all work. But you want to check backwards compatibility between different versions?

[thehowl]

But you want to check backwards compatibility between different versions?

We're on our thursday review calls; the answer is yes, that's what we're looking for :)

[jefft0]

Hi @piux2 . I got some GNOT on test3.gno.land and entered the following. (My account was already registered with the node, so I didn't run the Register command again.)

./build/gnokey maketx addpkg jefft0 -pkgpath gno.land/r/demo/jefft0_test2_boards -pkgdir ../../../../test3/gno/examples/gno.land/r/demo/boards -deposit 50000000ugnot -gas-fee 1000000ugnot -gas-wanted 10000000 -broadcast -chainid test3 -remote test3.gno.land:36657
./build/gnokey maketx call -pkgpath gno.land/r/demo/jefft0_test2_boards -func CreateBoard -args test_snake_case -gas-fee "1000000ugnot" -gas-wanted "10000000" -broadcast -chainid test3 -remote test3.gno.land:36657 jefft0

As expected, the CreateBoard command prints "(1 gno.land/r/demo/jefft0_test2_boards.BoardID)" and you can see the result. To test send, the query of the account of piux2 ./build/gnokey query auth/accounts/g1fj9jccm3zjnqspq7lp2g7lj4czyfq0s35600g9 -remote test3.gno.land:36657 shows:

data: {
  "BaseAccount": {
    "address": "g1fj9jccm3zjnqspq7lp2g7lj4czyfq0s35600g9",
    "coins": "999000000ugnot",
    "public_key": null,
    "account_number": "20877",
    "sequence": "0"
  }
}

I enter:

./build/gnokey maketx send -gas-fee "1ugnot" -gas-wanted "5000000" -to "g1fj9jccm3zjnqspq7lp2g7lj4czyfq0s35600g9" -send "1000000ugnot" -broadcast -chainid test3 -remote test3.gno.land:36657 jefft0

Now the piux2 account has 1000000000ugnot . (You're welcome.)

In summary, these commands show the gnokey client in this PR correctly working with the test3 node using addpkg, call, send, func and deposit.