Add a partial support for vote program #167

unordered-set · 2022-01-08T01:59:09Z

Transaction generated by this code: https://explorer.solana.com/tx/5GepwTxXrGpPPtLbY1VtKomHLqTRd3FozwK14G22yUPQuL9MBa5CeCkxuxY1Mcb9jHgvFF3ncCRZVbCu2ZVbquTu?cluster=testnet

michaelhly · 2022-01-08T05:23:12Z

@unordered-set can you write a test?

codecov · 2022-01-08T05:23:59Z

Codecov Report

Merging #167 (717a3c1) into master (f996c07) will decrease coverage by 0.09%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #167      +/-   ##
==========================================
- Coverage   91.06%   90.97%   -0.10%     
==========================================
  Files          37       39       +2     
  Lines        2698     2725      +27     
==========================================
+ Hits         2457     2479      +22     
- Misses        241      246       +5

unordered-set · 2022-01-09T16:17:16Z

It looks I've found an issue, but not sure where exactly.

$ solana withdraw-from-vote-account --dump-transaction-message CWqJy1JpmBcx7awpeANfrPk6AsQKkmego8ujjaYPGFEk A1V5gsis39WY42djdTKUFsgE5oamk4nrtg16WnKTuzZK --authorized-withdrawer withdrawer.json 2 --blockhash Add1tV7kJgNHhTtx3Dgs6dhC7kyXrGJQZ2tJGW15tLDH --sign-only

Blockhash: Add1tV7kJgNHhTtx3Dgs6dhC7kyXrGJQZ2tJGW15tLDH
Transaction Message: AgEBBcaUWaMv01yl0qeT4Dkw0FHdbvzYEdTlyXqXUPuOYBZ3OoXlJ9RH/kj2LaCcgccSvTWPYki2akUdJpF3vg1pnXCrFQYO0flqHdOoQpaNxOZ8eSlkLWHns0kvxLHtDo6WbYXY7vSg6/O3UeAhW76fJJpKta0mIrV4Npw88NZHELAWB2FIHTV0dLt8TXYk69O9s9g1XnPREEP8DaNTgAAAAACPGtBoxLsWOZpH76a8WvdcTpdvMxRMwlQikhOtRYHMbAEEAwIDAQwDAAAAAJQ1dwAAAAA=
Signers (Pubkey=Signature):
 ENAvhKRf2FF6iQxVRkrTyGiiFhw4Gv1GEkRzzjQKpTzE=5sEaoWwmLqPHffSk3fN6VzyDavgLHAZvt5T24LZ6KeTSg2y1FxUW2sjra94XX72wmDJ4VRRfxx4i55gPGHmJ955y
 4wT4iL6ALVQ7BypjAUAUbe8tBRM5ebVdJhYWKpABRpzj=kHSFttLmTMBDUvSXrm9Y8WYJXwqnCr9iaNxbvp5KG9rcL7PEBa1t5vKE5ZRR4jfyik9TzN3CT2a4S7H9GYKdZtN

The message above is decoded correctly, but it has a fee-payer different from the withdrawer. So I've updated it to:

$ solana withdraw-from-vote-account --dump-transaction-message CWqJy1JpmBcx7awpeANfrPk6AsQKkmego8ujjaYPGFEk A1V5gsis39WY42djdTKUFsgE5oamk4nrtg16WnKTuzZK --authorized-withdrawer withdrawer.json 2 --blockhash Add1tV7kJgNHhTtx3Dgs6dhC7kyXrGJQZ2tJGW15tLDH --sign-only -k withdrawer.json

Blockhash: Add1tV7kJgNHhTtx3Dgs6dhC7kyXrGJQZ2tJGW15tLDH
Transaction Message: AQABBDqF5SfUR/5I9i2gnIHHEr01j2JItmpFHSaRd74NaZ1wqxUGDtH5ah3TqEKWjcTmfHkpZC1h57NJL8Sx7Q6Olm2F2O70oOvzt1HgIVu+nySaSrWtJiK1eDacPPDWRxCwFgdhSB01dHS7fE12JOvTvbPYNV5z0RBD/A2jU4AAAAAAjxrQaMS7FjmaR++mvFr3XE6XbzMUTMJUIpITrUWBzGwBAwMBAgAMAwAAAACUNXcAAAAA
Signers (Pubkey=Signature):
 4wT4iL6ALVQ7BypjAUAUbe8tBRM5ebVdJhYWKpABRpzj=4hg9T95XXEH4pvye4yJRfJbkYVRJSiDp4wrZf6Mkq2uu3KVV5bR1K5UPRw2MBACY9Gpw1T9YH5Em7SvfMMTUMT6g

Nice, only one withdrawer. But now the message fails to deserialize.

>>> import base64
>>> two_signers = base64.b64decode(b'AgEBBcaUWaMv01yl0qeT4Dkw0FHdbvzYEdTlyXqXUPuOYBZ3OoXlJ9RH/kj2LaCcgccSvTWPYki2akUdJpF3vg1pnXCrFQYO0flqHdOoQpaNxOZ8eSlkLWHns0kvxLHtDo6WbYXY7vSg6/O3UeAhW76fJJpKta0mIrV4Npw88NZHELAWB2FIHTV0dLt8TXYk69O9s9g1XnPREEP8DaNTgAAAAACPGtBoxLsWOZpH76a8WvdcTpdvMxRMwlQikhOtRYHMbAEEAwIDAQwDAAAAAJQ1dwAAAAA=')
>>> one_signer = base64.b64decode(b'AQABBDqF5SfUR/5I9i2gnIHHEr01j2JItmpFHSaRd74NaZ1wqxUGDtH5ah3TqEKWjcTmfHkpZC1h57NJL8Sx7Q6Olm2F2O70oOvzt1HgIVu+nySaSrWtJiK1eDacPPDWRxCwFgdhSB01dHS7fE12JOvTvbPYNV5z0RBD/A2jU4AAAAAAjxrQaMS7FjmaR++mvFr3XE6XbzMUTMJUIpITrUWBzGwBAwMBAgAMAwAAAACUNXcAAAAA')
>>> import solana.transaction as txlib
>>> txlib.Transaction.deserialize(two_signers)
MessageHeader(num_required_signatures=16, num_readonly_signed_accounts=176, num_readonly_unsigned_accounts=22)
NumAccounts=7
<solana.transaction.Transaction object at 0x7f17f99757f0>
>>> txlib.Transaction.deserialize(one_signer)
MessageHeader(num_required_signatures=142, num_readonly_signed_accounts=150, num_readonly_unsigned_accounts=109)
NumAccounts=5868136825186135045
... freezes

so the way how num accounts is computed is wrong somewhere (either on solana-cli or on solana-py). Checking.

$ solana` --version
solana-cli 1.9.4 (src:8ce65878; feat:3258470607)

kevinheavey · 2022-01-09T16:55:05Z

txlib.Transaction.deserialize(one_signer) hangs for me too.

What are you using to get this MessageHeader and NumAccounts stuff?

unordered-set · 2022-01-09T17:01:54Z

These are debug prints I placed into the library (which is located in this repo)

unordered-set · 2022-01-09T17:21:54Z

Ok, these are binary dumps of txs:

(.venv) stromsund@DESKTOP-EMLQ922:~/solana-py$ hexdump -C two_signers.bin
00000000  02 01 01 05 c6 94 59 a3  2f d3 5c a5 d2 a7 93 e0  |......Y./.\.....|
00000010  39 30 d0 51 dd 6e fc d8  11 d4 e5 c9 7a 97 50 fb  |90.Q.n......z.P.|
00000020  8e 60 16 77 3a 85 e5 27  d4 47 fe 48 f6 2d a0 9c  |.`.w:..'.G.H.-..|
00000030  81 c7 12 bd 35 8f 62 48  b6 6a 45 1d 26 91 77 be  |....5.bH.jE.&.w.|
00000040  0d 69 9d 70 ab 15 06 0e  d1 f9 6a 1d d3 a8 42 96  |.i.p......j...B.|
00000050  8d c4 e6 7c 79 29 64 2d  61 e7 b3 49 2f c4 b1 ed  |...|y)d-a..I/...|
00000060  0e 8e 96 6d 85 d8 ee f4  a0 eb f3 b7 51 e0 21 5b  |...m........Q.![|
00000070  be 9f 24 9a 4a b5 ad 26  22 b5 78 36 9c 3c f0 d6  |..$.J..&".x6.<..|
00000080  47 10 b0 16 07 61 48 1d  35 74 74 bb 7c 4d 76 24  |G....aH.5tt.|Mv$|
00000090  eb d3 bd b3 d8 35 5e 73  d1 10 43 fc 0d a3 53 80  |.....5^s..C...S.|
000000a0  00 00 00 00 8f 1a d0 68  c4 bb 16 39 9a 47 ef a6  |.......h...9.G..|
000000b0  bc 5a f7 5c 4e 97 6f 33  14 4c c2 54 22 92 13 ad  |.Z.\N.o3.L.T"...|
000000c0  45 81 cc 6c 01 04 03 02  03 01 0c 03 00 00 00 00  |E..l............|
000000d0  94 35 77 00 00 00 00                              |.5w....|
000000d7
(.venv) stromsund@DESKTOP-EMLQ922:~/solana-py$ hexdump -C one_signer.bin
00000000  01 00 01 04 3a 85 e5 27  d4 47 fe 48 f6 2d a0 9c  |....:..'.G.H.-..|
00000010  81 c7 12 bd 35 8f 62 48  b6 6a 45 1d 26 91 77 be  |....5.bH.jE.&.w.|
00000020  0d 69 9d 70 ab 15 06 0e  d1 f9 6a 1d d3 a8 42 96  |.i.p......j...B.|
00000030  8d c4 e6 7c 79 29 64 2d  61 e7 b3 49 2f c4 b1 ed  |...|y)d-a..I/...|
00000040  0e 8e 96 6d 85 d8 ee f4  a0 eb f3 b7 51 e0 21 5b  |...m........Q.![|
00000050  be 9f 24 9a 4a b5 ad 26  22 b5 78 36 9c 3c f0 d6  |..$.J..&".x6.<..|
00000060  47 10 b0 16 07 61 48 1d  35 74 74 bb 7c 4d 76 24  |G....aH.5tt.|Mv$|
00000070  eb d3 bd b3 d8 35 5e 73  d1 10 43 fc 0d a3 53 80  |.....5^s..C...S.|
00000080  00 00 00 00 8f 1a d0 68  c4 bb 16 39 9a 47 ef a6  |.......h...9.G..|
00000090  bc 5a f7 5c 4e 97 6f 33  14 4c c2 54 22 92 13 ad  |.Z.\N.o3.L.T"...|
000000a0  45 81 cc 6c 01 03 03 01  02 00 0c 03 00 00 00 00  |E..l............|
000000b0  94 35 77 00 00 00 00                              |.5w....|
000000b7

kevinheavey · 2022-01-09T17:23:21Z

Not sure what to do with those 😅 I'm trying to find the line that's hanging

unordered-set · 2022-01-09T17:26:48Z

Not sure what to do with those 😅 I'm trying to find the line that's hanging

The hanging line is here: https://github.com/michaelhly/solana-py/blob/master/src/solana/message.py#L180 as num accounts is huge. So I'm trying to figure out if it is an issue with solana-cli serialization or this library deserialization

unordered-set · 2022-01-09T17:33:22Z

So, in case of two accounts reading with offset 64*2+1 10 b0 16 07 - this is correct. In case of one account: offset 1+64: 8e 96 6d 85. 85 is wrong. So either something is wrong with serialization or the library is not parsing the format properly.

t-nelson · 2022-01-09T17:46:06Z

Ok, these are binary dumps of txs:

(.venv) stromsund@DESKTOP-EMLQ922:~/solana-py$ hexdump -C two_signers.bin
00000000  02 01 01 05 c6 94 59 a3  2f d3 5c a5 d2 a7 93 e0  |......Y./.\.....|
00000010  39 30 d0 51 dd 6e fc d8  11 d4 e5 c9 7a 97 50 fb  |90.Q.n......z.P.|
00000020  8e 60 16 77 3a 85 e5 27  d4 47 fe 48 f6 2d a0 9c  |.`.w:..'.G.H.-..|
00000030  81 c7 12 bd 35 8f 62 48  b6 6a 45 1d 26 91 77 be  |....5.bH.jE.&.w.|
00000040  0d 69 9d 70 ab 15 06 0e  d1 f9 6a 1d d3 a8 42 96  |.i.p......j...B.|
00000050  8d c4 e6 7c 79 29 64 2d  61 e7 b3 49 2f c4 b1 ed  |...|y)d-a..I/...|
00000060  0e 8e 96 6d 85 d8 ee f4  a0 eb f3 b7 51 e0 21 5b  |...m........Q.![|
00000070  be 9f 24 9a 4a b5 ad 26  22 b5 78 36 9c 3c f0 d6  |..$.J..&".x6.<..|
00000080  47 10 b0 16 07 61 48 1d  35 74 74 bb 7c 4d 76 24  |G....aH.5tt.|Mv$|
00000090  eb d3 bd b3 d8 35 5e 73  d1 10 43 fc 0d a3 53 80  |.....5^s..C...S.|
000000a0  00 00 00 00 8f 1a d0 68  c4 bb 16 39 9a 47 ef a6  |.......h...9.G..|
000000b0  bc 5a f7 5c 4e 97 6f 33  14 4c c2 54 22 92 13 ad  |.Z.\N.o3.L.T"...|
000000c0  45 81 cc 6c 01 04 03 02  03 01 0c 03 00 00 00 00  |E..l............|
000000d0  94 35 77 00 00 00 00                              |.5w....|
000000d7
(.venv) stromsund@DESKTOP-EMLQ922:~/solana-py$ hexdump -C one_signer.bin
00000000  01 00 01 04 3a 85 e5 27  d4 47 fe 48 f6 2d a0 9c  |....:..'.G.H.-..|
00000010  81 c7 12 bd 35 8f 62 48  b6 6a 45 1d 26 91 77 be  |....5.bH.jE.&.w.|
00000020  0d 69 9d 70 ab 15 06 0e  d1 f9 6a 1d d3 a8 42 96  |.i.p......j...B.|
00000030  8d c4 e6 7c 79 29 64 2d  61 e7 b3 49 2f c4 b1 ed  |...|y)d-a..I/...|
00000040  0e 8e 96 6d 85 d8 ee f4  a0 eb f3 b7 51 e0 21 5b  |...m........Q.![|
00000050  be 9f 24 9a 4a b5 ad 26  22 b5 78 36 9c 3c f0 d6  |..$.J..&".x6.<..|
00000060  47 10 b0 16 07 61 48 1d  35 74 74 bb 7c 4d 76 24  |G....aH.5tt.|Mv$|
00000070  eb d3 bd b3 d8 35 5e 73  d1 10 43 fc 0d a3 53 80  |.....5^s..C...S.|
00000080  00 00 00 00 8f 1a d0 68  c4 bb 16 39 9a 47 ef a6  |.......h...9.G..|
00000090  bc 5a f7 5c 4e 97 6f 33  14 4c c2 54 22 92 13 ad  |.Z.\N.o3.L.T"...|
000000a0  45 81 cc 6c 01 03 03 01  02 00 0c 03 00 00 00 00  |E..l............|
000000b0  94 35 77 00 00 00 00                              |.5w....|
000000b7

These look like transaction messages not transactions. Messages don't have the signatures array prepended

kevinheavey · 2022-01-09T17:54:19Z

Yeah if I call Message.deserialize(one_signer) it gives me a message with this header:

MessageHeader(num_required_signatures=1, num_readonly_signed_accounts=0, num_readonly_unsigned_accounts=1)

unordered-set · 2022-01-09T17:56:08Z

Yes, I see, it was a coincidence, that it was also correctly interpreted as transaction in the first byte. Guys , thanks a lot and sorry for bothering you

kevinheavey · 2022-01-09T17:57:48Z

No worries, thank you for speaking out when there appeared to be a problem!

Guess we can get back to this PR now

t-nelson · 2022-01-09T18:05:51Z

Rust transaction sanitization is done here. Might pay to have similar logic and raise more helpful errors

unordered-set · 2022-01-09T23:40:17Z

@kevinheavey @michaelhly , Added tests, do you mind making one more review iteration please?

michaelhly

LGTM!

unordered-set · 2022-01-11T01:27:16Z

Thank you @michaelhly ! I just don't have write permissions to click merge :)

Add a partial support for vote program

1163718

unordered-set added 2 commits January 9, 2022 23:33

Add tests

0b113dd

Add some comments

23f0bbd

Fix linting errors

717a3c1

michaelhly approved these changes Jan 10, 2022

View reviewed changes

michaelhly merged commit 291286e into michaelhly:master Jan 11, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a partial support for vote program #167

Add a partial support for vote program #167

unordered-set commented Jan 8, 2022

michaelhly commented Jan 8, 2022

codecov bot commented Jan 8, 2022 •

edited

Loading

unordered-set commented Jan 9, 2022 •

edited

Loading

kevinheavey commented Jan 9, 2022 •

edited

Loading

unordered-set commented Jan 9, 2022

unordered-set commented Jan 9, 2022 •

edited

Loading

kevinheavey commented Jan 9, 2022

unordered-set commented Jan 9, 2022

unordered-set commented Jan 9, 2022

t-nelson commented Jan 9, 2022

kevinheavey commented Jan 9, 2022

unordered-set commented Jan 9, 2022

kevinheavey commented Jan 9, 2022

t-nelson commented Jan 9, 2022

unordered-set commented Jan 9, 2022 •

edited

Loading

michaelhly left a comment

unordered-set commented Jan 11, 2022

Add a partial support for vote program #167

Add a partial support for vote program #167

Conversation

unordered-set commented Jan 8, 2022

michaelhly commented Jan 8, 2022

codecov bot commented Jan 8, 2022 • edited Loading

Codecov Report

unordered-set commented Jan 9, 2022 • edited Loading

kevinheavey commented Jan 9, 2022 • edited Loading

unordered-set commented Jan 9, 2022

unordered-set commented Jan 9, 2022 • edited Loading

kevinheavey commented Jan 9, 2022

unordered-set commented Jan 9, 2022

unordered-set commented Jan 9, 2022

t-nelson commented Jan 9, 2022

kevinheavey commented Jan 9, 2022

unordered-set commented Jan 9, 2022

kevinheavey commented Jan 9, 2022

t-nelson commented Jan 9, 2022

unordered-set commented Jan 9, 2022 • edited Loading

michaelhly left a comment

Choose a reason for hiding this comment

unordered-set commented Jan 11, 2022

codecov bot commented Jan 8, 2022 •

edited

Loading

unordered-set commented Jan 9, 2022 •

edited

Loading

kevinheavey commented Jan 9, 2022 •

edited

Loading

unordered-set commented Jan 9, 2022 •

edited

Loading

unordered-set commented Jan 9, 2022 •

edited

Loading