Fix getAccountsWithOptionalFeePayer

[ChikinDeveloper]

Fix getAccountsWithOptionalFeePayer.
Had issues signing transactions with phantom wallet, took 3 days to find out why haha
Source code for transaction encoding
The programIds needs to be placed at the end the accounts list

[codecov-commenter]

Codecov Report

Merging #184 (cd0bc1a) into master (fbcb760) will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #184      +/-   ##
==========================================
+ Coverage   80.37%   80.39%   +0.02%     
==========================================
  Files         143      143              
  Lines        3108     3112       +4     
==========================================
+ Hits         2498     2502       +4     
  Misses        610      610              

Impacted Files	Coverage Δ
packages/solana/lib/src/encoder/extensions.dart	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fbcb760...cd0bc1a. Read the comment docs.

[ookami-kb]

Hey @ChikinDeveloper

Can you please add some tests? Also, this order should already be taken care of by the fact that AccountMeta is Comparable with the order defined here.

[Ltei]

Hi @ookami-kb, added a test
It didn't work for a transaction with multiple instructions

[ookami-kb]

I see. Are there some docs on why the program IDs should be placed at the end of the account list? The only information I can find is this one:

The addresses that require signatures appear at the beginning of the account address array, with addresses requesting write access first and read-only accounts following. The addresses that do not require signatures follow the addresses that do, again with read-write accounts first and read-only accounts following.

And this order is respected by AccountMeta's compareTo method.

[ChikinDeveloper]

No, there isn't any doc about it, in fact the transaction will work even if you don't place the program ids at the end.
The problem is that for the same transaction the encoded message will differ from a message encoded with the official solana librarie.
We had problems to sign a transaction with phantom : we pass a transaction object and phantom encodes it (using the official solana library) and signs.
When we tried to send the signed transaction by encoding it with your library, we got a "Signature verification failure" error, because Phantom had signed a different message.

[ookami-kb]

Yeah, I see. Well, that can be a problem – relying on the implementation details of a specific library. There's no guarantee that this implementation won't change in the future.

By the way, as I see it from the source code, they rather sort signatures in the lexicographical order, not just "program ids go last".

Can you please provide more details on your use case? I'm just trying to understand where's the connection between dart and JS libraries.

[ChikinDeveloper]

Just checked the rust version, it's the same they place program ids at the end source.

Indeed, I forgot to check this part, good job spotting that!

We have a flutter web app, and try to interact with Phantom chrome extension. So we need to convert from dart to js and back in order to call Phantom with a javascript Transaction object.
There is no way to sign multiple transactions at once (adapter source code) other than passing a list of js Transaction objects, and some wallets like coin98 doesn't even have a "signMessage" method.

I can imagine an other (very specific) use case :
Imagine I have an flutter android app that uses your library, and a javascript backend/api
The api has one method that sends a signature (from a private dev wallet) for a specific solana transaction (the api method only takes the user wallet as parameter)
It would lead to the same kind of issue.

I understand that there's no guarantee that this implementation won't change in the future. And it isn't even needed since the transactions will work as long as you sort signed/writeable accounts (and feePayer).
But I don't see any drawback so why not ? haha

Moreover if the implementation change in the future, this may lead to issues with all the websites that uses the library, if their version differs from the version used to a wallet they connect to. So I guess it should be written somewhere in the solana doc (in a developer standard section or something like that)

[ookami-kb]

But I don't see any drawback so why not ?

The drawback that I see there is that the dart library would depend on the undocumented implementation of the js library. Which in the future will lead to the situation that a specific js library version is only compatible with a specific dart library version.

Also, while the Rust library indeed places program IDs at the end, it doesn't do this lexicographic sorting, as far as I see it. So Rust and JS implementations are different as well. So the same situation can in theory happen with mixing dart <-> rust code.

I can imagine an other (very specific) use case :
Imagine I have an flutter android app that uses your library, and a javascript backend/api
The api has one method that sends a signature (from a private dev wallet) for a specific solana transaction (the api method only takes the user wallet as parameter)
It would lead to the same kind of issue.

In that case, I would rather say that since API accepts a signature and uses format-specific serialization, it should return a compiled message instead of relying on client knowledge about implementation details of the back-end. Otherwise, it just looks like bad architecture.

---

In your case, I would say it's better to add some layer that would actually transform the instructions by reordering accounts to match the order from JS library. I understand that it will introduce some overhead, but I don't think that one library should rely on some "random" implementation details from another library, especially when they are:

• undocumented;
• inconsistent with other (Rust) libraries.

[ChikinDeveloper]

The drawback that I see there is that the dart library would depend on the undocumented implementation of the js library. Which in the future will lead to the situation that a specific js library version is only compatible with a specific dart library version.
What I meant when I say that I don't see any drawback, is that right now you are 100% sure that your library is not compatible with the js library.
While with these changes you are just unsure about how long your library will be compatible.

Here is the commit that added locale lexicographical sorting I say "Fix non deterministic writeable account order" so I guess it has been added to prevent this kind of issue. (But maybe sorting by locale isn't the best idea ever haha)

In that case, I would rather say that since API accepts a signature and uses format-specific serialization, it should return a compiled message instead of relying on client knowledge about implementation details of the back-end. Otherwise, it just looks like bad architecture.
I can understand that. To my mind there should be some kind of standard.

[ChikinDeveloper]

I found this : solana-labs/solana#21722