Feat: Optional Positional Accounts

[stegaBOB]

Problem

At the moment, the only way to handle accounts that may or may not be passed in to your program is through the remaining_accounts vec. The downside of this is there is no way to easily handle checks and deserialization with the remaining accounts vec, and programs would need to keep track of what accounts should be passed in and it's actually so horrible.

Solution

Implement support for "optional" accounts in the Accounts struct. This would allow for significantly more flexibility while also providing safeguards from shooting oneself in the foot with remaining_accounts

#[derive(Accounts)]
struct OptionalTest<'info> {
    payer: Signer<'info, Pool>,
    ...
    #[account(init, payer = payer, space = AccountStruct::LEN)
    maybe_new: Option<Account<'info, AccountStruct>>,
    // System can now be optional and is tied to whether `maybe_new` is passed in
    system_program: Option<Program<'info, System>>
}

In the example above, the maybe_new account is only created if it is passed in. If not, &ctx.accounts.maybe_new will return None.

How this works?

The key to make everything work is the fact that repeated accounts don't add to the size of the transaction due to how Solana serializes accounts in transactions. We use this fact to denote an optional account with the program Id. When the accounts are passed in, if an account should be ignored, clients will send the program Id instead. In addition, with this implementation, adding on optional accounts at the end of an Accounts struct wouldn't be a breaking change and the program will treat the missing account as None (since older clients will not be passing in this new account).

Thanks to @febo for working on this with me!

[Henry-E]

Best to keep the version changes to a separate PR, assuming I can get it merged quick enough for you

[stegaBOB]

Best to keep the version changes to a separate PR, assuming I can get it merged quick enough for you

I'll revert that for now. Merging the changes seems to have broken a bunch of things :(

[Henry-E]

👏 big round of applause for such a massive contribution here.

Looks good to merge but as with all things smart contract related should treated extremely carefully. The only major issue previously found with anchor was in the init_if_needed code, which allows for optional accounts in a slightly different way than presented here. So anyone coming across this, please be very considerate as to whether you truly need optional accounts in your program or if the client could handle the optional logic instead.

Thanks again for all the hard work @stegaBOB!

[dhruvja]

Are optional accounts allocated in stack by default? If i want to box those accounts, how would i able to do that?
I tried this

/// This failed, due to access violation which i think is because it ran out of memory
escrow_account: Option<Box<Account<'info, TokenAccount>>>, 
// But This worked
escrow_account: Box<Account<'info, TokenAccount>>

How can i allocate optional accounts in heap?