Application transactions

[jannotti]

Allow AVM 1.0 programs to "execute transactions"

This is a pretty big change so I will comment on individual pieces by reviewing my own code, but the high level summary is.

1. An app now has authority over an account which is the hash of its appid.
2. The app can create transactions with three new opcodes, tx_begin, tx_field, and tx_submit.
3. If the transaction so created would need to be authorized by that app account from 1, the effects occur.

So far, only pay and axfer are allowed. acfg and afrz are under consideration, but have separate issues written.

This makes ApplyData a recursive datastructure, so that the "inner" transactions of app calls can be encoded in EvalDelta and, themselves, have ApplyData. However, this PR does not yet expose that new inner transactions list to the REST API. Another issue covers that need.

[jannotti]

LedgerForLogic needs to be able to "Perform" a transaction now.

[jannotti]

Cleaner ways to set up complicated balance tables for testing. This was, coincidentally, recently requested from an outside contributor.

[tsachiherman]

just a wild idea - the address we're going to pick is a pseudo-account public address, right ?
if so, can we pick an address that has no private key ?
another thought is to have that address be programmable - like the Rekey/AuthAddr. I'm not sure what this is good for, but I think that someone could find a good use for that.

[jannotti]

I think we're just putting our faith in the magic of cryptographic hashes, right? The intent is that the newly allocated address does not have a private key (or, equivalently, such a key exists, but nobody can find it). It's the same reasoning behind logicsig's controlling an address that we certainly hope does already belong to somebody.

I wrote all that before really realizing what you meant - I'll leave it in case it's helpful to someone else. I now realize you're asking if, upon app creation, maybe we can pick our app address, as long as the account is not in use. I think that's insecure because I may know your public address before you fund it (this would be particularly true of a templatized logic sig that has not been instantiated yet, it would also be true if I watch the transaction pool). I should not be able to make my app use that address, or else when you "come into existence" the app will control your funds.

As for setting the address. I think the rule you would want is that you can only give the app authority over an account that you yourself have authority. That is, you certainly don't want to reset an app address to a big foundation address and ask it for money! So I believe that we accomplish this by approaching it from the opposite direction. We allow you to rekey your account to the app's address. And then it has authority. But, of course, you can only rekey that account because you own it.

[jasonpaulos]

I think @tsachiherman is actually asking about choosing an invalid ed25519 curve point such that no private key exists that could match the address. I believe the MainNet fee sink and rewards pool are invalid points, though I'm not sure how to derive such an invalid point.

[tsachiherman]

Correct - what I meant was to select a ed25519 which no private key exists for. This is a post-quantum guarantee equivalent, since no crypto operation would generate that point. The x coordinates on ed25519 curve are mostly on the positive side; in fact, if you'll pick a number on the negative side that is smaller than -486662, I believe that there won't be any solution for that. ( i.e. that number would not be on the curve.. )
In fact, you don't even have to use a hash here; you can just use the -486662 - AppIndex. But I do realize it might not be as obfuscated as a hash ;-)

[jannotti]

I had no idea that the public key was used so directly - I sort of assumed it seeded a PRNG that was then used to pick points. I assume by "negative", you mean the 32 bytes is treated as a two-complement u256. So really we could pretty just flip the high bit?

I am far too timid to do anything like that without three or four cryptographers telling me it's a good idea.

[tsachiherman]

I completely agree that we should consult a cryptographer or two here. The randomness you're referring to is used when generating the private key; the public key is a pretty trivial calculation from there :

int
crypto_sign_ed25519_seed_keypair(unsigned char *pk, unsigned char *sk,
                                 const unsigned char *seed)
{
    ge25519_p3 A;

    crypto_hash_sha512(sk, seed, 32);
    sk[0] &= 248;
    sk[31] &= 127;
    sk[31] |= 64;

    ge25519_scalarmult_base(&A, sk);
    ge25519_p3_tobytes(pk, &A);

    memmove(sk, seed, 32);
    memmove(sk + 32, pk, 32);

    return 0;
}

Note that there is another randomness taking place when verifying a signature ( irrelevant in here ).

I do like the idea that you've had of using a hash function, as it "distributes" the account addresses as well, btw.

[algorandskiy]

Nice work and looks like a clever integration into existing code. I guess a little more code for updating ApplyData with side effects left, and quite a bit tests in ledger.
I do not like some code duplication in Perform and tx_submit but have no better ideas on how to get rid of it.

[tsachiherman]

Adding a controlled address is cool. Is there any application benefit for supporting multiple application controlled accounts for a single application ?

[jannotti]

There is at least one - suppose you wanted to escrow algos from many accounts, and always be wiling to give them back, with the rewards they've earned in the meantime. Since rewards are impossible to calculate in contract, you'd need to segregate your deposits, rather than keep them in one pot.

I originally planned on having a way to create any number of app accounts, for exactly this reason (and some more obscure ideas). But I think that the ability to rekey to the app addr is powerful enough. To use such a contract, a depositor would make a new account, opt it into the contract, and then, in an atomic group:

1. make a deposit
2. call the contract to bind the new address to the depositor (by writing the deposit address in address's app local state)
3. rekey the new account to the app account

Now, when depositing or withdrawing, the caller tells the contract where his deposit account is. So the contract can use these extra accounts that were ceded over to it because of rekeying. Which also prevents the caller from messing with them ever again.

(If you really like cleanliness, you'd have an operation to ask the contract to close out the account if you wanted to withdraw it all.)

[tsachiherman]

I think that this is a good change, but I'm not sure it has to be coupled with the rest of this PR.

[algorandskiy]

I guess all my remarks are resolved:

1. fee credit is calculated based on MinFee proto param in order to be deterministic. Max inner txn is set to 16.
2. min balance checks are applied to inner txns and accounts appear in cow's modifiedAccounts() that is confirmed in tests

TxnCounter update might be done in a separate PR

[jasonpaulos]

Ah, my comment was ambiguous. I mean the description field here should probably have value OK, since I believe it's purpose is to describe what a 200 response means. For example see the respones for /v2/applications/{application-id}.

[jasonpaulos]

This is to test that an app can't overspend assets, correct? So why isn't the error "insufficient balance" or similar?

[jannotti]

It's actually: "underflow on subtracting 101 from sender amount 100" or some such. I suppose I could construct the entire message.

[jasonpaulos]

One tiny request, but other than this and updating the old LogItem tests we talked about, this is looking good

[jasonpaulos]

We should add "format": "byte" here too to convey this is a base64 encoded binary string. This can maybe be done in a separate PR though.

[tolikzinovyev]

What is the solution for this one?

[jannotti]

It turned out that the allocbounds are not our only protection. We don't decode streamed msgpack messages. We take them off the wire, with bounded size, and then decode.

[tolikzinovyev]

Given what Nickolai said about msgpack pre-allocating an array with a decoded integer size, I think there can an amplification attack where a few decoded bytes allocate 16 new SignedTxnWithAD objects.

Something like this when decoding SignedTxnWithAD:

dt itx 16 dt itx 16 dt itx 16 ...