Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FLIP-0099: Storage Fees #99

Merged
merged 24 commits into from
Apr 5, 2021
Merged

FLIP-0099: Storage Fees #99

merged 24 commits into from
Apr 5, 2021

Conversation

janezpodhostnik
Copy link
Contributor

@janezpodhostnik janezpodhostnik commented Oct 13, 2020

Smart contract is here: onflow/flow-core-contracts#76
Documentation task is here: #194

@janezpodhostnik janezpodhostnik self-assigned this Oct 13, 2020
@vercel
Copy link

vercel bot commented Oct 13, 2020

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/onflow/flow-docs/E6KSLP3GDKXmu79bJxmvUMhc6V23
✅ Preview: https://flow-docs-git-janez-storage-fees-onflow.vercel.app

Copy link
Contributor

@psiemens psiemens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great draft! I knew very little about this problem and the proposed solution, but reading this gave me a great overview. 👏

- absolute: get all registers for each address (touched in the current `View`) and sum their size to get `storage_used`
- differential: on the current `View`, keep track of register size changes per address and sum them with the previous `storage_used`.

The decision was made to go with the differential approach, because iterating through all key value pairs in storage for each account touched would be to expensive and would not scale well.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, this choice makes sense 👍

flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
2. NewTransactionSequenceNumberChecker
3. NewTransactionFeeDeductor
4. NewTransactionInvocator <- actual transaction
5. *There are no further register (view) writes (or deletes) after this point. All fungible token moves already happened. All storage size changes already happened*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

3. NewTransactionFeeDeductor
4. NewTransactionInvocator <- actual transaction
5. *There are no further register (view) writes (or deletes) after this point. All fungible token moves already happened. All storage size changes already happened*
6. **NewStorageLimiter** <- for all participating account that have touched registers, compute account size (if over limit raise error)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I'm glad the transaction pipeline will be put to use.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just learned about this, great idea to make it extensible like that! 👏


The basic principle of this approach is to update accounts `storage_used` every time a register size changes, by adding the size change to current `storage_used`.

- During `View.Set` and `View.Delete` if storage size changes update `storage_used`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this require an extra read before every write?

I suppose in most cases, a register is either being written for the first time or is being written after previously being read earlier in the transaction. This is getting into implementation details, but I wonder if the View could keep a clever cache of register sizes as it performs every read in order to prevent future redundant lookups.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A cache to reduce reads is a good idea. It also reduces the amount of times the spock needs to be updated, which is one of the concerns I had.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reads are used to generate SPoCKs so if cache is to be used it must be deterministic

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the order and number of times a register is read is an important source of entropy for SPoCKs, if we want to put it behind a cache we need @tarakby's approval.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't have it already, I think we should add a cheap look-up to check the size of each storage register without loading its contents. This is going to be useful for LOTS of things (for example, any wallet software that helps a user manage their storage usage when their account starts to fill up!). To be clear, I'm not suggesting we need to expose this to Cadence yet (although that would be awesome), but we might as well build it internally for now, knowing that it will be useful to expose externally later.

Co-authored-by: Peter Siemens <peterjsiemens@gmail.com>
with this the migration pseudo-code can be written as:

```py
def migrate(key, value) -> (keyType, valueType)[]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work on using python for explaining the logic. 👏

Copy link
Member

@turbolent turbolent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, well done @janezpodhostnik! 👏

I've added some suggestions, but nothing major, they're all minor/cosmetic

flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
- the payer pays for the minimum storage needed by the address
- checks if `storage_capacity` is already present on the address (if so panic)
- sets the `storage_capacity` to `minimumAccountStorage`
- using `FlowFees` deducts `flowPerAccountCreation` from the payer
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would maintain that only base transaction fees (the cost of inclusion and computation) should be implicitly paid by the payer. All other fees (including storage costs and account creation) should be explicitly paid by passing in a FLOW vault. Of course, the payer can pay these amounts, but we shouldn't require it.

(BTW- It seems like we currently charge the payer implicitly for creating a new account, which I consider incorrect. Personally, I feel strongly enough about this that I'd rather have account creation and storage payment be inconsistent rather than move us towards having more things paid for implicitly. Assuming, of course, we don't have time to change the way account creation is paid for.)

flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
flips/20201310-storage-fees.md Outdated Show resolved Hide resolved
@psiemens psiemens changed the title Storage fees FLIP FLIP: Storage Fees Nov 24, 2020
@psiemens psiemens changed the title FLIP: Storage Fees FLIP 99: Storage Fees Nov 24, 2020
@turbolent
Copy link
Member

@janezpodhostnik Maybe we can merge this now, given that it is now implemented and released?

Copy link
Member

@turbolent turbolent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic work! 🌊 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FLIP Flow Improvement Proposal
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants