Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EIP 103 (Serenity): Blockchain rent #35

Open
vbuterin opened this issue Nov 25, 2015 · 62 comments
Open

EIP 103 (Serenity): Blockchain rent #35

vbuterin opened this issue Nov 25, 2015 · 62 comments
Assignees
Labels

Comments

@vbuterin
Copy link
Contributor

@vbuterin vbuterin commented Nov 25, 2015

Motivation

The Ethereum platform gives users access to a number of computational resources: particularly computational steps, memory, logs / bloom filter usage and permanent storage in the Ethereum state. For most of these resources, the gas system is roughly incentive-compatible, although price-fixing the exchange rates between these resources is, in the current framework, unfortunately required. Storage, however, is unique in that it is not ephemeral; once storage is filled, it must be stored by all full nodes forever until/if it is deliberately cleared. The cost of storage is bytes * time, but users are only paying for bytes.

A partial solution exists for Ethereum 1.0 in the form of the SSTORE clearing refund, where a user gets up to a net 10000 gas back for converting a filled storage slot into an empty slot. However, this approach is imperfect for two reasons:

  1. During a block where transaction inclusion demand does not reach the gas limit (or even where the opportunity-cost gasprice is cheaper than average), miners have the incentive to fill up storage slots so as to use them as a "bank" that expensive transactions can use to grant themselves additional gas at a time when gasprice is expensive; essentially, conducting inter-temporal gas arbitrage with the additional negative externality of requiring all full nodes to store extra data in the meantime.
  2. There is no incentive to use storage slots for a short period of time instead of a long period of time.

Proposal

A proposal for solving this is to introduce "blockchain rent": rather than charging for bytes, actually charge for bytes * time. Here is a proposed protocol for doing so:

  • In the account RLP, keep track of the variables last_accessed and total_storage_bytes. last_accessed is meant to represent the block number at which storage was most recently changed, and total_storage_bytes is meant to represent the total number of bytes in account storage.
  • When you perform an SSTORE(k, v) operation on the account, let fee = (TOTFEE[block.number] - TOTFEE[last_accessed]) * account.total_storage_bytes, where TOTFEE[n] is the sum of the data storage fees from block 0 to block n (eg. TOTFEE[n] = n * c for some constant c would work as a basic policy but one can come up with more dynamic policies that change per block). Set account.balance -= fee, account.last_accessed = block.number and account.total_storage_bytes += len(v) + (len(k) if v != '' else 0) - len(old value of account.storage[k]) - (len(k) if old value of account.storage[k] != '' else 0). If account.balance < 0, destroy the account.
  • Create a special type of ping transaction which performs a dummy SSTORE(0, account.storage[0]) operation on any desired account. Miners can run this if they see an account that needs to be deleted in order to process fees and delete it if it is in fact bankrupt. If and only if a ping leads to an account bankruptcy, it requires no gas.

For example, suppose that TOTFEE[n] = n * 10 (ie. 10 wei per block per second), a block was last accessed in block 321070, the current block is 312085, the account has a pre-existing balance of 1000, and it only has two keys in storage: {"cow": "dog", "horse": "doge"}. We should have account.last_accessed = 321070 and account.total_storage_bytes = 3 + 3 + 5 + 4 = 15. Now, suppose we have a transaction that sends 10000 wei and, in the process of executing code, sets SSTORE("moose", "deer"). We compute:

  • Balance increased from 1000 to 11000
  • fee = (TOTFEE[312085] - TOTFEE[312070]) * 15 = (3120850 - 3120700) * 15 = 2250
  • Balance decreased from 11000 to 8750 due to the fee
  • account.total_storage_bytes += len("deer") + len("moose") - 0 - 0, ie. increased by 9 for a new total of 15 + 9 = 24. The values subtracted are both zero because there was nothing pre-existing in that key.
  • account.last_accessed = 312085

Setting TOTFEE

The hardest challenge in this scheme is setting the TOTFEE function. One could set this to a static value (ie. TOTFEE[n] = c * n), but this is essentially a form of price-fixing and so is arguably hard to set in a way that is economically efficient. A somewhat more effective strategy is to set a space limit (arguably, this is no less reasonable than setting a gas limit), and targeting TOTFEE based on that value. For example, consider a policy where we have some function for setting a space limit L (eg. a flat L = 2**35 or a linearly growing L = 2**10 * (block.timestamp - genesis.timestamp)). Then, every block, we set TOTFEE[n] = TOTFEE[n-1] + 2**(20 * L / (L - total_state_size)) (ie. the fee is 2**20 wei ~= 0.001 shannon initially, then when the total state size increases to 5% of the maximum it doubles, when it increases to 10% of the maximum it doubles again, and so forth, with the doubling intervals decreasing in size and ultimately converging to a singularity at L). In theory, the amount of storage usage should increase until it hits an equilibrium equal to the marginal demand for storage at a supply somewhere less than L (in the case of the above example, roughly L / 2 is plausible); the main tradeoff to be made is in the steepness of the supply function. Steeper supply functions are more robust to mistakes made in the initial supply-function-setting process, but they also lead to more volatile rent charges.

Why should I want to support a proposal that adds in a "weird new tax" when storage usage per second is free now?

  • The SSTORE gas costs may go down, benefiting dapp developers that only use storage for short time durations
  • The disk space requirements for a full node will go down
@vbuterin vbuterin self-assigned this Nov 25, 2015
@frozeman frozeman added the type: Core label Nov 25, 2015
@alexvandesande
Copy link

@alexvandesande alexvandesande commented Nov 25, 2015

In that proposal, the only way someone else can pay a contract's fees is to send money to it, correct? Should there be a way in which someone could send money to a contract but that the contract itself could not spend to anything other than fees? The scenario I'm thinking is one where someone created a contract that always does something with any money it received and keeps a balance of zero because it was programmed at a time that that shouldn't matter.

For example: say I build a contract that whenever anyone sends money to it, it automatically sends it 50% to Bob and 50% to Alice. It's a very useful contract for both Bob and Alice and they would like to keep it alive, but they can't send money to it in order to pay its fees because it will always forward all the deposits back to them. Even if the contract was programmed with the rent scenario in mind, how can Bob and Alice even know how much is enough for the contract? If say, the contract is programmed to keep 0.1% of anything that it forwards, and if storage costs are low and the contract is used a lot, it might create a scenario where the contract has fees to pay it's rent for the next 30 years and Bob and Alice cannot retrieve any of these funds..

About how to set the fee, can't that be left to the miners? Each contract might offer a price and miners choose to accept it or not and contracts can only be deleted if no miner in the last N blocks have accepted their rent?

@vbuterin
Copy link
Contributor Author

@vbuterin vbuterin commented Nov 25, 2015

In that proposal, the only way someone else can pay a contract's fees is to send money to it, correct?

Correct.

One option to solve the problem is for the contract to keep an internal "shadow balance" in storage, and then have a function for getting the difference between balance and shadow balance; someone wanting to send to the contract to fill it up would just call this function beforehand in order to learn how much to send, and then send that amount on top of what they were going to send.

About how to set the fee, can't that be left to the miners? Each contract might offer a price and miners choose to accept it or not and contracts can only be deleted if no miner in the last N blocks have accepted their rent?

The economics of this seem iffy; particularly, in this exact scheme, note that each miner receives 100% of the gain from including a contract but only pays a small fraction of the cost, so you'll get a much-larger-than-optimal quantity of contract inclusion.

@vbuterin vbuterin changed the title Blockchain rent EIP 103: Blockchain rent Nov 25, 2015
@vbuterin vbuterin changed the title EIP 103: Blockchain rent EIP 103 (Serenity): Blockchain rent Nov 25, 2015
@chriseth
Copy link
Contributor

@chriseth chriseth commented Nov 25, 2015

Having to keep contracts funded with ether all the time seems a bit awkward, although I can certainly see the need for and benefits of that. Which alternatives are there? Is it possible to pay for storage using gas? Can a contract perhaps deduct a certain amount of gas during the SSTORE that is used to hold the value in storage?
Would it be possible to set up a "Storage fund contract" that holds a certain amount of ether but is used by a group of contracts which can then retrieve just the required amount of ether to fund their existence?

Correctly balancing storage and computation fee is also a major concern, because we do not really know how costs for storage and computing will change in the future. Also with EIP-101 in mind - would it make sense to use a "storage token" that is different from ether and used to pay for storage? Perhaps some of the logic could be moved into the storage token contract.

Oh and perhaps a side note: Contract code can be used to emulate storage:
The only content of contract A's storage is a pointer to another contract P. It uses EXTCODECOPY to load data from P and whenever it wants to modify the data, it creates a new contract and updates the pointer. So I think we should also consider blockchain rent for code.

@vbuterin
Copy link
Contributor Author

@vbuterin vbuterin commented Nov 25, 2015

I suppose you can set up a system where a contract accumulates a "debt" of gas fees and that debt has to be paid off before you can interact with the contract (ie. same as above, except you do debt += fee, and when you send a transaction to a contract you do msg.gas -= min(debt, msg.gas); debt -= min(debt, msg.gas) first), and if the debt goes higher than the block gas limit then the contract can get deleted. However, any mechanism by which gas can be "stored" potentially allows for inter-temporal arbitrage (ie. paying down debts only when gas is marginally free); not sure about this but it might be the inevitable tradeoff for currency neutrality.

Another approach here is a hybrid strategy: allow temporary storage using some kind of ether or gas fees, at low cost, and then have permanent storage at high cost.

@simondlr
Copy link

@simondlr simondlr commented Nov 25, 2015

Another approach here is a hybrid strategy: allow temporary storage using some kind of ether or gas fees, at low cost, and then have permanent storage at high cost.

I'm a fan of this. I feel there's something to be said for allowing some permanent storage on a blockchain. The "set & forget" nature of the "world computer" supported by various participants across the world is a big benefit. It's potentially dangerous to have some parts of a burgeoning contract ecosystem disappear, because rent wasn't paid (by whomever). Could lead to some cascading dapp failures. Also, who is set up to continue paying rent? If a build a dapp and rely on a contract that's paying rent, I want to be pretty damn well sure it's not going to drop off, or worse that I & others have to pay rent for it to keep going. If a developer says they will pay rent and they leave or die, will a community step up to continue paying rent?

@Smithgift
Copy link

@Smithgift Smithgift commented Nov 25, 2015

I'm greatly in favor of some kind of blockchain rent. There's an odd morality in essentially forcing future users to pay for current use.

I agree with chriseth about contract code being used as storage. If this is a possibility, it would be better to create a specific perma-store opcode. (Call it owning a part of the blockchain rather than renting it.) However, what does it mean if a contract has some owning-storage and some renting-storage and it can't pay the rent? Does the whole thing cease to exist? Or just the rented data? Either could be a landmine for a DApp developer.

Every currently existing DApp will have to have a new incentive structure to self-perpetuate. For example, demanding a small fee when you use it. In itself this isn't bad, but nearly all old contracts will disintegrate short generous donations. In a hybrid system, we could rule that all old DApps have retroactively used owning-storage, and that new DApps can use renting-storage as they please.

If EIP 101 happens, how will this interact with the new ETH-contract? There will be a contract at 0x00... whose data is altered by the protocol and not by itself. I suppose the ping transaction could actually be a normal transaction sent to 0x00... but I'm not sure how the contract would be able to delete other contracts in an elegant manner.

@PeterBorah
Copy link

@PeterBorah PeterBorah commented Nov 25, 2015

I'm pretty strongly opposed to this, unless there's absolutely no way around it. One of the unique properties of Ethereum is that contracts are trustworthy. I can build my contract on top of another one, and feel confident that that will continue to work.

In a blockchain rent world, that guarantee goes away. You now have to be paranoid that any contract you rely on might disappear at any time. Careful contracts will try to keep a fund around to rescue contracts they depend on, but that's never going to be a foolproof process. I have visions of a tiny utility contract used in a fundamental layer of contracts being forgotten about, and years later running out of funds, causing a quarter of active contracts to stop working. That's a failure you can't even recover from because of code immutability.

It also potentially opens up some attack vectors: I can call a contract a bunch of times, with the intention of filling up its storage and thereby making it too expensive to maintain. This costs me ether, of course, but it's only a one-time cost, while it might continue to cost the contract forever. And it might have disproportionate effects due to the cascading failure problem.

Finally, though probably least importantly, it adds a pretty serious amount of complexity to the process of creating a contract. Right now, the illusion that you're writing a normal web script is pretty good. After this change, there will be a whole host of complex issues around analyzing your code to determine future storage usage, checking dependencies to determine their likely reliability, etc.

@PeterBorah
Copy link

@PeterBorah PeterBorah commented Nov 25, 2015

Instead, we should be aiming to keep contract storage below Moore's Law. If we need to increase storage costs to do so, so be it.

@vbuterin
Copy link
Contributor Author

@vbuterin vbuterin commented Nov 25, 2015

Ok, so it seems like parametrizing contract storage lifetime with 0 = current log, ∞ = current sstore and 0 < k < ∞ = new hybrid option might be a route worth exploring, perhaps where temp storage is encapsulated in a clear way so it isn't mistaken for permanent storage. For one trivial example of where storage with lifetime 1 might be useful, consider cheques.

@PeterBorah
Copy link

@PeterBorah PeterBorah commented Nov 25, 2015

I think I don't understand what you mean by "lifetime 1". A cheque might go a long time without being cashed, so it will need a lifetime of longer than 1 block.

I'm definitely more open to a hybrid approach, though, if the unreliability of temporary storage is emphasized.

@wanderer
Copy link
Member

@wanderer wanderer commented Nov 25, 2015

on TOTFEE would it make sense to replace c as some function of the average gasPrice over x previous block?

EDIT: I kinda don't like targeting a total_state_size b/c we cannot predict hardware storage costs. But we can attempt infer the total system cost by the gasPrice.

@alanpog
Copy link

@alanpog alanpog commented Nov 25, 2015

One of the unique properties of Ethereum is that contracts are trustworthy

I don't think trustworthiness in this context means necessarily permanent contracts, but I'd characterize it as having certainty, at any time Ta that the contract will still exist at time Tb, for the Tb that matters to you.

There's huge value in being able to predict, at contract inception, the future cost of rent (as a function of storage), and then design a contract that will provably survive under certain conditions, irrespective of exogenous variables like rent price.

One way to accomplish this certainty is through derivatives on the price of rent (simple forwards). But that would likely be too cumbersome for the average contract.

Another alternative is a rent insurance market where contracts can buy "lifetime" or at least long-dated rent for a fixed cost. Rent insurance contracts could even have variable storage allowances (i.e. fix a price for a flexible storage size, not a precise amount), as moral hazard can be significantly mitigated by the immutability of the contract's code. Of course insurance markets have tail risk and may need to be bailed out (or crash) if not regulated.

@Smithgift
Copy link

@Smithgift Smithgift commented Nov 25, 2015

@vbuterin Are we talking arbitrarily-settable (by the contract) k? I could see that being very useful. I'm working on a game where the players will probably not care about the state a week afterwards.

For the sake of argument, suppose you prepaid storage costs in current gas (i.e. SSTORE costs an additional (c * k) gas, with some (big) flat rate for permanent hosting). The system would have to have some way of detecting/clearing/deleting the unpaid, but at least it's reasonably possible to determine when a contract will lose its storage.

@vbuterin
Copy link
Contributor Author

@vbuterin vbuterin commented Nov 25, 2015

I think I don't understand what you mean by "lifetime 1". A cheque might go a long time without being cashed, so it will need a lifetime of longer than 1 block.

I am specifically referring to the use case of paying for contract functions in subcurrency; in this case cheques only need to last for long enough to persist between the parent and child execution, ie. just the block during which the ancestor transaction is included. For other cases, yes, cheques should probably have a medium-length expiry.

@vbuterin Are we talking arbitrarily-settable (by the contract) k?

Yes.

@PeterBorah
Copy link

@PeterBorah PeterBorah commented Nov 25, 2015

That will work until scalability creates mandatory asynchrony. :)

It's worth pointing out that you as the user don't have control over when your transactions are included in a block. If there's a big gas price spike, or a networking issue, or just bad luck, you might miss even a deadline you thought was generous.

I am skeptical that self-destructing data is ever a good idea, but as long as there is the option for permanent data as well, I can just ignore the existence of the unreliable version, and advocate that others do the same, the way we ignore half of Javascript. Not ideal, but acceptable.

@alanpog
Copy link

@alanpog alanpog commented Nov 25, 2015

Ok, so it seems like parametrizing contract storage lifetime with 0 = current log, ∞ = current sstore and 0 < k < ∞ = new hybrid option might be a route worth exploring

@vbuterin , wouldn't accepting anything different than k=0 just reduce the scale of (but not solve) the problem we have now (lack of incentive compatibility for storage) as any price for k>0 would be a guess that gets more uncertain as k -> ∞ ?

edit: what I mean by accepting is setting today a price for. In this sense the original EIP proposal would only accept k=0 pricing.

@vbuterin
Copy link
Contributor Author

@vbuterin vbuterin commented Nov 25, 2015

Yes, accepting anything other than k=0 would still lead to some degree of lack of incentive compatibility or inter-temporal arbitrage. But judging from comments, very many people consider this an acceptable price to pay for certainty.

@alexvandesande
Copy link

@alexvandesande alexvandesande commented Nov 25, 2015

I have visions of a tiny utility contract used in a fundamental layer of contracts being forgotten about, and years later running out of funds, causing a quarter of active contracts to stop working.

This is something that can really happen, specially since solidity now supports libraries, which could create a cascading sheet of dependencies that no one is paying attention until one of the sub contracts get deleted.

@simonjanin
Copy link

@simonjanin simonjanin commented Nov 25, 2015

I think this is a great proposal.

But there is something that bothers me a little, which is a related argument to @alanpog's : at the time of creation, a contract with permanent storage has a cost C[t], which will be linearly more expensive than renting one for a given amount of time. The thing is, if the gas prices change non-linearly, then this makes for a very asymmetrical situation in the long run -- where miners/validators would have to store the data of the first contract permanently essentially for free, while newly created contracts would pay non-linearly more; i.e. C[t+k] >= λ*(C[t])^(1 + ε) for ε > 0.

In a word, the argument is that if the gas prices change super-linearly over time, then this is not only unfair to the new miners/validators, it also makes it very difficult to insure against (insurance relies on costs being at most linear in the number of objects insured).

If we know for sure that only linear changes of gas prices are possible, then this is fine.

@pipermerriam
Copy link
Member

@pipermerriam pipermerriam commented Nov 25, 2015

I'd like to echo some of what @PeterBorah and @alexvandesande said.

I'm really hesitant about much of what this proposes. I'd like this a lot more if we keep a permanent storage system alongside a new ephemeral storage system. However, switching to a purely paid storage system opens up attack vectors and the potential for disastrous cascading failures and it isn't immediately clear to me how these issues are going to be resolvable.

@niran
Copy link

@niran niran commented Nov 25, 2015

What if there were a way to revive an evicted contract? There's consensus on what the contract's state was when it was evicted, and if I'm willing to pay the gas costs and rent to get miners to store it and keep it around again, that could work around some of the terrifying scenarios where dependencies disappear. Transactions would just fail until someone revived the dependency contract.

If the revive mechanism is a bounty that any miner of future blocks can collect, most miners can continue to only store the active state while bounty hunter miners maintain quick access to older state that might be revived.

@pipermerriam
Copy link
Member

@pipermerriam pipermerriam commented Nov 25, 2015

What if there were a way to revive an evicted contract?

I'm not sure this solves anything. This assumes a lot of things about people paying attention to these things, as well as there being a benevolent source of funds to pay for ongoing storage.

@alexvandesande
Copy link

@alexvandesande alexvandesande commented Nov 25, 2015

@alanpog

Another alternative is a rent insurance market where contracts can buy "lifetime" or at least long-dated rent for a fixed cost.

@chriseth

Also with EIP-101 in mind - would it make sense to use a "storage token" that is different from ether and used to pay for storage?

@vbuterin

I suppose you can set up a system where a contract accumulates a "debt" of gas fees and that debt has to be paid off before you can interact with the contract

Notice that all these things are being implemented in swarm/ipfs. It would probably be cautionary to see how these solutions pan out in the long term and then see how it can be used in the case of the blockchain. Swarm doesn't implement a token, but rather a debt system where nodes keep debts to each other and if they are past a limit and aren't paid these nodes are cut out. It also implements an insurance market where you can buy "promises to store" something and you risk a deposit if that ever goes offline.

If nodes could rely on swarm/ipfs instead of having full nodes then they could store locally only chunks of the blockchain that are often requested and leave other chunks on the "cloud". This could solve the issue of blockchain rent on the client level, instead of the protocol level.

@niran
Copy link

@niran niran commented Nov 25, 2015

This assumes a lot of things about people paying attention to these things, as well as there being a benevolent source of funds to pay for ongoing storage.

The dapp pays attention to those things. The users are the benevolent source of funds. Consider a prediction market. Every transaction has fees, so the client software lets users know how much they're paying to take actions. If the user is attempting to redeem some of their winning shares but a required contract has been evicted, the client software includes the revival fees as part of the transaction fees. Alternatively, it lets the user participate in a crowdfund to revive the contract.

The perpetual blockchain storage world we live in is a seductive mirage. It is unsustainable to force future users to subsidize storage. Transaction costs in such a system will be higher than those in blockchains that charge rent, so it will be outcompeted. We are perfectly capable of building systems that collect enough revenue to fund their operations—this is how the whole world works.

Unrecoverable eviction does seem problematic, though.

@nmushegian
Copy link

@nmushegian nmushegian commented Nov 25, 2015

throwing support behind @PeterBorah and @pipermerriam .

This EIP has the potential to introduce a lot of technical debt for DAOs working on ethereum.

The "rent vs own" discussion is the only one I would entertain, since then you could say all contracts prior to the upgrade own their storage and the semantics don't change. Even making ownership a special case for ETH 1.0 contracts is acceptable, as inelegant as that is.

@Smithgift
Copy link

@Smithgift Smithgift commented Nov 25, 2015

Thinking about @niran's idea for reviving contracts.

Let's call this process archiving. When a contract is left untouched (by whatever metric) for long enough it can be archived, at which point it is replaced in the state by a notice of the block the contract was archived. It can be added back by providing a SPV proof of that contract's state at the block it was archived. Note that we still can't remove the existence of the contract entirely from the state, because there needs to be some way of determining its last true state to prevent an attacker from providing an old but still techncially valid SPV proof.

I think the idea has merit, but I'm not sure what odd technical problems it might present.

@nmushegian: If we do go with ownership, I would discourage it being a special ETH-1.0 case. Otherwise, smart investors would create thousands of contracts before the upgrade, and sell their permastorage post-upgrade.

@pipermerriam
Copy link
Member

@pipermerriam pipermerriam commented Nov 28, 2015

This has gotten to be quite a thread of conversation. I'm going to attempt to summarize the ideas that have been presented thus far in hopes that we can pull this together into something actionable.

  • The current state of storage being permanent is likely not priced correctly.
  • Having a mechanism for permanent contract storage will reduce the complexity in contract development.
  • In the event that a contract can't pay for itself, an archival or deletion mechanism may be necessary.
  • Any mechanism which causes contracts to end up being archived or deleted may result in cascading failures that could have serious ripple effects. This effects the trustless aspect of ethereum contracts as you can no longer blindly trust that they will stay around forever.
  • IPFS/Swarm/Sharding are potential down-the-road solutions
  • The pricing for storage should potentially be a dynamic formula that adjusts based on market conditions in order to derive the appropriate price for storage.
  • This is not necessarily a one-or-the-other choice as there could be mechanisms for both temporary storage and permanent storage.
  • Theoretically benevolent entities could be setup that ensure important contracts remain funded
@niran
Copy link

@niran niran commented Nov 28, 2015

I think @alexvandesande's comments lead to an important point: permanent storage without ongoing costs doesn't actually exist in today's Ethereum, nor can it ever. The optimal mining strategy is to prune state that is unprofitable to maintain. The simplest heuristic for this is to prune the state of the least recently used contracts. Miners who do this will ignore transactions that pay less than the cost of restoring that contract's state to accessible storage, either via fee payments or transfers from the contract's balance triggered by the transaction. If all miners use a similar strategy, blockchain rent would be imposed de facto: zero rent for transactions on recently used contracts, but the cost of restoring the contract's state for transactions on heuristically evicted contracts.

Formalizing rent or state archiving now seems unnecessary to me. Contract writers should be aware that permanent storage without ongoing costs is an illusion, so their users will have to pay a miner to restore the state if the contract doesn't have a built in economic mechanism to do so.

@pipermerriam
Copy link
Member

@pipermerriam pipermerriam commented Nov 28, 2015

permanent storage without ongoing costs doesn't actually exist in today's Ethereum, nor can it ever.

That is a very strong statement and I don't see the evidence to back it up. Please note that I'm not arguing the opposing point of view, but rather stating that we don't know how things will play out. It is very possible that either all storage will require ongoing rent or it may turn out that we can find mechanisms to allow truly permanent or effectively permanent storage.

@niran
Copy link

@niran niran commented Nov 28, 2015

My evidence is just that there's no way to force a miner to store everything. They're going to store the data that makes them money, which is less than everything. The rest of the conclusions follow from this.

If storage becomes so cheap that it's always profitable to keep everything around, this stops being an issue. If we think this is unlikely, as I do, then contract authors need to prepare for it.

@pipermerriam
Copy link
Member

@pipermerriam pipermerriam commented Nov 28, 2015

A miner cannot include transactions in blocks which reference storage for contracts it's chosen to delete so it's easy to make the alternative argument that a rational miner figures out how to store that data.

I'm not saying that this is what will be what happens. Rather that we don't know what will happen and we should be careful when making very important decisions if our supporting reasoning is based on an assertion about what people will or will not do in the coming years. Predicting the future is difficult and technology changes the rules of the game quickly sometimes.

@Smithgift
Copy link

@Smithgift Smithgift commented Nov 28, 2015

@niran: It's true that you can't force a miner to store anything. It's also impossible to force a miner to agree 2 ETH + 2 ETH = 4 ETH. The point of archiving being part of the consensus is that the protocol can punish miners who forgot contracts that everyone else has to remember.

While a non-formal rent system like this is elegant, there are issues. Other miners have to verify a transaction. If another miner creates a block using a pruned contract, they have three options:

  1. Take it on faith that the miner of the new block executed the contract accurately. This creates a kind of moral hazard should enough miners do this. (See: the unintentional forks from SPV mining in bitcoin recently.) Should the block be incorrect, and at least 51% are trusting or on the attacker's side, light clients may now be deluded as to what really happened in the contract, which could quickly spread across the entire state.
  2. Ignore the block until it's sufficiently deep. This may lead to effective censorship of an older contract if 51% of the miners will not mine on a chain that has a dubious block, which means that the chain is never deep enough for the block to become undubious. Even miners that trust/verify/don't care may still ignore the block on the basis that they don't want to be on the losing chain.
  3. Locate the archive of the contract and verify that it is a real block. This involves spending money and time for which the miner gets nothing in return, except, theoretically, being on the right chain if enough others do the same. A malicious miner could create wild goose chases for contracts that don't exist.

There's also the option of rejecting the block completely, in which case we have a fork.

Note also that any miner may pick any one of these strategies, The effects of all three being done in varying amounts may lead to instability, if nothing else. True, a modified p2p protocol where the miner shoves the contract's state along with the block may help some of these issues. But how does the miner know whether the other miners have the contract or not? If it guesses wrong, it either wastes both their time or the block is rejected.

NINJA EDIT: If the transaction is high value enough, a miner might reasonably assume that other miners dug up the same archive. There's troll potential in making an enormous contract, letting it be forgotten, then pinging with some ultra-expensive gas which turns out to be the equivalent of function() { throw; } The miner may not appreciate the small amount of gas he got from it relative to the expense of getting it.

Actually, the true cost of revival is (pretending we live in a world with equal-powered miners) something like (number of miners * cost of acquiring the archive), because the effective reward is averaged between every miner. This is true of every revival system where the user does not provide the contract's state.

Non-formal rent seems to have merit, but something about it makes me uneasy.

@etherReal
Copy link

@etherReal etherReal commented Nov 28, 2015

You can't have everything where would you put it?

👍 to @Smithgift above.

“My evidence is just that there's no way to force a miner to store everything. They're going to store the data that makes them money, which is less than everything. The rest of the conclusions follow from this.”

If we agree with @niran and assume that miners will drop what they can't profit from and then follow @jmlubin's reasoning to advocate rent or back taxes then there will be contracts that sink down the archive layers into the distant and potentially inaccessible past, whether or not they have a use to someone or everyone. Contract availability will be a function not of utility or gas value to any given user but to the ability to pay to embed rent in the contract or to stump up an ever increasing resurrection fee.

We may then see sets of contracts huddled in shop doorways with “will work for food” signs all along the block chain. This is a great way to bring back a class division justified by “meritocracy”, you might think of it as censorship of the poor.

I think the solution to this problem should be incorporated into Casper. The solution below offers an off the top of my head way to incorporate a redistribution of ether into the mechanics of the mining game.

To Illustrate my point lets look at this from a highly non technical metaphorical perspective:-

Let us take the underlying market mechanics and put it in the context of a real world market.

[Like mining,] Employment is administered as a 'market'. People get rewarded for their work based on the (market embedded) remuneration strategy of their employers. On the one hand there is competition between applicants to fill these roles and on the other the demand of employers to get a given job done. This hides a host of assumptions and game mechanics (e.g. class interests / cultural norms etc.) that are not explicit in the market mechanism itself.

There is an alternative method of administering work in the physical world called the Balanced Job Complex, which I shall now describe as it might be implemented for archiving or sharding in Casper.

As part of their 'bond' each Miner must prove that they have stored a randomly assigned share of archiving tasks (Assuming this to be potentially computationally and storage expensive but may never be rewarded) as some proportion of their “work”. All miners are also given a random share of the most regularly rewarding and empowering tasks (contracts that are very popular). Mining is thus coordinated with everyone's involvement in both 'rewarding' and 'burdensome' tasks on a “fair” basis. The burdens and benefits of work are then allotted by Casper so that each nodes ability to participate in democratic decision-making within the network resists class division.

This will not prevent the data base growing infinitely big but it will mean that at least in the medium term (for the next 5 years?) no mater how big the entirety of the system grows, not every full node has to store all of it.

P.S. I am sure this is a forking nightmare but it's all I've got.

@sillytuna
Copy link

@sillytuna sillytuna commented Nov 28, 2015

Rent should be charged for code and storage. There should be a (segregated?) pool of ETH that >each contract maintains that pays this rent.

Here's an example use case where it has to be clear how it'll work:

A contract containing a large number of 'users' is much more space efficient than a contract per registration. User data could be 32-128 bytes depending upon the details.

Who pays for the cost of the now large contract persisting? Each individual user item, which could be a Solidity array, isn't going to be individually tracked, it's an all or nothing affair I would assume. The contract could store additional data for who has paid renewal fees, and could then zero out anyone who hasn't (has its own implications in terms of data organisation - there is no simple/cheap way to 'compact' a solidity array for example IIRC).

Can we charge each user up front 'renewal' fees to guarantee availability for e.g. 7 years, an example period for documentation retention?

Who is paid those fees and how predictable will they be?

Will centralised services be needed to issue reminders and track renewal requirements, similar to domain name services?

What about where the blockchain is queried for older data only? Will we have archive nodes which allow that for some kind of fee?

What about the risk of cascaded failures - how can we be very transparent about renewal requirements for libraries and utilities? It's all very well saying we could each set up a kind of on-chain foundation but I think that's rather pie in the sky because it takes organisation - people aren't that organised.

I'm also concerned that decentralised contracts will then have the risk that they only work with centralised patronage.

@niran
Copy link

@niran niran commented Nov 29, 2015

@Smithgift Great points.

The point of archiving being part of the consensus is that the protocol can punish miners who forgot contracts that everyone else has to remember.

Formal archival just removes the incentive to ignore a restored contract by providing the entire state of the contract when the contract is restored so there's no need for all the miners to fetch it on their own. This doesn't have to occur at the consensus level, though. It can be replicated in contract code. If there's a hint that no old data besides headers is necessary to validate the transaction, there's no incentive to either blindly trust or avoid the transaction when mining new blocks. Contract writers could set a flag on functions that causes them to terminate and consume all of the provided gas if the function attempts to read from the contract's storage. Miners could safely execute these even if they've discarded the state.

Formal rent provides an avenue for an incentive to keep the state around, but there's no guarantee that the contract will pay rent. Miners will still discard the state of contracts they don't expect to pay rent, even if it's formalized. Might as well avoid formalizing it and let miners and contract writers coevolve to a point where they both have accurate expectations about what state will be kept locally and how profitable a contract's state will be.

@chriseth
Copy link
Contributor

@chriseth chriseth commented Nov 29, 2015

Just a random thought about the cost of archiving (sorry if it has been brought up already):

Since all miners / validators have to execute all transactions, the reward for executing an opcode is currently set to be X * n, where X is the cost to execute it by a single miner and n is the number of miners. Assuming equal mining power and reward being equal to the actual costs, this evens out the costs on average for all miners.

So our general model is "mining reward equals overall network costs".

Of course storage has costs over time, because it cannot be reused, and thus our fixed price per SSTORE model is flawed - the reason for this EIP.

Reviving contracts from an archive is a different story, though: For keeping a contract archived, it is enough if only a small number of archive nodes keep the contract. Also the influence of time is not as important anymore - it costs money to keep disks working, but it is much cheaper if you do not expect the data to be accessed often. So we have the interesting fact that the gas costs of an opcode should scale with the size of the network, but the costs for keeping data archived scales with time and not with the number of nodes, and I would argue that the time-scaling factor is even quite low compared to the fixed costs.

Now let us look at the costs for retrieving the data from an archive: Of course, not all full nodes will need to retrieve the data from their own archive, rather they will initially all use the tiny number of archive nodes but the data will replicate quite fast through the network - in the same way a newly created block will spread. Spreading data in the network scales extremely well with the number of nodes (see bittorrent), so again: The costs for reviving a contract can be considered constant.

Of course I am assuming that the archive data is correctly merkle-hashed and cannot be spoofed, but that is easy to achieve in a logarithmically scaling way.

So unless I am mistaken, I would say that almost-permanent storage of data at (comparatively) almost no cost is quite possible.

@pipermerriam
Copy link
Member

@pipermerriam pipermerriam commented Nov 29, 2015

Can anyone speak to the feasibility of Swarm/IPFS based storage. It seems to me that if it was easy for miners to configure locally stored blockchain data vs IPFS based blockchain data then you end up with a model under which miners are incentivized to cache chain data that contains frequently accessed information and assuming there is a Swarm style incentive system for ensuring that the less used blocks are guaranteed available, then miners will always have a way to get at that data.

Things that may complicate this that I'm unsure how to measure.

  • How does network latency effect this? Does the network cost and/or Swarm cost of retrieving chain data from IPFS make this not economically feasible?
  • Does this require Swarm? Can it be implemented now with direct IPFS support for storage of chain data knowing that the size of the blockchain is likely be manageable for the near-term future and that we can iterate on the specifics of ensuring that the entire chain is available?

Thoughts?

Edit:

A lot of my perspective on this comes from believing that Moore's Law + clever decentralization/distribution can make storage cheap enough that permanent storage won't be a problem.

@Smithgift
Copy link

@Smithgift Smithgift commented Nov 29, 2015

@niran: Depending on how fast rent accumulates, a deadbeat contract might be archived before anyone gets to it. If there's an incentive to archive, then miners would hold off on dumping a contract until they can archive it and get rewarded.

@chriseth: If the transaction comes with the archive in tow, the cost to the miners is no more than it takes to receive it and verify it. If the archive is in kilobytes, or even megabytes, it may be too small to meter. If it's some ginormous archive of a spam contract in measured in gigabytes, the miner might ignore it on principle. I mean, if you had a contract that huge but it was actually used, it'd probably be something like Augur and never fall off the blockchain in the first place.

Is all a contract's state in the same node in the state trie? If so, all an archive is an ordinary SPV proof+the state.

@pipermerriam: I was always picturing the archive system using, on some level, swarm or IPFS. For swarm, an archive will hopefully fit into a single chunk, and it might be quite the juicy chunk if the contract was once popular. Someone who can't pay the rent but would rather the contract remain in existence could hire a swarm node to store it. Or they could just host it themselves.

Discussion point: there's probably a power law distribution of contract popularity. There's going to be those greeter contracts which will be forgotten after a day, and then mildly popular contracts that might one day be revived, and then that old critical contract that's used once in a blue moon and it falls off and the world breaks. The most popular contracts will probably never disappear, short some failure in incentives.

@Smithgift
Copy link

@Smithgift Smithgift commented Nov 29, 2015

@vbuterin sorta mentioned this here, but I think it's worth repeating. In a post-EIP 101 world, if the contract's code is known, and a contract's address is the hash of its code, then a kind of archiving/revival becomes trivial: just recreate the contract, and it's back at the same address. Any storage would be lost, but perhaps that would be the rent/own distinction: a contract owns its address/code but its storage is temporary. That wouldn't be an issue for that tiny library contract that is forgotten and then breaks everything when it runs out of rent money.

This wouldn't work so well for ETH 1.0 contracts, which have different addresses than their code, but those could all be grandfathered in.

@pipermerriam
Copy link
Member

@pipermerriam pipermerriam commented Nov 29, 2015

just recreate the contract, and it's back at the same address. Any storage would be lost

I expect the majority of contracts would be entirely broken without their associated storage.

@Smithgift
Copy link

@Smithgift Smithgift commented Nov 29, 2015

@pipermerriam: It's a kind of archive, not a true one in the sense we've been discussing previously.

@subtly
Copy link
Member

@subtly subtly commented Nov 30, 2015

@niran @chriseth Worth noting that "forgetting" data requires consensus and thus an expensive operation, as-is marking or moving data to archive and rent agreements. Pruning data which hasn't been paid for simply changes the problem from "subsidizing cost of permanent storage" to "subsidizing cost of permanent erasure". How is a delete operation different than a permanent store operation?

Scalability is going to open a can of worms with respect to storage and it seems the objective of this EIP is to reduce costs for storage when permanent storage isn't necessary and to reduce disk space requirements of a full node. In this regard, I think rent as an option and a contract being able to delete storage is a big step forward. @vbuterin And with rent as an option, perhaps there isn't a need to refund gas when deleting from permanent storage, as the "loss" is an incentive to rent storage.

@joeykrug
Copy link

@joeykrug joeykrug commented May 8, 2016

How does this work for things like subcurrencies? Say I have a subcurrency and people make a bunch of accounts on it they don't use --- then active users are forced to pay for all their storage even if they aren't using it? Why not delete data after x time if it's not paid for as opposed to forcing people to pay for others' data?

Another issue is: someone can just spend some money to saddle a contract with data and then walk away and never pay for it, bankrupting the contract at worst, and at best, screwing over everyone who is honest users of the contract. I don't like the idea of trusting people to not be Byzantine jerks.

Edit: seems this is a non issue with proposal #87

@jpritikin
Copy link

@jpritikin jpritikin commented Aug 1, 2016

I am concerned about the need to explicitly refuel contracts. If SSTORE is the only way to refuel then we'll gradually need to re-SSTORE all the data that we want to keep on a regular basis. This complicates life for contract creators: In addition to the difficult work of programming the functionality of contracts, contract creators will need to devise a plan to get their contracts refueled on a regular basis. What about allowing SLOAD to refuel the storage rent? That way, storage that is used (read or written) automatically gets refueled (or re-rented) without any additional explicit refueling process. The goal of this suggestion is to make storage rent as automatic as possible to avoid the need to explicitly micro-manage storage rental fees.

Some other mechanism would need to be devised for long-term "alarm clock" style contracts.

@tawaren
Copy link

@tawaren tawaren commented Mar 15, 2017

In my opinion any mechanism that can lead to an unspecified deletion of a contract or unpreventable state change in general is a bad idea (sadly this already can happen with the selfdestruct and coinbase, at least this always leads to an increase in the balance and tracking the balance separately in storage solves this, even if suboptimally).

My argument against such mechanism is that I think "The code is Law" is one of Ethereums big selling points (at least from my view). For "The code is Law" a contracts behavior should only be defined by its code and the messages send to it in the past. Further every contract interacting with another contract must be sure that the same is true for the other contract and a sudden disappearing that is not specified in its code is from my viewpoint unacceptable and a fundamental philosophical change. With a blockchain rent in place a contract may vanish from one point in time to another without receiving a message triggering it (even if a message must be send for deletion, It would lead to behavior not specified in the executed code (namely an contract deletion))

One thing that is so promissing by "The code is Law" philosophy is that it allows for autonomous contracts, that after they are developed does not need any maintainer or developer that looks after it and just work as defined by their code forever (except a backward incompatible change happens)

The option of an opt-in solution where the option for an ethernal storage still exist is better but still introduces a complete new dimension to Ethereum. Currently each opcode does only have an immediate effect but with a rent system temporal effects are introduced. An opcode can suddenly mean something like: the storage slot X is changed to Y and somewhere in the future this contract may be deleted. As a programmer and programming language designer the existence of such an opcode horrifies me.

I personally think, that something like this would be a so fundamental change that Ethereum would leave a niche behind fillable by another blockchain or by an old version of Ethereum where we still had "The code is Law".

@jamesray1
Copy link
Contributor

@jamesray1 jamesray1 commented Nov 10, 2017

Referenced here: https://ethresear.ch/t/the-stateless-client-concept/172.

With a stateless client concept an advantage is: "All of the thorny questions about state storage economics that lead to the need for designs like rent (eg. #35 http://github.com/ethereum/EIPs/issues/872 http://github.com/ethereum/EIPs/issues/882) and even the current complex SSTORE cost/refund scheme disappear, and blockchain economics can focus purely on pricing bandwidth and computation, a much simpler problem)".

@wjmelements
Copy link

@wjmelements wjmelements commented Nov 18, 2017

Rent-free storage is an attractive feature in a distributed VM/application platform, but it should be more expensive, while freeing state should be more lucrative.

  • An evicted contract's state can still be derived from the blockchain, which still has to be stored by every node. Denying service is inferior to providing it.
  • Contract suicide should refund the recipient for the freed state, in addition to transferring contract funds.
  • The cost of permanent storage decreases with time. Future storage cost is unpredictable, but predictably diminishing with time.
  • Sharding will decrease the storage cost per node. Storage size should be considered when sharding.
  • IPFS/Swarm should offset the growth of permanent storage with the proper incentives.
  • Heavily increasing the SSTORE gas price and refund would better-incentivize proper contract storage without threatening the utility of the EVM.
@hashguide
Copy link

@hashguide hashguide commented Feb 1, 2018

What about having a separate type of node that has a higher incentive to store and execute contracts than the miners to verify blocks. There can be set gas limits for each type of node usage in this case so anyone deploying or writing/reading smart contracts have to pay slightly higher fees than one just sending plain transaction to a friend or exchange. I realize this has nothing to do with this proposal, but it's another option to solve the issue at hand. Just not sure if it is possible to implement on an existing blockchain with so many working contracts the way it is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.