Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PM Ticket Expiration #267

Closed
eladmallel opened this issue Dec 13, 2018 · 7 comments
Closed

PM Ticket Expiration #267

eladmallel opened this issue Dec 13, 2018 · 7 comments
Labels
mvs Minimum Viable Streamflow pm Probabilistic Micropayments question streamflow Streamflow

Comments

@eladmallel
Copy link
Contributor

Why do we want ticket expiration?

  1. Minimize double spending by B
    1. By design B cannot know for any given ticket if it's a winning ticket until O reveals its recipientRand value, which happens upon redemption on-chain. With no bound on when O redeems tickets, B can easily hand out winning tickets of a value greater than B's deposit and accidentally double spend.
    2. B can decide to maliciously double spend on many Os, and until enough Os redeem enough winning tickets to trigger double spend detection, B's additional utility keep growing. Ticket expiration is a way to curb B's additional utility for such behavior.
  2. Delegators being squeezed by O and B colluding: without expirations O can hold on to many tickets to accrue a significant chunk of revenue, then update their fee sharing params on-chain, and only once those changes take effect redeem all the tickets. Expiration can not only curb this behavior, but further ensure that the right delegators are rewarded for the work (if we agree that the right delegators to reward are those that delegated onto O at the round when the ticket was created).

(for more background on some of these issues see livepeer/prob-pay#5 and livepeer/prob-pay#6).

Assuming we've established that we want to have an expiration mechanism, moving on to its design.

Ticket expiration design

  • Expiration must rely on a verifiable data point that is infeasible to know ahead of time, otherwise colluding Bs and Os can easily issue future-dated tickets.
  • The best source of such data in our case seems to be Ethereum block hashes.
  • This problem becomes more challenging since currently in Ethereum we can only query for the block hash of the last 256 blocks.

Initial solution approach:

  • The foundation for our workaround is to rely on smart contract storage for block hashes lookup.
  • To have that we need some mechanism for storing block hashes regularly.
  • The most cost-effective solution we have now is to leverage the existing round initialization mechanism.
  • At every round initialization we would store a mapping from round number to the previous blockhash.
  • Bs can query this mapping to get the latest values, and include them in each PM ticket.
  • Both O and TicketBroker can reference that same map for their ticket validation process:
    • Do the round number and round hash in the ticket match what's stored on-chain?
    • Are we still within the ticket expiration period?

Round-based expirations challenges

Rounds are currently about 24-hours long. This granularity presents us with a few challenges:

  • If we set the expiration period to be one round, we open the protocol to edge cases of tickets expiring very quickly if B gets the latest block hash just before a new round is initialized (worst case could be one block).
  • From this we deduce that the expiration period should be at least 2 rounds.
  • However, a 2-round expiration means that the worst case for exposing double spending becomes 48 hours, and that’s suboptimal as well.
  • A potential mitigation could be shortening rounds from 24 hours to 12 hours, but that would come at the expense of not giving delegators reasonable time to learn of any delegation params changes (fee share, etc.) - which is an intention currently baked into the protocol.

As mentioned, the ideal solution would be to rely on block hashes, and there are EIPs that present such a possibility, but we have no certainty on if and when they will be included in a mainnet Ethereum version.

What is the best compromise for now?

@eladmallel eladmallel added question mvs Minimum Viable Streamflow pm Probabilistic Micropayments streamflow Streamflow labels Dec 13, 2018
@dob
Copy link
Member

dob commented Dec 14, 2018

I'm not sure that the max 48 hour expiration window is such a negative. The default behavior for O is still to cash in a winner as quickly as possible, and therefore if B has dramatically overspent, the likelihood is that they would be exposed very quickly. If they've only mildly overspent, well, their penalty escrow should cover that.

The 2 round option feels ok to me. The only attack/downside that I immediately see would be the "oscillating Orchestrator" who shows up, attracts stake with attractive params, changes those params causing D's to leave, then cashes payments in the next round. If D's prioritize any sense of history or longevity in nodes, or are patient enough to ride out two rounds, they get their fees anyway. The attack seems like more work/cost than it's worth.

@j0sh
Copy link

j0sh commented Dec 14, 2018

Just another data point in favor of expiration: a variation of the first point can be used by O as a form of squeezing to grief Bs. In particular, O could accumulate just enough tickets to ensure that B would be slashed. Having an expiration minimizes the "float" that B has at risk.

At some point, B should probably maintain an internal approximation of its estimated outstanding float if O does not cooperate in notifying B of a winning ticket.


The only drawback to using a round mapping is that it depends on global round initialization. I'm keen on minimizing any requirement for such (tragedy-of-the-commons for mainnet, and has been unreliable on testnets).

One thought is to use earlier winning tickets to ratchet the expiration, as long as the blockhash (or perhaps just a block number) is accessible somewhere in that data. This has a bit of a bootstrapping problem, but I think we should be able to use any winning ticket for this, not just O's own winning tickets. Orchestrators already should be listening for winning ticket events, to monitor the deposit balance of active Bs.

Could we do something similar with reward calls? Especially if we can subsume initializeRound within reward -- eg, the first O to call reward for the round updates the round->blockhash mapping.

@eladmallel
Copy link
Contributor Author

One thought is to use earlier winning tickets to ratchet the expiration

To clarify, do you mean something like upon receiving a winning ticket in the TicketBroker contract, we would simply add to our mapping of all historic winning tickets the block info, and that would be our "clock"?

If so, I think that's very interesting. One advantage we have from using rounds relates to B's need to get the "last recorded blockhash" to provide the best expiration data for tickets.

Using rounds, we can construct the contract mapping for easier lookup, i.e. roundNumber -> blockhash, because round numbers increment more slowly. If we used winning tickets as the triggering event, what would the mapping look like? Ideas:

  • It could be a mapping from ticket hash to blockhash, and B can rely on querying redemption events from the contract to get the recent winning ticket hash.
  • We could have an additional variable of the latest hash, which we would update every time we update the clock mapping. Downside is increased gas costs.

wdyt?

@eladmallel
Copy link
Contributor Author

I'm not sure that the max 48 hour expiration window is such a negative. The default behavior for O is still to cash in a winner as quickly as possible, and therefore if B has dramatically overspent, the likelihood is that they would be exposed very quickly. If they've only mildly overspent, well, their penalty escrow should cover that.

A few thoughts:

  • I agree that O's default behavior mitigates this problem, but I think it's ideal to design the protocol such that it works really well without assuming benevolent client implementations, and if we don't assume that to me the 48-hour window remains problematic for double spend detection.
  • Re:penalty escrow, just point out that in our current implementation we slash the entire penalty escrow no matter how much B overspends. If you think we should change this behavior we should definitely open a discussion on that as well!

@j0sh
Copy link

j0sh commented Dec 18, 2018

round numbers increment more slowly

The (potentially) increased granularity from using winning tickets seemed to be a nice aspect, but I suppose it can go either way.

It could be a mapping from ticket hash to blockhash, and B can rely on querying redemption events from the contract to get the recent winning ticket hash.

Ah, I was thinking O sets the expiry as part of the PM parameters to alleviate B of the need for blockchain monitoring. This also dovetails with expiring the data that O is required to cache per recipientRand such as senderNonce, but of course this is technically a different type of expiration that can be enforced entirely on the goclient.

Having B set the expiry has some nice properties: the expiration could ratchet as the stream goes on. This is one benefit for the increased granularity offered by using winning tickets as they come in from the network.

In any case, if we can look up the block hash and/or block number from any on-chain metadata that's available when doing a ticket lookup, without incurring additional storage writes, then we could use that. Otherwise, we'd have to write to a storage slot -- which I agree is expensive.

We could also write to a storage slot during the reward call. We can optimize this so it's only done once per round -- the first reward caller for the round would spend just a little bit more gas to do so. Of course, it's a bit of a slippery slope if we ever start adding extra steps here.

Again, my motive for suggesting all this is to mitigate the requirement for a global initializeRound. Anything we can do to avoid that is a win.

we slash the entire penalty escrow no matter how much B overspends

Burning the escrow is a good approach. Otherwise we incentivize squeezing attacks on B, where O hoards tickets in hopes of triggering that big slash and reaping a disproportionate reward.

@yondonfu
Copy link
Member

yondonfu commented Dec 21, 2018

Here is an update on the current proposed roadmap for this feature based on this thread and an offline discussion between @eladmallel and I:

We prefer not to use the approach of storing block hashes on winning ticket redemption because:

  • If we store the latest block number for which a block hash has been stored, then ticket redemption requires 2 additional storage writes: 1 for writing the block hash and 1 for writing the current block number. As a result, the ticket redemption gas cost increases by 40000 gas which is not ideal given that we want to try to keep ticket redemption gas cost as low as possible.
  • If we don't store the latest block number for which a block hash has been stored, then the client will have to query for the latest ticket redemption event which will provide the latest block number. This still requires an additional storage write which is 20000 gas.
  • We lose the ability to direct ticket pay outs into the fee pool for the round during which a ticket is created and thus delegators that delegated to O during that round will not be compensated for helping O get selected for work during that round.
  • Relying on ticket redemptions for the ticks in a universal clock that both B and O use means that we have irregular time intervals between ticks. Furthermore, the ticks slow down during times of low network demand and speed up during times of high network demand. In the worst case, the ticks stop for a period of time.

We prefer not to directly rely on block numbers i.e. attach a block hash and a block number to a ticket and the Broker checks if the block hash is valid for the block number and use a maximum ticket validity period of 256 blocks which is around 1 hour given ~15 second block times (since contracts cannot look up block hashes past currentBlock - 256) because:

  • If block times ever decrease in the future, the maximum ticket validity period decreases as well. In this scenario, there is not a lot of flexibility with adjusting validity periods to allow Os to opportunistically redeem tickets during times of low gas prices
  • The ~1 hour validity period puts a lot more pressure on the client to force ticket redemption transactions to confirm on-chain quickly. During times of increasing gas price and network congestion, the client needs to make sure to constantly bump gas prices if a transaction is left pending
  • While there are EIPs that propose increasing the range of blocks for which a contract can query block hashes for, at the very earliest they could be included in a post-Constantinople hard fork if at all. Furthermore, they do not seem to allow for querying for block hashes from the full range of past blocks, but only a certain past blocks. See: Simpler alternative to BLOCKHASH extension (#210) ethereum/EIPs#1218. It is probably best not to create dependencies on upgrades with unclear timelines.

We propose the use of round initialization to store block hashes that can be used to prove the existence of a block at the time of ticket creation. The workflow for B would be as follows:

  1. Query the RoundsManager for currentBlockHash for the current initialized round. Note: If the current round is not initialized, B will query the RoundsManager for the block hash of the last initialized round - this behavior is to ensure that B is never blocked from streaming if we are in a transitionary period during which the current round is not initialized yet.
  2. B creates a ticket with currentBlockHash and currentRound.
  3. O validates the ticket by checking that currentBlockHash is indeed the block hash for currentRound by querying the RoundsManager.
  4. If the ticket wins, O redeems the ticket on-chain. The Broker will check that currentBlockHash is indeed the block hash for currentRound by querying the RoundsManager. The ticket pay out can then be sent to O's fee pool for currentRound (which results in fees directed to delegators that delegated to O at the time of ticket creation).

The upsides of this approach are:

  • Greater flexibility around ticket validity period relative to the method of relying on block numbers directly
  • We are able to direct fees to delegators that delegated to a O during ticket creation

The downsides of this approach are:

  • The validity period needs to be at least 2 rounds (i.e. 2 days) in order to prevent cases where a ticket is created toward the end of a round and it expires very quickly
  • A 2 round minimum for the validity period might not ideal for Bs because they could see on-chain deposit updates from winning tickets less often and for Os because there is a larger time period for double spends to occur
  • We add additional logic to global round initialization which is not guaranteed to be executed at the very beginning of a round. One potential additional incentive for more timely round initialization is that if O is being paid by B and the round is not initialized, B will use the previous round thereby reducing the validity period of the ticket from 2 to 1 rounds. So it could be in O's best interest to initialize the round if it is in this situation.

Our conclusion is that while using round initialization to store block hashes might not be the most ideal solution, it still beats out the other available solutions at the moment.

If anyone has any additional thoughts/concerns we please continue the discussion here!

If there are no additional thoughts/concerns, we can proceed with the following plan:

  • For MVS, remove expiration checks on tickets for the sake of reducing scope.
  • For SS, update RoundsManager to store block hashes upon round initialization and update LivepeerETHTicketBroker to validate the creation round numbers in tickets using the attached block hash. Also, update the client to include these fields in the tickets and add client side validation step for O.

@j0sh
Copy link

j0sh commented Feb 13, 2019

Thanks for taking the time to dissect alternatives so thoroughly. Is round initialization anticipated to be used for anything other than storing the block hash?

If the block hash is the only reason for round initialization, then it might be nice to continue exploring other avenues to avoid round initialization. Performing global round initialization to accommodate an incidental effect of the payment mechanism feels heavy.

One possible approach is to set the block hash during the first reward call.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mvs Minimum Viable Streamflow pm Probabilistic Micropayments question streamflow Streamflow
Projects
None yet
Development

No branches or pull requests

4 participants