Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use go-statemachine + FSMs in retrieval market #124

Merged
merged 4 commits into from Feb 26, 2020

Conversation

hannahhoward
Copy link
Collaborator

Goals

  • persist retrieval deals on both client and provider
  • remove common logic around transitioning states to make code more readable
  • make it simply to decouple steps in retrieval market to enable easier resuming of state (not in this PR but coming soon)

Implementation

  • Use go-statemachine and the FSM module defined in this PR Easy FSM Add-on: Makes making Finite State Machines with go-statemachine easier go-statemachine#4
  • Define the state machine's event logic in a single place, separate from the state handlers -- see provider_fsm.go & client_fsm.go
  • Simplify state handlers that no longer have to modify state but just dispatch events
  • Update provider & client to construct statemachines from a data store so they are saved on disk
  • Dispatch several new more informative events for different things that can happen in a retrieval deal
  • Add one new extra step to the retrieval deal process -- make sure that once the last payment is sent, the provider receives and processes it (this felt missing from the deal flow)
  • Support seperating payment responses from block responses (not implemented here, but feels important on the way to retrieval deal flow v1 w/ graphsync)

For discussion

  • still need tests for Finalize on client and provider
  • think ProviderDealIdentifier should be renamed ProviderDealID probably
  • need to complete persisting nextDealID or this becomes mostly non-functional :) but didn't want to add to already large PR

convert retrieval market to using go-statemachine, specifically the FSM module
Copy link
Contributor

@shannonwells shannonwells left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks okay, I don't see anything blocking.

}

// Process Blocks
totalProcessed := uint64(0)
completed := false
completed := deal.Status == rm.DealStatusBlocksComplete
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if completed is true, do we still need to consume Blocks? Or will response.Blocks be empty and the for loop never executes? My memory is fuzzy.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it will be empty but it's a good point -- we really shouldn't be consuming blocks period once all blocks are consumed -- arguably if anything is in response.Blocks it should error at that point.

if err != nil {
return errorFunc(xerrors.Errorf("consuming block: %w", err))
return ctx.Event(rm.ClientEventConsumeBlockFailed, err)
}
totalProcessed += processed
if done {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couldn't you just say processed, completed, err := environment.ConsumeBlock... and then if completed { break } ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh yea, probably :) had not considered

@@ -28,133 +26,100 @@ func errorFunc(err error) func(*rm.ClientDealState) {

// ClientHandlerFunc is a function that handles a client deal being in a specific state
// It processes the state and returns a modification function for a deal
type ClientHandlerFunc func(ctx context.Context, environment ClientDealEnvironment, deal rm.ClientDealState) func(*rm.ClientDealState)
type ClientHandlerFunc func(ctx fsm.Context, environment ClientDealEnvironment, deal rm.ClientDealState) error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this is no longer returning a func, "Func" should be removed from the name. Also the comment should be updated.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it

require.Empty(t, dealState.Message)
require.Equal(t, dealState.PaymentRequested, abi.NewTokenAmount(0))
require.Equal(t, dealState.FundsSpent, big.Add(defaultFundsSpent, defaultPaymentRequested))
require.Equal(t, dealState.BytesPaidFor, defaultTotalReceived)
require.Equal(t, dealState.CurrentInterval, defaultCurrentInterval+defaultIntervalIncrease)
require.Equal(t, dealState.Status, retrievalmarket.DealStatusCompleted)
require.Equal(t, dealState.Status, retrievalmarket.DealStatusFinalizing)
})

t.Run("not enough funds left", func(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this would really benefit from either table tests or helpers to reduce copy-pasta, or both.

})
}

func TestProcessNextResponse(t *testing.T) {
ctx := context.Background()
node := testnodes.NewTestRetrievalClientNode(testnodes.TestRetrievalClientNodeParams{})

eventMachine, err := fsm.NewEventMachine(retrievalmarket.ClientDealState{}, "Status", clientstates.ClientEvents)
require.NoError(t, err)
environment := func(netParams testnet.TestDealStreamParams,
responses []consumeBlockResponse) clientstates.ClientDealEnvironment {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above re: table tests

@hannahhoward hannahhoward merged commit 5ee324c into master Feb 26, 2020
@dirkmc dirkmc mentioned this pull request Dec 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants