Skip to content

Prefix frontend dispatch#501

Merged
eddyashton merged 59 commits intomicrosoft:masterfrom
eddyashton:prefix_frontend_dispatch2
Nov 4, 2019
Merged

Prefix frontend dispatch#501
eddyashton merged 59 commits intomicrosoft:masterfrom
eddyashton:prefix_frontend_dispatch2

Conversation

@eddyashton
Copy link
Member

This is an extension of #422, created as a separate branch and PR as it includes additional refactoring.

It aims to separate the parsing of transport-level framing information from the semantic transaction processing. In particular extracting an actor from the transaction message data, rather than from the certificate which established the underlying channel.

In our current framed JSON-RPC protocol the actor is prefixed on the method, so "method": "users/getCommit" will be dispatched by the endpoint to the users frontend, and by that frontend to its getCommit method handler. This requires a full parse of the JSON-RPC object before we can do the valid_caller check - in a flatbuffer or HTTP endpoint it should be possible to do a partial parse (of only the header) to extract the actor and similar framing information, and pass the remainder unprocessed data verbatim to the frontend.

The key code change here is that RpcFrontend::process now takes a single context object, populated with parsed details. This is largely to make API changes easier, so we can tweak this struct's contents (to narrow down what the frontend requires) without altering hundreds of calling points. This context is split into a common SessionContext which may persist over many transactions, and the remaining fields populated per-RPC. The unit tests call process rather than process_json, and have a gestures towards standardisation.

For future PRs:

  • Cert changes
  • Consolidation of process_forwarded and process_pbft
  • Reduction of forwarded special-casing (how much framing should be re-parsed/re-validated on executor?)

@ghost
Copy link

ghost commented Nov 4, 2019

images

@ghost
Copy link

ghost commented Nov 4, 2019

images

@ghost
Copy link

ghost commented Nov 4, 2019

images

@ghost
Copy link

ghost commented Nov 4, 2019

images

@ghost
Copy link

ghost commented Nov 4, 2019

images

@eddyashton
Copy link
Member Author

Sorry I have to be that guy: we need to update the docs accordingly.
At least: https://microsoft.github.io/CCF/developers/example.html and https://microsoft.github.io/CCF/users/issue_commands.html

And also update the RPC API spec?

I've updated a few places in the docs that show direct JSON queries, and added a brief description in the issue_commands doc.

@ghost
Copy link

ghost commented Nov 4, 2019

images

@eddyashton eddyashton merged commit 7ee977b into microsoft:master Nov 4, 2019
@eddyashton eddyashton deleted the prefix_frontend_dispatch2 branch November 4, 2019 15:10
eddyashton added a commit to eddyashton/CCF that referenced this pull request Mar 24, 2020
* wip

* WIP

* wip

* More prefixing

* More prefixing

* Prefixes

* Formatting

* Format

* p

* Revert cert changes

* Fix error cases

* wip

* Pass new args to frontend process everywhere

* Move detect_pack up to jsonrpc.h

* Automatically prefix with sni

* Extract common unpack_rpc

* Don't give unpacked to pbft (yet)

* Minor test fixes

* Forward readonly_hint all the way

* WIP

* PoC: RPCContext holds everything, pre-parsed

* Push even more details into RPCContext

* Distinguish SessionContext

* Start extending PoC into frontend_test

* Combine and standardise tests

* More frontend_test fixes

* frontend_tests pass

* Begin update of membervoting_test, remove process_json

* Next test case passes

* Churn

* Progress

* Most of membervoting_test

* No more process_json

* Fix luageneric_test

* All compile

* Re-use parsing logic in RPCEndpoint

* Doh

* Improve logging, serialise method for forwarded tx

* Remove some unnecessary changes

* Store the correct thing in voting_history

* Remove more dead code

* Update PBFT compilation paths

* Improve logging

* PR feedback

* Upgrade to LOG_DEBUG

* Pass actor with create request

* Don't reparse

* Reparse where necessary

* non-None default prefices

* Juggle lines

* Add prefix in docs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants