feat: allow validator to request blinded versions for locally produced and selected blocks #6227
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
twoeths
reviewed
Dec 22, 2023
|
need to make sure we test possible values of |
g11tech
force-pushed
the
blobs-inclproof
branch
2 times, most recently
from
1d0e4a7
to
3ceb6bf
Compare
g11tech
force-pushed
the
blinded-local
branch
from
9dd2995
to
225de55
Compare
g11tech
commented
Dec 23, 2023
| logger.verbose("Assembling block with produceBlockV3 ", { | ||
| fork, | ||
| builderSelection, | ||
| const produceEngineOrBuilderBlock: ServerApi<routes.validator.Api>["produceBlockV3"] = |
There was a problem hiding this comment.
changes in produceEngineOrBuilderBlock (old produceBlockV3) are because of formatting changes introduced because of name length changes may be, but this function doesn't essentially change and produceBlockV3 is redefined which uses now this and handles blinding of local payload if requested
g11tech
commented
Dec 23, 2023
| @@ -643,6 +654,55 @@ export function getValidatorApi({ | |||
| } | |||
| }; | |||
|
|
|||
| const produceBlockV3: ServerApi<routes.validator.Api>["produceBlockV3"] = async function produceBlockV3( | |||
There was a problem hiding this comment.
this is the real code segment introduced here, old produceBlockV3 is renamed to produceEngineOrBuilderBlock and this one now further undertakes the task of returning blinded response for local is requested by the validator via query param
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## unstable #6227 +/- ##
============================================
- Coverage 80.95% 80.94% -0.02%
============================================
Files 185 185
Lines 17935 17954 +19
Branches 1078 1080 +2
============================================
+ Hits 14519 14532 +13
- Misses 3389 3395 +6
Partials 27 27 |
g11tech
force-pushed
the
blinded-local
branch
from
dc3d4cf
to
af12468
Compare
g11tech
force-pushed
the
blinded-local
branch
from
af12468
to
d8b2318
Compare
Performance Report✔️ no performance regression detected Full benchmark results
|
7 tasks
…d and selected blocks lint fix the types and tests
g11tech
force-pushed
the
blinded-local
branch
from
d8b2318
to
1f7aad0
Compare
This was referenced Dec 26, 2023
twoeths
reviewed
Dec 28, 2023
| // of local block and subsequent publishing via publishBlindedBlock | ||
| clientOptions: { | ||
| useProduceBlockV3: false, | ||
| "builder.selection": "maxprofit", |
There was a problem hiding this comment.
need to set blindedLocal: true here?
There was a problem hiding this comment.
no this doesn't use produceBlockV3 flow but calls older method of produceBlindedBlockV2 which also blinds local and return it in the blinded fashion just like produceBlockV3 does but with less metadata in the response
twoeths
approved these changes
Dec 30, 2023
|
🎉 This PR is included in v1.14.0 🎉 |
ensi321
pushed a commit
to ensi321/lodestar
that referenced
this pull request
Jan 22, 2024
…d and selected blocks (ChainSafe#6227) * feat: allow validator to request blinded versions for locally produced and selected blocks lint fix the types and tests * add blinded local combinations to sim * debug and fix issues
wemeetagain
added a commit
that referenced
this pull request
Jun 10, 2024
* Add config route definitions * Add debug route definitions * Add events route description * Add lightclient route definitions * Flatten function params * Type safety for optional params in write / parse req * Method args are optional if only optional props * Fix genesisValidatorsRoot type issue * Revert requiring all params in write / parse req * Update http client errors * Add lodestar route definitions * Add node route definitions * Add proof route definitions * Add builder route definitions * Add validator route definitions * Application method response can be void * Generic options can be passed to application methods * Default endpoint request type has body property * Improve types of transform methods * Export server types from index (to be removed) * Update config api impl * Update lightclient api impl * Update events api impl * Update lodestar api impl * Update proof api impl * Update node api impl * Update debug api impl * Update state api impl * Update pool api impl * Update blocks api impl * Partially update validator api impl * Update beacon routes export * Align submitPoolBlsToExecutionChange method args * Filters are always a object * Update errors messages * Add beacon client methods * Add missing routeId label to stream time metric * Fix json casing in codecs * Apply remaining changes from #6227 * Produce block apis only have version meta * Add block values meta to all produce block apis * Apply changes from #6337 * Handle unsafe version in WithMeta and WithVersion * Restore server api error * Update fastify route types * Update server routes / handlers * Remove unnecessary type cast * Restore per route clients * Fix beacon route types * Remove option to patch fetch from http client * Update eventstream client, remove fetch override Fallback does not work like this, see #6180 for proper solution * Use StringType for validator status until #6059 * Remove empty fetch.ts file * Add a few todos * Update builder client and server routes * Update beacon exports * Update api index exports * Update builder index imports * Improve type safety of schema definitions * Add headers to fastify schema * Fix schema definition type * Add missing schemas to route definitions * Fix response codec type * Remove response codec type casts * Fix casing in json only codec * Reuse EmptyResponseCodec * Update base rest api server * Update keymanager routes, client and server * Reuse data types in keymanager impl * Do not await setting headers, not a promise * Improve type safety of empty codecs * Only require to implement supported req methods * Handle requests that only support one format * Handle responses that only support one format * Add json / ssz only req codecs * Update only support errors * Fix assertion * Set correct accept header if only supports one format * Fix eslint / prettier issues * More formatting fixes * Fix fallback request retries in case of http errors * Formatting of res.error * Add add retry functionality to http client (from #6387) * Update rewards routes and server (#6178 and #6260) * Allow to omit body in ssz req if not defined * Always set metadata headers in response * Cache wire format in api response * Only call raw body for json meta * Update api package tests (wip) * Test json and ssz format in generic server tests * Add a bunch of todos * Fix a few broken route definitions * Fix partial config test * Another todo note * Stringify body of json requests * Override default response json method * Validate external consensus version headers in request * Add error handling todo * Skip body schema validation for ssz request bodies * Clean up generic server tests * Pass node, proof, events generic tests * Use enum for media types * Fix a bunch of route definitions * Add justified to blockid type * Properly handle booleans, remove block values codec * Create Uint8Array test data without allocating Buffer * Let fastify handle Buffer conversion * Convert Buffer to Uint8Array in content type parser * Fix build issues * Fix fork type in builder routes * Add some notes * Properly parse request headers * Fix incorrect type assumptions in transform * Generic server tests are passing (except lightclient) * Correctly handle APIs with empty responses * Update getHeader return type to reflect no bid responses * Do not append '?' to URL if query string is empty * Let server handler set status code for parsing errors * Remove unused import * Rename function, request specific * Completely drop ssz support from getSpec * Spec tests are passing against latest releases * Drop unused fastify route config * Drop ssz request from builder routes, not yet supported * Remove import * Apply change from #6695 * Update execution optimistic meta * Apply changes from #6645 * Add workaround to fix epoch committees type issue * Add todo to fix inefficient state conversion * Convert committee to normal array * Apply changes from #6655 * Align args of validators endpoints * Convert indices to str in rewards apis * Update api spec version of README badges * Revert table formatting changes * Make this accessible for class-basd API implementations * Throw err if metadata is accessed for failed response * Add assertOk to api response * Tweak api error message * Update operationIds match spec value * Add missing version to blob sidecars metadata * Test headers and ssz bodies against spec * Minor reordering of code in spec parsing * submitBlindedBlock throws err if fork is not execution * responseOk might be undefined * Remove statusOk from route definition * Remove stale comment * Less build errors in beacon-node * getBlobSidecars return version from server impl * Update validator produce block impl * More expressive pool method args * Application methods might be undefined in mock implementations * Adress open TODOs in server handler * Api response methods are synchronous now * Fix all remaining build issues * Use more performant from/toHex in server api impls * Clean up some TODOs * Fix ApiError type * Errors related to parsing return a 400 status code * Simplify method binding * Forward api context to application methods * There is no easy way to make generic opts typesafe * Better separation of server / client code * Fix comment about missing builder bid * Remove todo, not worth the change / extra indentation * Rename route definitions functions * Return 400 if data passed to keymanager is invalid * Properly handle response metadata headers * Fix lint issues * Add header jsdoc * Move metadata related code into separate file * Remove ssz from POST requests without body * Only set content-type header if body exists * Fix headers extra * POST requests without body are handled similar to GET requests * Fix http client options tests * Improve validation and type safety of JSON metadata * Add type guard for request without body * Differentiate based on body instead of GET vs POST * More renaming * Simplify RequestCode type * Review routes, improve validation * Remaining local diff * Fix accept header handling if only support one wire format * Update 406 error to more closely match spec example * Enforce version header via custom check instead of schema * Use ssz as default request wire format * Log failure to receive head event to verbose * Do not set default value for context * Update getClient return type to better align with method name * Consistent pattern to get route definitions * Dedupe api client type for builder and keymanager * Fix fallback logic if server returns http error * Update head event error logging * Retry 415 errors with JSON and cache SSZ not supported * Use fetch spy to assert call times * Update comment * Update getLightClientUpdatesByRange endpoint meta * Do not forward ssz bytes of blinded block to publishBlock * Fix lightclient e2e tests * Version header in publishBlock api is optional * Reduce type duplication * Add option to override request init in route definition * Add JsonOnlyResp codec * Validate boolean str value from headers * Document default wire formats * Simplify merging of inits in http client * Remove type hacks from fetchBeaconHealth * Reduce call stack in http client * Add .ssz() equivalent method for json to api response * More http client tests * Ensure topics query is provided to eventstream api * Validate request content type in handler Fastify does not cover all edge cases * Review routes, fix param docs, no empty comments * Fix typo * Add note about builder spec not supporting ssz * Consistently move keymanager jsdoc to routes * Sanitize user provided init values before merging * Remove unused ssz only codec * Allow passing wire formats as string literals * chore: review proof routes (#6843) Review proof routes * chore: review lightclient routes (#6842) Review lightclient routes * chore: review node routes (#6844) Review node routes * feat: add cli flags to configure http wire format (#6840) * Review PR, mostly cosmetic changes * Fix event stream error handling --------- Co-authored-by: Cayman <caymannava@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
if bn and vc are in different networks which we have had seen the need of, the communication size between validator and bn for proposals is a real issue adding unnecessary latency. This PR adds a capability to allow validator to request blinded versions of locally produced and selected blocks.
This PR adds this flag to the request so that the produceBlockV3 flow remains complaint to the spec and the local "blinded" capability aware validators would be able to use this feature, as some validators (vouch etc) or other validators who want to have full blocks for engine so as to assume statelessness can continue to function in the same way
Closes #6219