New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
replace receive slot with event stream #13563
Merged
Merged
Changes from 4 commits
Commits
Show all changes
107 commits
Select commit
Hold shift + click to select a range
29a552b
WIP
james-prysm 86966f5
event stream wip
james-prysm de96792
returning nil
james-prysm 033448e
temp removing some tests
james-prysm 23ec534
Merge branch 'develop' into remove-stream-slot
james-prysm 91ea02a
Merge branch 'develop' into remove-stream-slot
james-prysm b074356
wip health checks
james-prysm b5670b6
Merge branch 'develop' into remove-stream-slot
james-prysm b76e446
fixing conficts
james-prysm f3a0534
updating fields based on linting
james-prysm 38bce44
fixing more errors
james-prysm 87c62c9
Merge branch 'develop' into remove-stream-slot
james-prysm 1fb9a3e
fixing mocks
james-prysm 8b0d2ed
fixing more mocks
james-prysm 17eb642
fixing more linting
james-prysm be92c89
removing white space for lint
james-prysm 0e403d1
fixing log format
james-prysm 34af48f
gaz
james-prysm aa49ecc
reverting changes on grpc
james-prysm 800ce23
fixing unit tests
james-prysm 048bbcc
Merge branch 'develop' into remove-stream-slot
james-prysm 1595264
Merge branch 'develop' into remove-stream-slot
james-prysm 1859840
adding in tests for health tracker and event stream
james-prysm 73022ae
adding more tests for streaming slot
james-prysm 9be3711
Merge branch 'develop' into remove-stream-slot
james-prysm 47c3f28
gaz
james-prysm c179901
Merge branch 'develop' into remove-stream-slot
james-prysm 1b6581b
Update api/client/event/event_stream.go
james-prysm b7cab70
review comments
james-prysm 4ad4a46
Merge branch 'develop' into remove-stream-slot
james-prysm 0982c0b
Merge branch 'develop' into remove-stream-slot
james-prysm dd0b4d9
Update validator/client/runner.go
james-prysm 7464d9a
Update validator/client/validator.go
james-prysm 0f4467f
Update validator/client/validator.go
james-prysm 6a828bb
Update validator/client/validator.go
james-prysm ddd8560
Update validator/client/validator.go
james-prysm 68ceefa
Update validator/client/beacon-api/beacon_api_validator_client.go
james-prysm 40c566a
Update validator/client/validator.go
james-prysm 02ba06a
Update validator/client/validator.go
james-prysm 37da651
addressing radek comments
james-prysm 37da93f
Merge branch 'develop' into remove-stream-slot
james-prysm be240c4
Merge branch 'develop' into remove-stream-slot
james-prysm 371aa4c
Update validator/client/validator.go
james-prysm 1c3e369
Merge branch 'develop' into remove-stream-slot
james-prysm e802f62
addressing review feedback
james-prysm 44d5e67
moving things to below next slot ticker
james-prysm 4f4b473
fixing tests
james-prysm 2127691
Merge branch 'develop' into remove-stream-slot
james-prysm 472d211
update naming
james-prysm 0eb554c
adding TODO comment
james-prysm e7776fc
Merge branch 'develop' into remove-stream-slot
james-prysm 1d03da3
Merge branch 'develop' into remove-stream-slot
james-prysm 4d4eafa
Update api/client/beacon/health.go
james-prysm 264282c
addressing comments
james-prysm 1e8901c
Merge branch 'develop' into remove-stream-slot
james-prysm d698395
merging develop
james-prysm ba6c524
Merge branch 'develop' into remove-stream-slot
james-prysm e880031
fixing broken linting
james-prysm 49af08b
fixing more import issues
james-prysm 86f5f48
fixing more import issues
james-prysm 243c833
linting
james-prysm 182a683
Merge branch 'develop' into remove-stream-slot
james-prysm cc08106
updating based on radek's comments
james-prysm dde4265
addressing more comments
james-prysm ab89e47
fixing nogo error
james-prysm 2c63dba
Merge branch 'develop' into remove-stream-slot
james-prysm 32417a3
Merge branch 'develop' into remove-stream-slot
james-prysm f0c3903
Merge branch 'develop' into remove-stream-slot
james-prysm 263a51b
fixing duplicate import
james-prysm 9d08460
gaz
james-prysm 65b85ae
adding radek's review suggestion
james-prysm 2f80739
Merge branch 'develop' into remove-stream-slot
james-prysm 8e90fbf
Merge branch 'develop' into remove-stream-slot
james-prysm a35a12b
Merge branch 'develop' into remove-stream-slot
james-prysm 54bfd46
Merge branch 'develop' into remove-stream-slot
james-prysm 6729ae6
Merge branch 'develop' into remove-stream-slot
james-prysm 8c5200b
Merge branch 'develop' into remove-stream-slot
james-prysm bc8e9e2
Update proto/prysm/v1alpha1/node.proto
james-prysm 45693f2
preston review comments
james-prysm 98aa51e
Merge branch 'develop' into remove-stream-slot
james-prysm c8bebda
Update api/client/event/event_stream.go
james-prysm 4c4b669
Update validator/client/validator.go
james-prysm da6fd76
addressing some more preston review items
james-prysm 42356da
fixing tests for linting
james-prysm 79200f4
fixing missed linting
james-prysm 2d77069
Merge branch 'develop' into remove-stream-slot
james-prysm e3fd6b8
updating based on feedback to simplify
james-prysm aa144a0
adding interface check at the top
james-prysm d2f7df4
reverting some comments
james-prysm ae1720b
cleaning up intatiations
james-prysm 024624f
Merge branch 'develop' into remove-stream-slot
james-prysm c12e68b
Merge branch 'develop' into remove-stream-slot
james-prysm be7963f
reworking the health tracker
james-prysm 6a8a9b0
fixing linting
james-prysm 0c7bb37
Merge branch 'develop' into remove-stream-slot
james-prysm 0d21889
fixing more linting to adhear to interface
james-prysm 7141f63
adding interface check at the the top of the file
james-prysm e249fb9
fixing unit tests
james-prysm c49d870
attempting to fix dependency cycle
james-prysm 8ae7314
Merge branch 'develop' into remove-stream-slot
james-prysm fe599f8
Merge branch 'develop' into remove-stream-slot
james-prysm effc418
addressing radek's comment
james-prysm 414103e
Merge branch 'develop' into remove-stream-slot
james-prysm 92e813a
Update validator/client/beacon-api/beacon_api_validator_client.go
james-prysm 3b92c0b
adding more tests and feedback items
james-prysm 4c5ba7b
fixing TODO comment
james-prysm db14009
Merge branch 'develop' into remove-stream-slot
james-prysm File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
load("@prysm//tools/go:def.bzl", "go_library") | ||
|
||
go_library( | ||
name = "go_default_library", | ||
srcs = ["eventStream.go"], | ||
importpath = "github.com/prysmaticlabs/prysm/v4/api/client/event", | ||
visibility = ["//visibility:public"], | ||
deps = [ | ||
"//api:go_default_library", | ||
"@com_github_pkg_errors//:go_default_library", | ||
"@com_github_sirupsen_logrus//:go_default_library", | ||
], | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,141 @@ | ||
package event | ||
|
||
import ( | ||
"bufio" | ||
"context" | ||
"net/http" | ||
"net/url" | ||
"strings" | ||
|
||
"github.com/pkg/errors" | ||
"github.com/prysmaticlabs/prysm/v4/api" | ||
log "github.com/sirupsen/logrus" | ||
) | ||
|
||
const ( | ||
EventHead = "head" | ||
EventBlock = "block" | ||
EventAttestation = "attestation" | ||
EventVoluntaryExit = "voluntary_exit" | ||
EventBlsToExecutionChange = "bls_to_execution_change" | ||
EventProposerSlashing = "proposer_slashing" | ||
EventAttesterSlashing = "attester_slashing" | ||
EventFinalizedCheckpoint = "finalized_checkpoint" | ||
EventChainReorg = "chain_reorg" | ||
EventContributionAndProof = "contribution_and_proof" | ||
EventLightClientFinalityUpdate = "light_client_finality_update" | ||
EventLightClientOptimisticUpdate = "light_client_optimistic_update" | ||
EventPayloadAttributes = "payload_attributes" | ||
EventBlobSidecar = "blob_sidecar" | ||
EventError = "error" | ||
) | ||
|
||
var DefaultEventTopics = []string{EventHead} | ||
|
||
type EventStreamClient interface { | ||
Subscribe(eventsChannel chan<- *Event) | ||
} | ||
|
||
type Event struct { | ||
EventType string | ||
Data []byte | ||
} | ||
|
||
// EventStream is responsible for subscribing to the Beacon API events endpoint | ||
// and dispatching received events to subscribers. | ||
type EventStream struct { | ||
ctx context.Context | ||
httpClient *http.Client | ||
host string | ||
topics []string | ||
} | ||
|
||
func NewEventStream(ctx context.Context, httpClient *http.Client, host string, topics []string) (*EventStream, error) { | ||
// Check if the host is a valid URL | ||
_, err := url.ParseRequestURI(host) | ||
if err != nil { | ||
return nil, err | ||
} | ||
if len(topics) == 0 { | ||
return nil, errors.New("no topics provided") | ||
} | ||
|
||
return &EventStream{ | ||
ctx: ctx, | ||
httpClient: httpClient, | ||
host: host, | ||
topics: topics, | ||
}, nil | ||
} | ||
|
||
func (h *EventStream) Subscribe(eventsChannel chan<- *Event) { | ||
allTopics := strings.Join(h.topics, ",") | ||
log.Info("Starting listening to Beacon API events on topics: " + allTopics) | ||
fullUrl := h.host + "/eth/v1/events?topics=" + allTopics | ||
req, err := http.NewRequestWithContext(h.ctx, http.MethodGet, fullUrl, nil) | ||
if err != nil { | ||
eventsChannel <- &Event{ | ||
EventType: EventError, | ||
Data: []byte(errors.Wrap(err, "Failed to create HTTP request").Error()), | ||
} | ||
} | ||
req.Header.Set("Accept", api.EventStreamMediaType) | ||
req.Header.Set("Connection", api.KeepAlive) | ||
resp, err := h.httpClient.Do(req) | ||
if err != nil { | ||
eventsChannel <- &Event{ | ||
EventType: EventError, | ||
Data: []byte(errors.Wrap(err, "Failed to perform HTTP request").Error()), | ||
} | ||
} | ||
|
||
defer func() { | ||
if closeErr := resp.Body.Close(); closeErr != nil { | ||
log.WithError(closeErr).Error("Failed to close events response body") | ||
close(eventsChannel) | ||
} | ||
}() | ||
// Create a new scanner to read lines from the response body | ||
scanner := bufio.NewScanner(resp.Body) | ||
|
||
var eventType, data string // Variables to store event type and data | ||
|
||
// Iterate over lines of the event stream | ||
for scanner.Scan() { | ||
select { | ||
case <-h.ctx.Done(): | ||
log.Info("Context canceled, stopping event stream") | ||
close(eventsChannel) | ||
return | ||
default: | ||
line := scanner.Text() | ||
// Handle the event based on your specific format | ||
if line == "" { | ||
// Empty line indicates the end of an event | ||
if eventType != "" && data != "" { | ||
// Process the event when both eventType and data are set | ||
eventsChannel <- &Event{EventType: eventType, Data: []byte(data)} | ||
} | ||
|
||
// Reset eventType and data for the next event | ||
eventType, data = "", "" | ||
continue | ||
} | ||
|
||
if line[0:7] == "event: " { | ||
// Extract event type from the "event" field | ||
eventType = line[7:] | ||
} else if line[0:6] == "data: " { | ||
// Extract data from the "data" field | ||
data = line[6:] | ||
} | ||
} | ||
} | ||
|
||
if err := scanner.Err(); err != nil { | ||
eventsChannel <- &Event{ | ||
EventType: EventError, | ||
Data: []byte(errors.Wrap(err, "Error reading response body").Error()), | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is a span enough...