Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configurable size of validate queue #255

Merged
merged 2 commits into from
Jan 27, 2020

Commits on Jan 21, 2020

  1. Configurable size of validate queue

    validateWorker() reads from validateQ and invokes validate function
    that performs validation of the message. Signature validation is performed
    synchronously. The number of validate workers defaults to the number of CPUs
    and can be updated with WithValidateWorkers function. With no additional user
    validators, signature validation is the bottleneck when receiving new messages.
    
    Increasing the number of validating workers does not help given the context
    switching and bottleneck nature of this spot. As stated in WithValidateWorkers
    documentation, this function should be used rather to limit the number of workers
    to devote less CPU time for synchronous validation. On the other hand, with the
    default size of `validateQ`, some applications built on a top of libp2p may
    experience throttled validation and lost messages.
    
    This problem is addressed by WithValidateQueueSize allowing to configure the buffer
    size for synchronous validation. Application developers knowing the nature of their
    protocols can set this value to minimise the possibility of throttled synchronous
    validation and dropped messages. Configurable buffer size allows to gracefully
    handle peaks of messages and, from the other side, the number of concurrent
    synchronous workers is still limited by validateWorkers property so the receiver
    should not get congested.
    pdyraga committed Jan 21, 2020
    Configuration menu
    Copy the full SHA
    499109b View commit details
    Browse the repository at this point in the history

Commits on Jan 27, 2020

  1. go fmt on pubsub.go

    pdyraga committed Jan 27, 2020
    Configuration menu
    Copy the full SHA
    ae00326 View commit details
    Browse the repository at this point in the history