Parallec Aggregates HTTP Responses from 8000 Servers

Yuanteng (Jeff) Pei edited this page Nov 28, 2015 · 5 revisions

Watch Demo: 8,000 web server HTTP response aggregation to memory in 12 seconds / to ElasticSearch in 16 seconds in a single VM.

Aggregate to memory in 12 seconds

parallec HTTP Response Aggregation from 8000 servers in 12 seconds to memory

Aggregate to Elastic Search in 15.6 seconds

parallec HTTP Response Aggregation from 8000 servers in 15.6 seconds to Elastic Search

Test Environment

The Parallec spark server is running on a 4 core CPU VM with 24GB RAM.

For the 8000 servers 12/16 seconds demo, these are uniform HTTPS GET call request ; request very small string /XX; responses typically are less than 1KB

Note that the speed varies based on network speed, API response time, the slowest servers, timeout, and concurrency settings.