Add CPU_COUNT env variable / get MAX_THREADS for load_gen#2586
Conversation
| defaults['threads'] = 8 | ||
| try: | ||
| maxThreads = multiprocessing.cpu_count() | ||
| defaults['threads'] = multiprocessing.cpu_count() * 2 |
There was a problem hiding this comment.
Doesn't this count the number of cpus on the application server and not the load server (where it is used)?
|
@msmith-techempower please review changes. works for me on vagrant with 2 virtual cores: |
NateBrady23
changed the title
| uwsgi --ini ${IROOT}/config.ini --cutelyst-app ${IROOT}/cutelyst-benchmarks/src/libcutelyst_benchmarks.so ${PROCESS_OR_THREAD} $CPU_COUNT & | ||
| else | ||
| ${CROOT}/bin/cutelyst-wsgi --ini ${CROOT}/config.ini -a ${CROOT}/benchmarks/src/libcutelyst_benchmarks.so ${PROCESS_OR_THREAD} $MAX_THREADS --socket-timeout 0 ${BALANCER} & | ||
| ${IROOT}/bin/cutelyst-wsgi --ini ${IROOT}/config.ini -a ${IROOT}/cutelyst-benchmarks/src/libcutelyst_benchmarks.so ${PROCESS_OR_THREAD} $CPU_COUNT --socket-timeout 0 & |
There was a problem hiding this comment.
This undid some other recent changes in the cutelyst config. I suggest starting over with a copy of this file from master, then doing a search and replace for MAX_THREADS with CPU_COUNT.
| g++-4.8 -O3 -DNDEBUG -std=c++0x -L${BOOST_LIB} -I${BOOST_INC} -L${WT_LIB} -I${WT_INC} -o benchmark.wt benchmark.cpp -lwt -lwthttp -lwtdbo -lwtdbomysql -lboost_thread -lboost_system | ||
|
|
||
| ./benchmark.wt -c wt_config.xml -t ${MAX_THREADS} --docroot . --http-address 0.0.0.0 --http-port 8080 --accesslog=- --no-compression | ||
| ./benchmark.wt -c wt_config.xml -t ${CPU_COUNT} --docroot . --http-address 0.0.0.0 --http-port 8080 --accesslog=- --no-compression & |
There was a problem hiding this comment.
Could you remove the trailing & again? I removed it intentionally here NateBrady23@84d97ea
There was a problem hiding this comment.
Ah, sorry about that. When I was fixing conflicts I missed that in the diff.
| g++-4.8 -O3 -DNDEBUG -DBENCHMARK_USE_POSTGRES -std=c++0x -L${BOOST_LIB} -I${BOOST_INC} -L${WT_LIB} -I${WT_INC} -o benchmark_postgres.wt benchmark.cpp -lwt -lwthttp -lwtdbo -lwtdbopostgres -lboost_thread -lboost_system | ||
|
|
||
| ./benchmark_postgres.wt -c wt_config.xml -t ${MAX_THREADS} --docroot . --http-address 0.0.0.0 --http-port 8080 --accesslog=- --no-compression | ||
| ./benchmark_postgres.wt -c wt_config.xml -t ${CPU_COUNT} --docroot . --http-address 0.0.0.0 --http-port 8080 --accesslog=- --no-compression & |
|
|
||
| # Benchmark options | ||
| parser.add_argument('--concurrency-levels', default=[8, 16, 32, 64, 128, 256], help='Runs wrk benchmarker with different concurrency value (type int-sequence)', action=StoreSeqAction) | ||
| parser.add_argument('--query-levels', default=[1, 5,10,15,20], help='Database queries requested per HTTP connection, used during query test (type int-sequence)', action=StoreSeqAction) |
There was a problem hiding this comment.
With this, concurrency_levels and query_levels will no longer be configurable, right? If so, how about removing those lines from benchmark.cfg.example?
|
|
|
@michaelhixson Ok, I believe I've addressed everything. There might be another framework out there but I think I got them all. Some are defining there own MAX_THREADS variable so i couldn't just replace all. Didn't want those to lose their meaning. |
|
LGTM then |
|
@michaelhixson fyi, the concurrency and query levels need to stay in the |
3 participants
Removes the command line options for threads, concurrency_levels, and query_levels. The latter two remain in the
benchmark.cfgfile.Note
WIP: Moving away from making a
max_threadsenv variable available and instead will make acpu_countvariable available to frameworks. Will continue to use cpu_count*4 for the-toption of thewrkcommands.