Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Pycon 2012 Tutotial
Python Go CSS Shell C Perl Other
Branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
__pycache__
bigfile variosu mods
bigfiledata
c
cpp variosu mods
cpubound
go
keynote
log
optimal
pascal
presentation
pypackages/dry
urlclient
util
#wing#
README
README_FIRST
TODO
__init__.py
bigfile.go~
bigfile_brute.py
bigfile_brute_regex.py
bigfile_chunks_futures_threadpool.py
bigfile_chunks_gevent.py
bigfile_chunks_gevent_pipeline2.py
bigfile_chunks_mp.py
bigfile_chunks_mp_pipeline2.py
bigfile_chunks_threads.py
bigfile_chunks_threads.pyc
bigfile_coroutines.py
bigfile_pipeline_1.py
bigfile_pipeline_2.py
bigfile_test.bash
bigfile_test.bash~
blocking_io_1.py
coroutine_1.py
coroutine_loop.py
coroutinedec.py
coroutinedec.pyc
create_20K_threads.py
dining_philosophers.py
echo_client_gevent.py
echo_server_gevent.py
fibonacci_iterative.py
futures_errors~
generators_1.py
greenlet_1.py
greenlet_2.py
greenlet_example_1.py
main_bigfile_test.bash
main_bigfile_test.bash~
main_urlclient_main.bash~
main_urlclient_real.bash
main_urlclient_real.bash~
main_urlclient_systimes.bash
non_blocking_io_1.py
non_blocking_io_2.py
p_pycon2012parallelconcurrent.wpr
p_pycon2012parallelconcurrent.wpu
ptime.py
read_text_file.py
scratch.bash
scratch.py
scratch.py~
set_kernel.bash
set_limits.bash
settings.bash
settings.bash~
settings.py
settings.pyc
sockets_open.bash
sockets_open.bash~
spec.txt
stop_scaling_frequency.bash
stop_scaling_frequency.bash~
test.py
test_echo_client.bash~
test_gevent
urlclient_futures_pool.py
urlclient_gevent_individual.py
urlclient_gevent_pool.py

README

Increase system default settings
--------------------------------
Params that configure max open connections.
Edit /etc/sysctl.conf and add
    net.core.somaxconn=131072
    fs.file-max=131072

then on the command line:
    sudo sysctl -p

Edit /usr/include/linux/limits.h
    NR_OPEN = 65536
    
Edit /etc/security/limits.conf
    your_username  soft   nofile          65535
    your_username hard    nofile          65535    

limits.conf is read at login, so you should be able to logout and back in for the new
limits to take effect. 

As an alternative, you can run set_kernel.bash to set the kernel parameters at
runtime with even larger values:

    echo "10152 65535" > /proc/sys/net/ipv4/ip_local_port_range
    sysctl -w fs.file-max=128000
    sysctl -w net.ipv4.tcp_keepalive_time=300
    sysctl -w net.core.somaxconn=250000
    sysctl -w net.ipv4.tcp_max_syn_backlog=2500
    sysctl -w net.core.netdev_max_backlog=2500
    ulimit -n 10240
    
    
Stop scaling frequency
------------------------
CPU frequency scaling enables the operating system to scale the CPU speed up or 
down in order to save power. You need to stop this before running any time
based tests.

#! /bin/bash
for i in /sys/devices/system/cpu/cpu[0-9]
do
    echo performance > $i/cpufreq/scaling_governor
done

xferlogs
----------
You need to gunzip the xferlogs before running any tests.
Something went wrong with that request. Please try again.