Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Pycon 2012 Tutotial
Python Go CSS Shell C Perl Other
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
__pycache__
bigfile
bigfiledata
c variosu mods
cpp
cpubound variosu mods
go
keynote
log
optimal
pascal
presentation
pypackages/dry variosu mods
urlclient
util variosu mods
#wing#
README
README_FIRST
TODO
__init__.py
bigfile.go~
bigfile_brute.py
bigfile_brute_regex.py variosu mods
bigfile_chunks_futures_threadpool.py variosu mods
bigfile_chunks_gevent.py
bigfile_chunks_gevent_pipeline2.py
bigfile_chunks_mp.py
bigfile_chunks_mp_pipeline2.py
bigfile_chunks_threads.py variosu mods
bigfile_chunks_threads.pyc
bigfile_coroutines.py
bigfile_pipeline_1.py
bigfile_pipeline_2.py
bigfile_test.bash
bigfile_test.bash~
blocking_io_1.py
coroutine_1.py
coroutine_loop.py
coroutinedec.py
coroutinedec.pyc
create_20K_threads.py variosu mods
dining_philosophers.py variosu mods
echo_client_gevent.py
echo_server_gevent.py
fibonacci_iterative.py
futures_errors~
generators_1.py
greenlet_1.py variosu mods
greenlet_2.py variosu mods
greenlet_example_1.py
main_bigfile_test.bash with xferlogs and pypackages
main_bigfile_test.bash~
main_urlclient_main.bash~
main_urlclient_real.bash
main_urlclient_real.bash~ with xferlogs and pypackages
main_urlclient_systimes.bash
non_blocking_io_1.py
non_blocking_io_2.py
p_pycon2012parallelconcurrent.wpr
p_pycon2012parallelconcurrent.wpu
ptime.py
read_text_file.py
scratch.bash with xferlogs and pypackages
scratch.py variosu mods
scratch.py~ variosu mods
set_kernel.bash with xferlogs and pypackages
set_limits.bash init
settings.bash
settings.bash~ variosu mods
settings.py
settings.pyc variosu mods
sockets_open.bash
sockets_open.bash~ init
spec.txt
stop_scaling_frequency.bash with xferlogs and pypackages
stop_scaling_frequency.bash~
test.py
test_echo_client.bash~
test_gevent
urlclient_futures_pool.py variosu mods
urlclient_gevent_individual.py
urlclient_gevent_pool.py

README

Increase system default settings
--------------------------------
Params that configure max open connections.
Edit /etc/sysctl.conf and add
    net.core.somaxconn=131072
    fs.file-max=131072

then on the command line:
    sudo sysctl -p

Edit /usr/include/linux/limits.h
    NR_OPEN = 65536
    
Edit /etc/security/limits.conf
    your_username  soft   nofile          65535
    your_username hard    nofile          65535    

limits.conf is read at login, so you should be able to logout and back in for the new
limits to take effect. 

As an alternative, you can run set_kernel.bash to set the kernel parameters at
runtime with even larger values:

    echo "10152 65535" > /proc/sys/net/ipv4/ip_local_port_range
    sysctl -w fs.file-max=128000
    sysctl -w net.ipv4.tcp_keepalive_time=300
    sysctl -w net.core.somaxconn=250000
    sysctl -w net.ipv4.tcp_max_syn_backlog=2500
    sysctl -w net.core.netdev_max_backlog=2500
    ulimit -n 10240
    
    
Stop scaling frequency
------------------------
CPU frequency scaling enables the operating system to scale the CPU speed up or 
down in order to save power. You need to stop this before running any time
based tests.

#! /bin/bash
for i in /sys/devices/system/cpu/cpu[0-9]
do
    echo performance > $i/cpufreq/scaling_governor
done

xferlogs
----------
You need to gunzip the xferlogs before running any tests.
Something went wrong with that request. Please try again.