Erlang messaging server optimized to send 1 message to 40k subscribers to a topic in < 1s
Switch branches/tags
Nothing to show
Latest commit 8dca95a Jul 17, 2009 @wagerlabs wagerlabs courtesy of cliff moon
Permalink
Failed to load latest commit information.
ebin base Jul 15, 2009
priv base Jul 15, 2009
src courtesy of cliff moon Jul 17, 2009
Emakefile quiet Jul 17, 2009
Makefile bump max procs Jul 17, 2009
NOTES mac stats with improved timestamp Jul 17, 2009
README.textile Mac OSX notes Jul 16, 2009

README.textile

Janus

Janus is a messaging server optimized to unicast over TCP to thousands of clients subscribed to topics of interest.

The ultimate goal is to maintain a latency of less than 2 seconds for 20 thousand clients on Amazon EC2 (small instance).

License

Janus is available under the MIT license.

Bounty and rules

1) Unicast to 20K clients in < 1s (max latency) nets you a $1000 bounty if you are the first person to achieve this. Unicast in < 2s (max latency) nets you a $500 bounty.

2) Final proof has to be from EC2, one small instance for the server and one for the bots.

3) {packet, 0}, -smp disabled

4) TCP, not UDP.

5) No change of the protocol unless I agree to it.

Note that SMP is disabled since Amazon EC2 small instances have a single core.

Installation

Janus requires at least OTP R13B01.

Create a soft link from janus/mochiweb to your Mochiweb installation.

Running

1) ‘make run1’ to start the server
2) ‘make sh’ in a different window to run clients
3) ‘bot:test(flashbot, 20000).’ to run 20k bots on the same machine
4) ‘bot:test(flashbot, 20000, ’host’, 8081).’ if the server is somewhere else

You should see output like this if everything goes well (MacBook Pro Core2Duo, 2.93Ghz, SSD):


(debug@biggie.local)5> bot:test(flashbot,10000).

=INFO REPORT==== 15-Jul-2009::19:44:20 ===
setup: 5055.70ms, n: 10000, run: 7524.29ms
    1.8690ms | min   
  500.0000ms | 2125   -  21.25%
 1000.0000ms | 5010   -  50.10%
 1500.0000ms | 2865   -  28.65%
 1269.7850ms | max   
ok
(debug@biggie.local)5> bot:test(flashbot,20000).

=INFO REPORT==== 15-Jul-2009::19:44:55 ===
setup: 14293.01ms, n: 20000, run: 21956.94ms
    2.4850ms | min   
  500.0000ms | 478    -   2.39%
 1000.0000ms | 2283   -  11.42%
 1500.0000ms | 7154   -  35.77%
 2000.0000ms | 1574   -   7.87%
 2500.0000ms | 3301   -  16.50%
 3000.0000ms | 2779   -  13.89%
 3500.0000ms | 2431   -  12.16%
 3277.0740ms | max   
ok

Notes

On Leopard:


cat /etc/sysctl.conf 
kern.maxfiles=102400
kern.maxfilesperproc=40960
net.inet.ip.portrange.hifirst=10000
net.inet.ip.portrange.first=10000

grep ulimit ~/.bash_profile
ulimit -n 30720 

Also, you won’t be able to go past 1024 clients with Erlang
on Mac OSX, even with kernel poll enabled (+K true).

Edit FD_SETSIZE in /usr/include/sys/select.h, i.e. change
__DARWIN_FD_SETSIZE in /usr/include/sys/_structs.h
and recompile Erlang with —enable-kernel-poll.

On Linux:


echo "8192 61000" > /proc/sys/net/ipv4/ip_local_port_range
echo "<your user name>             -       nofile          32768" >>
/etc/security/limits.conf
ulimit -n 32768