We created these to compare the Warp web server and the Yesod web framework with other web servers and frameworks.
Here is a blog of the PONG benchmark: http://www.yesodweb.com/blog/2011/03/preliminary-warp-cross-language-benchmarks
In our comparison, we tried to find the fastest servers in other languages. For python we used tornado. For Ruby we used Goliath. Here is an overview of Ruby deployment options. For Java we used winstone, although there may well be a better option.
In the future we would like to benchmark against Erlang and Nginx.
These benchmarks are designed to be ran on a debian/Ubuntu EC2 server. There is an existing install script for every aspect of the installation.