Complete benchmarks seem misleading

[stephenoneal]

Since you're varying the ORM based on the http framework the benchmarks seem vary misleading. Some were using pee_wee, some sqlalchemy, and django was using django. Couldn't you make the benchmarks all use pee_wee so you're bench-marking the web delivery part of the framework in isolation of ORM performance and features?

[ikalnytskyi]

Seconded on this. Or even use pure SQL queries in order to avoid cost of abstractions.

[cypreess]

@stephenoneal @ikalnytskyi I do not find this misleading. I assume that this test is going to help you decide what exactly to use. Very often frameworks (like django) comes with builtin or at least standard/prefered ORM. When you choosing specific framwork you are then usually bound to specific ORM - and this is very practical test then.

[ikalnytskyi]

@cypreess I'm sorry to say so, but I completely disagree on this:

When you choosing specific framwork you are then usually bound to specific ORM - and this is very practical test then.

I'd say Django is exception here, though no one force you to use Django ORM. I've heard about using SQLAlchemy instead in some REST API project.

All those frameworks are not tied to ORM:

• flask
• pecan
• pyramid
• bottle
• webpy
• falcon

and you will find even more. I only listed the most popular ones (except async ones, as they usually do not have ORM).

[jozuenoon]

I would add here also:

• tornado

[klen]

No ORM anymore