Framework Overhead Label is confusing

[tback]

The framework overhead graph is labeled 'higher is better' where it should be labeled lower is better.

[RockinRoel]

No, if you look at the results, you'll see that indeed higher is better. It's just a bit confusing to call it overhead, because more overhead is worse.

[tback]

thanks for pointing that out

[michaelhixson]

For what it's worth, we did struggle to come up with better words for this. At least people have a general understanding of what "framework overhead" means, even if that chart seems backwards at first glance. We couldn't think of a good, understandable synonym for "negative framework overhead". The word "efficiency" was not a candidate because eventually we want to use that for some sort of developer-time efficiency measure.

[KevinHoward]

Would the term "Utilization" be a better than "Overhead"? Usually when you are talking about overhead you are talking about expenses involved to complete a job or task. What this graph represents is how well each Framework utilizes its Platform compared to other frameworks on the same platform, correct? Just a thought.

[bhauer]

Thanks to everyone for the additional discussion on this. I apologize for the confusion, and admit that it isn't surprising. As Michael Hixson points out, although we settled on the words "Framework Overhead" here, it was a somewhat reluctant decision.

I wanted something succinct (small enough to fit on a user interface tab). Efficiency is short and best describes the information being rendered, but we have elsewhere used the word efficiency to refer to a developer's efficiency when working with a framework. While that notion of efficiency is a subjective matter and not something that we plot anywhere on our results site, we nevertheless wanted to reserve the word "efficiency" for that notion to avoid future confusion if and when we discuss developer efficiency more.

Utilization isn't precisely what I want because, at least to my mind, it implies a measurement of how much time is spent using framework code versus the code of the test implementation.

As it is rendered today, it is an inverse depiction of the framework's overhead:

framework performance / platform performance

We could render the chart differently:

platform performance / framework performance

That would yield a chart that ranges from 0 to the maximum multiplier of overhead for the worst-overhead framework. It would make a smaller bar better.

I'd like to retain the name Framework Overhead, but I'm open to an alternative rendering like the above. Any additional thoughts?

[LadyMozzarella]

Closing due to inactivity, reopen if this is something that should be considered again.

[rnz]

Can describe in more detail how exactly the overhead is calculated? Why not build a straight diagram - less is better
And maybe add separate columns about resource consumption - CPU time, RAM consumption?