Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding support for Table.Reader protocol #369

Closed
wants to merge 5 commits into from

Conversation

akoutmos
Copy link
Contributor

@akoutmos akoutmos commented Aug 7, 2022

First of all, hello and thanks for all the hard work you have put into Benchee!

I took on implementing Benchee support in Livebook (see the GitHub issue here: livebook-dev/kino#132) and one of the ideas that José and Jonatan had was to add Table support in Benchee. This PR adds Table as an optional dependency and also provides the implementation of the Table.Reader protocol if the dependency is pulled down by the user.

Let me know what you think and we'll take it from there!
Thanks 😃

lib/benchee/suite.ex Outdated Show resolved Hide resolved
lib/benchee/suite.ex Outdated Show resolved Hide resolved
lib/benchee/suite.ex Outdated Show resolved Hide resolved
@jonatanklosko
Copy link

jonatanklosko commented Aug 8, 2022

Thanks @akoutmos!

As further motivation for adding this, note that this will also allow Explorer.DataFrame.new(suite) :)

akoutmos and others added 3 commits August 8, 2022 10:00
Co-authored-by: Jonatan Kłosko <jonatanklosko@gmail.com>
Co-authored-by: Jonatan Kłosko <jonatanklosko@gmail.com>
@jonatanklosko
Copy link

Hey @PragTob! What do you think about this integration? We already support Table.Reader in myxql, postgrex, explorer, to name a few.

Also, we are ready to release kino_benchee, but it relies on this feature. Let us know if there's anything we can do to move forward :)

@benkeil
Copy link

benkeil commented Dec 6, 2022

Would be awesome if we can merge this PR ☺️

@PragTob
Copy link
Member

PragTob commented Dec 22, 2022

Hello hello folks and sorry, let me just repost (but with different bunny pic) what I posted in the other thread: livebook-dev/kino#132 (comment)

👋

From a long slumber the @PragTob awakens. Sorry, lots of stress, bunny health, own health etc... and while bunny health is also... let's say medium right now... I'm looking at it now :) 💚

I want to eventually accept the PR and ship it. I'm still medium torn on including support for essentially a third party thing that is none trivial within benchee itself (vs. a separate package to provide it) - but it seems like that's what most people are doing right now and the Table.Reader protocol seems general enough/not tied only to livebook to convince me that it's reasonable to do so. :)

IMG_20221016_093021

Copy link
Member

@PragTob PragTob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, left some minor comments. I'll probably take it on myself to fix those and change some minor stuff around since it took me so long to get here 😅

Thanks for your work and patience 💚

@@ -47,3 +47,93 @@ defimpl DeepMerge.Resolver, for: Benchee.Suite do
Map.merge(original, override, resolver)
end
end

if Code.ensure_loaded?(Table.Reader) do
defimpl Table.Reader, for: Benchee.Suite do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be inclined to move this to its own file just for some separation as it is somehow meaty :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haha I myself implemented the deep_merge resolver here so... maybe not so that it starts breaking the pattern 😅

alias Benchee.Scenario

def init(suite_results) do
columns = get_columns_from_suite(suite_results)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the code base we usually refer to this as suite or well just the suite, so I think that's fine 🤔


memory_fields = Enum.map(fields_per_type, fn field -> "memory_#{field}" end)
reductions_fields = Enum.map(fields_per_type, fn field -> "reductions_#{field}" end)
run_time_fields = Enum.map(fields_per_type, fn field -> "run_time_#{field}" end)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Theoretically we'd only want to collect these if theses measurements have been run, right? We could use Scenario.data_processed? to check although I see that is missing the reductions implementation.

Or are these auto hidden if empty? I really don't know.

@PragTob
Copy link
Member

PragTob commented Dec 22, 2022

I'm trying to follow up on this in #377

@PragTob
Copy link
Member

PragTob commented Dec 22, 2022

I'm gonna close this one (for now) as I've followed up and extended on it with what I feel like are good/meaningful extensions over here: #377

reviews welcome!

@PragTob PragTob closed this Dec 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants