Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bench builder API #7324

Merged
merged 3 commits into from
Jul 19, 2023
Merged

Bench builder API #7324

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
731a773
Bench API with a builder
JaroslavTulach Jul 18, 2023
74ac8d7
Using Bench builder in Vector/Operations
JaroslavTulach Jul 18, 2023
8151182
Remove _ _ arguments as they are specified by the group configuration
JaroslavTulach Jul 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions distribution/lib/Standard/Test/0.0.0-dev/src/Bench.enso
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,68 @@
from Standard.Base import all
import Standard.Base.Runtime.Ref.Ref


type Bench_Options
## PRIVATE
Impl iter_size num_iters
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved

size : Integer -> Bench_Options
size self v = Bench_Options.Impl v self.num_iters

iter : Integer -> Bench_Options
iter self v = Bench_Options.Impl self.iter_size v

to_text self = "[iter_size=" + self.iter_size.to_text + ", num_iters=" + self.num_iters.to_text + "]"

type Bench_Builder
## PRIVATE
Impl builder

group : Text -> Bench_Options -> (Group_Builder -> Any) -> Any
group self (name:Text) (configuration:Bench_Options) fn =
b = Vector.new_builder
fn (Group_Builder.Impl b)
self.builder.append <| Bench.Group name configuration b.to_vector

type Group_Builder
## PRIVATE
Impl builder

specify : Text -> Any -> Bench
specify self (name:Text) ~benchmark =
self.builder.append <| Bench.Spec name (_ -> benchmark)


type Bench
All (groups : Vector Bench)
Group (name:Text) (configuration:Bench_Options) (specs : Vector Bench)
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
Spec (name:Text) (code : Any -> Any)

build : (Bench_Builder -> Any) -> Bench
build fn =
b = Vector.new_builder
fn (Bench_Builder.Impl b)
Bench.All b.to_vector

options : Bench_Options
options = Bench_Options.Impl -1 -1

fold : Any -> (Any -> Bench -> Bench -> Any) -> Any
fold self value fn = case self of
Bench.All groups -> groups.fold value (v-> g-> g.fold v fn)
Bench.Group _ _ specs -> specs.fold value (v-> s-> fn v self s)
Bench.Spec _ _ -> fn value self self

run_main self =
count = self.fold 0 v-> _-> _-> v+1
IO.println <| "Found " + count.to_text + " cases to execute"

self.fold Nothing _-> g-> s->
c = g.configuration
IO.println <| "Benchmarking " + s.name + " configuration: " + c.to_text
Bench.measure (s.code 0) s.name c.iter_size c.num_iters
IO.println <| "Benchmarking of " + s.name + " finished"

## Measure the amount of time it takes to execute a given computation.

Arguments:
Expand Down
47 changes: 29 additions & 18 deletions test/Benchmarks/src/Vector/Operations.enso
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,31 +16,42 @@ num_iterations = 10

# The Benchmarks ==============================================================

bench =
collect_benches group_builder =
random_vec = Utils.make_random_vec vector_size
random_vec_2 = Utils.make_random_vec 100000
random_gen = Java_Random.new 123456

Bench.measure (Base.Vector.new vector_size i->i) "New Vector" iter_size num_iterations
Bench.measure (Base.Vector.new vector_size _->42) "New Constant" iter_size num_iterations
Bench.measure (Base.Vector.new vector_size _->random_gen.nextLong) "New Random" iter_size num_iterations
Bench.measure (Base.Vector.fill vector_size 42) "Fill Constant" iter_size num_iterations
Bench.measure (Base.Vector.fill vector_size random_gen.nextLong) "Fill Random (constant)" iter_size num_iterations
Bench.measure (random_vec + [1]) "Append Single" iter_size num_iterations
Bench.measure (random_vec + random_vec_2) "Append Large" iter_size num_iterations
Bench.measure (random_vec.reduce (+)) "Sum" iter_size num_iterations
Bench.measure ((random_vec.drop (First 20)).reduce (+)) "Drop First 20 and Sum" iter_size num_iterations
Bench.measure ((random_vec.drop (Last 20)).reduce (+)) "Drop Last 20 and Sum" iter_size num_iterations
Bench.measure (random_vec.filter (x -> x % 3 == 1)) "Filter" iter_size num_iterations
Bench.measure (random_vec.filter_with_index (i-> x-> (i+x) % 3 == 1)) "Filter With Index" iter_size num_iterations

Bench.measure (random_vec . map (x -> x + random_gen.nextLong) . filter (x -> x % 3 == 1)) "Map & Filter" iter_size num_iterations
Bench.measure (random_vec.partition (x -> x % 3 == 1)) "Partition" iter_size num_iterations
Bench.measure (random_vec.partition_with_index (i-> x-> (i+x) % 3 == 1)) "Partition With Index" iter_size num_iterations
bench_measure ~act name _ _ = group_builder.specify name act

bench_measure (Base.Vector.new vector_size i->i) "New Vector" iter_size num_iterations
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
bench_measure (Base.Vector.new vector_size _->42) "New Constant" iter_size num_iterations
bench_measure (Base.Vector.new vector_size _->random_gen.nextLong) "New Random" iter_size num_iterations
bench_measure (Base.Vector.fill vector_size 42) "Fill Constant" iter_size num_iterations
bench_measure (Base.Vector.fill vector_size random_gen.nextLong) "Fill Random (constant)" iter_size num_iterations
bench_measure (random_vec + [1]) "Append Single" iter_size num_iterations
bench_measure (random_vec + random_vec_2) "Append Large" iter_size num_iterations
bench_measure (random_vec.reduce (+)) "Sum" iter_size num_iterations
bench_measure ((random_vec.drop (First 20)).reduce (+)) "Drop First 20 and Sum" iter_size num_iterations
bench_measure ((random_vec.drop (Last 20)).reduce (+)) "Drop Last 20 and Sum" iter_size num_iterations
bench_measure (random_vec.filter (x -> x % 3 == 1)) "Filter" iter_size num_iterations
bench_measure (random_vec.filter_with_index (i-> x-> (i+x) % 3 == 1)) "Filter With Index" iter_size num_iterations

bench_measure (random_vec . map (x -> x + random_gen.nextLong) . filter (x -> x % 3 == 1)) "Map & Filter" iter_size num_iterations
bench_measure (random_vec.partition (x -> x % 3 == 1)) "Partition" iter_size num_iterations
bench_measure (random_vec.partition_with_index (i-> x-> (i+x) % 3 == 1)) "Partition With Index" iter_size num_iterations

stateful_fun x =
s = State.get Number
State.put s+x
Bench.measure (State.run Number 0 <| random_vec.each stateful_fun) "Each" iter_size num_iterations
bench_measure (State.run Number 0 <| random_vec.each stateful_fun) "Each" iter_size num_iterations

bench =
options = Bench.options . size iter_size . iter num_iterations

all = Bench.build builder->
builder.group "Vector Operations" options group_builder->
collect_benches group_builder

all . run_main

main = bench
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
Loading