Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve foldr performance ~4x on large lists #180

Merged
merged 3 commits into from
Dec 23, 2020

Conversation

milesfrain
Copy link
Contributor

Performance is improved by using an inlined reverse function, rather than leveraging foldl.

Both versions appear to be identical, so it's a bit of a bummer that the concise version is so much slower.

There's a moderate speedup for all list sizes, but it's most significant on large lists. Here are the benchmarking results:

Before:

foldr: list (0 elems)
mean   = 1.52 μs
stddev = 416.18 ns
min    = 1.22 μs
max    = 11.11 μs
---
foldr: list (1 elems)
mean   = 2.90 μs
stddev = 22.78 μs
min    = 977.00 ns
max    = 631.81 μs
---
foldr: list (10 elems)
mean   = 5.96 μs
stddev = 21.03 μs
min    = 2.00 μs
max    = 563.57 μs
---
foldr: list (100 elems)
mean   = 16.16 μs
stddev = 50.97 μs
min    = 10.32 μs
max    = 1.54 ms
---
foldr: list (1000 elems)
mean   = 126.53 μs
stddev = 77.85 μs
min    = 101.90 μs
max    = 1.40 ms
---
foldr: list (10000 elems)
mean   = 1.36 ms
stddev = 301.37 μs
min    = 1.17 ms
max    = 3.69 ms
---
foldr: list (100000 elems)
mean   = 22.66 ms
stddev = 3.63 ms
min    = 20.18 ms
max    = 43.71 ms

After:

foldr: list (0 elems)
mean   = 1.20 μs
stddev = 542.72 ns
min    = 772.00 ns
max    = 10.38 μs
---
foldr: list (1 elems)
mean   = 1.66 μs
stddev = 3.04 μs
min    = 806.00 ns
max    = 96.76 μs
---
foldr: list (10 elems)
mean   = 5.40 μs
stddev = 19.95 μs
min    = 1.05 μs
max    = 392.62 μs
---
foldr: list (100 elems)
mean   = 11.22 μs
stddev = 16.66 μs
min    = 4.67 μs
max    = 332.93 μs
---
foldr: list (1000 elems)
mean   = 53.27 μs
stddev = 40.41 μs
min    = 44.55 μs
max    = 529.10 μs
---
foldr: list (10000 elems)
mean   = 525.74 μs
stddev = 192.72 μs
min    = 450.25 μs
max    = 2.29 ms
---
foldr: list (100000 elems)
mean   = 4.66 ms
stddev = 2.23 ms
min    = 2.09 ms
max    = 12.26 ms

Also refactored the benchmarking code.

Comment on lines 103 to +109
instance foldableList :: Foldable List where
foldr f b = foldl (flip f) b <<< rev
where
rev = foldl (flip Cons) Nil
rev = go Nil
where
go acc Nil = acc
go acc (x : xs) = go (x : acc) xs
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the core change. Wondering if the reverse function should be moved from Data.List to this file to deduplicate code.

Copy link
Contributor

@hdgarrood hdgarrood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! I’m not concerned about the duplication of reverse, since it’s so little code and it might look odd in the docs to say that reverse is re-exported from Data.List.Types whereas almost none of the other List functions are.

@milesfrain
Copy link
Contributor Author

This slower reverse pattern is duplicated in a few other places (just search for foldl (flip, for example:

Tuple one Nothing -> foldl (flip (:)) Nil (one : memo)

Nothing -> (foldl (flip (:)) Nil memo)

traverse f = map (foldl (flip (:)) Nil) <<< foldl (\acc -> lift2 (flip (:)) acc <<< f) (pure Nil)

rev = foldl (flip Cons) Nil

<#> case _ of NonEmptyList (x :| xs) → foldl (flip nelCons) (pure x) xs

So thinking these should all be replaced with a common rev helper function in this file (along with some comments about why it's written longform).

I'm also wondering if we should do more benchmarking in the browser (blocked by purescript/purescript-minibench#16) with bundling optimizations and see if there's still a performance difference. Ideally, I'd like to keep the original concise code.

@thomashoneyman thomashoneyman merged commit 41f63aa into purescript:master Dec 23, 2020
@milesfrain milesfrain deleted the foldr-perf branch December 23, 2020 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants