Enable ParMapN for IO up to arity 9 #1951

nomisRev · 2020-01-22T09:29:35Z

Small life of quality improvement for IO that enables IO.parMapN without an explicit CoroutineContext which will run on the default IO dispatchers.

1Jajen1

Looks good, definitly nice to have 👍

1Jajen1 · 2020-01-22T11:03:33Z

modules/fx/arrow-fx/src/main/kotlin/arrow/fx/IOParMap.kt

+
+  fun <A, B, C> parMapN(ctx: CoroutineContext, fa: IOOf<A>, fb: IOOf<B>, f: (A, B) -> C): IO<C> = IO.Async(true) { conn, cb ->
+    // Used to store Throwable, Either<A, B> or empty (null). (No sealed class used for a slightly better performing ParMap2)
+    val state = AtomicRefW<Any?>(null)


Out of curiosity: How would a sealed class negatively impact performance? Is it the extra wrapping of the throwable and either cases?

Very valid question! It's only slightly if simply used for parMapN but parMapN is what powers traverse so it can potentially be infinite wrappings if you traverse over Sequence.

The slightly more complex code is hidden & private.

👍

Btw: traverse over an infinite sequence with parApplicative won't even start any code because it will fold the entire sequence first, you need to have lazyAp to actually execute code before folding is complete 🙈

This is not even true atm because it just stackoverflows, which is a bug (I'll open a ticket, because I have no idea how to fix it...). But still even without the stackoverflow parApplicative cannot be used on infinite sequences.

I made a fix for IO's parallel ops not being stack-safe a week ago. So with this change, and your lazyAp change why wouldn't it work?

#1928

Because of this:

arrow/modules/fx/arrow-fx/src/main/kotlin/arrow/fx/typeclasses/ParApplicative.kt

Line 23 in 3051b67

override fun <A, B> Kind<F, A>.lazyAp(ff: () -> Kind<F, (A) -> B>): Kind<F, B> = ap(ff())

Without properly implementing this method the applicative cannot short-circuit before the entire IO is constructed. Only after executing ap fully for every element it will start evaluating the IO. If you don't want this behaviour you need to defer calling the argument in lazyAp somehow. E.q. Option does this by first checking if it is Some or None and runs the function only if it is Some. Same for Either, Validated and the normal IO applicative

pakoito

LGTM!

* Prettier show instances (#1937) * Add nicer show instances for all types in core * Linter + fixes * Small changes * Escape strings + fixes * Unused import * Fix kleisli test * Undo change in AndThen This wasn't intentional Co-authored-by: Simon Vergauwen <nomisRev@users.noreply.github.com> * Config: publish artifacts from release/0.11.0 branch (#1955) * Rename Applicative#map to #mapN and tupled to tupledN (#1950) * Renamed map overloads in Apply to mapN and deprecated old ones * Renamed tupled overloads in Apply to tupledN and deprecated old ones * Improved ReplaceWith for deprecated map * Replaced ff with f (variable name) Co-authored-by: Alberto Ballano <aballano@users.noreply.github.com> Co-authored-by: Simon Vergauwen <nomisRev@users.noreply.github.com> * Doc: update badges to show both WIP versions (#1959) * Doc: update badges to show both WIP versions * Update README.md * Update README.md Co-authored-by: Juan Valencia <juanvalencia@calvellido.es> * Replaced nuclear war examples with cooking examples (#1915) * Replaced nuclear war examples with cooking examples * remove :ank markup for code that cannot compile * DOCS: Error Handling - remove Try or replace Try with Either where appropriate Co-authored-by: Juan Valencia <juanvalencia@calvellido.es> Co-authored-by: Raúl Raja Martínez <raulraja@gmail.com> * Official Gradle Wrapper Validation GitHub Action (#1930) Signed-off-by: Jonathan Leitschuh <Jonathan.Leitschuh@gmail.com> Co-authored-by: Rachel M. Carmena <rachelcarmena@users.noreply.github.com> * Remove ObservableK Traverse/MonadFilter instances as they're not lawful. (#1964) * Remove ObservableK Traverse instances as they're not lawful. Fixes #1882 * Remove MonadFilter instance from Observable as it fails due to notion of errors Co-authored-by: Raúl Raja Martínez <raulraja@gmail.com> * Add MonadIO to fx and instances to fx-mtl (#1943) * Add MonadIO to fx and instances to fx-mtl * Add resource MonadIO instance and add MonadIO to docs Co-authored-by: Simon Vergauwen <nomisRev@users.noreply.github.com> Co-authored-by: Paco <pakoito@users.noreply.github.com> * Enable ParMapN for IO up to arity 9 (#1951) * Refactor ParMapN to enable arity up to 9 * Use correct ParMapN Co-authored-by: Raúl Raja Martínez <raulraja@gmail.com> * Deprecate bind operator component1 (#1965) * Deprecating binding using operator component1 since it has problems capturing the value * Removed usages of component1 Co-authored-by: Alberto Ballano <aballano@users.noreply.github.com> Co-authored-by: Simon Vergauwen <nomisRev@users.noreply.github.com> Co-authored-by: Jannis <overesch.jannis@gmail.com> Co-authored-by: Simon Vergauwen <nomisRev@users.noreply.github.com> Co-authored-by: Rachel M. Carmena <rachelcarmena@users.noreply.github.com> Co-authored-by: danieh <danimontoya_86@hotmail.com> Co-authored-by: Alberto Ballano <aballano@users.noreply.github.com> Co-authored-by: Juan Valencia <juanvalencia@calvellido.es> Co-authored-by: kylegoetz <kylegoetz@users.noreply.github.com> Co-authored-by: Raúl Raja Martínez <raulraja@gmail.com> Co-authored-by: Jonathan Leitschuh <jonathan.leitschuh@gmail.com> Co-authored-by: Paco <pakoito@users.noreply.github.com>

* Prettier show instances (#1937) * Add nicer show instances for all types in core * Linter + fixes * Small changes * Escape strings + fixes * Unused import * Fix kleisli test * Undo change in AndThen This wasn't intentional Co-authored-by: Simon Vergauwen <nomisRev@users.noreply.github.com> * Config: publish artifacts from release/0.11.0 branch (#1955) * Rename Applicative#map to #mapN and tupled to tupledN (#1950) * Renamed map overloads in Apply to mapN and deprecated old ones * Renamed tupled overloads in Apply to tupledN and deprecated old ones * Improved ReplaceWith for deprecated map * Replaced ff with f (variable name) Co-authored-by: Alberto Ballano <aballano@users.noreply.github.com> Co-authored-by: Simon Vergauwen <nomisRev@users.noreply.github.com> * Doc: update badges to show both WIP versions (#1959) * Doc: update badges to show both WIP versions * Update README.md * Update README.md Co-authored-by: Juan Valencia <juanvalencia@calvellido.es> * Replaced nuclear war examples with cooking examples (#1915) * Replaced nuclear war examples with cooking examples * remove :ank markup for code that cannot compile * DOCS: Error Handling - remove Try or replace Try with Either where appropriate Co-authored-by: Juan Valencia <juanvalencia@calvellido.es> Co-authored-by: Raúl Raja Martínez <raulraja@gmail.com> * Official Gradle Wrapper Validation GitHub Action (#1930) Signed-off-by: Jonathan Leitschuh <Jonathan.Leitschuh@gmail.com> Co-authored-by: Rachel M. Carmena <rachelcarmena@users.noreply.github.com> * Remove ObservableK Traverse/MonadFilter instances as they're not lawful. (#1964) * Remove ObservableK Traverse instances as they're not lawful. Fixes #1882 * Remove MonadFilter instance from Observable as it fails due to notion of errors Co-authored-by: Raúl Raja Martínez <raulraja@gmail.com> * Add MonadIO to fx and instances to fx-mtl (#1943) * Add MonadIO to fx and instances to fx-mtl * Add resource MonadIO instance and add MonadIO to docs Co-authored-by: Simon Vergauwen <nomisRev@users.noreply.github.com> Co-authored-by: Paco <pakoito@users.noreply.github.com> * Enable ParMapN for IO up to arity 9 (#1951) * Refactor ParMapN to enable arity up to 9 * Use correct ParMapN Co-authored-by: Raúl Raja Martínez <raulraja@gmail.com> * Deprecate bind operator component1 (#1965) * Deprecating binding using operator component1 since it has problems capturing the value * Removed usages of component1 Co-authored-by: Alberto Ballano <aballano@users.noreply.github.com> Co-authored-by: Simon Vergauwen <nomisRev@users.noreply.github.com> * Add automatic sync for release/0.11.0 (#1967) * Fix recently added doc links (#1970) * Clean dead configuration (#1972) Co-authored-by: Jannis <overesch.jannis@gmail.com> Co-authored-by: Simon Vergauwen <nomisRev@users.noreply.github.com> Co-authored-by: danieh <danimontoya_86@hotmail.com> Co-authored-by: Alberto Ballano <aballano@users.noreply.github.com> Co-authored-by: Juan Valencia <juanvalencia@calvellido.es> Co-authored-by: kylegoetz <kylegoetz@users.noreply.github.com> Co-authored-by: Raúl Raja Martínez <raulraja@gmail.com> Co-authored-by: Jonathan Leitschuh <jonathan.leitschuh@gmail.com> Co-authored-by: Paco <pakoito@users.noreply.github.com>

* Prettier show instances (#1937) * Add nicer show instances for all types in core * Linter + fixes * Small changes * Escape strings + fixes * Unused import * Fix kleisli test * Undo change in AndThen This wasn't intentional Co-authored-by: Simon Vergauwen <nomisRev@users.noreply.github.com> * Config: publish artifacts from release/0.11.0 branch (#1955) * Rename Applicative#map to #mapN and tupled to tupledN (#1950) * Renamed map overloads in Apply to mapN and deprecated old ones * Renamed tupled overloads in Apply to tupledN and deprecated old ones * Improved ReplaceWith for deprecated map * Replaced ff with f (variable name) Co-authored-by: Alberto Ballano <aballano@users.noreply.github.com> Co-authored-by: Simon Vergauwen <nomisRev@users.noreply.github.com> * Doc: update badges to show both WIP versions (#1959) * Doc: update badges to show both WIP versions * Update README.md * Update README.md Co-authored-by: Juan Valencia <juanvalencia@calvellido.es> * Replaced nuclear war examples with cooking examples (#1915) * Replaced nuclear war examples with cooking examples * remove :ank markup for code that cannot compile * DOCS: Error Handling - remove Try or replace Try with Either where appropriate Co-authored-by: Juan Valencia <juanvalencia@calvellido.es> Co-authored-by: Raúl Raja Martínez <raulraja@gmail.com> * Official Gradle Wrapper Validation GitHub Action (#1930) Signed-off-by: Jonathan Leitschuh <Jonathan.Leitschuh@gmail.com> Co-authored-by: Rachel M. Carmena <rachelcarmena@users.noreply.github.com> * Remove ObservableK Traverse/MonadFilter instances as they're not lawful. (#1964) * Remove ObservableK Traverse instances as they're not lawful. Fixes #1882 * Remove MonadFilter instance from Observable as it fails due to notion of errors Co-authored-by: Raúl Raja Martínez <raulraja@gmail.com> * Add MonadIO to fx and instances to fx-mtl (#1943) * Add MonadIO to fx and instances to fx-mtl * Add resource MonadIO instance and add MonadIO to docs Co-authored-by: Simon Vergauwen <nomisRev@users.noreply.github.com> Co-authored-by: Paco <pakoito@users.noreply.github.com> * Enable ParMapN for IO up to arity 9 (#1951) * Refactor ParMapN to enable arity up to 9 * Use correct ParMapN Co-authored-by: Raúl Raja Martínez <raulraja@gmail.com> * Deprecate bind operator component1 (#1965) * Deprecating binding using operator component1 since it has problems capturing the value * Removed usages of component1 Co-authored-by: Alberto Ballano <aballano@users.noreply.github.com> Co-authored-by: Simon Vergauwen <nomisRev@users.noreply.github.com> * Add automatic sync for release/0.11.0 (#1967) * Fix recently added doc links (#1970) * Clean dead configuration (#1972) * Fix recursive flatten for stdlib List (#1969) * Create special flatten function to work with partial wrapping such as List<List<A>> Added test for stdlib List and Listk flatten * RaceN arity9 (#1968) * Merged IORacePair & IORaceTriple into IORace and implemented `raceN` up until arity 9 * Nits from PR * Fix build Co-authored-by: Jannis <overesch.jannis@gmail.com> Co-authored-by: Rachel M. Carmena <rachelcarmena@users.noreply.github.com> Co-authored-by: danieh <danimontoya_86@hotmail.com> Co-authored-by: Alberto Ballano <aballano@users.noreply.github.com> Co-authored-by: Juan Valencia <juanvalencia@calvellido.es> Co-authored-by: kylegoetz <kylegoetz@users.noreply.github.com> Co-authored-by: Raúl Raja Martínez <raulraja@gmail.com> Co-authored-by: Jonathan Leitschuh <jonathan.leitschuh@gmail.com> Co-authored-by: Paco <pakoito@users.noreply.github.com>

Refactor ParMapN to enable arity up to 9

faf8dc2

nomisRev mentioned this pull request Jan 22, 2020

["Request"] Enable IO racing up to arity 9 #1952

Closed

1Jajen1 approved these changes Jan 22, 2020

View reviewed changes

nomisRev and others added 4 commits January 22, 2020 12:23

Merge branch 'master' into sv-io-parMapN

c01c407

Merge branch 'master' into sv-io-parMapN

0f8116d

Merge remote-tracking branch 'origin/master' into sv-io-parMapN

a40b20e

Merge branch 'master' into sv-io-parMapN

d418ca4

pakoito approved these changes Jan 23, 2020

View reviewed changes

nomisRev and others added 2 commits January 23, 2020 10:10

Merge branch 'master' into sv-io-parMapN

a38edce

Use correct ParMapN

4cf5b0b

nomisRev merged commit 4781868 into master Jan 23, 2020

nomisRev deleted the sv-io-parMapN branch January 23, 2020 10:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable ParMapN for IO up to arity 9 #1951

Enable ParMapN for IO up to arity 9 #1951

nomisRev commented Jan 22, 2020

1Jajen1 left a comment

1Jajen1 Jan 22, 2020

nomisRev Jan 22, 2020

1Jajen1 Jan 22, 2020

nomisRev Jan 22, 2020 •

edited

1Jajen1 Jan 22, 2020

pakoito left a comment

Enable ParMapN for IO up to arity 9 #1951

Enable ParMapN for IO up to arity 9 #1951

Conversation

nomisRev commented Jan 22, 2020

1Jajen1 left a comment

Choose a reason for hiding this comment

1Jajen1 Jan 22, 2020

Choose a reason for hiding this comment

nomisRev Jan 22, 2020

Choose a reason for hiding this comment

1Jajen1 Jan 22, 2020

Choose a reason for hiding this comment

nomisRev Jan 22, 2020 • edited

Choose a reason for hiding this comment

1Jajen1 Jan 22, 2020

Choose a reason for hiding this comment

pakoito left a comment

Choose a reason for hiding this comment

nomisRev Jan 22, 2020 •

edited