Experiment with tyre #93

Open
wants to merge 3 commits into
from

Conversation

Projects
None yet
4 participants
@Drup
Member

Drup commented Nov 27, 2016

This was just an amusing experiment. All the test passes.

I'm pretty sure we could make to_string works too, but I didn't tried (~scheme would be tricky).

If you are interested, I could clean it up and remove the dead code.

@avsm

This comment has been minimized.

Show comment
Hide comment
@avsm

avsm Nov 28, 2016

Member

Interesting! Is there a performance difference between the two?

Member

avsm commented Nov 28, 2016

Interesting! Is there a performance difference between the two?

@avsm

This comment has been minimized.

Show comment
Hide comment
@avsm

avsm Nov 28, 2016

Member

Also need to add tyre to the opam list or CI fails:

# ocamlfind: Package `tyre' not found
# W: Field 'pkg_tyre' is not set: Command ''/home/opam/.opam/4.02.3/bin/ocamlfind' query -format %d tyre > '/tmp/oasis-00e164.txt'' terminated with error code 2
Member

avsm commented Nov 28, 2016

Also need to add tyre to the opam list or CI fails:

# ocamlfind: Package `tyre' not found
# W: Field 'pkg_tyre' is not set: Command ''/home/opam/.opam/4.02.3/bin/ocamlfind' query -format %d tyre > '/tmp/oasis-00e164.txt'' terminated with error code 2
@Drup

This comment has been minimized.

Show comment
Hide comment
@Drup

Drup Nov 28, 2016

Member

I haven't tried! is there a nice benchmark suite somewhere?

We are doing one less call to Re.exec, there is one less string copying and there are quite a few less groups, but the extraction might be a tiny bit less efficient (more intermediate tuples), so I'm not sure how it'll turn out.

Member

Drup commented Nov 28, 2016

I haven't tried! is there a nice benchmark suite somewhere?

We are doing one less call to Re.exec, there is one less string copying and there are quite a few less groups, but the extraction might be a tiny bit less efficient (more intermediate tuples), so I'm not sure how it'll turn out.

@samoht

This comment has been minimized.

Show comment
Hide comment
@samoht

samoht Nov 28, 2016

Member

That looks pretty nice!

Member

samoht commented Nov 28, 2016

That looks pretty nice!

@avsm

This comment has been minimized.

Show comment
Hide comment
@avsm

avsm Nov 28, 2016

Member

I haven't tried! is there a nice benchmark suite somewhere?

Interesting -- we do need a simple test suite at some point; I guess Core_bench is still the best one to use here. Wonder if flambda has any effect on those intermediate return tuples...

Member

avsm commented Nov 28, 2016

I haven't tried! is there a nice benchmark suite somewhere?

Interesting -- we do need a simple test suite at some point; I guess Core_bench is still the best one to use here. Wonder if flambda has any effect on those intermediate return tuples...

@Drup

This comment has been minimized.

Show comment
Hide comment
@Drup

Drup Nov 28, 2016

Member

Wonder if flambda has any effect on those intermediate return tuples...

So, I discussed with @chambart about that ... flambda has trouble inlining just the right thing (without inlining far too many things unconditionally). So, for now, flambda doesn't really help. However, he also said: "it would be cool, but it doesn't matter in practice".

Member

Drup commented Nov 28, 2016

Wonder if flambda has any effect on those intermediate return tuples...

So, I discussed with @chambart about that ... flambda has trouble inlining just the right thing (without inlining far too many things unconditionally). So, for now, flambda doesn't really help. However, he also said: "it would be cool, but it doesn't matter in practice".

@Drup

This comment has been minimized.

Show comment
Hide comment
@Drup

Drup Nov 28, 2016

Member

(Side note: I could totally create the perfect optimal extraction code with metaocaml/macros ;)

Member

Drup commented Nov 28, 2016

(Side note: I could totally create the perfect optimal extraction code with metaocaml/macros ;)

@chambart

This comment has been minimized.

Show comment
Hide comment
@chambart

chambart Nov 28, 2016

By the way I came up with some pattern to control how to inline this kind of code correctly with the current flambda. But you probably don't want to use that. It's seriously unreadable and horribly inefficient without flambda.

By the way I came up with some pattern to control how to inline this kind of code correctly with the current flambda. But you probably don't want to use that. It's seriously unreadable and horribly inefficient without flambda.

@Drup

This comment has been minimized.

Show comment
Hide comment
@Drup

Drup Dec 2, 2016

Member

So, I started going benchmark experiments. benchmarks are in the bench branch, They are very primitive, but should cover most kind of practical uris. You can run them with tyre in the bench+tyre branch.

I was quite unsatisfied with the numbers, so I started doing some optimization work. I finally got the right idea this morning in the shower. :D

Modified tyre is Drup/tyre#12. Those are only internal changes, the API doesn't change.

┌──────────────────┬──────────┬─────────┬────────────┐
│ Name             │ Time/Run │ mWd/Run │ Percentage │
├──────────────────┼──────────┼─────────┼────────────┤
│ parsing/small    │   1.16us │ 145.00w │     45.48% │
│ parsing/ipv6     │   2.03us │ 177.00w │     79.62% │
│ parsing/complete │   2.55us │ 430.01w │    100.00% │
│ parsing/query    │   1.04us │ 134.00w │     40.57% │
│ parsing/path     │   1.79us │ 338.01w │     69.95% │
│ parsing/urn      │   1.26us │ 142.01w │     49.46% │
└──────────────────┴──────────┴─────────┴────────────┘

Released Tyre:
┌──────────────────┬────────────┬─────────┬──────────┬──────────┬────────────┐
│ Name             │   Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │
├──────────────────┼────────────┼─────────┼──────────┼──────────┼────────────┤
│ parsing/small    │ 1_097.43ns │ 230.01w │          │          │     35.23% │
│ parsing/ipv6     │ 1_909.14ns │ 258.01w │          │          │     61.29% │
│ parsing/complete │ 3_115.01ns │ 540.01w │    0.14w │    0.14w │    100.00% │
│ parsing/query    │   983.30ns │ 214.01w │          │          │     31.57% │
│ parsing/path     │ 2_124.17ns │ 444.02w │    0.11w │    0.11w │     68.19% │
│ parsing/urn      │ 1_417.99ns │ 221.01w │          │          │     45.52% │
└──────────────────┴────────────┴─────────┴──────────┴──────────┴────────────┘

Modified Tyre:
┌──────────────────┬────────────┬─────────┬────────────┐
│ Name             │   Time/Run │ mWd/Run │ Percentage │
├──────────────────┼────────────┼─────────┼────────────┤
│ parsing/small    │ 1_007.78ns │ 140.01w │     37.12% │
│ parsing/ipv6     │ 1_814.09ns │ 168.01w │     66.82% │
│ parsing/complete │ 2_714.99ns │ 405.01w │    100.00% │
│ parsing/query    │   854.43ns │ 129.00w │     31.47% │
│ parsing/path     │ 1_848.83ns │ 333.01w │     68.10% │
│ parsing/urn      │ 1_236.65ns │ 161.00w │     45.55% │
└──────────────────┴────────────┴─────────┴────────────┘
Member

Drup commented Dec 2, 2016

So, I started going benchmark experiments. benchmarks are in the bench branch, They are very primitive, but should cover most kind of practical uris. You can run them with tyre in the bench+tyre branch.

I was quite unsatisfied with the numbers, so I started doing some optimization work. I finally got the right idea this morning in the shower. :D

Modified tyre is Drup/tyre#12. Those are only internal changes, the API doesn't change.

┌──────────────────┬──────────┬─────────┬────────────┐
│ Name             │ Time/Run │ mWd/Run │ Percentage │
├──────────────────┼──────────┼─────────┼────────────┤
│ parsing/small    │   1.16us │ 145.00w │     45.48% │
│ parsing/ipv6     │   2.03us │ 177.00w │     79.62% │
│ parsing/complete │   2.55us │ 430.01w │    100.00% │
│ parsing/query    │   1.04us │ 134.00w │     40.57% │
│ parsing/path     │   1.79us │ 338.01w │     69.95% │
│ parsing/urn      │   1.26us │ 142.01w │     49.46% │
└──────────────────┴──────────┴─────────┴────────────┘

Released Tyre:
┌──────────────────┬────────────┬─────────┬──────────┬──────────┬────────────┐
│ Name             │   Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │
├──────────────────┼────────────┼─────────┼──────────┼──────────┼────────────┤
│ parsing/small    │ 1_097.43ns │ 230.01w │          │          │     35.23% │
│ parsing/ipv6     │ 1_909.14ns │ 258.01w │          │          │     61.29% │
│ parsing/complete │ 3_115.01ns │ 540.01w │    0.14w │    0.14w │    100.00% │
│ parsing/query    │   983.30ns │ 214.01w │          │          │     31.57% │
│ parsing/path     │ 2_124.17ns │ 444.02w │    0.11w │    0.11w │     68.19% │
│ parsing/urn      │ 1_417.99ns │ 221.01w │          │          │     45.52% │
└──────────────────┴────────────┴─────────┴──────────┴──────────┴────────────┘

Modified Tyre:
┌──────────────────┬────────────┬─────────┬────────────┐
│ Name             │   Time/Run │ mWd/Run │ Percentage │
├──────────────────┼────────────┼─────────┼────────────┤
│ parsing/small    │ 1_007.78ns │ 140.01w │     37.12% │
│ parsing/ipv6     │ 1_814.09ns │ 168.01w │     66.82% │
│ parsing/complete │ 2_714.99ns │ 405.01w │    100.00% │
│ parsing/query    │   854.43ns │ 129.00w │     31.47% │
│ parsing/path     │ 1_848.83ns │ 333.01w │     68.10% │
│ parsing/urn      │ 1_236.65ns │ 161.00w │     45.55% │
└──────────────────┴────────────┴─────────┴────────────┘
@Drup

This comment has been minimized.

Show comment
Hide comment
@Drup

Drup Dec 2, 2016

Member

Those benchmarks are on 4.03 without flambda, I haven't tried with flambda just yet.

Member

Drup commented Dec 2, 2016

Those benchmarks are on 4.03 without flambda, I haven't tried with flambda just yet.

@Drup

This comment has been minimized.

Show comment
Hide comment
@Drup

Drup Dec 2, 2016

Member

4.04.0+flambda without any annotation, Not great:


No Tyre -- flambda -O3:
┌──────────────────┬────────────┬─────────┬────────────┐
│ Name             │   Time/Run │ mWd/Run │ Percentage │
├──────────────────┼────────────┼─────────┼────────────┤
│ parsing/small    │ 1_121.15ns │ 137.01w │     42.00% │
│ parsing/ipv6     │ 2_102.49ns │ 169.00w │     78.77% │
│ parsing/complete │ 2_669.27ns │ 422.01w │    100.00% │
│ parsing/query    │   988.94ns │ 126.00w │     37.05% │
│ parsing/path     │ 1_916.19ns │ 330.01w │     71.79% │
│ parsing/urn      │ 1_323.07ns │ 134.00w │     49.57% │
└──────────────────┴────────────┴─────────┴────────────┘

Modified Tyre -- flambda
┌──────────────────┬──────────┬─────────┬──────────┬──────────┬────────────┐
│ Name             │ Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │
├──────────────────┼──────────┼─────────┼──────────┼──────────┼────────────┤
│ parsing/small    │   1.30us │ 231.01w │          │          │     37.38% │
│ parsing/ipv6     │   2.14us │ 259.01w │          │          │     61.26% │
│ parsing/complete │   3.49us │ 541.01w │    0.13w │    0.13w │    100.00% │
│ parsing/query    │   1.08us │ 215.01w │          │          │     30.98% │
│ parsing/path     │   2.22us │ 445.01w │          │          │     63.53% │
│ parsing/urn      │   1.56us │ 218.01w │          │          │     44.67% │
└──────────────────┴──────────┴─────────┴──────────┴──────────┴────────────┘

Modified Tyre -- flambda -03
┌──────────────────┬────────────┬─────────┬──────────┬──────────┬────────────┐
│ Name             │   Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │
├──────────────────┼────────────┼─────────┼──────────┼──────────┼────────────┤
│ parsing/small    │ 1_190.57ns │ 226.01w │          │          │     36.63% │
│ parsing/ipv6     │ 1_986.79ns │ 254.01w │          │          │     61.12% │
│ parsing/complete │ 3_250.56ns │ 536.01w │    0.13w │    0.13w │    100.00% │
│ parsing/query    │   972.47ns │ 210.01w │          │          │     29.92% │
│ parsing/path     │ 2_168.34ns │ 440.01w │          │          │     66.71% │
│ parsing/urn      │ 1_444.04ns │ 213.01w │          │          │     44.42% │
└──────────────────┴────────────┴─────────┴──────────┴──────────┴────────────┘
Member

Drup commented Dec 2, 2016

4.04.0+flambda without any annotation, Not great:


No Tyre -- flambda -O3:
┌──────────────────┬────────────┬─────────┬────────────┐
│ Name             │   Time/Run │ mWd/Run │ Percentage │
├──────────────────┼────────────┼─────────┼────────────┤
│ parsing/small    │ 1_121.15ns │ 137.01w │     42.00% │
│ parsing/ipv6     │ 2_102.49ns │ 169.00w │     78.77% │
│ parsing/complete │ 2_669.27ns │ 422.01w │    100.00% │
│ parsing/query    │   988.94ns │ 126.00w │     37.05% │
│ parsing/path     │ 1_916.19ns │ 330.01w │     71.79% │
│ parsing/urn      │ 1_323.07ns │ 134.00w │     49.57% │
└──────────────────┴────────────┴─────────┴────────────┘

Modified Tyre -- flambda
┌──────────────────┬──────────┬─────────┬──────────┬──────────┬────────────┐
│ Name             │ Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │
├──────────────────┼──────────┼─────────┼──────────┼──────────┼────────────┤
│ parsing/small    │   1.30us │ 231.01w │          │          │     37.38% │
│ parsing/ipv6     │   2.14us │ 259.01w │          │          │     61.26% │
│ parsing/complete │   3.49us │ 541.01w │    0.13w │    0.13w │    100.00% │
│ parsing/query    │   1.08us │ 215.01w │          │          │     30.98% │
│ parsing/path     │   2.22us │ 445.01w │          │          │     63.53% │
│ parsing/urn      │   1.56us │ 218.01w │          │          │     44.67% │
└──────────────────┴──────────┴─────────┴──────────┴──────────┴────────────┘

Modified Tyre -- flambda -03
┌──────────────────┬────────────┬─────────┬──────────┬──────────┬────────────┐
│ Name             │   Time/Run │ mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │
├──────────────────┼────────────┼─────────┼──────────┼──────────┼────────────┤
│ parsing/small    │ 1_190.57ns │ 226.01w │          │          │     36.63% │
│ parsing/ipv6     │ 1_986.79ns │ 254.01w │          │          │     61.12% │
│ parsing/complete │ 3_250.56ns │ 536.01w │    0.13w │    0.13w │    100.00% │
│ parsing/query    │   972.47ns │ 210.01w │          │          │     29.92% │
│ parsing/path     │ 2_168.34ns │ 440.01w │          │          │     66.71% │
│ parsing/urn      │ 1_444.04ns │ 213.01w │          │          │     44.42% │
└──────────────────┴────────────┴─────────┴──────────┴──────────┴────────────┘
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment