specialization cost

[johnynek]

I am pretty dubious the specialization on Boolean and Byte is going to help at all, but it is going to increase the method count of all the traits that specialize in that way, unless I am mistaken.

The reason I don't think it is needed is that boxing a Boolean is just an if/else without allocation, and boxing a Byte is just looking up a 256 length static array (which should be in cache if you are doing this a bunch, I would expect).

It might be nice to have some kind of linter to check that we specialize all the traits the same way (just Short, Int, Long, Double, Float?).

[non]

I will run some benchmarks right now to try to quantify what the cost (if any) is, and will report back here with the results.

(Your argument makes sense -- I believe in "trust but verify".)

[non]

So I just modified one of the Spire benchmarks we use to test specialization (spire.benchmark.AddBenchmarks) which is not normally specialized for Byte. I compared the cost of a direct implementation, a non-specialized implementation, and an implementation that was specialized on Byte.

The results were:

[info]             benchmark     us linear runtime
[info]        AddBytesDirect  117.5 =
[info]       AddBytesGeneric  850.7 ====
[info]      AddBytesFullspec  118.2 =

This appears to indicate an 8x slowdown for bytes. This doesn't prove that instances are being allocated: it's possible that this is the cost of using a lookup table. However it does indicate a penalty that we will pay for removing specialization.

I'm still open to discussing whether this is worth it or not.

[tixxit]

So, my personal opinion is that I don't see a ton of value in specializing below Int. Namely, I propose sticking with Int, Long, Float, and Double unless we have a compelling case otherwise. The cost of specialization especially blows up when classes have 2 or more specialized parameters. Scala's Function1 and Function2 don't go beyond these types either.

[VladUreche]

How about using miniboxing to achieve specialization? I feel it's reasonably stable now, and with the 0.4 release it should address all the quirks of specialization (quirks metabug here), allowing you to simplify the code. And it would certainly reduce the bytecode size, as benchmarks have shown.

Also, if you need any help or bugfixes to make it work, I'd be glad to lend a hand. ;)

[non]

Hi @VladUreche! I have been waiting for you to recommend something like this! That's great news!

I think as a first step I might publish an alternate miniboxed jar in addition to the "standard" one. Do you think that makes sense? That way, users could choose whether they wanted to introduce miniboxing into their build or not.

[tixxit]

Hrmm... @non I think I'll take a stab at that idea... Don't think it'll be too bad.

[non]

@tixxit Awesome, thanks!

[VladUreche]

I think as a first step I might publish an alternate miniboxed jar in addition to the "standard" one. Do you think that makes sense? That way, users could choose whether they wanted to introduce miniboxing into their build or not.

@non: Yep, that sounds very good! Having released jars built with miniboxing and specialization side by side will not only provide choice to the users, but will also allow us to benchmark them side by side and iron out any performance regressions that may occur. So yes, let's do that!

@tixxit: Awesome! Please let me know if you encounter any issues, I will fix them asap.

[johnynek]

See #30

[non]

After a quick poll of Spire users (at least those who are on the ML and responded) it seems like no one is really relying on Byte or Short.

My concrete proposal to move forward would be to use @sp(Int, Long, Float, Double) in all cases except for Eq, PartialOrder, and Order. I'd like to leave those alone since they are much more general.

What do you think?

[tixxit]

👍

[johnynek]

+1