Use Spire's typeclasses. #99

Closed
avibryant opened this Issue Jan 7, 2013 · 23 comments

Comments

Projects
None yet
6 participants
Collaborator

avibryant commented Jan 7, 2013

Not trolling. The definitions of Semigroup, Monoid, VectorSpace etc in https://github.com/non/spire are not obviously incompatible with us, and it might be nicer for the broader Scala community if we could unify.

Contributor

non commented Jan 7, 2013

Hey, thanks for thinking of Spire! If you have features you need from those type classes please let me know.

Contributor

tixxit commented Jan 7, 2013

One difference is that we (Spire) don't parameterize vectors on the scalar type in Module, VectorSpace, etc. Is this a problem?

Collaborator

avibryant commented Jan 7, 2013

Spire seems to require 2.10, which probably makes it a non-starter for us for a while. Is extracting a 2.9-compatible version of the alegbra typeclasses that we could both depend on infeasible?

Contributor

non commented Jan 7, 2013

It is definitely possible to extract the algebra part. I was thinking of backporting all of Spire to 2.9.x but if that fails (or is prohibitive) we could definitely extract spire.algebra.

Contributor

non commented Jan 8, 2013

If I was working on a 2.9 version, do I need to target 2.9.0? Could I target something more recent instead? (I'm not sure if there is any significant drift, maybe 2.9.0 will be just as easy. I'm just curious.)

Collaborator

avibryant commented Jan 8, 2013

AFAIK everyone using algebird is on either 2.9.1 or 2.9.2.

Contributor

non commented Jan 8, 2013

Great, thanks!

Contributor

non commented Jan 8, 2013

OK, so https://github.com/non/spire/tree/2.9-backport is building and passing tests. I need to take some more time to make sure specialization is working, and do some documentation clean up. But it should be in good enough shape for some initial integration work.

I'll comment as soon as I have a reasonable 2.9.x milestone published.

Collaborator

johnynek commented Jan 10, 2013

Thanks for doing this. When the 2.9.2 milestone is available, let us know.

Collaborator

johnynek commented Jan 10, 2013

@non one question: we have a lot of monoids/groups etc.. On built in collections. We resolve these implicitly by putting them in the Monoids object. Will you accept the same in spire so we can get easy implicit resolution for built in java and scala types?

Contributor

non commented Jan 10, 2013

Sure. As long as it has a well-defined identity/operator (which pretty much all collections do) there should be no problem. I'll take a look at what Algebird has and go about adding them.

Collaborator

johnynek commented Jan 10, 2013

@non thanks. PS: be careful with the map-(semigroup/ring). The means of joining the keys matters a lot for performance.

Also take a look at: https://github.com/twitter/algebird/blob/develop/src/main/scala/com/twitter/algebird/GeneratedAbstractAlgebra.scala

Those are generated by a script.

We use these pretty heavily in scalding.

Contributor

tixxit commented Jan 10, 2013

I saw that and will make sure we have the necessary generated tuple-versions. Thanks!

Contributor

tixxit commented Jan 10, 2013

I'll add benchmarks in Spire for Map semigroup op too and make sure there are no perf. regressions.

Contributor

tixxit commented Jan 22, 2013

Hey, so we've implemented many more instances for Scala collections, including Map and tuples (the tuples are autogenerated on compile). We've made a tentative 0.3.0-RC1 release for 2.10 and will backport the changes shortly. Before we do the backport, would you mind taking a look and seeing if there is anything you feel we should change?

One thing that is different, is that we only do vector addition with Modules/VectorSpaces (so Vector(1, 2) + Vector(3, 4) === Vector(4, 6)). Default Monoid for all Seq-likes is concatenation (so Vector(1, 2) |+| Vector(3, 4) === Vector(1, 2, 3, 4)). You can use Module[List[Int], Int].additive to get a Monoid[List[Int]] that does vector addition. If you'd like to be able to do vector-style addition when the scalar is only in, eg, a Monoid, then we can implement an optional import for this.

The map implementation is like yours though, where, eg, Monoid[Map[K, V]] uses a Semigroup[V] to combine values.

https://github.com/non/spire/blob/2.10.0/src/main/scala/spire/algebra/map.scala
https://github.com/non/spire/blob/2.10.0/src/main/scala/spire/algebra/seq.scala
https://github.com/non/spire/blob/2.10.0/project/GenProductTypes.scala

Collaborator

johnynek commented Jan 23, 2013

I get 404 when I click the map/seq links above, and I'll take your word about the tuple generated ones (actually looking at the generated code is probably more fun).

About the Vector difference, I guess that's a fine approach too. The rule that we do element-wise operations on Vector but not on List was always a bit ad-hoc.

About the 2.9 backport... Honestly, my guess is that the internal cost of porting our code from algebird to spire will be significant enough that we probably won't get to it before we upgrade to 2.10. So, realistically, it might be better for you to save the effort of backporting if you are doing it on our behalf.

What do you think of some of the approximation structures? CountMinSketch, etc...?

Also, what is your policy about contributions? It looks like only 5 people have ever committed to the spire project. Do you welcome pull requests?

Contributor

non commented Jan 23, 2013

Now that I've (almost) completed the backport I will probably try to keep it up. There are places where Spire on 2.9 would be nice for me personally. But it's true that Spire is a lot nicer on 2.10.

I'll try to look at CountMinSketch soon, so I can say more about it. Maybe Tom has opinions?

About contributions: Spire is released under an MIT license. As long as the submitter is fine with that then we welcome pull requests from anyone.

P.S. The link for those source files actually changed to:

https://github.com/non/spire/blob/2.10.0/core/src/main/scala/spire/algebra/map.scala
https://github.com/non/spire/blob/2.10.0/core/src/main/scala/spire/algebra/seq.scala

Collaborator

avibryant commented Jan 23, 2013

To the extent that it makes sense to keep these projects separate (ie, assuming we aren't just trying to merge them), I think the approximation structures are definitely on the algebird side of the line... @tixxit and @non can comment, but it feels like Spire is more about setting up these typeclasses for existing Scala types, and Algebird is more about developing new, more exotic data structures that can also be tied into the same framework.

Contributor

tixxit commented Jan 23, 2013

Yes, I agree with @avibryant -- Spire's focus is mostly on the type classes themselves and also new numeric types (ie. Complex, Rational, Real, FPFilter, Number, etc.), so more exotic/fun data structures may be a bit out of scope (though I am looking forward to Avi's talk at NEScala).

Contributor

tixxit commented Jan 23, 2013

And, as Erik said, we definitely welcome PRs, as long as you are fine with the fairly liberal license.

mhlr commented Nov 3, 2014

So what happened? Everybody seemed in agreement, the Spire prep work was done but almost 2 years later the issue is stil open and there are no PRs.

Contributor

non commented Nov 3, 2014

So it's basically my fault. The algebra project is almost there but I never got everyone's complete agreement, and I also didn't do the Spire integration I needed.

It's on my radar and I'm hoping to get it working. Thanks for the nudge.

Collaborator

sritchie commented Oct 24, 2016

tackled by #523

sritchie closed this Oct 24, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment