Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Added Tag support and changed the Numeric <-> String Bijections to Numeric <-> String @@ Rep[Numeric] #44

Merged
merged 8 commits into from

9 participants

@jedws

not finished, still needs the String -> Option[String @@ Rep[X]] machinery and tests, but basic support is there and working:

scala> import com.twitter.bijection._
import com.twitter.bijection._

scala> Bijection[Int, String](100)
<console>:11: error: diverging implicit expansion for type com.twitter.bijection.Bijection[Int,String]
starting with method inverseOf in trait LowPriorityBijections
              Bijection[Int, String](100)
                                ^

scala> Bijection[Int, String @@ Rep[Int]](100)
res1: com.twitter.bijection.package.@@[String,com.twitter.bijection.Rep[Int]] = 100
@jedws jedws referenced this pull request
Closed

What a Bijection is not #41

@jedws

erm, needs further work yet…

@sritchie
Collaborator

Awesome, man. Looking forward to seeing this.

@johnynek
Collaborator

Looks great, as soon as the tests pass (probably with some changes) we'll merge.

jedws added some commits
@jedws jedws getting the tests compiling by adding the correct tags b382fcd
@jedws jedws had to comment these out for the moment, the StringJoinBijection is g…
…oing to require some more work to support preservation of the tags
3c0eda9
@jedws jedws tags for the standard String Bijections 0d8b571
@jedws jedws tag the URL and UUID bijections in the StringBijectionLaws f31b30b
@jedws jedws Adds "toRep[A]" syntax with import Rep._ to provide the ability to ta…
…ke a naked type and lift it into its Rep if it matches. eg.

scala> "123".toRep[Int]
res2: Option[com.twitter.bijection.package.@@[java.lang.String,com.twitter.bijection.Rep[Int]]] = Some(123)

scala> "123a".toRep[Int]
res3: Option[com.twitter.bijection.package.@@[java.lang.String,com.twitter.bijection.Rep[Int]]] = None
1a9ee52
@jedws

K, tests pass, and I've added some standard A.toRep[B] syntax that can turn a naked A value into an Option[A @@ Rep[B]].

The only regression is that you cannot now go via StringJoinBijection to get an Int <-> List[Int] and List[Int <-> String]. These are suspect bijections anyway. See StringBijectionLaws lines 57

@krishnanraman

How do you propose to handle edge cases like -

scala> "01".toInt
res1: Int = 1

scala> "001".toInt
res2: Int = 1

scala> "001.001".toFloat
res3: Float = 1.001

scala> "1.001".toFloat
res4: Float = 1.001

So I have basically shown String=>Int & String=>Float ( & a host of other numeric types ) are many-to-one functions ( for eg. {"001","1"} mapping to 1 ) because of the way the invert() is coded up in the int2string, inside NumericBijections.scala. So its not just injective on the forward & non-surjective on the return trip, the return trip is actually multivalued...which complicates things quite a bit.
A fix would be parsing to disallow 0-prefixed strings....I coded that up but it breaks other things. I guess for now we can atleast document this behavior for 0-prefixed strings.

@johnynek johnynek merged commit f52088b into twitter:develop
@johnynek
Collaborator

Thanks a ton, Jed!

This is a great approach.

Next issue is to update the readme.

@johnynek
Collaborator

@krishnanraman The cases you supply could be dealt with by returning None in the toRep method of HasRep (if I remember correctly).

Your real question, I think, is how to prove to the compiler that a String is a String @@ Rep[Int].

@krishnanraman

@johnynek I wouldn't go down that route because the set of cases is quite large.
I just came up with a few more -
scala> "1.000000000000000000000000000001".toFloat
res5: Float = 1.0
!!!
So its not just 0-prefixed, its also long strings with trailing zeros ending in a non-zero.
You can then combine those cases in various permutations !!!
So there's too many weird cases in Float => String & Double=>String
Not worth the hassle of returning None for every single one of them.
Better to simply document it in the readme for now.

@jedws

@krishnanraman I did not attempt to cover this case. This comes to the heart of the problem that these are not real bijections as the OP of #41 points out. IOW what we have now is a non-surjective injection.

However, at least for the ones I provided here it is fairly easy to make the HasRep return only the canonical String @@ Rep[X] value simply by calling toString in the rep. I have a separate PR to prepare that does just this. This does not solve but should make the current crop of X <=> String @@ Rep[X] truly bijective.

@krishnanraman

So here's a rather lame fix that takes care of 0-prefixed numbers, long floats with trailing zeros & the like.
When someone gives you an x, do a round trip on x.
If the round trip doesn't equate to x, return None.
eg. "001"->1->"1" , "1" != "001", so we return None.
This works, but essentially to do a single invert correctly, you'd have to first do an apply(invert(x))==x ...
Comments ?

@jedws

@krishnanraman that is kind of what I added in #46 except that the A => Option[A @@ B] functions canonicalize the tagged value. So, "001" => "1" tagged with Rep[Int]

That means that the relationship between say Int <=> String @@ Rep[Int] is both surjective and injective. The String => Option[String @@ Rep[Int]] isn't required to be surjective.

@retronym

Please be aware that this type tagging trick triggers some odd behaviour in scalac. I consider it to be a bit experimental. (Yeah, I know I added it to Scalaz, but our userbase has an appetite for these bleeding edge things).

In particular, def foo(a: Int @@ X) = List(a, a) will fail at runtime: scalac gets confused about whether to use a reference or primitive array to carry the varargs.

@etorreborre

That's interesting news Jason, doesn't that mean that we should better use value classes now with Scala 2.10?

@oxbowlakes

def foo(a: Int @@ X) = List(a, a)
"scalac gets confused about whether to use a reference or primitive array to carry the varargs."

What varargs?

@ijuma

To answer the question, I assume Jason is talking about the varargs parameter of List.apply.

@retronym

Yep, Value Classes are a safer alternative, although you encounter boxing when adding them the arrays or collections, so it isn't quite the same. Perhaps you can post any followup questions to the scalaz mailing list so we don't spam twitter's issue tracker any further.

@aloiscochard

@retronym is it something we could except to be fix in scalac? is there any issue open about this one?

@johnynek
Collaborator

@retronym I don't mind spamming this issue since we pulled the patch and thus problems and solutions are relevant.

Does the bug only get triggered when tagging primitives or does it also show up on classes?

We took the value class approach originally (with GZippedBytes, Base64String, etc...), but we stopped short of doing it with String since it's such a common case (maybe that's a bad argument).

I looked at the jvm code created by using the tag, and it seems to get removed completely (2.9.2), so the Bijection[Int, String @@ Rep[Int]] gets compiled to Bijection.

@retronym

Here's the bug:

https://issues.scala-lang.org/browse/SI-5183
https://gist.github.com/cace1fcb319fbb776f6e

I've only seen issues when tagging primitive types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jan 9, 2013
  1. @jedws

    Issue 41: Added support for Tagging and changed the Numeric <-> Strin…

    jedws authored
    …g Bijections to use String @@ Rep[Numeric] as proof.
  2. @jedws
Commits on Jan 10, 2013
  1. @jedws
  2. @jedws

    had to comment these out for the moment, the StringJoinBijection is g…

    jedws authored
    …oing to require some more work to support preservation of the tags
  3. @jedws
  4. @jedws
  5. @jedws

    Adds "toRep[A]" syntax with import Rep._ to provide the ability to ta…

    jedws authored
    …ke a naked type and lift it into its Rep if it matches. eg.
    
    scala> "123".toRep[Int]
    res2: Option[com.twitter.bijection.package.@@[java.lang.String,com.twitter.bijection.Rep[Int]]] = Some(123)
    
    scala> "123a".toRep[Int]
    res3: Option[com.twitter.bijection.package.@@[java.lang.String,com.twitter.bijection.Rep[Int]]] = None
  6. @jedws
This page is out of date. Refresh to see the latest.
View
9 bijection-core/src/main/scala/com/twitter/bijection/ClassBijection.scala
@@ -19,11 +19,14 @@ package com.twitter.bijection
/**
* Bijection between Class objects and string.
*/
-object ClassBijection extends Bijection[Class[_], String] {
- override def apply(k: Class[_]) = k.getName
- override def invert(s: String) = Class.forName(s)
+class ClassBijection[T] extends Bijection[Class[T], String @@ Rep[Class[T]]] {
+ override def apply(k: Class[T]) = Tag(k.getName)
+ override def invert(s: String @@ Rep[Class[T]]) = Class.forName(s).asInstanceOf[Class[T]]
}
+object ClassBijection {
+ def apply[T](): Bijection[Class[T], String @@ Rep[Class[T]]] = new ClassBijection()
+}
/**
* Bijection to cast back and forth between two types.
* Note that this uses casting and can fail at runtime.
View
48 bijection-core/src/main/scala/com/twitter/bijection/NumericBijections.scala
@@ -70,40 +70,40 @@ trait NumericBijections {
/**
* Bijections between the numeric types and string.
*/
- implicit val byte2String: Bijection[Byte, String] =
- new Bijection[Byte, String] {
- def apply(b: Byte) = b.toString
- override def invert(s: String) = s.toByte
+ implicit val byte2String: Bijection[Byte, String @@ Rep[Byte]] =
+ new Bijection[Byte, String @@ Rep[Byte]] {
+ def apply(b: Byte) = Tag(b.toString)
+ override def invert(s: String @@ Rep[Byte]) = s.toByte
}
- implicit val short2String: Bijection[Short, String] =
- new Bijection[Short, String] {
- def apply(s: Short) = s.toString
- override def invert(s: String) = s.toShort
+ implicit val short2String: Bijection[Short, String @@ Rep[Short]] =
+ new Bijection[Short, String @@ Rep[Short]] {
+ def apply(s: Short) = Tag(s.toString)
+ override def invert(s: String @@ Rep[Short]) = s.toShort
}
- implicit val int2String: Bijection[Int, String] =
- new Bijection[Int, String] {
- def apply(i: Int) = i.toString
- override def invert(s: String) = s.toInt
+ implicit val int2String: Bijection[Int, String @@ Rep[Int]] =
+ new Bijection[Int, String @@ Rep[Int]] {
+ def apply(i: Int) = Tag(i.toString)
+ override def invert(s: String @@ Rep[Int]) = s.toInt
}
- implicit val long2String: Bijection[Long, String] =
- new Bijection[Long, String] {
- def apply(l: Long) = l.toString
- override def invert(s: String) = s.toLong
+ implicit val long2String: Bijection[Long, String @@ Rep[Long]] =
+ new Bijection[Long, String @@ Rep[Long]] {
+ def apply(l: Long) = Tag(l.toString)
+ override def invert(s: String @@ Rep[Long]) = s.toLong
}
- implicit val float2String: Bijection[Float, String] =
- new Bijection[Float, String] {
- def apply(f: Float) = f.toString
- override def invert(s: String) = s.toFloat
+ implicit val float2String: Bijection[Float, String @@ Rep[Float]] =
+ new Bijection[Float, String @@ Rep[Float]] {
+ def apply(f: Float) = Tag(f.toString)
+ override def invert(s: String @@ Rep[Float]) = s.toFloat
}
- implicit val double2String: Bijection[Double, String] =
- new Bijection[Double, String] {
- def apply(d: Double) = d.toString
- override def invert(s: String) = s.toDouble
+ implicit val double2String: Bijection[Double, String @@ Rep[Double]] =
+ new Bijection[Double, String @@ Rep[Double]] {
+ def apply(d: Double) = Tag(d.toString)
+ override def invert(s: String @@ Rep[Double]) = s.toDouble
}
/**
View
76 bijection-core/src/main/scala/com/twitter/bijection/Rep.scala
@@ -0,0 +1,76 @@
+package com.twitter.bijection
+
+import java.net.{ MalformedURLException, URL }
+import java.util.UUID
+
+/**
+ * Type tag used to indicate that an instance of a type such as String
+ * contains a valid representation of another type, such as Int or URL.
+ */
+trait Rep[A]
+
+/**
+ * Useful HasRep
+ */
+object Rep {
+ // TODO make implicit class in 2.10
+ implicit def ToRepOpsPimp[A](a: A) = new ToRepOps(a)
+
+ /**
+ * Adds toRep[B] syntax to elements of type A if there is an implicit HasRep[A, B] in scope.
+ */
+ class ToRepOps[A](a: A) {
+ def toRep[B](implicit ev: HasRep[A, B]): Option[A @@ Rep[B]] =
+ ev.toRep(a)
+ }
+
+ implicit val StringHasIntRep = new HasRep[String, Int] {
+ def toRep(s: String) = rep(s) { _.toInt }
+ }
+
+ implicit val StringHasLongRep = new HasRep[String, Long] {
+ def toRep(s: String) = rep(s) { _.toLong }
+ }
+
+ implicit val StringHasByteRep = new HasRep[String, Byte] {
+ def toRep(s: String) = rep(s) { _.toByte }
+ }
+
+ implicit val StringHasShortRep = new HasRep[String, Short] {
+ def toRep(s: String) = rep(s) { _.toShort }
+ }
+
+ implicit val StringHasFloatRep = new HasRep[String, Float] {
+ def toRep(s: String) = rep(s) { _.toFloat }
+ }
+
+ implicit val StringHasDoubleRep = new HasRep[String, Double] {
+ def toRep(s: String) = rep(s) { _.toDouble }
+ }
+
+ implicit val StringHasURLRep = new HasRep[String, URL] {
+ def toRep(s: String) = rep(s) { new URL(_) }
+ }
+
+ implicit val StringHasUUIDRep = new HasRep[String, UUID] {
+ def toRep(s: String) = rep(s) { UUID.fromString }
+ }
+
+ private val catching =
+ scala.util.control.Exception.catching(
+ classOf[NumberFormatException]
+ , classOf[MalformedURLException]
+ , classOf[IllegalArgumentException]
+ )
+
+ // catch and return option
+ private def rep[A, B](a: A)(partial: A => B): Option[A @@ Rep[B]] =
+ catching.opt(partial(a)) map (_ => Tag(a))
+}
+
+/**
+ * Type class for summoning the function that can check whether the instance can be tagged with Rep
+ */
+trait HasRep[A, B] {
+ def toRep(a: A): Option[A @@ Rep[B]]
+}
View
22 bijection-core/src/main/scala/com/twitter/bijection/StringBijections.scala
@@ -31,26 +31,26 @@ trait StringBijections {
}
// Some bijections with string from standard java/scala classes:
- implicit val url2String: Bijection[URL, String] =
- new Bijection[URL, String] {
- def apply(u: URL) = u.toString
- override def invert(s: String) = new URL(s)
+ implicit val url2String: Bijection[URL, String @@ Rep[URL]] =
+ new Bijection[URL, String @@ Rep[URL]] {
+ def apply(u: URL) = Tag(u.toString)
+ override def invert(s: String @@ Rep[URL]) = new URL(s)
}
implicit val symbol2String: Bijection[Symbol, String] =
new Bijection[Symbol, String] {
def apply(s: Symbol) = s.name
- override def invert(s: String) = Symbol(s)
+ override def invert(s: String ) = Symbol(s)
}
- implicit val uuid2String: Bijection[UUID, String] =
- new Bijection[UUID, String] {
- def apply(uuid: UUID) = uuid.toString
- override def invert(s: String) = UUID.fromString(s)
+ implicit val uuid2String: Bijection[UUID, String @@ Rep[UUID]] =
+ new Bijection[UUID, String @@ Rep[UUID]] {
+ def apply(uuid: UUID) = Tag(uuid.toString)
+ override def invert(s: String @@ Rep[UUID]) = UUID.fromString(s)
}
- implicit def class2String[T]: Bijection[Class[T], String] =
- CastBijection.of[Class[T], Class[_]] andThen ClassBijection
+ implicit def class2String[T]: Bijection[Class[T], String @@ Rep[Class[T]]] =
+ ClassBijection[T]()
}
object StringCodec extends StringBijections
View
20 bijection-core/src/main/scala/com/twitter/package.scala
@@ -25,4 +25,24 @@ package com.twitter
* libraries (Bijection[MyTrait, YourTrait]) and many other purposes.
*/
package object bijection {
+
+ /**
+ * Tagging infrastructure.
+ */
+ type Tagged[T] = { type Tag = T }
+
+ /**
+ * Tag a type `T` with `Tag`. The resulting type is a subtype of `T`.
+ *
+ * The resulting type is used to discriminate between type class instances.
+ */
+ type @@[T, Tag] = T with Tagged[Tag]
+
+ private[bijection] object Tag {
+ @inline def apply[A, T](a: A): A @@ T = a.asInstanceOf[A @@ T]
+
+ def subst[A, F[_], T](fa: F[A]): F[A @@ T] = fa.asInstanceOf[F[A @@ T]]
+
+ def unsubst[A, F[_], T](fa: F[A @@ T]): F[A] = fa.asInstanceOf[F[A]]
+ }
}
View
8 bijection-core/src/test/scala/com/twitter/bijection/CollectionLaws.scala
@@ -23,9 +23,9 @@ import org.scalacheck.Prop._
object CollectionLaws extends Properties("Collections")
with BaseProperties {
- implicit val listToVector = Bijection.toContainer[Int, String, List[Int], Vector[String]]
- property("round trip List[Int] -> Vector[String]") = roundTrips[List[Int], Vector[String]]()
+ implicit val listToVector = Bijection.toContainer[Int, String @@ Rep[Int], List[Int], Vector[String @@ Rep[Int]]]
+ property("round trip List[Int] -> Vector[String @@ Rep[Int]]") = roundTrips[List[Int], Vector[String @@ Rep[Int]]]()
- implicit val setToIter = Bijection.toContainer[Int, String, Set[Int], Iterable[String]]
- property("round trip Set[Int] -> Iterable[String]") = roundTrips[Set[Int], Iterable[String]]()
+ implicit val setToIter = Bijection.toContainer[Int, String @@ Rep[Int], Set[Int], Iterable[String @@ Rep[Int]]]
+ property("round trip Set[Int] -> Iterable[String @@ Rep[Int]]") = roundTrips[Set[Int], Iterable[String @@ Rep[Int]]]()
}
View
14 bijection-core/src/test/scala/com/twitter/bijection/NumericBijectionLaws.scala
@@ -41,12 +41,12 @@ with BaseProperties {
property("round trips float -> jfloat") = roundTrips[Float, JFloat]()
property("round trips double -> jdouble") = roundTrips[Double, JDouble]()
- property("round trips byte -> string") = roundTrips[Byte, String]()
- property("round trips short -> string") = roundTrips[Short, String]()
- property("round trips int -> string") = roundTrips[Int, String]()
- property("round trips long -> string") = roundTrips[Long, String]()
- property("round trips float -> string") = roundTrips[Float, String]()
- property("round trips double -> string") = roundTrips[Double, String]()
+ property("round trips byte -> string") = roundTrips[Byte, String @@ Rep[Byte]]()
+ property("round trips short -> string") = roundTrips[Short, String @@ Rep[Short]]()
+ property("round trips int -> string") = roundTrips[Int, String @@ Rep[Int]]()
+ property("round trips long -> string") = roundTrips[Long, String @@ Rep[Long]]()
+ property("round trips float -> string") = roundTrips[Float, String @@ Rep[Float]]()
+ property("round trips double -> string") = roundTrips[Double, String @@ Rep[Double]]()
property("round trips short -> Array[Byte]") = roundTrips[Short, Array[Byte]]()
property("round trips int -> Array[Byte]") = roundTrips[Int, Array[Byte]]()
@@ -58,6 +58,6 @@ with BaseProperties {
property("round trips Long -> Date") = roundTrips[Long, java.util.Date]()
property("as works") = forAll { (i: Int) =>
- i.as[String] == i.toString && (i.toString.as[Int] == i)
+ i.as[String @@ Rep[Int]] == Tag[String, Rep[Int]](i.toString) && (Tag[String, Rep[Int]](i.toString).as[Int] == i)
}
}
View
12 bijection-core/src/test/scala/com/twitter/bijection/StringBijectionLaws.scala
@@ -34,7 +34,7 @@ with BaseProperties {
for( l <- choose(-100L, 100L);
u <- choose(-100L, 100L)) yield (new UUID(l,u))
}
- property("round trip UUID -> String") = roundTrips[UUID, String]()
+ property("round trip UUID -> String") = roundTrips[UUID, String @@ Rep[UUID]]()
def toUrl(s: String): Option[URL] =
try { Some(new URL("http://" + s + ".com")) }
catch { case _ => None }
@@ -45,7 +45,7 @@ with BaseProperties {
.filter { _.isDefined }
.map { _.get }
}
- property("round trip URL -> String") = roundTrips[URL, String]()
+ property("round trip URL -> String") = roundTrips[URL, String @@ Rep[URL]]()
property("rts through StringJoinBijection") =
forAll { (sep: String, xs: List[String]) =>
@@ -54,9 +54,9 @@ with BaseProperties {
(!iter.exists(_.contains(sep))) ==> (iter == rt(iter)(sjBij))
}
- implicit val listOpt = StringJoinBijection.viaContainer[Int, List[Int]]()
- property("viaCollection List[Int] -> Option[String]") = roundTrips[List[Int], Option[String]]()
- implicit val listStr = StringJoinBijection.nonEmptyValues[Int, List[Int]]()
- property("viaCollection List[Int] -> String") = roundTrips[List[Int], String]()
+ // implicit val listOpt = StringJoinBijection.viaContainer[Int, List[Int]]()
+ // property("viaCollection List[Int] -> Option[String]") = roundTrips[List[Int], Option[String @@ Rep[List[Int]]]]()
+ // implicit val listStr = StringJoinBijection.nonEmptyValues[Int, List[Int]]()
+ // property("viaCollection List[Int] -> String") = roundTrips[List[Int], String @@ Rep[List[Int]]]()
}
View
6 bijection-core/src/test/scala/com/twitter/bijection/TupleBijectionLaws.scala
@@ -30,9 +30,9 @@ import org.scalacheck.Prop.forAll
object TupleBijectionLaws extends Properties("TupleBijections")
with BaseProperties {
- property("round trips (Int,Long) -> (String,String)") = roundTrips[(Int,Long), (String,String)]()
+ property("round trips (Int,Long) -> (String,String)") = roundTrips[(Int,Long), (String @@ Rep[Int],String @@ Rep[Long])]()
property("round trips (Int,Long,String) -> (String,String,String)") =
- roundTrips[(Int,Long,String), (String,String,String)]()
+ roundTrips[(Int,Long,String), (String @@ Rep[Int],String @@ Rep[Long],String)]()
property("round trips (Int,Long,String,Long) -> (String,String,String,Array[Byte])") =
- roundTrips[(Int,Long,String,Long), (String,String,String,Array[Byte])]()
+ roundTrips[(Int,Long,String,Long), (String @@ Rep[Int],String @@ Rep[Long],String,Array[Byte])]()
}
Something went wrong with that request. Please try again.