New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New generic-extras module #429

merged 2 commits into from Nov 2, 2016


None yet
2 participants
Copy link

travisbrown commented Nov 2, 2016

Adapted from the release notes I'm drafting right now.

Generic derivation overhaul

This release PR includes major changes to the implementation and arrangement of the generic module (#429). For the most part this shouldn't require any changes in your code, with one exception: derived instances for Shapeless's generic representations (records and coproducts) are no longer directly available. For example, previously you could write this:

scala> import, shapeless.record.Record
import shapeless.record.Record

scala> case class Foo(s: String, i: Int)
defined class Foo

scala> io.circe.Decoder[Foo]
res0: io.circe.Decoder[Foo] = io.circe.generic.decoding.DerivedDecoder$$anon$1@555bf2d

scala> io.circe.Decoder[Record.`'s -> String, 'i -> Int`.T]
res1: io.circe.Decoder[shapeless.::[String with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("s")],String]...

And circe would happily provide instances for both Foo and its generic representation (the Record thing). Now you only get the instance for Foo.

I'm not aware of anyone actually using this functionality, but if you are, and you need it, skip ahead to the section below on the circe-shapeless module.

There's one other minor change (fixing an oversight): the static return type of the semi-automatic deriveEncoder is now ObjectEncoder instead of just Encoder (#422).

If you notice any other changes in the behavior of circe-generic between 0.5 and 0.6 this PR, I'm considering that a bug, and would appreciate a report.

The overhaul accomplishes a couple of things. Most importantly, it makes the new configurable generic derivation (see the next section) possible, with relatively little fragility and duplication. It also has a pretty substantial impact on compile times—compiling circe's own test suite drops from around a minute and fifteen seconds to just under a minute on my machine, and the resulting class files are almost 10% smaller.


Configurable generic derivation

I've been promising people that this was right around the corner since January, and it's finally here:

import io.circe.parser.decode, io.circe.syntax._
import io.circe.generic.extras.Configuration,

sealed trait Stuff
case class Foo(thisIsAString: String, anotherField: Int = 13) extends Stuff
case class Bar(stuff: Stuff) extends Stuff

implicit val customConfig: Configuration = 

val doc = """{ "type": "Bar", "stuff": { "type": "Foo", "this_is_a_string": "abc" }}"""
val stuff: Stuff = Bar(Bar(Foo("xyz", 23)))

And then:

scala> decode[Stuff](doc)
res0: Either[io.circe.Error,Stuff] = Right(Bar(Foo(abc,13)))

scala> stuff.asJson
res2: io.circe.Json =
  "stuff" : {
    "stuff" : {
      "this_is_a_string" : "xyz",
      "another_field" : 23,
      "type" : "Foo"
    "type" : "Bar"
  "type" : "Bar"

You can even mix and match:

import io.circe.{ Decoder, ObjectEncoder }
import io.circe.parser.decode, io.circe.syntax._
import io.circe.generic.extras.Configuration

import io.circe.generic.{ semiauto => boring }
import io.circe.generic.extras.{ semiauto => fancy }

implicit val customConfig: Configuration = 

sealed trait Stuff
case class Foo(thisIsAString: String, anotherField: Int = 13) extends Stuff
case class Bar(thisIsAString: String, anotherField: Int = 13) extends Stuff

object Foo {
  implicit val decodeBar: Decoder[Bar] = fancy.deriveDecoder
  implicit val encodeBar: ObjectEncoder[Bar] = fancy.deriveEncoder

object Bar {
  implicit val decodeBar: Decoder[Bar] = boring.deriveDecoder
  implicit val encodeBar: ObjectEncoder[Bar] = boring.deriveEncoder

And then:

scala> val foo: Stuff = Foo("abc", 123)
foo: Stuff = Foo(abc,123)

scala> val bar: Stuff = Bar("xyz", 987)
bar: Stuff = Bar(xyz,987)

scala> val fooJson = foo.asJson
fooJson: io.circe.Json =
  "this_is_a_string" : "abc",
  "another_field" : 123,
  "type" : "Foo"

scala> val barJson = bar.asJson
barJson: io.circe.Json =
  "thisIsAString" : "xyz",
  "anotherField" : 987,
  "type" : "Bar"

scala> Decoder[Stuff].decodeJson(fooJson)
res4: io.circe.Decoder.Result[Stuff] = Right(Foo(abc,123))

scala> Decoder[Stuff].decodeJson(barJson)
res5: io.circe.Decoder.Result[Stuff] = Right(Bar(xyz,987))

And you don't have to worry about issues like this.

One footnote about encoding with defaults: unlike argonaut-shapeless and upickle, if you have an instance of a case class with a field value that's the same as that field's default value, it will be included in the JSON representation. This allows us to avoid issues related to equality, and seems to me more natural anyway. This is subject to change in a future version, though.

Enumeration ADTs

Currently the derived codecs for case objects result in either an empty object, or a field with an empty object value, depending on whether the value is statically typed as an ADT leaf or root:

scala> import, io.circe.syntax._
import io.circe.syntax._

scala> sealed trait Base; case object Foo extends Base
defined trait Base
defined object Foo

scala> Foo.asJson.noSpaces
res0: String = {}

scala> (Foo: Base).asJson.noSpaces
res1: String = {"Foo":{}}

This will probably always be the default behavior, since it's always safe and unambiguous, and since having derived encoders that return JSON values that aren't objects complicates things significantly.

Lots of people have asked for this to be configurable, though, because if you're using a sealed trait of case objects to represent something like an enumeration, you often want the values to be represented as strings. This is now possible with io.circe.generic.extras.semiauto:

import io.circe.{ Decoder, Encoder }
import io.circe.generic.extras.semiauto.{ deriveEnumerationDecoder, deriveEnumerationEncoder }
import io.circe.jawn._, io.circe.syntax._

sealed trait Suit extends Product with Serializable
case object Club extends Suit
case object Heart extends Suit
case object Spade extends Suit
case object Diamond extends Suit

object Suit {
  implicit val decodeSuit: Decoder[Suit] = deriveEnumerationDecoder
  implicit val encodeSuit: Encoder[Suit] = deriveEnumerationEncoder

And then:

scala> decode[Suit]("\"Club\"")
res0: Either[io.circe.Error,Suit] = Right(Club)

scala> List(Heart, Club, Diamond).asJson
res1: io.circe.Json =

If we had any case classes in our Suit ADT, the derivation would fail to compile.

Note that this must be explicitly invoked via deriveEnumerationX, and will not collide with any automatically derived decoders.

So why the "extra" part?

The approach I've taken to configuration here is at odds with the [design guidelines] for the project, which prohibit using implicit scope for things like configuration. I'm still working on what I see as a more principled approach, and don't want to commit the core io.circe.generic module to one approach or the other yet.

Right now io.circe.generic.extras is more or less a drop-in replacement for io.circe.generic with some extra functionality. It might become io.circe.generic before the 1.0 release, or it might remain around as something different, but the approach introduced here will continue to be maintained.


This comment has been minimized.

Copy link

travisbrown commented Nov 2, 2016

I'm writing this up for the release notes right now—will post a description here chopped out of that in a few minutes.


This comment has been minimized.

Copy link

codecov-io commented Nov 2, 2016

Current coverage is 81.43% (diff: 82.35%)

Merging #429 into master will decrease coverage by 0.13%

@@             master       #429   diff @@
  Files            63         78    +15   
  Lines          1959       2101   +142   
  Methods        1807       1948   +141   
  Messages          0          0          
  Branches        152        153     +1   
+ Hits           1598       1711   +113   
- Misses          361        390    +29   
  Partials          0          0          

Powered by Codecov. Last update 206a15d...8785781

@travisbrown travisbrown merged commit f04aab2 into master Nov 2, 2016

4 checks passed

codecov/patch 82.35% of diff hit (target 81.57%)
codecov/project Absolute coverage decreased by -0.13% but relative coverage increased by +0.78% compared to 206a15d
continuous-integration/travis-ci/pr The Travis CI build passed
continuous-integration/travis-ci/push The Travis CI build passed

@travisbrown travisbrown deleted the topic/generic-extras branch Nov 19, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment