use circe as serde lib for json: 200

[TebaleloS]

Release notes:

• Added Circe Json serde module library for Json for serialization.
• Added 'Circe generic` in the serde dependencies.
• Added CirceJsonImplicit class and defined implicit conversion for DTO's.
• Integrated Circe with the server module.
• Removed play Json as part of the server dependencies.
• Used for Circe to implicitly convert DTO's to Json throughout the project.
• Modified the test cases to suit the Circe format in the for http.
• Replaced jsonbArrayPutUsingString with jsonbArrayPut in DoobieImplicits for both Json and Jsonb objects.
• Reimplemented SerislizationUtils using circe format.
• Replaced Json4s with circe in the agent module.
• Modified SerializationUtilsUnitTest.

closes #200

[github-actions]

JaCoCo model module code coverage report - scala 2.13.11

Overall Project	59.93% -92.06%	❌
Files changed	14.72%	❌

File	Coverage
SerializationUtils.scala	26.37% -259.34%	❌
ZonedDateTimeSerializer.scala	0% -475%	❌

[github-actions]

JaCoCo agent module code coverage report - scala 2.13.11

Overall Project	83.19%	🍏

File	Coverage
HttpDispatcher.scala	38.14%	🍏

[github-actions]

JaCoCo server module code coverage report - scala 2.13.11

Overall Project	64.14% -77.25%	🍏
Files changed	71.09%	❌

File	Coverage
CirceJsonImplicits.scala	100% -1081.25%	🍏
FlowControllerImpl.scala	100%	🍏
BaseController.scala	100%	🍏
PartitioningForDB.scala	95.45% -381.82%	❌
PartitioningControllerImpl.scala	88.89%	🍏
CheckpointFromDB.scala	84.35%	🍏
WriteCheckpoint.scala	77.52%	🍏
CheckpointControllerImpl.scala	76%	🍏
GetPartitioningMeasures.scala	71.43%	🍏
GetPartitioningAdditionalData.scala	70.82%	🍏
CreateOrUpdateAdditionalData.scala	69.79% -37.5%	❌
CreatePartitioningIfNotExists.scala	69.47% -37.89%	❌
ErrorResponse.scala	69.39% -2932.65%	❌
GetPartitioningCheckpoints.scala	54.75%	🍏
GetFlowCheckpoints.scala	0% -2.26%	❌

[TebaleloS]

Release notes:

Added Circe Json serde module library for Json for serialization.
Added 'Circe generic` in the serde dependencies.
Added CirceJsonImplicit class and defined implicit conversion for DTO's.
Integrated Circe with the server module.
Removed play Json as part of the server dependencies.
Used for Circe to implicitly convert DTO's to Json throughout the project.
Modified the test cases to suit the Circe format in the for http.
Replaced jsonbArrayPutUsingString with jsonbArrayPut in DoobieImplicits for both Json and Jsonb objects.
Reimplemented SerislizationUtils using circe format.
Replaced Json4s usage with circe in the agent module.
Modified SerializationUtilsUnitTest.

[salamonpavel]

The whole object SerializationUtils should be removed (or kept only for third party types serde implicits). It's customary to define serde implicits in companion objects - for default de/serialization. If needed one could then provide alternative implementation(s) either in direct scope where the de/serialization is performed or create yet another object for non-default implementations inside the companion object and in import into direct scope only the desired implementation (the preffered approach in my opinion).

A simple example would be:

import io.circe.{Decoder, Encoder, HCursor, Json}
import io.circe.generic.semiauto.{deriveDecoder, deriveEncoder}

case class Person(name: String, age: Int)

object Person {
  implicit val defaultPersonEncoder: Encoder[Person] = deriveEncoder[Person]
  implicit val defaultPersonDecoder: Decoder[Person] = deriveDecoder[Person]

  object JsonImplicits {
    implicit val nonDefaultPersonEncoder: Encoder[Person] = new Encoder[Person] {
      final def apply(a: Person): Json = Json.obj(
        ("name", Json.fromString(a.name)),
        ("isAdult", Json.fromBoolean(a.age >= 18))
      )
    }

    implicit val nonDefaultPersonDecoder: Decoder[Person] = new Decoder[Person] {
      final def apply(c: HCursor): Decoder.Result[Person] =
        for {
          name <- c.downField("name").as[String]
          isAdult <- c.downField("isAdult").as[Boolean]
        } yield {
          val age = if (isAdult) 18 else 17
          Person(name, age)
        }
    }
  }
}

Usage would look like this:

import io.circe.syntax._
import io.circe.parser._

val person = Person("John Doe", 20)

// Using default serde from companion (always available in direct scope without the need to import it)
val defaultJson = person.asJson // implicitly looked up from companion Person.defaultPersonEncoder

// Using non-default serde, passed implicitly
import Person.JsonImplicits.nonDefaultPersonEncoder
val nonDefaultJson = person.asJson

// Using non-default serde, passed explicitly
val nonDefaultJson = person.asJson(Person.JsonImplicits.nonDefaultPersonEncoder)

[salamonpavel]

Do we still need this?

[salamonpavel]

It's not used anywhere anymore and could be removed.

[salamonpavel]

You could define private method in DoobieImplicits object as below in order to avoid repetition.

private def circeJsonListToPGJsonArrayString(jsonList: List[Json]): String = {
    val arrayElements = jsonList.map { x =>
      // Convert to compact JSON string and escape inner quotes
      val escapedJsonString = x.noSpaces.replace("\"", "\\\"")
      // Wrap in double quotes for the array element
      s"\"$escapedJsonString\""
    }

    // Join all elements into a single string wrapped in curly braces
    arrayElements.mkString("{", ",", "}")
  }

Usage:

implicit val jsonArrayPut: Put[List[Json]] = {
      Put.Advanced
        .other[PGobject](
          NonEmptyList.of("json[]")
        )
        .tcontramap { a =>
          val o = new PGobject
          o.setType("json[]")
          o.setValue(circeJsonListToPGJsonArrayString(a))
          o
        }
    }

[salamonpavel]

This object shouldn't exist. Encoder/Decoder instances should be defined in companions of given types or in either separate objects for third party types (or in SerializationUtils) but anyway it belongs to model module, not in module package in server module.
Previously we had data serialized with different libraries in agent and server modules, that's why we had to have these "*JsonImplicits" objects as we had to define the serde implicits outside of model module. That's not the case anymore.

[salamonpavel]

Please fix the formatting.

[salamonpavel]

Please add back the layer with rollback. These are the changes you have made when we were debugging the code on the call, but it should be reverted.