Redesign

README.md

@@ -2,7 +2,7 @@
 
 [comment]: # (Start Badges)
 
-[![Join the chat at https://gitter.im/47deg/fetch](https://badges.gitter.im/47deg/fetch.svg)](https://gitter.im/47deg/fetch?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![Build Status](https://travis-ci.org/47deg/fetch.svg?branch=master)](https://travis-ci.org/47deg/fetch) [![codecov.io](http://codecov.io/github/47deg/fetch/coverage.svg?branch=master)](http://codecov.io/github/47deg/fetch?branch=master) [![Maven Central](https://img.shields.io/badge/maven%20central-0.7.3-green.svg)](https://oss.sonatype.org/#nexus-search;gav~com.47deg~fetch*) [![License](https://img.shields.io/badge/license-Apache%202-blue.svg)](https://raw.githubusercontent.com/47deg/fetch/master/LICENSE) [![Latest version](https://img.shields.io/badge/fetch-0.7.3-green.svg)](https://index.scala-lang.org/47deg/fetch) [![Scala.js](http://scala-js.org/assets/badges/scalajs-0.6.17.svg)](http://scala-js.org) [![GitHub Issues](https://img.shields.io/github/issues/47deg/fetch.svg)](https://github.com/47deg/fetch/issues)
+[![Join the chat at https://gitter.im/47deg/fetch](https://badges.gitter.im/47deg/fetch.svg)](https://gitter.im/47deg/fetch?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![Build Status](https://travis-ci.org/47deg/fetch.svg?branch=master)](https://travis-ci.org/47deg/fetch) [![codecov.io](http://codecov.io/github/47deg/fetch/coverage.svg?branch=master)](http://codecov.io/github/47deg/fetch?branch=master) [![Maven Central](https://img.shields.io/badge/maven%20central-0.6.1-green.svg)](https://oss.sonatype.org/#nexus-search;gav~com.47deg~fetch*) [![License](https://img.shields.io/badge/license-Apache%202-blue.svg)](https://raw.githubusercontent.com/47deg/fetch/master/LICENSE) [![Latest version](https://img.shields.io/badge/fetch-0.6.1-green.svg)](https://index.scala-lang.org/47deg/fetch) [![Scala.js](http://scala-js.org/assets/badges/scalajs-0.6.15.svg)](http://scala-js.org) [![GitHub Issues](https://img.shields.io/github/issues/47deg/fetch.svg)](https://github.com/47deg/fetch/issues)
 
 [comment]: # (End Badges)
 
@@ -38,77 +38,99 @@ Or, if using Scala.js (0.6.x):
 Fetch is a library for making access to data both simple & efficient. Fetch is especially useful when querying data that
 has a latency cost, such as databases or web services.
 
+## Create a runtime
+
+Since `Fetch` relies on `IO` from the `cats-effect` library, we'll need a runtime for executing our `IO` instances. This includes a `ContextShift[IO]` used for running the `IO` instances and a `Timer[IO]` that is used for scheduling, let's go ahead and create them, we'll use a `java.util.concurrent.ScheduledThreadPoolExecutor` with a couple of threads to run our fetches.
+
+```scala
+import java.util.concurrent._
+import scala.concurrent.ExecutionContext
+import cats.effect.{ IO, Timer, ContextShift }
+
+val executor = new ScheduledThreadPoolExecutor(2)
+val executionContext: ExecutionContext = ExecutionContext.fromExecutor(executor)
+
+implicit val timer: Timer[IO] = IO.timer(executionContext)
+implicit val cs: ContextShift[IO] = IO.contextShift(executionContext)
+```
+
 ## Define your data sources
 
-To tell `Fetch` how to get the data you want, you must implement the `DataSource` typeclass. Data sources have `fetchOne` and `fetchMany` methods that define how to fetch such a piece of data.
+To tell Fetch how to get the data you want, you must implement the `DataSource` typeclass. Data sources have `fetch` and `batch` methods that define how to fetch such a piece of data.
 
 Data Sources take two type parameters:
 
-1. `Identity` is a type that has enough information to fetch the data. For a users data source, this would be a user's unique ID.
-2. `Result` is the type of data we want to fetch. For a users data source, this would the `User` type.
+<ol>
+<li><code>Identity</code> is a type that has enough information to fetch the data</li>
+<li><code>Result</code> is the type of data we want to fetch</li>
+</ol>
 
 ```scala
+import cats.Parallel
 import cats.data.NonEmptyList
 
 trait DataSource[Identity, Result]{
   def name: String
-  def fetchOne(id: Identity): Query[Option[Result]]
-  def fetchMany(ids: NonEmptyList[Identity]): Query[Map[Identity, Result]]
+  def fetch(id: Identity): IO[Option[Result]]
+  def batch(ids: NonEmptyList[Identity])(
+    implicit P: Parallel[IO, IO.Par]
+  ): IO[Map[Identity, Result]]
 }
 ```
 
-We'll implement a dummy data source that can convert integers to strings. For convenience, we define a `fetchString` function that lifts identities (`Int` in our dummy data source) to a `Fetch`. 
+Returning `IO` instances from the fetch methods allows us to specify if the fetch must run synchronously or asynchronously and use all the goodies available in `cats` and `cats-effect`.
+
+We'll implement a dummy data source that can convert integers to strings. For convenience, we define a `fetchString` function that lifts identities (`Int` in our dummy data source) to a `Fetch`.
 
 ```scala
+import scala.concurrent.duration._
+import cats.Parallel
 import cats.data.NonEmptyList
 import cats.instances.list._
+import cats.syntax.all._
 import fetch._
 
 implicit object ToStringSource extends DataSource[Int, String]{
   override def name = "ToString"
-  
-  override def fetchOne(id: Int): Query[Option[String]] = {
-    Query.sync({
-      println(s"[${Thread.currentThread.getId}] One ToString $id")
-      Option(id.toString)
-    })
+
+  override def fetch(id: Int): IO[Option[String]] = {
+    IO(println(s"--> [${Thread.currentThread.getId}] One ToString $id")) >>
+    IO.sleep(10.milliseconds) >>
+    IO(println(s"<-- [${Thread.currentThread.getId}] One ToString $id")) >>
+    IO(Option(id.toString))
   }
-  override def fetchMany(ids: NonEmptyList[Int]): Query[Map[Int, String]] = {
-    Query.sync({
-      println(s"[${Thread.currentThread.getId}] Many ToString $ids")
-      ids.toList.map(i => (i, i.toString)).toMap
-    })
+
+  override def batch(ids: NonEmptyList[Int])(
+    implicit P: Parallel[IO, IO.Par]
+  ): IO[Map[Int, String]] = {
+    IO(println(s"--> [${Thread.currentThread.getId}] Batch ToString $ids")) >>
+    IO.sleep(10.milliseconds) >>
+    IO(println(s"<-- [${Thread.currentThread.getId}] Batch ToString $ids")) >>
+    IO(ids.toList.map(i => (i, i.toString)).toMap)
   }
 }
 
-def fetchString(n: Int): Fetch[String] = Fetch(n) // or, more explicitly: Fetch(n)(ToStringSource)
+def fetchString(n: Int): Fetch[String] = Fetch(n, ToStringSource)
 ```
 
 ## Creating and running a fetch
 
 Now that we can convert `Int` values to `Fetch[String]`, let's try creating a fetch.
 
 ```scala
-import fetch.syntax._
-
 val fetchOne: Fetch[String] = fetchString(1)
 ```
 
-We'll run our fetches to the ambient `Id` monad in our examples. Note that in real-life scenarios you'll want to run a fetch to a concurrency monad such as `Future` or `Task`, synchronous execution of a fetch is only supported in Scala and not Scala.js and is meant for experimentation purposes.
+Let's run it and wait for the fetch to complete, we'll use `IO#unsafeRunTimed` for testing purposes, which will run an `IO[A]` to `Option[A]` and return `None` if it didn't complete in time:
 
 ```scala
-import cats.Id
-import fetch.unsafe.implicits._
-import fetch.syntax._
+Fetch.run(fetchOne).unsafeRunTimed(5.seconds)
+// --> [92] One ToString 1
+// <-- [93] One ToString 1
+// res0: Option[String] = Some(1)
 ```
 
-Let's run it and wait for the fetch to complete:
-
-```scala
-fetchOne.runA[Id]
-// [260] One ToString 1
-// res3: cats.Id[String] = 1
-```
+As you can see in the previous example, the `ToStringSource` is queried once to get the value of 1.
 
 ## Batching
 
@@ -123,39 +145,42 @@ val fetchThree: Fetch[(String, String, String)] = (fetchString(1), fetchString(2
 When executing the above fetch, note how the three identities get batched and the data source is only queried once.
 
 ```scala
-fetchThree.runA[Id]
-// [260] Many ToString NonEmptyList(3, 1, 2)
-// res5: cats.Id[(String, String, String)] = (1,2,3)
+Fetch.run(fetchThree).unsafeRunTimed(5.seconds)
+// --> [92] Batch ToString NonEmptyList(1, 2, 3)
+// <-- [93] Batch ToString NonEmptyList(1, 2, 3)
+// res1: Option[(String, String, String)] = Some((1,2,3))
 ```
 
+Note that the `DataSource#batch` method is not mandatory, it will be implemented in terms of `DataSource#fetch` if you don't provide an implementation.
+
 ## Parallelism
 
 If we combine two independent fetches from different data sources, the fetches can be run in parallel. First, let's add a data source that fetches a string's size.
 
-This time, instead of creating the results with `Query#sync` we are going to do it with `Query#async` for emulating an asynchronous data source.
-
 ```scala
 implicit object LengthSource extends DataSource[String, Int]{
   override def name = "Length"
-  
-  override def fetchOne(id: String): Query[Option[Int]] = {
-    Query.async((ok, fail) => {
-      println(s"[${Thread.currentThread.getId}] One Length $id")
-      ok(Option(id.size))
-    })
+
+  override def fetch(id: String): IO[Option[Int]] = {
+    IO(println(s"--> [${Thread.currentThread.getId}] One Length $id")) >>
+    IO.sleep(10.milliseconds) >>
+    IO(println(s"<-- [${Thread.currentThread.getId}] One Length $id")) >>
+    IO(Option(id.size))
   }
-  override def fetchMany(ids: NonEmptyList[String]): Query[Map[String, Int]] = {
-    Query.async((ok, fail) => {
-      println(s"[${Thread.currentThread.getId}] Many Length $ids")
-      ok(ids.toList.map(i => (i, i.size)).toMap)
-    })
+  override def batch(ids: NonEmptyList[String])(
+    implicit P: Parallel[IO, IO.Par]
+  ): IO[Map[String, Int]] = {
+    IO(println(s"--> [${Thread.currentThread.getId}] Batch Length $ids")) >>
+    IO.sleep(10.milliseconds) >>
+    IO(println(s"<-- [${Thread.currentThread.getId}] Batch Length $ids")) >>
+    IO(ids.toList.map(i => (i, i.size)).toMap)
   }
 }
 
-def fetchLength(s: String): Fetch[Int] = Fetch(s)
+def fetchLength(s: String): Fetch[Int] = Fetch(s, LengthSource)
 ```
 
-And now we can easily receive data from the two sources in a single fetch. 
+And now we can easily receive data from the two sources in a single fetch.
 
 ```scala
 val fetchMulti: Fetch[(String, Int)] = (fetchString(1), fetchLength("one")).tupled
@@ -164,17 +189,21 @@ val fetchMulti: Fetch[(String, Int)] = (fetchString(1), fetchLength("one")).tupl
 Note how the two independent data fetches run in parallel, minimizing the latency cost of querying the two data sources.
 
 ```scala
-fetchMulti.runA[Id]
-// [260] One ToString 1
-// [261] One Length one
-// res7: cats.Id[(String, Int)] = (1,3)
+Fetch.run(fetchMulti).unsafeRunTimed(5.seconds)
+// --> [93] One ToString 1
+// --> [92] One Length one
+// <-- [93] One ToString 1
+// <-- [92] One Length one
+// res2: Option[(String, Int)] = Some((1,3))
 ```
 
 ## Caching
 
 When fetching an identity, subsequent fetches for the same identity are cached. Let's try creating a fetch that asks for the same identity twice.
 
 ```scala
+import cats.syntax.all._
+
 val fetchTwice: Fetch[(String, String)] = for {
   one <- fetchString(1)
   two <- fetchString(1)
@@ -184,19 +213,27 @@ val fetchTwice: Fetch[(String, String)] = for {
 While running it, notice that the data source is only queried once. The next time the identity is requested it's served from the cache.
 
 ```scala
-fetchTwice.runA[Id]
-// [260] One ToString 1
-// res8: cats.Id[(String, String)] = (1,1)
+Fetch.run(fetchTwice).unsafeRunTimed(5.seconds)
+// --> [93] One ToString 1
+// <-- [92] One ToString 1
+// res3: Option[(String, String)] = Some((1,1))
 ```
+
+
+
+
+---
+
 ## Fetch in the wild
 
 If you wish to add your library here please consider a PR to include it in the list below.
 
 [comment]: # (Start Copyright)
+
 # Copyright
 
 Fetch is designed and developed by 47 Degrees
 
 Copyright (C) 2016-2018 47 Degrees. <http://47deg.com>
 
-[comment]: # (End Copyright)
+[comment]: # (End Copyright)

build.sbt

@@ -8,7 +8,7 @@ lazy val root = project
   .in(file("."))
   .settings(name := "fetch")
   .settings(moduleName := "root")
-  .aggregate(fetchJS, fetchJVM, fetchMonixJVM, fetchMonixJS, debugJVM, debugJS, twitterJVM)
+  .aggregate(fetchJS, fetchJVM, debugJVM, debugJS)
 
 lazy val fetch = crossProject
   .in(file("."))
@@ -19,16 +19,6 @@ lazy val fetch = crossProject
 lazy val fetchJVM = fetch.jvm
 lazy val fetchJS  = fetch.js
 
-lazy val monix = crossProject
-  .in(file("monix"))
-  .dependsOn(fetch % "compile->compile;test->test")
-  .settings(name := "fetch-monix")
-  .jsSettings(sharedJsSettings: _*)
-  .crossDepSettings(commonCrossDependencies ++ monixCrossDependencies: _*)
-
-lazy val fetchMonixJVM = monix.jvm
-lazy val fetchMonixJS  = monix.js
-
 lazy val debug = (crossProject in file("debug"))
   .settings(name := "fetch-debug")
   .dependsOn(fetch)
@@ -38,22 +28,17 @@ lazy val debug = (crossProject in file("debug"))
 lazy val debugJVM = debug.jvm
 lazy val debugJS  = debug.js
 
-lazy val twitter = crossProject
-  .in(file("twitter"))
-  .settings(name := "fetch-twitter")
-  .dependsOn(fetch % "compile->compile;test->test")
-  .crossDepSettings(commonCrossDependencies ++ twitterUtilDependencies: _*)
-
-lazy val twitterJVM = twitter.jvm
-
 lazy val examples = (project in file("examples"))
   .settings(name := "fetch-examples")
   .dependsOn(fetchJVM)
   .settings(noPublishSettings: _*)
   .settings(examplesSettings: _*)
+  .settings(Seq(
+    resolvers += Resolver.sonatypeRepo("snapshots")
+  ))
 
 lazy val docs = (project in file("docs"))
-  .dependsOn(fetchJVM, fetchMonixJVM, debugJVM)
+  .dependsOn(fetchJVM, debugJVM)
   .settings(name := "fetch-docs")
   .settings(docsSettings: _*)
   .settings(noPublishSettings)