2.6_testers2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name=\"top\"></a><img src=\"images/chisel_1024.png\" alt=\"Chisel logo\" style=\"width:480px;\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Module 2.6: Testers2\n",
    "**Prev: [Control Flow](2.3_control_flow.ipynb)**<br>\n",
    "**Next: [Sequential Logic](2.4_sequential_logic.ipynb)**\n",
    "\n",
    "## Motivation\n",
    "The Chisel team has been working on an improved testing framework. Imaginatively titled \"testers2\", it provides the following improvements .\n",
    "\n",
    "- suitable for both unit tests and system integration tests\n",
    "- designed for composable abstractions and layering\n",
    "- highly usable, encouraging unit tests by making it as easy, painless (avoiding boilerplate and other nonsense), and useful as possible to write them\n",
    "\n",
    "### Planned\n",
    "- ablity to target multiple backends and simulators (possibly requiring a link to Scala, if the testvector is not static, or using a limited test constructing API subset, when synthesizing to FPGA)\n",
    "- will be included in base chisel3, to avoid packaging and dependency nightmares\n",
    "\n",
    "\n",
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "val path = System.getProperty(\"user.dir\") + \"/source/load-ivy.sc\"\n",
    "interp.load.module(ammonite.ops.Path(java.nio.file.FileSystems.getDefault().getPath(path)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import chisel3._\n",
    "import chisel3.util._\n",
    "import chisel3.experimental._\n",
    "import chisel3.experimental.BundleLiterals._\n",
    "import chisel3.tester._\n",
    "import chisel3.tester.RawTester.test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">This bootcamp requires some slight differences from the imports you might see \n",
    "elsewhere for chisel. The `import chisel3.tester.RawTester.test` brings in \n",
    "version of `test(...)` below that is designed specifically for the bootcamp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# Basic Tester implementation\n",
    "\n",
    "Testers2 starts with the same basic operations as iotesters. Here's a brief summary of the basic\n",
    "functionality mapping between the older iotesters and the new testers2\n",
    "\n",
    "|        | iotesters             | testers2            |\n",
    "| :----  | :---                  | :---                |\n",
    "| poke   | poke(c.io.in1, 6)     | c.io.in1.poke(6.U)    |\n",
    "| peek   | peek(c.io.out1)       | c.io.out1.peek(6.U)   |\n",
    "| expect | expect(c.io.out1, 6)  | c.io.out1.expect(6.U) |\n",
    "| step   | step(1)               | c.io.clock.step(1)  |\n",
    "| initiate | Driver.execute(...) { c => | test(...) { c => |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by looking at the simple pass through module from 2.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "// Chisel Code, but pass in a parameter to set widths of ports\n",
    "class PassthroughGenerator(width: Int) extends Module { \n",
    "  val io = IO(new Bundle {\n",
    "    val in = Input(UInt(width.W))\n",
    "    val out = Output(UInt(width.W))\n",
    "  })\n",
    "  io.out := io.in\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the old style a simple test would look like this\n",
    "\n",
    "```scala\n",
    "val testResult = Driver(() => new Passthrough()) {\n",
    "  c => new PeekPokeTester(c) {\n",
    "    poke(c.io.in, 0)     // Set our input to value 0\n",
    "    expect(c.io.out, 0)  // Assert that the output correctly has 0\n",
    "    poke(c.io.in, 1)     // Set our input to value 1\n",
    "    expect(c.io.out, 1)  // Assert that the output correctly has 1\n",
    "    poke(c.io.in, 2)     // Set our input to value 2\n",
    "    expect(c.io.out, 2)  // Assert that the output correctly has 2\n",
    "  }\n",
    "}\n",
    "assert(testResult)   // Scala Code: if testResult == false, will throw an error\n",
    "println(\"SUCCESS!!\") // Scala Code: if we get here, our tests passed!\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test(new PassthroughGenerator(16)) { c =>\n",
    "    c.io.in.poke(0.U)     // Set our input to value 0\n",
    "    c.io.out.expect(0.U)  // Assert that the output correctly has 0\n",
    "    c.io.in.poke(1.U)     // Set our input to value 1\n",
    "    c.io.out.expect(1.U)  // Assert that the output correctly has 1\n",
    "    c.io.in.poke(2.U)     // Set our input to value 2\n",
    "    c.io.out.expect(2.U)  // Assert that the output correctly has 2\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">Just to illustrate the way testers2 advances the clock we can\n",
    "add some `step` operations to the previous examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test(new PassthroughGenerator(16)) { c =>\n",
    "    c.io.in.poke(0.U)     // Set our input to value 0\n",
    "    c.clock.step(1)    // advance the clock\n",
    "    c.io.out.expect(0.U)  // Assert that the output correctly has 0\n",
    "    c.io.in.poke(1.U)     // Set our input to value 1\n",
    "    c.clock.step(1)    // advance the clock\n",
    "    c.io.out.expect(1.U)  // Assert that the output correctly has 1\n",
    "    c.io.in.poke(2.U)     // Set our input to value 2\n",
    "    c.clock.step(1)    // advance the clock\n",
    "    c.io.out.expect(2.U)  // Assert that the output correctly has 2\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## What to notice in the above example\n",
    "\n",
    "Testers2 test method requires a bit less boiler plate. What was the `PeekPokeTester` is now\n",
    "built into the process.\n",
    "\n",
    "The `poke` and `expect` methods are now part of each individual `io` element.\n",
    "This gives important hints the the tester to make better checking of types.\n",
    "The `peek` and `step` operations are also now methods on `io` elements.\n",
    "\n",
    "Another difference is that values poked and expected are Chisel literals.\n",
    "Although pretty simple here, it also provides stronger checking in more advanced and interesting examples.\n",
    "This will be further enhanced with coming improvements in the ability to specify `Bundle` literals\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Modules with Decoupled Interfaces\n",
    "In this section we will look at some of the tester2's tools for working with `Decoupled` interfaces.\n",
    "`Decoupled` takes a chisel data type and provides it with `ready` and `valid` signals.\n",
    "Testers2 provides some nice tools for automating and reliably testing these interfaces.\n",
    "\n",
    "## A queue example\n",
    "The `QueueModule` passes through data whose type is determined by `ioType`. There are `entries` state elements inside the `QueueModule` meaning it can hold that many elements before it exerts backpressure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "case class QueueModule[T <: Data](ioType: T, entries: Int) extends MultiIOModule {\n",
    "  val in = IO(Flipped(Decoupled(ioType)))\n",
    "  val out = IO(Decoupled(ioType))\n",
    "  out <> Queue(in, entries)\n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Note the `case` class modifer is not generally required but seems to be in order for\n",
    "this example to be re-used in multiple cells in Jupyter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EnqueueNow and expectDequeueNow\n",
    "*testers2* has some built in methods for dealing with circuits with Decoupled interfaces in the IOs. In this example we will see how to insert and extract values from the `queue`. \n",
    "\n",
    "| method | description |\n",
    "| :---   | :---        |\n",
    "| enqueueNow | Add (enqueue) one element to a `Decoupled` input interface |\n",
    "| expectDequeueNow | Removes (dequeues) one element from a `Decoupled` output interface |\n",
    "---\n",
    "\n",
    "\n",
    ">Note: There is some required boiler plate `initSource`, `setSourceClock`, etc that is necessary to ensure that the `ready` and `valid` fields are\n",
    "all initialized correctly at the beginning of the test.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test(QueueModule(UInt(9.W), entries = 200)) { c =>\n",
    "    // Example testsequence showing the use and behavior of Queue\n",
    "    c.in.initSource()\n",
    "    c.in.setSourceClock(c.clock)\n",
    "    c.out.initSink()\n",
    "    c.out.setSinkClock(c.clock)\n",
    "    \n",
    "    val testVector = Seq.tabulate(200){ i => i.U }\n",
    "\n",
    "    testVector.zip(testVector).foreach { case (in, out) =>\n",
    "      c.in.enqueueNow(in)\n",
    "      c.out.expectDequeueNow(out)\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EnqueueSeq and DequeueSeq \n",
    "Now we are going to introduce two new methods that deal with enqueuing and dequeuing operations in single operations.\n",
    "\n",
    "| method | description |\n",
    "| :---   | :---        |\n",
    "| enqueueSeq | Continues to add (enqueue) elements from the `Seq` to a `Decoupled` input interface, one at a time, until the sequence is exhausted |\n",
    "| expectDequeueSeq | Removes (dequeues) elements from a `Decoupled` output interface, one at a time, and compares each one to the next element of the `Seq` |\n",
    "---\n",
    "> Note: The example below works fine but, as written, the `enqueueSeq` must finish before the `expectDequeueSeq` can begin. This example would fail if the `testVector`'s size is made larger than the queue depth, because the queue would fill up and not be able to complete the `enqueueSeq`. Try it yourself to see what the failure looks like. In the next section we will show to construct this type of test properly.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test(QueueModule(UInt(9.W), entries = 200)) { c =>\n",
    "    // Example testsequence showing the use and behavior of Queue\n",
    "    c.in.initSource()\n",
    "    c.in.setSourceClock(c.clock)\n",
    "    c.out.initSink()\n",
    "    c.out.setSinkClock(c.clock)\n",
    "    \n",
    "    val testVector = Seq.tabulate(100){ i => i.U }\n",
    "\n",
    "    c.in.enqueueSeq(testVector)\n",
    "    c.out.expectDequeueSeq(testVector)\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> One more important take away from the last section is that the functions we just saw, `enqueueNow`, \n",
    "`enqueueSeq`, `expectDequeueNow`, and `expectDequeueSeq` are not complicated special case logic in testers2.\n",
    "Rather they are examples of the kinds of harness building that testers2 encourages you to build from the testers2 primitives. To see how these methods are implemented check out [TestAdapters.scala](https://github.com/ucb-bar/chisel-testers2/blob/d199c5908828d0be5245f55fce8a872b2afb314e/src/main/scala/chisel3/tester/TestAdapters.scala)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fork and Join in testers2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section we will look at running sections of a unit test concurrently. In order to do this we will introduce two new features of testers2.\n",
    "\n",
    "| method | description |\n",
    "| :---   | :---        |\n",
    "| fork   | launches a concurrent code block, additional forks can be run concurrently to this one via the .fork appended to end of the code block of the preceeding fork |\n",
    "| join | re-unites multiple related forks back into the calling thread |\n",
    "---\n",
    "\n",
    "In the example below two `fork`s are chained together, and then `join`ed. In the first `fork` block the `enqueueSeq` will continue to add elements until exhausted. The second `fork` block will `expectDequeueSeq` on each cycle when data is available.\n",
    "\n",
    ">The threads created by fork are run in a deterministic order, largely according to their order as specified in code, and certain bug-prone operations that depend on other threads are forbidden with runtime checks. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "test(QueueModule(UInt(9.W), entries = 200)) { c =>\n",
    "    // Example testsequence showing the use and behavior of Queue\n",
    "    c.in.initSource()\n",
    "    c.in.setSourceClock(c.clock)\n",
    "    c.out.initSink()\n",
    "    c.out.setSinkClock(c.clock)\n",
    "    \n",
    "    val testVector = Seq.tabulate(300){ i => i.U }\n",
    "\n",
    "    fork {\n",
    "        c.in.enqueueSeq(testVector)\n",
    "    }.fork {\n",
    "        c.out.expectDequeueSeq(testVector)\n",
    "    }.join()\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Fork and Join with GCD\n",
    "In this section we will use the fork join methods to implement tests of *Greatest Common Denominator* **GCD**.\n",
    "Let's start by defining our IO bundles. We are going to add a bit of boiler plate here to allow us to use `Bundle` *literals*. As the comments say, it is hoped that we will soon have support for autogeneration of the literal support code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class GcdInputBundle(val w: Int) extends Bundle {\n",
    "  val value1 = UInt(w.W)\n",
    "  val value2 = UInt(w.W)\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class GcdOutputBundle(val w: Int) extends Bundle {\n",
    "  val value1 = UInt(w.W)\n",
    "  val value2 = UInt(w.W)\n",
    "  val gcd    = UInt(w.W)\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's look at a *Decoupled* version of **GCD**. We've modified it a bit here to use the `Decoupled` wrapper that adds a `ready` and a `valid` signal to the input and output Bundle. The `Flipped` wrapper takes the `Decoupled` `GcdInputBundle` which by default is created as an output and converts each field to the opposite direction (recursively). The data elements of the bundled arguments to `Decoupled` are placed in the top level field `bits`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "/**\n",
    "  * Compute GCD using subtraction method.\n",
    "  * Subtracts the smaller of registers x and y from the larger until register y is zero.\n",
    "  * value input register x is then the Gcd\n",
    "  * returns a packet of information with the two input values and their GCD\n",
    "  */\n",
    "class DecoupledGcd(width: Int) extends MultiIOModule {\n",
    "\n",
    "  val input = IO(Flipped(Decoupled(new GcdInputBundle(width))))\n",
    "  val output = IO(Decoupled(new GcdOutputBundle(width)))\n",
    "\n",
    "  val xInitial    = Reg(UInt())\n",
    "  val yInitial    = Reg(UInt())\n",
    "  val x           = Reg(UInt())\n",
    "  val y           = Reg(UInt())\n",
    "  val busy        = RegInit(false.B)\n",
    "  val resultValid = RegInit(false.B)\n",
    "\n",
    "  input.ready := ! busy\n",
    "  output.valid := resultValid\n",
    "  output.bits := DontCare\n",
    "\n",
    "  when(busy)  {\n",
    "    // during computation keep subtracting the smaller from the larger\n",
    "    when(x > y) {\n",
    "      x := x - y\n",
    "    }.otherwise {\n",
    "      y := y - x\n",
    "    }\n",
    "    when(y === 0.U) {\n",
    "      // when y becomes zero computation is over, signal valid data to output\n",
    "      output.bits.gcd := x\n",
    "      output.bits.value1 := xInitial\n",
    "      output.bits.value2 := yInitial\n",
    "      output.bits.gcd := x\n",
    "      output.valid := true.B\n",
    "      busy := false.B\n",
    "    }\n",
    "  }.otherwise {\n",
    "    when(input.valid) {\n",
    "      // valid data available and no computation in progress, grab new values and start\n",
    "      val bundle = input.deq()\n",
    "      x := bundle.value1\n",
    "      y := bundle.value2\n",
    "      xInitial := bundle.value1\n",
    "      yInitial := bundle.value2\n",
    "      busy := true.B\n",
    "      resultValid := false.B\n",
    "    }\n",
    "  }\n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our test looks pretty much the same as the earlier Queue tests.\n",
    "But there's more going on because the computation take multiple cycles so the input enqueue process is blocked as each GCD is computed.\n",
    "The good news is that test side of this is simple and consistent across different Decoupled circuits.\n",
    "\n",
    "Also introduced here is the new Chisel3 `Bundle` literal notation. Consider the line\n",
    "```scala\n",
    "new GcdInputBundle(16)).Lit(_.value1 -> x.U, _.value2 -> y.U)\n",
    "```\n",
    "`GcdInputBundle` defined above has two fields `value1` and `value2`.\n",
    "We create a bundle literal by first creating a bundle then calling its `.Lit` method.\n",
    "That method takes a variable argument list of key/value pairs, where the key (e.g. `_.value`) is the field name and the value (e.g. x.U) is a chisel hardware literal, the Scala `Int` x is converted into a Chisel `UInt` literal.\n",
    "The `_.` in front of the field name is necessary to bind the name value to the bundle internals. \n",
    "\n",
    ">This may not be the perfect notation but in extensive development discussions it was viewed as\n",
    "the best balance between minimizing boilerplate and the notational limitations available in Scala.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test(new DecoupledGcd(16)) { dut =>\n",
    "  dut.input.initSource().setSourceClock(dut.clock)\n",
    "  dut.output.initSink().setSinkClock(dut.clock)\n",
    "\n",
    "  val testValues = for { x <- 1 to 10; y <- 1 to 10} yield (x, y)\n",
    "  val inputSeq = testValues.map { case (x, y) =>\n",
    "    (new GcdInputBundle(16)).Lit(_.value1 -> x.U, _.value2 -> y.U)\n",
    "  }\n",
    "  val resultSeq = testValues.map { case (x, y) =>\n",
    "    new GcdOutputBundle(16).Lit(_.value1 -> x.U, _.value2 -> y.U, _.gcd -> BigInt(x).gcd(BigInt(y)).U)\n",
    "  }\n",
    "\n",
    "  fork {\n",
    "    dut.input.enqueueSeq(inputSeq)\n",
    "  }.fork {\n",
    "    dut.output.expectDequeueSeq(resultSeq)\n",
    "  }.join()\n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# You're done!\n",
    "\n",
    "[Return to the top.](#top)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Scala",
   "language": "scala",
   "name": "scala"
  },
  "language_info": {
   "codemirror_mode": "text/x-scala",
   "file_extension": ".scala",
   "mimetype": "text/x-scala",
   "name": "scala",
   "nbconvert_exporter": "script",
   "version": "2.12.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}