TaggedUnion implementation

[ronan-lashermes]

Work-in-progress for #1214

Context, Motivation & Description

This is a first attempt at a TaggedUnion class leveraging the new union feature.
This is not targeting a merge for now, but I would welcome feedback on this implementation.
Requires more tests / polish.

Checklist

• Unit tests were added
• API changes are or will be documented:
  • using Scaladoc comments: /** */?
  • on RTD?
  • thanks to a new tracking issue on RTD?

Usage

A tagged union is a Sum Type. Use it as a Bundle, but a tagged union value is one value among the descriptors.

case class RWRequest() extends TaggedUnion {
    val read = ReadRequest()
    val write = WriteRequest()
}

val rw = RWRequest()
// rw is either read or write

There are two main methods to deal with tagged unions.

choose select the union variant.

rw.choose(rw.read) {
  r: ReadRequest => {
     //here you can write to the tagged union, seeing it as a ReadRequest
    r := ...
  }
}

The first argument selects the variant, the second provides an object with the correct variant type for your usage.

among to "read" the union.

rw.among {
  case (rw.read, r: ReadRequest) => {
    // you can read the tagged union as a ReadRequest
    ... := r
  }
  case (rw.write: w: WriteRequest) => {
    // you can read the tagged union as a WriteRequest
    ... := w
  }
}

Why do you need a 2-tuple in among ? To deal with tagged union with multiple variants of the same type.

Pros/cons

Pros:

• finally a TaggedUnion : you can leverage the type system for a correct implementation of a choice among variants

Cons:

• ergonomy is a bit complex. I did not find any simpler way to do it while staying correct.
• naming is not working. The union signals are not properly named and I did not find a way to overcome that.
• assume that all variants have the same direction. This is not properly tested, some assert are missing here I think.
• often requires a default value for the union payload => need for properly placed assignDontCare that may be hidden inside the TaggedUnion implementation.
• probably some edge cases not properly dealt with (e.g. TaggedUnion without any variant)

Longer example

Here is a simple example to describe a read/write port. You should be able to copy/write this example and get it elaborated.

package taggedunion

import spinal.core._
import spinal.lib._

case class ReadRequest() extends Bundle {
    val address = UInt(32 bits)
}

case class ReadResponse() extends Bundle {
    val value = Bits(32 bits)
}

case class WriteRequest() extends Bundle {
    val address = UInt(32 bits)
    val value = Bits(32 bits)
}

case class RWRequest() extends TaggedUnion {
    val read = ReadRequest()
    val write = WriteRequest()
}

case class RWResponse() extends TaggedUnion {
    val read = ReadResponse()
}



case class MemoryController() extends Component {
    val io = new Bundle {
        val request = master(Stream(RWRequest()))
        val response = slave(Flow(RWResponse()))

        val doReq = in Bool()
        val rw = in Bool()

        val answer = out Bits(32 bits)
    }

    io.request.payload.assignDontCare()


    io.request.valid := False
    when(io.doReq) {
        io.request.valid := True
        when(io.rw) { //bad !
            io.request.payload.choose(io.request.payload.write) {
                wReq: WriteRequest => {
                    wReq.address := 2
                    wReq.value := 0
                }
            }
        }
        .otherwise {
            io.request.payload.choose(io.request.payload.read) {
                rReq: ReadRequest => {
                    rReq.address := 1
                }
            }
        }
    }

    io.answer.assignDontCare()

    when(io.response.valid) {
        io.response.payload.among {
            case (io.response.payload.read, rRes: ReadResponse) => {
                io.answer := rRes.value
            }
        }
    }
    
}

object MemoryControllerVerilog extends App {
    Config.spinal.generateVerilog(MemoryController())
}

[Dolu1990]

Nice :D

I pushed fixes for the naming and the HardType.unionSeq.

About the choose, there is one less verbose alternative syntax which could be :

    def choose(callback: PartialFunction[Data, Unit]): Unit = {
        for((name, k) <- unionDescriptors if callback.isDefinedAt(k)){
            val variant = tagUnionDescriptors(name)
            this.tag := variant
            callback(this.unionPayload.aliasAs(k))
            return
        }
        SpinalError(s"not a member of this TaggedUnion")
    }

   Which can be used as 
   
      rw.choose { case r: WriteRequest =>
        r.y := 255
      }

I think partialfunction it can also be used for the among to produce :

rw.among {
  case r: ReadRequest => {
    // you can read the tagged union as a ReadRequest
    ... := r
  }
  case w: WriteRequest => {
    // you can read the tagged union as a WriteRequest
    ... := w
  }
}

So, to be clear, it isn't a replacement of the original among, as it only rely on type, which may not be enough in all cases.
But still a nice short cut for many of cases.

[ronan-lashermes]

Removing explicitly naming the variant in choose and among was my first version but may raise issues :

case class SameTypeUnion() extends TaggedUnion {
    val a1 = TypeA()
    val a2 = TypeA()
}

is not usable anymore. But this case could be replaced by

case class VariantBundle extends Bundle {
    val a = TypeA()
    val variant = SpinalEnum() // a1 or a2
}

Yet, it implies that TaggedUnion must refuse multiple variants with the same type.
More problematic :

case class AlmostSameTypeUnion() extends TaggedUnion {
    val a1 = UInt(8 bits)
    val a2 = UInt(16 bits)
}

implies to distinguish between the two variants inside the case.

[ronan-lashermes]

I have to check if we can propose the two possibilities (with and without explicit variants)

[ronan-lashermes]

Ok, new versions for among and choose :

• renaming the old versions with explicit variants : choose -> chooseVariant and among -> amongVariants
• now choose and among are based on type only and will throw an error if incorrect usage (E.g. cannot be used with a TaggedUnionn with several variants with same type).

when(io.doReq) {
        io.request.valid := True
        when(io.rw) {
            io.request.payload.choose {
                case wReq: WriteRequest => {
                    wReq.address := 2
                    wReq.value := 0
                }
            }
        }
        .otherwise {
            io.request.payload.chooseVariant(io.request.payload.read) {
                rReq: ReadRequest => {
                    rReq.address := 1
                }
            }
        }
    }

when(io.response.valid) {
        io.response.payload.among {
            case rRes: ReadResponse => {
                io.answer := rRes.value
            }
            case wRes: WriteResponse => {
                when(wRes.success) {
                    io.answer := 0
                }
                .otherwise {
                    io.answer := 1
                }
            }
        }
    }

Different method names XXX and XXXVariant are required because the type system has trouble to differentiate them otherwise.

[Dolu1990]

@ronan-lashermes The TaggedUnionTesterCocotbBoot failed ?

[ronan-lashermes]

I had to interrupt my work. But I am still working on it as soon as I have a little time (coming week should be ok).

[Dolu1990]

No worries ^^

[ronan-lashermes]

Ok, tester fixed and first documentation draft proposed.
Still not sure about :

• names "choose" and "among"
• we need to assign (e.g. assignDontCare) outside of the methods choose and among to avoid latches, there is something to improve here.

[Dolu1990]

def choose(callback: PartialFunction[Data, Unit]): Unit

I'm thinking, it would also probably possible to have something like :

import scala.reflect.{ClassTag, classTag}
def choose[T <: Data : ClassTag](body : T => Unit) : Unit = ...

myTaggedUnion.choose[MyElementType].xyz

probably same for ammong

names "choose" and "among"

to be do thing in a more scala way, it could be :

• among become apply, so you can then silently use it
• choose become update

[ronan-lashermes]

Thanks for the suggestions.
Now both choose and chooseVariant are named update (solved the type system issue) :

when(io.rw) {
            io.request.payload.update {
                wReq: WriteRequest => {
                    wReq.address := 2
                    wReq.value := 0
                }
            }
        }
        .otherwise {
            io.request.payload.update(io.request.payload.read) {
                rReq: ReadRequest => {
                    rReq.address := 1
                }
            }
        }

And both among and amongVariants has been merged into a more accepting apply:

case class SameTypeUnion() extends TaggedUnion {
    val a1 = TypeA()
    val a2 = TypeA()
    val b = TypeB()
}

val tu = SameTypeUnion()
tu {
  case bVal: TypeB => {
    // read tu as of TypeB when b variant selected
    v := bVal.valid
  }
  case (tu.a1, aVal: TypeA) => {
  //...
  }
  case (tu.a2, aVal: TypeA) => {
  //...
  }
}

I still need to write the tests though.

[Dolu1990]

Cool thanks :)

[andreasWallner]

Two quick question (and a nitpick):

• How would I currently init a Reg(TaggedUnion) to have a specific reset value? Did I overlook that in the code?
• would it be possible to have a way of just comparing the active member of the union in something like when(myUnion.is(ta.a1))? I don't see any issues with that (not saying you have to add it now, just whether we can add it in the future)

[andreasWallner]

From the looks of it this (and some of the next members) should be val instead of var? I don't see us reassigning them

[ronan-lashermes]

Yep, you are right the first 3 vars should be vals

[ronan-lashermes]

So, last commit added the is(data) function as you mentioned, good idea.

[ronan-lashermes]

For the init, this is where I may need a little guidance.
Is there a special trait to implement for that ? Where can I find an example ?
Related is the need to assign outside of update (even if with assignDontCare) or we get latches.

[ronan-lashermes]

So I did try to write some tests, but I can’t get it to pass locally. But I get no meaningful log, the error could be a missing dependency for all I know...
Is there a guide on how to setup tests ? (I mainly tried to reverse engineer the CI and the other tests).

[Dolu1990]

So I did try to write some tests, but I can’t get it to pass locally. But I get no meaningful log, the error could be a missing dependency for all I know... Is there a guide on how to setup tests ? (I mainly tried to reverse engineer the CI and the other tests).

Mostly, you need iverilog, ghdl, cocotb, verilator installed.
But often the version of those tools may have breakdown XD

The doc about those is the CI, unfortunatly.

Where things go bad ?

[ronan-lashermes]

I still missing dependencies (python packages) for my local tests, but I gradually improve on that. It’s just a bit long with trial and errors.
I am more advanced using this project CI. I just fixed an issue with the VHDL tester root name.
And for the verilog tester I get

MODULE=TaggedUnionTester TESTCASE= TOPLEVEL=TaggedUnionTester TOPLEVEL_LANG=verilog \
         /usr/bin/vvp -M /usr/local/lib/python3.10/dist-packages/cocotb/libs -m libcocotbvpi_icarus   sim_build/sim.vvp 
     -.--ns INFO     gpi                                ..mbed/gpi_embed.cpp:76   in set_program_name_in_venv        Did not detect Python virtual environment. Using system-wide Python interpreter
     -.--ns INFO     gpi                                ../gpi/GpiCommon.cpp:101  in gpi_print_registered_impl       VPI registered
     0.00ns INFO     cocotb                             Running on Icarus Verilog version 11.0 (stable)
     0.00ns INFO     cocotb                             Running tests with cocotb v1.7.2 from /usr/local/lib/python3.10/dist-packages/cocotb
     0.00ns INFO     cocotb                             Seeding Python random module with 1702891737
     0.00ns INFO     cocotb.regression                  Found test TaggedUnionTester.test1
     0.00ns INFO     cocotb.regression                  running test1 (1/1)
     0.00ns INFO     cocotb.TaggedUnionTester           Cocotb test boot TaggedUnion
     1.00ns INFO     cocotb.regression                  test1 failed
                                                        Traceback (most recent call last):
                                                          File "/__w/SpinalHDL/SpinalHDL/tester/src/test/python/spinal/TaggedUnionTester/TaggedUnionTester.py", line 47, in test1
                                                            assertEquals(ref.io_o_unionPayload, dut.io_o_unionPayload, "io_o_unionPayload")
                                                          File "/__w/SpinalHDL/SpinalHDL/tester/src/test/python/cocotblib/misc.py", line 47, in assertEquals
                                                            if int(a) != int(b):
                                                          File "/usr/local/lib/python3.10/dist-packages/cocotb/handle.py", line 915, in __int__
                                                            return int(self.value)
                                                          File "/usr/local/lib/python3.10/dist-packages/cocotb/binary.py", line 552, in __int__
                                                            return self.integer
                                                          File "/usr/local/lib/python3.10/dist-packages/cocotb/binary.py", line 372, in integer
                                                            return self._convert_from(self._str)
                                                          File "/usr/local/lib/python3.10/dist-packages/cocotb/binary.py", line 252, in _convert_from_unsigned
                                                            return int(x.translate(_resolve_table), 2)
                                                          File "/usr/local/lib/python3.10/dist-packages/cocotb/binary.py", line 84, in __missing__
                                                            return self.resolve_x(key)
                                                          File "/usr/local/lib/python3.10/dist-packages/cocotb/binary.py", line 61, in resolve_error
                                                            raise ValueError(
                                                        ValueError: Unresolvable bit in binary string: 'x'
     1.00ns INFO     cocotb.regression                  **************************************************************************************
                                                        ** TEST                          STATUS  SIM TIME (ns)  REAL TIME (s)  RATIO (ns/s) **
                                                        **************************************************************************************
                                                        ** TaggedUnionTester.test1        FAIL           1.00           0.00        707.29  **
                                                        **************************************************************************************
                                                        ** TESTS=1 PASS=0 FAIL=1 SKIP=0                  1.00           0.19          5.36  **
                                                        **************************************************************************************

Still trying to make sense of this error message.

[Dolu1990]

Ahhh you did you testbench in cocotb ?
It mean that you are trying to convert a hardware value which contains 'X' into a python integer, but X can't be translated

Have you seen the following for tooling install ? :

SpinalHDL/tools.sh

Line 4 in 68b6158

sudo apt install -y git make autoconf build-essential flex libfl-dev bison help2man # First time prerequisites

[ronan-lashermes]

Locally my issue is that cocotb uses find_libpython and can’t find the module even if present. But this is most probably due to my local environment setup which is nix-based (great when it works, else a nightmare...).

For the test, yeah I tried to replicate the BundleTester with a cocotb testbench.
Hmmm, quite normal that we got X in TaggedUnions, when the variant does not use the full union bits.
Is there an elegant way to deal with it with cocotb, or do you suggest to use a verilog testbench instead ?

[Dolu1990]

For the test, yeah I tried to replicate the BundleTester with a cocotb testbench.

So in general, i moved away from cocotb and now only using SpinalSim :)

But this is most probably due to my local environment setup which is nix-based (great when it works, else a nightmare...).

Ahhh i don't know nix so well, it is very package oriented right ? (i kinda always try to get things from source)

Hmmm, quite normal that we got X in TaggedUnions, when the variant does not use the full union bits.

Yes, and in old version of cocotb the default was that X were translated into 0.
But it isn't the default anymore. So to still keep it, there is a trick :

    from cocotblib.misc import cocotbXHack
    cocotbXHack()

Is there an elegant way to deal with it with cocotb, or do you suggest to use a verilog testbench instead ?

SpinalSim <3, else the trick above i would say

[Dolu1990]

Hi ^^

How things are going ?

[ronan-lashermes]

I actually got a working SpinalSim going. Unfortunately, I need a few more days to find the time to iron things out and push the final tester.

I also progressed on the Init problem, that may require slight modifications to existing init : the function signature is currently init(hardware: T, value: T), where we need in our case: init(hardware: T, value: V), with both T and V: Data (in TaggedUnion, the type of the union is different than the types of its possible values).
I need to experiment a bit to find a satisfying solution.

[Dolu1990]

Hoo right, i haven't thinked about the init situation XD

[ronan-lashermes]

Ok, here a new version with a working SpinalSim tester and a first try at init (in 6b15880).
Reg(MyTaggedUnion()).init(unionVariant) generates the correct verilog BUT :

• I did not find a way to test the reset value with SpinalSim
• The trick is to define the init fuction with a new type signature => it introduces some breaking changes : got two init that were not working in Phase.scala because implicit conversion were not working. This needs to been tested more, maybe find a way to get same implicit conversion as before).
• No support for RegInit, or Reg(hardware, init value) for now.

Does that seems like the correct direction for you ?

[Dolu1990]

Ok, here a new version with a working SpinalSim tester and a first try at init (in 6b15880). Reg(MyTaggedUnion()).init(unionVariant) generates the correct verilog BUT :

• I did not find a way to test the reset value with SpinalSim
• The trick is to define the init fuction with a new type signature => it introduces some breaking changes : got two init that were not working in Phase.scala because implicit conversion were not working. This needs to been tested more, maybe find a way to get same implicit conversion as before).
• No support for RegInit, or Reg(hardware, init value) for now.

Does that seems like the correct direction for you ?

Hi ^^
How the (implicit converter: ANY => T): works ? Do you have some examples ?
What's about instead providing the special init implementations in the TaggedUnion class itself or via new implicit functions specificaly for ?

[ronan-lashermes]

For the converter: the issue is with inits such as value.init(0), with (value: T). 0 is not of type T. It was working when there is an implicit converter from Int to T (in this case). So the new init makes this converter (ANY -> T) explicit to deal with init values that are not Data specifically.

It works in nearly all cases but a corner case with SpinalEnum (cf Phases.scala). I am still not sure of what implicit converter was called. These corner cases are tied to type system warnings for the same lines in the master branch.

The gist of this modification is to be able to use RegInit, accepting any input as long as there is a converter at hand.
The special init for TaggedUnion would work, with less sweat, but it would require special initialization methods (e.g. RegInit won't work when initializing from a variant).

I will try to improve things a bit (and explore the special TaggedUnion init direction).

[ronan-lashermes]

Ok I pushed a new try with a dedicated initVariant to init a TaggedUnion with a variant value in a register.
My limited test show coherent generated Verilog.
I have my doubts with respect to this line:

this.unionPayload.initFrom(variantData.asBits.aliasAs(this.unionPayload))

I convert the variant to the "neutral" unionPayload with aliasAs. But I am not sure that this is equivalent to

this.unionPayload.aliasAs(variantData).initFrom(variantData)

The latter cannot work since aliasAs does not transfer the isReg property.

[Dolu1990]

Ok I pushed a new try with a dedicated initVariant to init a TaggedUnion with a variant value in a register. My limited test show coherent generated Verilog. I have my doubts with respect to this line:

this.unionPayload.initFrom(variantData.asBits.aliasAs(this.unionPayload))

I convert the variant to the "neutral" unionPayload with aliasAs. But I am not sure that this is equivalent to

this.unionPayload.aliasAs(variantData).initFrom(variantData)

The latter cannot work since aliasAs does not transfer the isReg property.

Hmm, i'm not sure to understand why using aliasAs ?

Why not this.unionPayload.init(variantData.asBits.resized) ? i may have missed something

[ronan-lashermes]

Your solution gives the same Verilog as mine. My line is just a bit more complex ;)
The issue is that generates something along the line

unionPayload <= {variant_x,variant_y};

where I would prefer

unionPayload[7:0] <= variant_x;
unionPayload[15:8] <= variant_y;

The explicit assignation seems to me less error prone.

[Dolu1990]

The explicit assignation seems to me less error prone.

As long as it works ^^

Can't merge the branch, as there is merge conflicts

[ronan-lashermes]

Ok, I will look into it. And perform some more tests...