Hello Community,
My team is planing to tapout a chipyard based design, and we have a different Clock Scheme from the default chipyard design:
domain1, CORE,SBUS,CBUS: 800MHZ
domain2, MBUS,L2Wrapper:400MHZ
domain3: PBUS,FBUS: 100MHZ.
All these clock domains are rational crossed. They are phase aligned with each other. Therefore, taking UART as an example, interrupt initiated by uart(at PBUS) has to cross to IBUS(the same clock domain with SBUS), see code below:
(intXType match {
case _: SynchronousCrossing => where.ibus.fromSync
case _: RationalCrossing => where.ibus.fromRational
case _: AsynchronousCrossing => where.ibus.fromAsync
}) := uart.intXing(intXType)
There are two dangerous bugs in code above:
1,The domain crossing logic is applied twice, the uart.intXing(intXType) will be converted into IntSyncRationalCrossingSink() :*=* IntSyncNameNode(name) :*=* scope { IntSyncNameNode(name) :*=* IntSyncCrossingSource(alreadyRegistered) } :*=* node , whileibus.fromRational also has domain crossing facilities:
/** Collects interrupts from internal and external devices and feeds them into the PLIC */
class InterruptBusWrapper(implicit p: Parameters) extends ClockSinkDomain {
override def shouldBeInlined = true
val int_bus = LazyModule(new IntXbar) // Interrupt crossbar
private val int_in_xing = this.crossIn(int_bus.intnode)
private val int_out_xing = this.crossOut(int_bus.intnode)
def from(name: Option[String])(xing: ClockCrossingType) = int_in_xing(xing) :=* IntNameNode(name)
def to(name: Option[String])(xing: ClockCrossingType) = IntNameNode(name) :*= int_out_xing(xing)
def fromAsync: IntInwardNode = from(None)(AsynchronousCrossing(8,3))
def fromRational: IntInwardNode = from(None)(RationalCrossing(direction = Flexible))
def fromSync: IntInwardNode = int_bus.intnode
def toPLIC(xing: ClockCrossingType = NoCrossing): IntOutwardNode = to(Some("toPLIC"))(xing)
}
I don't really think this is the intended behavior, I think domain corssing stuff should only be applied once, that is the IntSource should be at PBUS domain while the IntSink should be at IBUS domain.
2, The crossing domain collateral being applied twice make the Interrupt Domain crossing moderately dangerous, when applying the first RationalCrossing collateral(coming out of PBUS), the IntSyncAsyncCrossingSink is neither at the domain of PBUSnor the IBUS, it is at where which is the BaseSubsystem, aka the DigitalTop, the ModuleImpof DigitalTop is LazyRawModuleImp, which mean there is no implicit clock and reset for this DigitalTop, therefore anonymous children of DigitalTop will not receive clock or reset, this means the IntSyncAsyncCrossingSink in where will not be approperly clocked. :
def attachTo(where: Attachable)(implicit p: Parameters): TLUART = where {
val name = s"uart_${UART.nextId()}"
val tlbus = where.locateTLBusWrapper(controlWhere)
val divinit = (tlbus.dtsFrequency.get / device.initBaudRate).toInt
val uartClockDomainWrapper = LazyModule(new ClockSinkDomain(take = None, name = Some("TLUART")))
val uart = uartClockDomainWrapper { LazyModule(new TLUART(tlbus.beatBytes, device, divinit)) }
uart.suggestName(name)
tlbus.coupleTo(s"device_named_$name") { bus =>
val blockerOpt = blockerAddr.map { a =>
val blocker = LazyModule(new TLClockBlocker(BasicBusBlockerParams(a, tlbus.beatBytes, tlbus.beatBytes)))
tlbus.coupleTo(s"bus_blocker_for_$name") { blocker.controlNode := TLFragmenter(tlbus, Some("UART_Blocker")) := _ }
blocker
}
uartClockDomainWrapper.clockNode := (controlXType match {
case _: SynchronousCrossing =>
tlbus.dtsClk.map(_.bind(uart.device))
tlbus.fixedClockNode
case _: RationalCrossing =>
tlbus.clockNode
case _: AsynchronousCrossing =>
val uartClockGroup = ClockGroup()
uartClockGroup := where.allClockGroupsNode
blockerOpt.map { _.clockNode := uartClockGroup } .getOrElse { uartClockGroup }
})
(uart.controlXing(controlXType)
:= TLFragmenter(tlbus, Some("UART"))
:= blockerOpt.map { _.node := bus } .getOrElse { bus })
}
(intXType match {
case _: SynchronousCrossing => where.ibus.fromSync
case _: RationalCrossing => where.ibus.fromRational
case _: AsynchronousCrossing => where.ibus.fromAsync
}) := uart.intXing(intXType)
uart
}
Can anyone in the team confirm my understanding is right? If I am right, I wonder what's the typical domain crossing scehmes used by chipyard or rocketchip teams that has been tapout verified. The reason for choosing PBUS at lower clock speed is that the device at PBUS is slow, therefore it's not very power effcient to make it 800MHZ(in our case). I also wonder if the rational crossing type has been used or tested at all?
Any help will be appreciated, Thanks again, I am sorry to post issues recently and disrupt you guys.
@jerryz123 @sequencer @tianrui-wei
Hello Community,
My team is planing to tapout a chipyard based design, and we have a different Clock Scheme from the default chipyard design:
domain1, CORE,SBUS,CBUS: 800MHZ
domain2, MBUS,L2Wrapper:400MHZ
domain3: PBUS,FBUS: 100MHZ.
All these clock domains are rational crossed. They are phase aligned with each other. Therefore, taking UART as an example, interrupt initiated by uart(at PBUS) has to cross to IBUS(the same clock domain with SBUS), see code below:
There are two dangerous bugs in code above:
1,The domain crossing logic is applied twice, the
uart.intXing(intXType)will be converted intoIntSyncRationalCrossingSink() :*=* IntSyncNameNode(name) :*=* scope { IntSyncNameNode(name) :*=* IntSyncCrossingSource(alreadyRegistered) } :*=* node, whileibus.fromRationalalso has domain crossing facilities:I don't really think this is the intended behavior, I think domain corssing stuff should only be applied once, that is the IntSource should be at
PBUSdomain while theIntSinkshould be atIBUSdomain.2, The crossing domain collateral being applied twice make the Interrupt Domain crossing moderately dangerous, when applying the first
RationalCrossingcollateral(coming out ofPBUS), theIntSyncAsyncCrossingSinkis neither at the domain ofPBUSnor theIBUS, it is atwherewhich is theBaseSubsystem, aka theDigitalTop, theModuleImpofDigitalTopisLazyRawModuleImp, which mean there is no implicit clock and reset for thisDigitalTop, therefore anonymous children ofDigitalTopwill not receiveclockorreset, this means theIntSyncAsyncCrossingSinkinwherewill not be approperly clocked. :Can anyone in the team confirm my understanding is right? If I am right, I wonder what's the typical domain crossing scehmes used by chipyard or rocketchip teams that has been tapout verified. The reason for choosing
PBUSat lower clock speed is that the device at PBUS is slow, therefore it's not very power effcient to make it 800MHZ(in our case). I also wonder if the rational crossing type has been used or tested at all?Any help will be appreciated, Thanks again, I am sorry to post issues recently and disrupt you guys.
@jerryz123 @sequencer @tianrui-wei