Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AxiCrossBar with Standard Axi4 Interface in Briey #406

Open
ic-hjx opened this issue May 6, 2024 · 15 comments
Open

AxiCrossBar with Standard Axi4 Interface in Briey #406

ic-hjx opened this issue May 6, 2024 · 15 comments

Comments

@ic-hjx
Copy link

ic-hjx commented May 6, 2024

Hello, recently I have been wanting to add an SPI peripheral IP with a standard AXI structure to Briey's AxiCrossBar, which is written in SystemVerilog. I have noticed that the Axi4Shared interface is predominantly used in Briey's AxiCrossBar, and I believe the AxiCrossBar should be able to support the use of both the Axi4 standard interface and the Axi4Shared interface simultaneously.
The main issue I have encountered is that I do not know how to use the function addPipelining on an object of the Axi4 standard interface, and I have not found any relevant examples so far.

def addPipelining(axi: Axi4)(ro: (Axi4ReadOnly, Axi4ReadOnly) => Unit)(wo: (Axi4WriteOnly, Axi4WriteOnly) => Unit): this.type = {
  val b = axi4SlaveToReadWriteOnly(axi)
  val rAxi = b(0).asInstanceOf[Axi4ReadOnly]
  val wAxi = b(1).asInstanceOf[Axi4WriteOnly]
  addPipelining(rAxi)(ro)
  addPipelining(wAxi)(wo)
  this
}

Regarding the AxiShared interface, I can see that the function is used as follows:

axiCrossbar.addPipelining(ram.io.axi)((crossbar, ctrl) => {
  crossbar.sharedCmd.halfPipe()  >>  ctrl.sharedCmd
  crossbar.writeData            >/-> ctrl.writeData
  crossbar.writeRsp              <<  ctrl.writeRsp
  crossbar.readRsp               <<  ctrl.readRsp
})

However, for the Axi4 type interface, I am not quite clear on how to wire it, so I am seeking some assistance and looking forward to your reply!

@Dolu1990
Copy link
Member

Dolu1990 commented May 6, 2024

Hi,

My guess is that for AXI it would be something like :

axiCrossbar.addPipelining(myaxi)((from, to) => {
  from.ar >>  to.ar
  from.r   << to.r
}) ((from, to) => {
  from.aw >>  to.aw
  from.w >>  to.w
  from.b << to.b
})

And then you can add custom stages on the top

@ic-hjx
Copy link
Author

ic-hjx commented May 7, 2024

Following your suggestion, I added the following code to Briey.scala:

axiCrossbar.addPipelining(spi.io.axi)((crossbar, ctrl) => {
  crossbar.ar >> ctrl.ar
  crossbar.r << ctrl.r
})((crossbar, ctrl) => {
  crossbar.aw >> ctrl.aw
  crossbar.w >> ctrl.w
  crossbar.b << ctrl.b
})

This code does not have any syntax errors, but an error occurred when compiling and running Briey. The first error is:

Exception in thread "main" java.lang.Exception: literal 0x3ffff can't fit in UInt(12 bits)

I analyzed that this error is caused by a mismatch between the address width of spi.io.axi and the address width of spi in the following code:

axiCrossbar.addSlaves(
  ram.io.axi       -> (0x80000000L,   onChipRamSize),
  spi.io.axi       -> (0x20000000L,   32 kB),
  apbBridge.io.axi -> (0xF0000000L,   1 MB)
)

Where val spi = Axi4SpiCtrl has an addressWidth of 12, but the address width for spi.io.axi is different:

val spi = Axi4SpiCtrl(
  addressWidth = 12,
  dataWidth = 32,
  idWidth = 2
)

So, I changed the addressWidth to 20 to get the compilation to pass. However, there is something strange that I noticed: I also tried to change the address size in spi.io.axi -> (0x20000000L, 32 kB), to 1 KB, but it still reported an error requiring a 20-bit address width.

Anyway, after resolving the above error, I encountered a new problem, and the log shows:

Exception in thread "main" spinal.core.SpinalExit:
Error detected in phase PhaseCreateComponent
********************************************************************************

toplevel/??? : Axi4ReadOnly has no master}
********************************************************************************

toplevel/??? : Axi4WriteOnly has no master}
********************************************************************************

Design's errors are listed above.
SpinalHDL compiler exit stack:

I don't quite understand how this issue arose.

The following is part of the code I used when adding the Axi4 interface IP:

val spi = Axi4SpiCtrl(
  addressWidth = 20,
  dataWidth = 32,
  //  byteCount = 32 kB,
  idWidth = 2
)

axiCrossbar.addSlaves(
  ram.io.axi       -> (0x80000000L,   onChipRamSize),
  spi.io.axi       -> (0x20000000L,   32 kB),
  apbBridge.io.axi -> (0xF0000000L,   1 MB)
)

axiCrossbar.addConnections(
  core.iBus       -> List(ram.io.axi),
  core.dBus       -> List(ram.io.axi, apbBridge.io.axi),
  spi.io.axi      -> List(ram.io.axi)
)

axiCrossbar.addPipelining(spi.io.axi)((crossbar, ctrl) => {
  crossbar.ar >> ctrl.ar
  crossbar.r << ctrl.r
})((crossbar, ctrl) => {
  crossbar.aw >> ctrl.aw
  crossbar.w >> ctrl.w
  crossbar.b << ctrl.b
})

object Axi4SpiCtrl {
  def getAxiConfig(addressWidth: Int, dataWidth: Int, idWidth: Int) = Axi4Config(
    addressWidth = addressWidth,
    dataWidth = dataWidth,
    idWidth = idWidth,
    useLock = false,
    useRegion = false,
    useCache = false,
    useProt = false,
    useQos = false,
    arUserWidth = 1,
    bUserWidth = 1,
    rUserWidth = 1,
    wUserWidth = 1,
    awUserWidth = 1
  )
}
case class Axi4SpiCtrl (addressWidth: Int, dataWidth : Int, idWidth : Int, arwStage : Boolean = false) extends Component{
  val axiConfig = Axi4SpiCtrl.getAxiConfig(addressWidth,dataWidth ,idWidth)

  val io = new Bundle {
    val axi = slave(Axi4(axiConfig))
  }
  /.../

}

@ic-hjx
Copy link
Author

ic-hjx commented May 7, 2024

I realized my mistake was in the following code snippet:

axiCrossbar.addConnections(
  core.iBus       -> List(ram.io.axi),
  core.dBus       -> List(ram.io.axi, apbBridge.io.axi),
  spi.io.axi      -> List(ram.io.axi)
)

I wanted to control the SPI module through instructions from the iBus and dBus, so I should have directed the connections of iBus and dBus to the Axi4Spi. Therefore, I modified the code to:

axiCrossbar.addConnections(
  core.iBus       -> List(ram.io.axi, spi.io.axi),
  core.dBus       -> List(ram.io.axi, apbBridge.io.axi, spi.io.axi)
)

After the modification, the compilation passed, and it was able to output the file Briey.v. I think this modification should be correct.

After studying the Axi4CrossbarFactory, I have some questions about the Axi4Shared interface it creates. In this interface, the write address channel and the read address channel of Axi4 share the IO. However, the standard Axi4 protocol should support simultaneous read and write operations on both the write address channel and the read address channel. It seems that the operation of Axi4Shared might prevent the write address channel and the read address channel from working simultaneously. Although this modification reduces the circuit area, it seems to come at the cost of communication speed.

Additionally, I noticed that dBus is structured with an Axi4Shared interface. Will the Axi4CrossbarFactory automatically resolve the interconnection issues between the Axi4Shared and Axi4 interfaces?

@Dolu1990
Copy link
Member

It seems that the operation of Axi4Shared might prevent the write address channel and the read address channel from working simultaneously.

Depend what you mean by simultaneously.
It doesn't prevent having multiple inflight read / write memory request. Just that address requests can't go through shared nodes the same cycle.

it seems to come at the cost of communication speed.

I would say, address channel shouldn't be a bottleneck, unless you have a lot of single beat burst.

Will the Axi4CrossbarFactory automatically resolve the interconnection issues between the Axi4Shared and Axi4 interfaces?

Yes it will bridge things

@ic-hjx
Copy link
Author

ic-hjx commented May 13, 2024

Thanks for your reply!
And in the past few days I have been trying to do some jobs with the Axi4Spi, when I was using VexRiscv to read data from a specified address via Axi4Spi, I encountered some strange errors. I wanted to read a 32-bit piece of data from the address 20000000, with ARSIZE set to 2 and ARLEN to 7 by the arbiter. I thought that ARSIZE=2 represents that the size of each data packet is 32 bits, and at this time, ARLEN should probably be equal to 0?
Additionally, when I tried to remove the AxSize signal using useSize=false (since the Axi4Spi IP I'm using does not have an AxSize channel), I encountered the following error:

[error] Exception in thread "main" spinal.core.SpinalExit: 
[error]  Error detected in phase PhaseCreateComponent
[error] ********************************************************************************
[error] ********************************************************************************
[error] (toplevel/??? :  UInt[3 bits]) can't drive toplevel/[Axi4ReadOnlyArbiter]/io_inputs_1_ar : Stream[Axi4Ar] because this last one doesn't has the corresponding pin
[error]     spinal.lib.bus.amba4.axi.Axi4Priv$.driveWeak(Axi4Channel.scala:366)
[error]     spinal.lib.bus.amba4.axi.Axi4Priv$.driveAx(Axi4Channel.scala:380)
[error]     spinal.lib.bus.amba4.axi.Axi4Ar$StreamPimper.drive(Axi4Channel.scala:414)
[error]     spinal.lib.bus.amba4.axi.Axi4ReadOnly.$greater$greater(Axi4ReadOnly.scala:22)
[error]     spinal.lib.bus.amba4.axi.Axi4ReadOnly.$less$less(Axi4ReadOnly.scala:20)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29$$anon$4$$anon$5$$anonfun$31.apply(Axi4Crossbar.scala:212)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29$$anon$4$$anon$5$$anonfun$31.apply(Axi4Crossbar.scala:209)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29$$anon$4$$anon$5.<init>(Axi4Crossbar.scala:209)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29$$anon$4.<init>(Axi4Crossbar.scala:203)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29.apply(Axi4Crossbar.scala:195)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29.apply(Axi4Crossbar.scala:194)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory.build(Axi4Crossbar.scala:194)
[error]     vexriscv.demo.Briey$$anon$3.<init>(Briey.scala:414)
[error]     vexriscv.demo.Briey.<init>(Briey.scala:253)
[error]     vexriscv.demo.BrieySim$$anonfun$main$4.apply(Briey.scala:516)
[error]     vexriscv.demo.BrieySim$$anonfun$main$4.apply(Briey.scala:516)
[error]     spinal.sim.JvmThread.run(SimManager.scala:51)
[error] ********************************************************************************
[error] ********************************************************************************
[error] (toplevel/??? :  UInt[3 bits]) can't drive toplevel/[Axi4WriteOnlyArbiter]/io_inputs_0_aw : Stream[Axi4Aw] because this last one doesn't has the corresponding pin
[error]     spinal.lib.bus.amba4.axi.Axi4Priv$.driveWeak(Axi4Channel.scala:366)
[error]     spinal.lib.bus.amba4.axi.Axi4Priv$.driveAx(Axi4Channel.scala:380)
[error]     spinal.lib.bus.amba4.axi.Axi4Aw$StreamPimper.drive(Axi4Channel.scala:401)
[error]     spinal.lib.bus.amba4.axi.Axi4WriteOnly.$greater$greater(Axi4WriteOnly.scala:27)
[error]     spinal.lib.bus.amba4.axi.Axi4WriteOnly.$less$less(Axi4WriteOnly.scala:25)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29$$anon$6$$anonfun$34.apply(Axi4Crossbar.scala:235)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29$$anon$6$$anonfun$34.apply(Axi4Crossbar.scala:234)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29$$anon$6.<init>(Axi4Crossbar.scala:234)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29.apply(Axi4Crossbar.scala:227)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory$$anonfun$29.apply(Axi4Crossbar.scala:194)
[error]     spinal.lib.bus.amba4.axi.Axi4CrossbarFactory.build(Axi4Crossbar.scala:194)
[error]     vexriscv.demo.Briey$$anon$3.<init>(Briey.scala:414)
[error]     vexriscv.demo.Briey.<init>(Briey.scala:253)
[error]     vexriscv.demo.BrieySim$$anonfun$main$4.apply(Briey.scala:516)
[error]     vexriscv.demo.BrieySim$$anonfun$main$4.apply(Briey.scala:516)
[error]     spinal.sim.JvmThread.run(SimManager.scala:51)
[error] ********************************************************************************
[error] ********************************************************************************
[error] Design's errors are listed above.
[error] SpinalHDL compiler exit stack : 
[error] 	at spinal.core.SpinalExit$.apply(Misc.scala:455)
[error] 	at spinal.core.SpinalError$.apply(Misc.scala:510)
[error] 	at spinal.core.internals.PhaseContext.checkPendingErrors(Phase.scala:177)
[error] 	at spinal.core.internals.PhaseContext.doPhase(Phase.scala:193)
[error] 	at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2$$anonfun$apply$142.apply(Phase.scala:2927)
[error] 	at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2$$anonfun$apply$142.apply(Phase.scala:2925)
[error] 	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
[error] 	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
[error] 	at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2.apply(Phase.scala:2925)
[error] 	at spinal.core.internals.SpinalVerilogBoot$$anonfun$singleShot$2.apply(Phase.scala:2861)
[error] 	at spinal.core.ScopeProperty$.sandbox(ScopeProperty.scala:71)
[error] 	at spinal.core.internals.SpinalVerilogBoot$.singleShot(Phase.scala:2861)
[error] 	at spinal.core.internals.SpinalVerilogBoot$.apply(Phase.scala:2856)
[error] 	at spinal.core.Spinal$.apply(Spinal.scala:413)
[error] 	at spinal.core.SpinalConfig.generateVerilog(Spinal.scala:179)
[error] 	at spinal.core.sim.SpinalSimConfig.compileCloned(SimBootstraps.scala:942)
[error] 	at spinal.core.sim.SpinalSimConfig.compile(SimBootstraps.scala:912)
[error] 	at vexriscv.demo.BrieySim$.main(Briey.scala:516)
[error] 	at vexriscv.demo.BrieySim.main(Briey.scala)
[error] Nonzero exit code returned from runner: 1
[error] (Compile / runMain) Nonzero exit code returned from runner: 1

@ic-hjx
Copy link
Author

ic-hjx commented May 14, 2024

I found that ARLEN was just assigned to be 8'h07, I don't understand the reason.

  assign axi4ReadOnlyDecoder_1_io_input_ar_payload_len = 8'h07; // @ Axi4Channel.scala l367

@Dolu1990
Copy link
Member

I thought that ARSIZE=2 represents that the size of each data packet is 32 bits, and at this time, ARLEN should probably be equal to 0?

Yes

Additionally, when I tried to remove the AxSize signal using useSize=false (since the Axi4Spi IP I'm using does not have an AxSize channel), I encountered the following error:

It is because one master has the size signal, the interconnect want to propagate it down to every ends.

@ic-hjx
Copy link
Author

ic-hjx commented May 14, 2024

My Axi4Spi controller IP currently does not support burst signal transmission. How can I inform the Axi4 Master in Briey that this slave does not support Burst signal transmission?

@Dolu1990
Copy link
Member

What we would need is a bridge between AXI with burst -> axi without burst, which either respond with an error, either adapt things.

The interconnect itself isn't handeling those kind of missmatch.

@ic-hjx
Copy link
Author

ic-hjx commented May 14, 2024

Okay, now I am trying to find out how the issue arises through simulation. My program attempts to read a 32-bit value from address 0x20000000. The master sets the arvalid signal to 1 only on the first read attempt. When attempting to read the value from address 0x20000000 repeatedly, arvalid remains at 0.

Since the slave does not receive the arvalid signal, it cannot determine whether the read address is correct and cannot update the state machine. This ultimately leads to the inability to assert the rvalid signal to tell the master that the read data is correct and to refresh the read value.

I think that even when reading from the same address, the arvalid signal should be asserted high each time to indicate the start of a read to the slave, right?

@ic-hjx
Copy link
Author

ic-hjx commented May 15, 2024

6B0DD7A2FE15266FEBB96A0E4CAC0882
So here is the slave wave with Axi4 AR and R channel, I just can't which signal is wrong that made s_axi_arvalid will never be High again

@Dolu1990
Copy link
Member

Hi,

I think that even when reading from the same address, the arvalid signal should be asserted high each time to indicate the start of a read to the slave, right?

Unless there is a burst requestwhich is the case here, see arlen.
The CPU is trying to refill a whole cache line here.
If you don't want this, you need to set that memory region as an "ioRegion" (in the vexriscv configuration)

@ic-hjx
Copy link
Author

ic-hjx commented May 15, 2024

Thank you! I can understand what you said about why ARVALID wouldn't go high in Burst mode, but I don't know how to specifically set a memory region as an IORegion. What should I do? Here is some setup about my current memory map:

val core = new Area{
    val config = VexRiscvConfig(
      plugins = cpuPlugins += new DebugPlugin(debugClockDomain)
    )

    val cpu = new VexRiscv(config)
    var iBus : Axi4ReadOnly = null
    var dBus : Axi4Shared = null
    for(plugin <- config.plugins) plugin match{
      case plugin : IBusSimplePlugin => iBus = plugin.iBus.toAxi4ReadOnly()
      case plugin : IBusCachedPlugin => iBus = plugin.iBus.toAxi4ReadOnly()
      case plugin : DBusSimplePlugin => dBus = plugin.dBus.toAxi4Shared()
      case plugin : DBusCachedPlugin => dBus = plugin.dBus.toAxi4Shared(true)
      case plugin : CsrPlugin        => {
        plugin.externalInterrupt := BufferCC(io.coreInterrupt)
        plugin.timerInterrupt := timerCtrl.io.interrupt
      }
      case plugin : DebugPlugin      => debugClockDomain{
        resetCtrl.axiReset setWhen(RegNext(plugin.io.resetOut))
        io.jtag <> plugin.io.bus.fromJtag()
      }
      case _ =>
    }
  }

val axiCrossbar = Axi4CrossbarFactory()
axiCrossbar.addSlaves(
  ram.io.axi       -> (0x80000000L,   onChipRamSize),
  spi.io.axi       -> (0x20000000L,   1 kB),
  apbBridge.io.axi -> (0xF0000000L,   1 MB)
)

 axiCrossbar.addConnections(
   core.iBus       -> List(ram.io.axi),
   core.dBus       -> List(ram.io.axi, apbBridge.io.axi, spi.io.axi)
 )

@Dolu1990
Copy link
Member

it is specified here :

ioRange = _(31 downto 28) === 0xF

But i would say, the simpler would just be to remap your spi.io.axi into the 0xF0100000L-0xFFFFFFFFL range

@ic-hjx
Copy link
Author

ic-hjx commented May 15, 2024

After mapping the starting address of the memory region used by SPI to 0xF1000000, my problem seems to be resolved. I will conduct further verification. Thank you very much for your help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants