Skip to content
Software model of various peripherals (ARM Generic PCI controller, Intel E1000 adapter, AHCI SATA device, Altera DMA engine)
C Assembly C++ Other
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.
mdepx @ 204ddf8


This application emulates various peripherals on CHERI platform:

  1. Altera Modular Scatter-Gather DMA engine (mSGDMA)
  2. ARM Generic ECAM PCI-controller
  3. Intel E1000 ethernet device
  4. AHCI controller and memory-backed Serial ATA device

CHERI CPU used by this application is synthesized on Altera FPGA and prototyped on Altera DE4 board.

This is bare-metal software, i.e. it runs on a dedicated CPU core of CHERI processor.

Build FreeBSD with a beri_manager driver and bm (BERI Manager) application

$ git clone -b dma
$ make -j4 TARGET=mips TARGET_ARCH=mips64 buildworld
$ make -j4 TARGET=mips TARGET_ARCH=mips64 KERNCONF=BERI_DE4_USBROOT buildkernel

Make rootfs image if you don't have one

$ make -j4 TARGET=mips TARGET_ARCH=mips64 -DNO_ROOT DESTDIR=${HOME}/world-mips64 installworld
$ make -j4 TARGET=mips TARGET_ARCH=mips64 -DNO_ROOT DESTDIR=${HOME}/world-mips64 distribution
$ cd ${HOME}/world-mips64
$ echo '# Altera JTAG UART' >> etc/ttys
$ echo 'ttyj0 "/usr/libexec/getty std.115200" xterm on secure' >> etc/ttys
$ echo 'ttyj1 "/usr/libexec/getty std.115200" xterm on secure' >> etc/ttys
$ echo 'hostname="mips64"' >> etc/rc.conf
$ echo 'sendmail_enable="NONE"' >> etc/rc.conf
$ echo 'cron_enable="NO"' >> etc/rc.conf
$ echo "/dev/da0  /       ufs     ro      1       1" > etc/fstab
$ echo "./etc/fstab type=file uname=root gname=wheel mode=0644" >> METALOG
$ echo "./etc/rc.conf type=file uname=root gname=wheel mode=0644" >> METALOG
$ makefs -B big -D -f 30000 -o version=2 -s 1200m mips64.img METALOG

Build device-model under FreeBSD

$ sudo pkg install llvm-cheri
$ git clone --recursive
$ cd device-model
$ gmake

Boot FreeBSD kernel on a 1st core of BERI system

$ cd ctsrd/cherilibs/trunk/tools/debug
$ ./berictl -j loadsof -z DE4_BERI_EPW.sof.bz2
$ ./berictl -j loadbin -z kernel.bz2 0x100000
$ ./berictl -j boot && ./berictl -j console

Receive device-model on a FreeBSD that runs on a 1st core

$ ifconfig atse0
$ /etc/rc.d/var start
$ nc -l 1234 > /var/tmp/dm.bin

On your host (development) machine, send binary

$ nc 1234 < dm.bin

Program device-model to DDR memory from a FreeBSD that runs on a 1st core. Start execution DM.

This enables peripheral window (register interface).

$ bm -rRl /var/tmp/dm.bin

Open a terminal to the device-model on your host (development) machine

$ nios2-terminal -i 2

To emulate Altera mSGMDA:

$ devctl disable atse1
$ devctl disable msgdma2
$ devctl disable msgdma3
$ sleep 1
$ devctl enable msgdma2
$ devctl enable msgdma3
$ devctl enable atse1
$ ifconfig atse1

To emulate PCIe bus and connected devices:

$ devctl disable pci0
$ devctl enable pci0

Note: both mSGMDA and e1000 PCI-e device use the same Altera FIFO, which means emulating both of them same time leads to undefined behaviour.

Example device-tree node for PCI-e device

pcie@7fb10000 {
	compatible = "pci-host-ecam-generic";
	device_type = "pci";
	#interrupt-cells = <1>;
	#address-cells = <3>;
	#size-cells = <2>;

	reg = <0x7fb10000 0xe0000>;
	reg-names = "PCI ECAM";

	//attributes      pci_addr   cpu_addr         size
	ranges =
	  <0x02000000 0 0x7fb20000 0x7fb20000 0 0x00020000	//e1000 bar 0
	   0x01000000 0 0x00000010 0x00000010 0 0x00010000	//e1000 bar 2
	   0x43000000 0 0x7fb40000 0x7fb40000 0 0x00010000	//e1000 bar 1
	   0x02000000 0 0x7fb50000 0x7fb50000 0 0x00010000>;	//ahci-hd bar 0


	interrupt-map = < 0x000 0 0  1  &beripic0  0x13 	//e1000
			  0x800 0 0  1  &beripic0  0x14 >; 	//ahci
	interrupt-map-mask = <0x800 0 0 7>;

FreeBSD kernel configuration file

device		pci
device		em
device		ahci

Physical memory organization

Start End Size Description
0x00000000 0x40000000 1 GB Entire physical memory of SoC
0x10000000 0x20000000 256 MB Reserved by FreeBSD for device-model
0x10000000 0x10800000 8 MB device-model static data
0x10800000 0x11000000 8 MB device-model malloc()
0x11000000 0x13000000 32 MB free space
0x13000000 0x20000000 208 MB AHCI SATA device memory block


IOMMU module translates addresses for various device-model peripherals and manages BERI TLB.

Only Altera FIFO-based models are currently evaluated.

To enable IOMMU module build device-model project with DM_IOMMU=1 environment variable.

On the FreeBSD side, add the following IOMMU devices to your DTS file:

               va_space0: iommu-va@7fb60000 {
                       compatible = "beri,iommu-va-space";
                       reg = <0x0 0x20000000>;

               va_space1: iommu-va@7fb60100 {
                       compatible = "beri,iommu-va-space";
                       reg = <0x20000000 0x20000000>;

               iommu0: iommu@7fb60000 {
                       compatible = "beri,iommu";
                       reg = <0x7fb60000 0x100>;
                       va-region = <&va_space0>;
                       status = "okay";

               iommu1: iommu@7fb60100 {
                       compatible = "beri,iommu";
                       reg = <0x7fb60100 0x100>;
                       va-region = <&va_space1>;
                       status = "okay";

Add the "xdma,iommu" property to the nodes of the models, for example

               dm_msgdma0: msgdma@7fb04080 {
                       xdma,iommu = <&iommu0>;

               dm_msgdma1: msgdma@7fb04000 { 
                       xdma,iommu = <&iommu1>;

alt text

You can’t perform that action at this time.