SDRAM controller for the ULX3S

[zipotron]

Hello again! I am studying how to enable the 32MB of SDRAM in the ULX3S, and I found in other projects interesting codes in VHDL that access to the ULX3S SDRAM (https://github.com/nullobject/sdram-fpga), but those are not Wishbone compatible (or at least in not mentioned). I found in the file "rtl/processor_templates/neorv32_ProcessorTop_MinimalBoot.vhd" that we have "MEM_EXT_EN" that suppose to be the connection with that SDRAM.
I would like to implement the controller to link the wishbone external memory with the VHDL ULX3S SDRAM. If its OK I start to do it and hopefully if my work allows me do a PR soon, but I will need a bit support. Do you like the idea?

[umarcor]

See #105.

[stnolting]

and I found in other projects interesting codes in VHDL that access to the ULX3S SDRAM (https://github.com/nullobject/sdram-fpga), but those are not Wishbone compatible (or at least in not mentioned)

That looks interesting! I am also searching for a small SDRAM controller to tinker with one of my board's memories. Maybe I will give it a try. From a first look, the interface seems to be pretty straightforward, so it shouldn't be a big deal to connect it to the processor's external bus interface.

I would like to implement the controller to link the wishbone external memory with the VHDL ULX3S SDRAM. If its OK I start to do it and hopefully if my work allows me do a PR soon, but I will need a bit support. Do you like the idea?

👍

We can help with the processor's bus interface, but for questions regarding the SDRAM controller you might be be better off contacting @nullobject.

[zipotron]

Thanks @umarcor !
@stnolting Yes, even better, I am trying to contact the Radiona team, as soon as I have something on what is possible to start working I will do PR. As usually, it will came as soon as my work allows me.

[zipotron]

Hello again @stnolting and @umarcor! I got some improvement regarding the SDRAM. I got in touch with the ULX3S guys, first I got a Verilog 8-bit controller from one of their project (I select that because came with really interesting test support) and translate to VHDL. Because I had some doubts about how to make it works, I ask them, and after we make it work the owner of the project offer me to adapt the test to work with https://github.com/nullobject/sdram-fpga, that is 32-bit controller and will be better choice for integrate in NeoRV32! On the way they had to fix some defects detected when we pass the test to the Nullobject controller, but now seams working!
Just ULX3S team advice to me that we have to make double check, there is a wired trick done in sdram.vhd line 441 that may cause troubles, you can see our conversations here https://gitter.im/ulx3s/Lobby
I would like to ask you to have a look of the code before to start adapting to the Wishbone intervace and integrate it, the code is here: https://github.com/zipotron/sdram-fpga
Looking forward opinions and suggestions!

[umarcor]

@zipotron, note that you can use Verilog sources together with NEORV32, through GHDL + Yosys. See example MixedLanguage for FOMU (https://github.com/stnolting/neorv32/tree/master/setups/osflow/board_tops) which is based on https://github.com/im-tomu/fomu-workshop/tree/master/hdl/mixed/blink. See https://github.com/stnolting/neorv32/blob/master/setups/osflow/Makefile#L92-L97. You should be able to create a WithSDRAM target by copying that snippet.

Nevertheless, I believe it is interesting to have the controller available in VHDL as well. I recommend channel https://gitter.im/vhdl/General for questions related to the language.

[zipotron]

Hi @umarcor , yes, I know, thanks for the info but as you can see, the exposed code have the controller in VHDL and the test in Verilog, and this is the part of the job I did. I put an effort in provide VHDL controller for follow the style, and for other reason more egoist, I am good Verilog coder, but just beginner in VHDL, and this kind of task give me the excuse for study it.
Thanks also for the channel, is going a be useful for future, but my doubt now is functional, not regarding the language.

[stnolting]

These are great news!

you can see our conversations here https://gitter.im/ulx3s/Lobby

I will have a look.

You source code also looks good - at least the code itself 😉
Unfortunately, I cannot play with that right now, but it would be interesting to test it with different memory chips.

[umarcor]

as you can see, the exposed code have the controller in VHDL and the test in Verilog, and this is the part of the job I did. I put an effort in provide VHDL controller for follow the style, and for other reason more egoist, I am good Verilog coder, but just beginner in VHDL, and this kind of task give me the excuse for study it. Thanks also for the channel, is going a be useful for future, but my doubt now is functional, not regarding the language.

Oh, I misunderstood. I thought the whole controller was in Verilog and you were converting it to VHDL. So, do you want to keep doing mixed-language simulation (keep the test in Verilog) or do you want to provide a complete solution in VHDL? In the latter case, authors of open source verification frameworks (https://larsasplund.github.io/github-facts/) such as OSVVM, VUnit or cocotb do participate in https://gitter.im/vhdl/General. So, you can head there not only for questions related to the language, but to ask about the functionality you want to achieve and what are the alternatives.

BTW, you can alt+click on a gitter message, and you'll get a direct link to it, instead of a link to the channel.

[zipotron]

@stnolting Well, I can help for one chip, the unique I have, that is in my ULX3S, but as soon as we have one, modify for work with others would be easy. As soon as my work allows me I will start to connect the controller with the Wishbone, and lets see!
@umarcor its a good advice! I will be around there! also thanks for the "alt+click" trick, I didn't know.

[zipotron]

Last thing, well, still early for ask this, but, I would like to have something for test the external RAM, like small program as a bootloader that play with leds if the memory is writable/readable...Anyone have already anything?

[stnolting]

If you have a UART (UART0 😉) interface you can use the sw/example/hex_viewer program. It allows to manually read/write/dump arbitrary memory locations:

[zipotron]

@stnolting This is perfect!

[zipotron]

Hello @stnolting , I am trying to execute the "Hex_viewer" in the ULX3S, I need a bit of support. I compiled the "Hex_viewer" going to the "neorv32/sw/example/hex_viewer" directory and executing "make install", It compiled OK and generated a .vhd in the "RTL" directory, then I went back to the root project directory and did "make -C setups/osflow BOARD=ULX3S MinimalBoot" and flash the .bit to the board. The first led blinks few second as expected and after that keep lighting. I connect the UART using Putty with 19200 speed, Putty report "connected, but dont shows any message, and I can not write command. But the board seams listening, if I type "echo -ne '\033[2J' > /dev/ttyUSB0" the UART led of the board blinks... Did I miss anything setting up the HEX_VIEWER?

[stnolting]

Hey @zipotron !
The ULX3S example setup uses the processor-internal bootloader - so make install will not initialize the internal instruction memory (per-synsthesis) with your application code. Use make exe to generate an executable that can be uploaded via the bootloader.

The first led blinks few second as expected and after that keep lighting

This is the bootloader status LED.
After reset, you should see the bootloader console in your terminal program. Better check your putty configuration again, it should be no parity-bit and one stop-bit.

If I remember correctly, the ULX3S board has a status LED connected to the USB-chip's TX pin, so you should see that flashing while the boatloader is transmitting it's console.

[zipotron]

@stnolting , Thanks a lot! Last question before start working on it and I will not bother you for a while! Where can I find examples and doc about uploading exe using the bootloader? Thanks!!!

[stnolting]

No worries - questions are always welcome! 👍

• 📚 User Guide: Uploading and Starting of a Binary Executable Image via UART
• 📚 Data Sheet: Bootloader

[zipotron]

@stnolting sorry, I promise no more question for a while but... I configured in my Linux the Putty as is specified in the doc, connect but dont shows nothing, when you type anything the TX UART led in the board blinks, the the board is responding... I reboot in Windows and install TeraTerm as the doc suggests... the same, I tried also with MobaXTerm, the same.
And thinking about the issue and the facts, I wanted to ask you. Is it possible that we did during porting a miscalculation of the system clock and the UART speed in the board is something different from 19200?
One point more! UART TX flash when I push any key, but is not flashing when I plug the board, neather when stop blinking the other led...

[stnolting]

Hmm strange... If you reset the processor and just type h in your terminal, does the status LED (the one at GPIO(0), not the TX LED) keep blinking or does it remain on after some seconds?

The ULX example setup if configured for 25MHz:

neorv32/setups/osflow/board_tops/neorv32_ULX3S_BoardTop_MinimalBoot.vhd

Lines 64 to 65 in cde357b

	-- configuration --
	constant f_clock_c : natural := 25000000; -- clock frequency in Hz

[stnolting]

Btw, which version of the processor are you using?

edit
I just found out that I broke some parts of the core with the updates from v1.6.2.4... 🙈
I have just fixed that. So please upgrade your code 😅

[zipotron]

@stnolting I just did a pull, and now the "hart" led is not blinking anymore and Putty don't connect...

[stnolting]

I just did a pull, and now the "hart" led is not blinking anymore

You are still using the plain, unmodified ULX3S setup from this repository, right?

Putty don't connect

If your terminal program does not connect there is a problem on your computer side. All the USB handling is done by the on-board FTDI chip and is not effected by the FPGA bitstream.

[zipotron]

You are still using the plain, unmodified ULX3S setup from this repository, right?

If you mean last version of master branch, yes.

If your terminal program does not connect there is a problem on your computer side. All the USB handling is done by the on-board FTDI chip and is not effected by the FPGA bitstream.

Make sense, but before the pull (was 3 days old code) was at least connecting and the (gpio0) led blinking, also, at that time I was trying with Windows also connecting but not showing text. And now nothing... I don't know what to think. Any suggestion @stnolting ?

[stnolting]

I do not think this is a bitstream issue. However, you could use the "last stable" version and try that: https://github.com/stnolting/neorv32/releases/tag/v1.6.2

Do you generate the bitstream by yourself or do you use the one from the project's implementation workflow?

Can you synthesize a setup just connecting rxd_input to txd_output (local UART echo)? This is the simplest way to check the UART connection is working.

[zipotron]

Hi @stnolting

Do you generate the bitstream by yourself or do you use the one from the project's implementation workflow?

Yes, I always synthesize locally.

Can you synthesize a setup just connecting rxd_input to txd_output (local UART echo)? This is the simplest way to check the UART connection is working.

I remember doing this when we were working in ULX3S and AlhambraII ports, actually was the verification way at that time, I never saw the UART interface in non of my boards, At that time I understood that was the verification way because something was pending to be implemented until see the UART interface.

Can you synthesize a setup just connecting rxd_input to txd_output (local UART echo)? This is the simplest way to check the UART connection is working.

Yes! I will try that! Also I will try luck to ask the ULX3S team if someone from their side can check also!

[stnolting]

Yes, I always synthesize locally.

You could try a bitstream from the Implementation workflow (for example the latest one: https://github.com/stnolting/neorv32/actions/runs/1378356656)

Yes! I will try that! Also I will try luck to ask the ULX3S team if someone from their side can check also!

👍

[zipotron]

You could try a bitstream from the Implementation workflow (for example the latest one: https://github.com/stnolting/neorv32/actions/runs/1378356656)

@stnolting , we have a problem, the same behavior, no blinking (gpio) led anymore and no UART connection. And I insist, before was blinking and connecting (just was not showing the bootloader menu). something changed in the last few day and break the ULX3S

[zipotron]

@stnolting I investigated
I checkout this commit: [sw/common] improved crt0 for bootloader case 0afa308 and is blinking the "gpio" led.
and in this commit: [rtl/core] updated pre-build memory images 9050319 stop working!
Something wrong with the new bootloader?

[stnolting]

That's interesting! Maybe I broke something updating the bootloader... The changes your are pointing at make use of the memory information from the SYSINFO module to setup the stack pointer: so basically, the bootloader's stack pointer setup is now "done" via the setting of the MEM_INT_DMEM_SIZE generic. I testes that new setup with severla DMEM sizes - without any problems. 🤔

I checkout this commit: [sw/common] improved crt0 for bootloader case 0afa308 and is blinking the "gpio" led.

Do you get the bootloader console via UART when using this version?

[stnolting]

Could you please re-run your synthesis using this neorv32_bootloader_image.vhd instead of the default one?
(Note that you need to remove the .TXT suffix)

• neorv32_bootloader_image.vhd.TXT

Maybe I screwed up the setup of the "global pointer"... I'm doing some more tests...

[stnolting]

Forget about my previous comment 😅
Please try this one: neorv32_bootloader_image.vhd.TXT

[zipotron]

@stnolting Hi! Sorry, during the week I just can be in this for few time at night... I tested!

Please try this one: neorv32_bootloader_image.vhd.TXT

I just tried, and I got:
Loading... ERROR_3

Also I saw that you did a commit! I am going I try that...
Not good also:
Loading... [EXC 0x00000005 0xffff0d58 0xffffffa8]

ERROR_1
The unique good point is I could connect the UART! (was my environment, permissions and Linux stuff)
I don't know how to help for solve this, too late and too much work today (I mean, the boring one that I got paid)...

[stnolting]

Great to see the bootloader connection is working again! 👍

Loading... ERROR_3

This error means there is a problem accessing the SPI flash (see 📚 Data Sheet: Bootloader Error Codes). Were you trying to boot from flash (bootloader command l) or storing an executable to flash (bootloader command s)?

ERROR_1

This means your executable is too big for the processor's IMEM. The default ULX3S setup implements 4kB of IMEM while the plain hex_viewer program requires about 8kB. I think the FPGA provides enough memory resources so you could increase IMEM size:

neorv32/setups/osflow/board_tops/neorv32_ULX3S_BoardTop_MinimalBoot.vhd

Line 78 in 935f43c

MEM_INT_IMEM_SIZE => 4*1024,

Another option would be to enable the compressed instructions ISA extension (set CPU_EXTENSION_RISCV_C generic true) and compile the hex_viewer program using make clean_all MARCH=rv32ic exe. This will reduce executable size by approx. ~30% - so you still need to increase IMEM size. 😉

Loading... [EXC 0x00000005 0xffff0d58 0xffffffa8]

This is really really bad! 😅
The bootloader encountered an unrecoverable exception - in this case it was trying to access the SPI module (at address 0xffffffa8) but there is no SPI module implemented in the ULX3S example setup. I think you might have executed the l command, right?

However, the bootloader should not terrify the user with some cryptic exception code here - it should just output ERROR_3 again (SPI/flash error). I will try to beautify that. 😉

[stnolting]

I just checked the ULX3S' FPGA datasheet - the Lattice ECP5 LFE5U-85 has plenty of block RAM (3744 kbit!!!) 🎉

How about a PR to upgrade the ULS3S setup's memory configuration? We could (should!) use the default memory configuration (i.e. 16kB IMEM, 8kB DMEM) so there is no need to adapt the default linker script when compiling executables for the board.

Just an idea (nice-to-have!) for a future setups:
The FPGA also provides large sysMEM blocks. I am not sure if the open-source synthesis flow can infer them (looking at you @umarcor - have you ever seen them being actually inferred? 😉). Anyway, we could add customized IMEM and DMEM modules just like in the UPduino setup to utilize those fallow memories.

edit
Forget about the last point... I just realized the sysMEM blocks are the "normal" block RAM modules and not fancy large-storage blocks like the ice40up's SPRAMs 😅

[zipotron]

How about a PR to upgrade the ULS3S setup's memory configuration?

Hi! Yes... as soon as I have a functional NeoRV32 in my ULX3S is will push the controller (that is already prepared and tested!)
I will try to find a windows of time this night to test all of last suggestions, If I can not, for sure weekend I am going a be in it hard!

[zipotron]

How about a PR to upgrade the ULS3S setup's memory configuration?

Umm, you mean a PR with the right memory configuration for execute the bootloader! Sure! As soon as I can jump on it!

[stnolting]

Adding an SDRAM controller to the board setup is something for the near future.

I was talking about some processor-internal updates. Just change the setup to use 16kB of instruction memory (IMEM) and 8kB of data memory (DMEM) instead of the current 4kB/8kB configuration. This setup also allows to use the project's software framework as-is (no modifications of the linker script required).

I can take care of that PR - basically, it is just changing two lines of code.

edit
✔️ done!

---

Regarding the SDRAM controller:
So, at some point you want to add support for FPGA-external memory - which is great! The problem with the current ULS3X example setup is that it does not expose the Wishbone interface at all! There are two ways to go:

1. Add SDRAM support to the repository's ULX3S example setup. This also means we have to add the SDRAM controller source codes and connect it to the core.
2. Just provide a Wishbone interface inside the example setup that is unused by default. Then you could use the setup as base platform, but you need to add the SDRAM controller locally.

[zipotron]

@stnolting you are fast! I am sorry, during the week I can not dedicate almost no time... But Saturday I will, I could see that you discard the first option because started with the Wishbone! Or shall we integrate directly in the core? Is just to know the strategy.

you need to add the SDRAM controller locally.

What do you mean with locally?
I could not test in the board the last commits, sorry, just open my personal computer for reply you, read your commits... and go to sleep, tomorrow I have to wake up unusually early. I will try to test tomorrow.

[stnolting]

I am sorry, during the week I can not dedicate almost no time...

No worries! Take your time.

Or shall we integrate directly in the core? Is just to know the strategy.

The PR (#189) is just something like a proof-of-concept. It adds an unconnected Wishbone bus to the setup that can be used by the user to attach custom modules - like your memory interface.

What do you mean with locally?

In the current state of that PR, you could download it an add the SDRAM controller locally on your machine without the SDRAM ctrl being explicitly added to the example setup in this repository.

I don't know if it is better to provide just a "Wishbone-ready" setup (which is the current state of #189) or to directly add the SDRAM controller to that setup. This is open for discussion 😉

[zipotron]

@stnolting I am testing!

This is really really bad! sweat_smile
The bootloader encountered an unrecoverable exception - in this case it was trying to access the SPI module (at address 0xffffffa8) but there is no SPI module implemented in the ULX3S example setup. I think you might have executed the l command, right?

No, didn't press any key to get that, and I am still getting the same in the master branch:

Loading...
[ERR 0x00000005 0xffff0d58 0xffffffa8]

ERROR_1

Even I tried to increase the memory to 32kb, the same, and if I override the image "neorv32_bootloader_image.vhd" the error still

ERROR_3

I checkout the branch "ulx3s_add_wishbone" and I couldn't synthesize using "make -C setups/osflow BOARD=ULX3S MinimalBoot"
Now I am free, and online, we can try fixes, and I can test!!

[stnolting]

Let's stay on the master branch at first.

No, didn't press any key to get that, and I am still getting the same in the master branch:

Does this appear right after resetting the processor?

ERROR_1

Can you post the whole bootloader output? Especially the hardware information it prints at the beginning.

[stnolting]

I checkout the branch "ulx3s_add_wishbone" and I couldn't synthesize using "make -C setups/osflow BOARD=ULX3S MinimalBoot"

Sorry, I have overseen that.
The PR uses a different design now. make -C setups/osflow BOARD=ULX3S Default should word.

[zipotron]

Does this appear right after resetting the processor?

Just after the "hart" led stop blinking

Can you post the whole bootloader output? Especially the hardware information it prints at the beginning.

That is the whole output!!! Just this 3 lines:

Loading...
[ERR 0x00000005 0xffff0d58 0xffffffa8]

ERROR_1

[stnolting]

Ah ok, I see.

This behavior is normal. After reset, the bootloader waits 8s for a user input. If there is no user input the bootloader starts the autoboot sequence and tries to fetch an executable via SPI from an external flash. Since there is no SPI module implemented in your design, you get this error: [ERR 0x00000005 0xffff0d58 0xffffffa8]

Can you upload an execute programs using the bootloader?
-> https://stnolting.github.io/neorv32/ug/#_uploading_and_starting_of_a_binary_executable_image_via_uart

To test this, please compile the sw/example/blink_led example program using make clean_all exe.

Connect to your FPGA board via UART, reset the processor and wait for the bootloader to show up. Hit any key to skip the autoboot sequence, then press u and upload the neorv32_exe.bin file from sw/example/blink_led. After the upload is complete, you can execute the program by pressing e.

---

edit

That is the whole output!!! Just this 3 lines:

That's all?? Even after resetting the processor again? There should be something like this:

<< NEORV32 Bootloader >>

BLDV: Oct  5 2021
HWV:  0x01060107
CLK:  0x05f5e100
MISA: 0x40901105
CPU:  0xc000072b
SOC:  0x1fff400f
IMEM: 0x00008000 bytes @0x00000000
DMEM: 0x00004000 bytes @0x80000000

Autoboot in 8s. Press key to abort.

[zipotron]

This behavior is normal. After reset, the bootloader waits 8s for a user input. If there is no user input the bootloader starts the autoboot sequence and tries to fetch an executable via SPI from an external flash. Since there is no SPI module implemented in your design, you get this error: [ERR 0x00000005 0xffff0d58 0xffffffa8]

Ouch! I am so sorry, I was expecting the menu pop-up after the led stop blinking! Yes, was working! I just try to push "H" key before the led stop blinking and is working! really sorry for make you work around for no reason, my fault. I just compile the Blink_led app and I am going a restart in Windows for uploading it! BTW, do you know any way to upload app from Linux?

[stnolting]

👍

Can you upload and execute the hex_viewer program? That would be the next step targeting SDRAM support.

BTW, do you know any way to upload app from Linux?

I only use graphical programs for Windows. On Linux I am just using the shell to interact with the UART's tty device. Not pretty, but handy 😉

[zipotron]

Blink_led app works!

Can you upload and execute the hex_viewer program? That would be the next step targeting SDRAM support.

Sure, I am going a try to do in Linux using the shell... OK! Works!
I think we have all necessary to start thinking in the SDRAM support, I will close this thread and we can start a new one when I push the first draft of the PR.
Thanks!

[stnolting]

Btw, we should continue the discussion regarding the Wishbone interface in #189

[Mecrisp]

@zipotron I am from the dark side, and I wrote a basic SDRAM controller for ULX3S in January. It is neither wishbone nor VHDL, but it is intended for teaching and well commented. You may use it for guidance: https://github.com/BrunoLevy/learn-fpga/blob/master/FemtoRV/RTL/SDRAM/muchtoremember.v

[zipotron]

@Mecrisp Hi! And thanks for came from the dark side and share your work! I had a look and seams really nice work. But unfortunately for my actual task don't help too much... I had already a a Verilog SDRAM controller (Is not mine but I tested and is working), and I wanted to re implement it in VHDL ( many reasons, like integrate with NeoRV32 without mixing VHDL and verilog, but the main reason, It was a personal chalenge). And I did it but never works... I suspect that could be a GHDL bug regarding "inout" ports, but couldnt verify yet. And last weeks I had some troubles at work, and lets be honest, I lose a lot of motivation.
@stnolting suggested me to use Lattice Diamond for investigate, but I am waiting for the account validation for get the free license already two weeks...
Just in case you have curiosity, we were discussing it in this thread: emb4fun/neorv32-examples#1 (comment)
Maybe I should go back to Verilog (In which I have more experience and knowledge) and then definitely use your controller!
Thanks a lot for care and help!!!