1280x720p with SDRAM achieved

[suarezvictor]

Hi, I was playing with litex since recently and achieved to display a pretty high resolution framebuffer based on a simple SDRAM like in ULX3S.
Resolutions are 1280x720p 16-bit color (60Hz refresh rate) and 1024x768 32-bit color (50Hz).
I had to use "reduced blanking" timings. They are not custom modes, I'm using recomended CVT timings except for 50Hz which has pixel clock proportional to the 60Hz standard but I guess al monitors will be happy at 50Hz (I tested results on two different monitors with success).
As far as I know there is no previous examples of such resolutions with a simple SDRAM, but since I'm new with litex I may be wrong...
My setup is a Cyclone IV on a Terasic DE0 Nano, 16-bit 143MHz SDRAM and a VGA output based on a resistors DAC (HDMI will be tested soon).
I'm willing to share code but it's currently a mess... Anyways it kinda works, the image of the random power-on bits from SDRAM appears on the screen and monitors show correct timings.
I'd also appreciate if someone can explain half-rate SDRAM access since I can use it to clock memory at 144Mhz but for the 16-bit case I use just half clock rate at 1:1. Does such halfrate module converts to 32 bit bus to simulate the lower external clock?
I had to implement the 16-bit framebuffer myself since litevideo only supports 32-bit color, but I'm unsure on how to use doubled SDRAM clocks for my 16-bit color framebuffer.
Writing a real image to the SDRAM will be my next experiment so I can share photos... but I'm not sure the base addreess of framebuffer.
Any help?

[enjoy-digital]

Hi @suarezvictor,

very interesting, thanks for sharing. Are you using LiteDRAM in your design? If so, in 1:1 rate? 1:2 rate would indeed allow you to have a 32-bit native interface and would lower the CPU clock frequency. Feel free to share your code even if you think this is not clean enough, it will be easier to understand your design and help.

[suarezvictor]

I'm using al of litex designs with a bit of tweaking
Where I can post it in a way that muy name as contributor will be preserved?
I'm using 1:2 for 1024x768 32bpp and 1:1 for 1280x720 16bpp. In first case I detected the píxel clock should be less (and not equal) 2/3 of píxel clock. In second case pixel clock has to be about same fraction (you may be able to explain)

[suarezvictor]

[suarezvictor]

Carification: in 1:2 case pixel clock has to be < 2/3 of CPU clock

[enjoy-digital]

@suarezvictor: We are trying to be very careful about keeping contributors listed and preserved for all contributions, so if your changes are not too specific to your hardware and are something we could merge, we'll of course be careful to preserve your name for your contribution.

[suarezvictor]

Perfect, let me add some comments and I'll post the sources on Monday

[suarezvictor]

HIGH RESOLUTION VIDEO DRIVER USING 16-bit SDRAM CHIP (1024x768@32bpp and 1280x720@16bpp)

terasic_de0nano.py.txt

Based on DE0-Nano board (Cyclone IV FPGA with a 143Mhz capable SDRAM, 16-bit bus)
A VGA resistor DAC is connected to respective pins on JP3 (search this source for this string: VGA PINS)

1024x768 characteristics:

Name: "1024x768@50Hz(RB)"
Pixel clock: 46.42MHz
Color depth: 32-bit (RGBA888)
Vertical rate: 50Hz
Sync polarity*: -hsync -vsync
Sys clock: 72.2MHz
SDRAM clock: 144.4MHz
Build Command: ./terasic_de0nano.py --with-video-framebuffer --sys-clk-freq=72222222 --sdram-rate=1:2 --build

1280x720 characteristics:

Name: "1280x720@60Hz(RB)"
Pixel clock: 61.9 MHz
Color depth: 16-bit (RGB565)
Vertical rate: 60Hz
Sync polarity: -hsync -vsync
Sys clock: 100MHz
SDRAM clock: 100MHz
Build command: ./terasic_de0nano.py --with-video-framebuffer --sys-clk-freq=100000000 --build

NOTES:

Timings are based on CVT standard with Reduced Blanking Time (put less pressure in pixel clock)
The timings are obtained by linux "cvt" tool, for example:
# cvt -r 1024 768 60
outputs:
Modeline "1024x768R" 56.00 1024 1072 1104 1184 768 771 775 790 +hsync -vsync

NOTE: besides +hsync is specified, testing shows we beed -hsync
Pixel clock is adjusted to 46.42MHz (instead of 56Mhz) to put less pressure on SDRAM and to mantain < 2/3 rate with SDRAM clock (unavoidable in current design)
The result is that framerate gets to 50Hz and not to the 60Hz standard. Anyways most (if not all) monitors will be happy to show 50Hz Vsync if they're capable of 60Hz.
The 50Hz obtained is not exact but very close, maybe output is 49ish-Hz (pixel clock is set to 46.42 instead of 46.77). To more exact adjusting, some more PLL trickery would be needed or blanking timings could be tweaked to sightly off-standard values. In regards to 1280x720, the pixel clock is also not exact but very close (61.9 is used instead of 64.02). Same criteria applies.

In case of 1280x720, the SDRAM is not maxed to their utmost clock capacity since that would require the 1:2 logic but in that case pixel clock would be too close to sys clock. Some trickery with clock domain crossig would be needed to achieve higher SDRAM clock rates (HIGHLY DESIRED FEATURE).

Remaining bandwidth for writing to SDRAM is completely untested, but should be enough for something useful.

PATCHING NEEDED

Instead of touching many sources of litex project, this sole source is enough to reproduce the results obtained. When needed, variable of the correspondig module that needs touching is changed, or functions copied and modified. For example, to add videomodes, video_timings variable from module litex.soc.cores.video is changed before use.

In regards to vsync and hsync polarity, un the current version of litex this is not specified in the video mode, but it would be very useful if such capacity is specified directly on the mode description. Additionally, in current version same polarity is assumed for vertical and horizontal when that should be independient. This source code could be used for adequate patching.

A very important correction done is about LiteDRAMDMAReader usage: fifo depth is greately decreased since the default makes the project unable to fit. A significant lower value is needed. Unfortunately this seems hardcoded at the VideoFrameBuffer creation so a patched one copied from original implementation is used (VideoFrameBufferN class). It's also parametrized by bit depth: a change from the 32-bit default to add support to 16-bit was easy and seemed to work well.

In regards to 16-bit mode, a conversion from 16-bit RGB565 to 24-bit RGB888 is needed but was not implemented (currently of the 16 bits, portions of 8 bits are used, since this project was mainly to test capabiliy of using a 16-bit SDRAM to achieve high color and resolutions). Providing support for that conversion should be trivial.

The "Video Terminal" (--with-video-terminal) mode is supported but cannot work above 800x600 since much exigence on block ram that Cyclone IV cannot provide. So test were conducted just with "Video Framebuffer" (--with-video-framebuffer)

VideoCustomPHY is also patched (i.e copied and modified) mainly to support exporting "de" (Data Enable) signal to a pin, and to allow vsync and hsync of contrary polarities.

_CRG was patched to support the separate "vga" clock domain in a similar way as the de10lite (terasic_de10lite.py)
NOTE: the PLL is configured with tight marginfs (1e-3) to allow better relationship between needed pixel clocks and sys/sdram clocks.

BaseSoC was patched to support adding the video terminal or framebuffer, and correspondingly the main function to such the parameters.

Default UART maybe moved to JP3 pins (this is for my own convenience), see comment: UART PINS

[suarezvictor]

Promised on monday, a little late but done: still monday in Buenos Aires ;-)

[enjoy-digital]

Thanks @suarezvictor, that's very interesting. The current VideoFramebuffer currently support the most common use-cases and your work will be useful to improve it and make it more generic. We'll try to remove some of the limitations you saw.

[suarezvictor]

Good To heard that. I can convert this proof of concept to a pull request is needed.
I'll apreciate some guidance on how to use 1:2 SDRAM access (for example SDRAM at 144mhz and CPU at 72mhz) but at same time using a 66Mhz pixel clock. Un current implementation pixel clock needs to be < 2/3 of sys clock but it would be needed to approach sys clock. How to cross domains at more than 2/3 or equivalently, more than 1/3 of SDRAM clock? Referring to sys clock is not needed in case RAM clock is 144mhz and needed pixel clock is 66mhz bit the 1:2 module seems not to allow that

[rdolbeau]

@enjoy-digital pointed this out to me after I submitted a PR to fully support RGB565 (16-bits) in the VideoFramebuffer in Litex (with DTS support so that Linux can use it), see enjoy-digital/litex#1053

[suarezvictor]

Would you show us the commands needed to see the linux driver working on some FPGA board?

[rdolbeau]

@suarezvictor Normally nothing is needed; the Litex framebuffer is described in the DTS file generated by Litex. Linux should automatically configure the 'simple-framebuffer' driver from the information in the DTS. It does for me in both RGBA8888 and RGB565 with Linux-on-Litex-Vexrisc.

[enjoy-digital]

Thanks @suarezvictor, @rdolbeau, the VideoOut core in LiteX should now allow RGB888 and RGB565 modes.