Add support for flashing boards

[zguig52]

Hi guys,

I'm working on flashspi and board ECPIX5. I had first an issue with specific clock pin as described here: YosysHQ/prjtrellis#158

I finished by just removing the associated core clk pinout and have this finally:
# SPIFlash
("spiflash", 0,
Subsignal("cs_n", Pins("AA2")),
Subsignal("mosi", Pins("AE2")),
Subsignal("miso", Pins("AD2")),
Subsignal("wp", Pins("AF2")),
Subsignal("hold", Pins("AE1")),
IOStandard("LVCMOS33")
),
("spiflash4x", 0,
Subsignal("cs_n", Pins("AA2")),
Subsignal("dq", Pins("AE2", "AD2", "AF2", "AE1")),
IOStandard("LVCMOS33")
),
Is it the right way to do?

Second, now I would like to flash the FPGA config in it and saw that the make.py --flash is to be done in project linux on vexrisc. Have you a few guidelines/conclusions on how is planned to be implemented this function so I can try to do it?

[zguig52]

What I understood yet (for validation):

• loading to the flash the FPGA SOC needs first to program the FPGA with code to make a bridge and being able to access the chipset pins in programming mode (from the JTAG/FTDI link)
• ECPIX5 can be easily programmed with OpenFPGAloader (http://docs.lambdaconcept.com/ecpix-5/features/debug.html), with this command: openFPGAloader -b ecpix5 -f top.bit
• Using OpenOCD, there is an intermediate code input required to generate the bridge (flash_proxy_basename init property / await a .bit file)
• This code seems to be a good base to produce this intermediate bridge : https://github.com/quartiq/bscan_spi_bitstreams/blob/master/lattice_bscan_spi.py

[zguig52]

Did something working with:
1: changing the programmer type of board ecpix5 to OpenFPGALoader (litex_boards -> platforms/ecpix5.py):
def create_programmer(self):
return OpenFPGALoader("ecpix5")
2: adding the flash function to openfpgaloader to make its prototype compatible with all other models (litex -> build/openfpgaloader.py):
def flash(self, address, bitstream_file):
#address not used, for function prototype compatibility with other flash tools yet
cmd = ["openFPGALoader", "--board", self.board, "--write-flash", "--bitstream", bitstream_file]
self.call(cmd)
3. implementing flash option for linux vexrisc (make.py):
In class Board:
def flash(self, address, filename):
prog = self.platform.create_programmer()
prog.flash(address, filename)
[...]
In main:
if args.flash:
board.flash(address="0x000000", filename=os.path.join(builder.gateware_dir, soc.build_name + board.bitstream_ext))

[enjoy-digital]

Hi @zguig52,

thanks for looking at this. Linux-on-LiteX-VexRiscv is heavily relying on LiteX-Boards (https://github.com/litex-hub/litex-boards) and we should probably add the flash support there by adding a --flash to the targets that would Flash the bitstream in SPI-Flash (silmilar to the --load command that loads the bitstream in SRAM). For ECP5, we are already using OpenOCD to load FPGA bistreams, so if using OpenOCD is possible I would have a preference for this. Once supported in LiteX-Boards, we could reuse this in Linux-on-LiteX-VexRiscv.

[zguig52]

After some digging on the net, I see that there was a pull request related here:
quartiq/bscan_spi_bitstreams#7
It is also discussed here:
litex-hub/litex-boards#61

[zguig52]

I did some tests with current code for flashing through OpenOCD, with the svf file coming from @ilya-epifanov (from previous pull requests). I saw that he was also in charge of most of OpenOCD script associated.
I am stuck currently with this line from lattice/programmer.py: "flash bank spi0 jtagspi 0 0 0 0 ecp5.spi0.proxy 0x32":

Debug: 135 2 command.c:143 script_debug(): command - ocd_command ocd_command type ocd_flash bank spi0 jtagspi 0 0 0 0 ecp5.spi0.proxy 0x32
Debug: 136 2 command.c:143 script_debug(): command - ocd_flash ocd_flash bank spi0 jtagspi 0 0 0 0 ecp5.spi0.proxy 0x32
Error: 138 2 tcl.c:1026 handle_flash_bank_command(): 'jtagspi' driver rejected flash bank at 0x00000000; usage: (null)
Debug: 139 2 command.c:143 script_debug(): command - ocd_command ocd_command type ocd_usage flash bank
Debug: 140 2 command.c:143 script_debug(): command - usage ocd_usage flash bank
User : 142 2 command.c:828 command_help_show_indent(): User : 143 2 command.c:865 command_help_show(): flash bankUser : 144 2 command.c:871 command_help_show(): User : 145 2 command.c:844 command_help_show_wrap(): bank_id driver_name base_address size_bytes chip_width_bytes
User : 146 2 command.c:828 command_help_show_indent(): User : 147 2 command.c:828 command_help_show_indent(): User : 148 2 command.c:828 command_help_show_indent(): User : 149 2 command.c:828 command_help_show_indent(): User : 150 2 command.c:828 command_help_show_indent(): User : 151 2 command.c:828 command_help_show_indent(): User : 152 2 command.c:844 command_help_show_wrap(): bus_width_bytes target [driver_options ...]
User : 153 2 command.c:828 command_help_show_indent(): User : 154 2 command.c:865 command_help_show(): flash banksUser : 155 2 command.c:874 command_help_show():
User : 156 2 command.c:828 command_help_show_indent(): User : 157 2 command.c:865 command_help_show(): flash listUser : 158 2 command.c:874 command_help_show():
User : 159 2 command.c:828 command_help_show_indent(): User : 160 2 command.c:865 command_help_show(): mflash bankUser : 161 2 command.c:871 command_help_show(): User : 162 2 command.c:844 command_help_show_wrap(): soc_type base_addr pin_id target

Did some tests also just to flash the svf file, by commenting the specific lines where defining the target and had errors on reset halt command:
Debug: 6070 2238 svf.c:1548 svf_run_command(): TDO read = 0x0200100
User : 6071 2238 command.c:544 command_print():
Time used: 0m2s118ms
User : 6072 2238 command.c:544 command_print(): svf file programmed successfully for 1963 commands with 0 errors
User : 6073 2238 command.c:687 command_run_line(): invalid command name "reset"

[trabucayre]

Maybe I'm wrong somewhere or not understood everything, but with an ECP5 (and unlike xilinx or intel) a bridge loaded in RAM in not required.
After sending an specific instruction the FPGA give a direct access to the SPI flash. This way is used by openFPGALoader and @gregdavill ecpprog to write the bitstream.
And, for at least, openFPGALoader, it's possible to write both firmware at the beginning of the flash and a binary file at an arbitrary location by using --offset.

[gregdavill]

@trabucayre Is correct the ECP5 JTAG logic contains a command that enables pass-through to the FLASH pins, so you don't have to load a bridge bitstream.

It is possible to use openOCD + a crafted SVF file. But it's quite slow, as the standard SVF protocol does not support conditional loops/branching. So after each page write and sector erase you really need to wait the maximum time as highlighted in your flash datasheet.

I have an example of python script that creates a flat SVF here: https://gist.github.com/gregdavill/4f9f536757966171ef974f98348bbacb

I also wrote a simple tool: https://github.com/gregdavill/ecpprog and I believe that openFPGALoader also works to perform FLASH writes through the JTAG interface with FT232H based programmers.

This approach is much faster because the busy flag in the flash is polled, so erase/write times are closer to typical values, rather than max.

[trabucayre]

I also wrote a simple tool: https://github.com/gregdavill/ecpprog and I believe that openFPGALoader also works to perform FLASH writes through the JTAG interface with FT232H based programmers.

@gregdavill true: openFPGALoader is able to perform FLASH writes with any JTAG compatible device

[trabucayre]

But it's quite slow, as the standard SVF protocol does not support conditional loops/branching. So after each page write and sector erase you really need to wait the maximum time as highlighted in your flash datasheet.

True again. But it's not limited to the flash: each time a wait until status bit is (un)set, SVF add a delay instead of polling for the bit.

[enjoy-digital]

Thanks @trabucayre, @gregdavill for the info/feedback. So let's forget OpenOCD for flashing on ECP5 then, @zguig52 we could switch the ECPIX5 to openFPGALoader or ecpprog and I'll do a pass on the different LiteX-Boards later to be sure we have flashing support for most of the supported boards.

[zguig52]

Hi, thanks all for the reply and support.

I did a quick survey on openFPGALoader capabilities yet WRT litex-boards.

Native supported boards:

openFPGALoader --list-boards
board name:    cable_name
acornCle215     
arty            digilent
colorlight      
crosslinknx_evn ft2232
cyc1000         ft2232
de0nano         usb-blaster
ecp5_evn        ft2232
ecpix5          ecpix5-debug
fireant         ft232
ice40_generic   ft2232
licheeTang      anlogicCable
littleBee       ft2232
machXO2EVN      ft2232
machXO3EVN      ft2232
machXO3SK       ft2232
nexysVideo      digilent_b
pipistrello     ft2232
qmtechCycloneV  
spartanEdgeAccelBoard 
tangnano        ft2232
ulx2s           ft232RL
ulx3s           ft231X
xtrx            

Supported FPGA:

openFPGALoader --list-fpga
IDCode      manufacturer  family         model               
0x81111043  lattice       ECP5           LFE5UM5G-25         
0x81112043  lattice       ECP5           LFE5UM5G-45         
0x81113043  lattice       ECP5           LFE5UM5G-85         
0xe12bb043  lattice       MachXO3LF      LCMX03LF-1300C      
0x100381b   Gowin         GW1N           GW1N-4              
0x10f0043   lattice       CrosslinkNX    LIFCL-17            
0x10f1043   lattice       CrosslinkNX    LIFCL-40-ES         
0x1111043   lattice       ECP5           LFE5UM-25           
0x1112043   lattice       ECP5           LFE5UM-45           
0x1113043   lattice       ECP5           LFE5UM-85           
0x129a043   lattice       XP2            LFXP2-8E            
0x12b5043   lattice       MachXO2        LCMXO2-7000HE       
0x12b9043   lattice       MachXO2        LCMXO2-640HC        
0x12bd043   lattice       MachXO2        LCMXO2-7000HC       
0x20f30dd   altera        cyclone 10 LP  10CL025             
0x2b150dd   altera        cyclone V      5CEA2               
0x3620093   xilinx        spartan7       xc7s15ftgb196-1     
0x362c093   xilinx        artix a7 50t   xc7a50t             
0x362d093   xilinx        artix a7 35t   xc7a35              
0x362f093   xilinx        spartan7       xc7s50              
0x37c4093   xilinx        spartan7       xc7s25              
0x900281b   Gowin         GW1N           GW1N-1              
0xa014c35   anlogic       eagle s20      EG4S20BG256         
0x1100581b  Gowin         GW1N           GW1NR-9             
0x110f1043  lattice       CrosslinkNX    LIFCL-40            
0x13631093  xilinx        artix a7 100t  xc7a100             
0x13636093  xilinx        artix a7 200t  xc7a200             
0x21111043  lattice       ECP5           LFE5U-12            
0x212e3043  lattice       MachXO3D       LCMX03D-9400HC      
0x310f0043  lattice       CertusNX       LFD2NX-17           
0x310f1043  lattice       CertusNX       LFD2NX-40           
0x41111043  lattice       ECP5           LFE5U-25            
0x41112043  lattice       ECP5           LFE5U-45            
0x41113043  lattice       ECP5           LFE5U-85            
0x44008093  xilinx        spartan6       xc6slx45            
0x612b2043  lattice       MachXO3LF      LCMX03LF-1300E      
0x612b3043  lattice       MachXO3LF      LCMX03LF-2100E      
0x612b4043  lattice       MachXO3LF      LCMX03LF-4300E      
0x612b5043  lattice       MachXO3LF      LCMX03LF-6900E      
0x612b6043  lattice       MachXO3LF      LCMX03LF-9400E      
0x612bb043  lattice       MachXO3LF      LCMX03LF-2100C      
0x612bc043  lattice       MachXO3LF      LCMX03LF-4300C      
0x612bd043  lattice       MachXO3LF      LCMX03LF-6900C      
0x612be043  lattice       MachXO3LF      LCMX03LF-9400C      

[zguig52]

Here the quickly done compatibility matrix (to be checked / validated / ...)

FPGACompatilbility.docx

[zguig52]

Best would be to have a one tool fit all, but not sure it will be possible. Maybe the best is to put a layer of abstraction in linux on vexrisc build / flash functions (as it seems to be already ?). Then only the list of requirements might be different depending on the board (openocd / openfpgaloader / ...). Related to OpenFPGALoader VS ecpprog, what are the major pro and cons that you would see related to each solution?

As I understood / saw, there are 3 main operations to be done:

• load gateware live (seems that OpenOCD do this quite good for all boards?)
• write gateware to flash when possible (seems that it is hard/inappropriate to do it everywhere with OpenOCD as appears in the discussion)
• write boot code to flash when possible

What do you think to keep the following operations and add this new one with this prototype:

• [NO CHANGE] board.load(1 mandatory parameter, the gateware file)
• [CHANGED] board.flash_gateware (1 mandatory parameter, the gateware file) / if specific offset is required to write the gateware, maybe this is managed internally by the board itself
• [NEW] board.flash_firmware (1 mandatory parameter, the boot.json?) / there is something related to the offset where to write that has to be managed automatically?

Other point I would like to discuss is about the build output files gateware names, I think it might be useful to add to the file name the configuration associated (or a subfolder for each conf) to the build to avoid always having to rebuild when parameters are changing (when changing the CPU count for ex, and so on).

[enjoy-digital]

In fact each boards already comes with a default programmer. So it's not really an issue if the same programmer is not used for all boards, but we should still try to be consistent and avoid using too much different programmers. For Xilinx FPGA, OpenOCD is working fine and we are also using it for JTAG UART, so we are going to keep it, but for Lattice, Intel and others we could eventually change. Not sure we should keep OpenOCD to load the gateware if another programmer will be used for Flashing.

For the operations, you are right. The current load_bitstream and flash methods of GenericProgrammer allow the two first operations you are listing and part of the third one (not handling a .json file). We could probably make sure LiteX-Boards have support for flashing the gateware and once done we could add a way to flash specific images.

For the build name, we could indeed add at least add the cpu count to it (ex ecpix5_cc1, ecpix5_cc2, etc... as done for the VexRiscv cluster name) and eventually extend it to others parameters in the future.

[zguig52]

OK great!

Still a few questions:

1. Flash board gateware:

• for implementation of load_bitstream and flash methods, shall I reuse the "generic openfpgaloader.py" or a new generic ecpprog.py, depending on the one we go for, or shall I go for a specific class in lattice/programmer.py?
• if we stick to the actual generic openfpgaloader.py, then, do we keep the one function load_bitstream on which we add the offset parameter and at board level, this function is called with different parameters depending on the use case or we have 2/3 functions with specific names for each operations (as discussed before)

2. Flash board firmware

• regarding to the "flash_firmware" operation, here are the things to take in account (for verification/correction/completion):

1. The BIOS/OpenSBI is the one in charge to read at a specific address in flash to search for a startup code: this is fixed / linked to the binary?
2. Depending on the board/components setup, the gateware is more or less heavy in size, so how would this start address be managed dynamically? Reserving a fixed size for gateware and always loading the firmware at a specific address?

Shall we first manage the "flash_gateware" operation with current functions prototype (flash with offset 0), and flash_firmware with a specific offset or shall we go directly to 2 different functions prototypes (as there is a big coupling to all boards files to be reviewed)?

[trabucayre]

Thanks for my "order list" :)
More seriously, some quick note for openFPGALoader:

• if the second column in board list is empty this mean user must provides manually the cable name (-c)
• orangeCrab is missing but supported (currently JTAG only)
• fomu is DFU not direct SPI access (I'm working on this protocol, will release this ASAP need a cleanup -> tested with orangeCrab)
• iceBreaker is supported (ice_generic board)
• since pipistrello is okay all spartan6 too, same for cyclone10 and IV.
• TinyFPGA has a dedicated protocol no ?
• zynq and socfpga are not supported (current limitation is one device in the chain)
• for xilinx (and, WIP, altera) a bridge is available for some devices to write flash

[enjoy-digital]

@zguig52 We should probably first make sure we can provide a flash method for the ECPIX5 (that was your original goal, with OpenFPGALoader if you want yes), then try to extend this to the different ECP5 boards (I could do it unless you want to) and others boards (could also do it) and once done, we could see how to flash multiple images at one (with a boot.json file, we could flash both gateware, firmware and eventual configuration data at once). Working first on the other aspects give us time to think and find a nice solution for the last operation :)

[mithro]

@enjoy-digital - Sadly, as OpenFPGALoader is licensed under the AGPL a large number of people contributing to this repository can not use the software (or even look at the software), hence we will need to support an alternative tool as well.

[trabucayre]

@enjoy-digital - Sadly, as OpenFPGALoader is licensed under the AGPL a large number of people contributing to this repository can not use the software (or even look at the software), hence we will need to support an alternative tool as well.

What is the best license and the way to change this?
The most important thing is not to limit access, but to allows absolutely freely use it for everyone.

[mithro]

What is the best license and the way to change this?
The most important thing is not to limit access, but to allows absolutely freely use it for everyone.

@trabucayre -- Changing the license of a project which has been accepting contributions but not using a CLA is a fairly complicated process which is not guaranteed to be successful (see the original GCC OpenRISC support issue). I would be happy to walk you through the process.

[trabucayre]

Yes of course!

[zguig52]

@enjoy-digital I did commits to add flash to ECPIX5 board only so you can have a check if this is what you were expecting in terms of implementation. If this is what you were expecting, I can also take in charge to add this function to all ECP5 boards, no problem (and maybe others that have not yet this method and that are supported by OpenFPGALoader)

[zguig52]

@enjoy-digital - Sadly, as OpenFPGALoader is licensed under the AGPL a large number of people contributing to this repository can not use the software (or even look at the software), hence we will need to support an alternative tool as well.

@mithro, what is the issue with AGPL licence use? Here we only call the binary file to load code, so like a proprietary software, the license is not "propagated", isn't it? When reading the AGPL licence, there is something, but only related to network usage of the software. I don't think we are in this use case.

@trabucayre , best licenses to support use in both free and proprietary software are MIT / BSD from what I know. It literally doesn't oblige anybody to contribute and this comes only from free will (to the contrary of GPL). Do you agree @mithro ?

[enjoy-digital]

Closing since no update. The general direction for this repository if to reuse as much as possible from LiteX and LiteX-boards repository. So boards with flashing directly supported in LiteX-Boards will also be directly supported here.