The aim of this project is to achieve real time dehazing for video (or image).
We utilize DE2-115 FPGA, ethernet connection, camera TRDB-D5M and display through VGA in this project.
With our own router, we achieve 10 FPS in RGB24 and 15 FPS in RGB16.
It;'s a short demo gif. The quality is not that good due to conversion from video to gif.
The diagram shows the connection of ethernet mode. In camera mode, the data and valid line should be from RAW2RGB module.
This project is the final project for NTUEE Digital Circuits Lab, 2021 Fall.
Contributor:
- Dehaze picture or video input via ethernet connection or camera (The algorithm is hardware-friendly)
You can visit CModel for more information
- Achieve real time usage
- Currently only support 640x480 image input
- For ethernet input, only support UDP protocol
- For ethernet input, only support RGB24/RGB16 bit input
- You can use either camera or ethernet as your image input (refer to step below)
- Default ethernet
- Connect VGA to display
- Connect TRDB-D5M or ethernet wire to input image
- Use switch 1 for displaying transmission map
- Use switch 2 for displaying dehazed image
- Button 0 to reset VGA & SDRAM & Camera
- Button 1 to reset Ethernet and LCD
Run FPGA and send image (or video) by python
For ethernet & FPGA usage, we refer to alexforencich's project. Reference here
Make sure your PC (python) is in the same local network as your FPGA
- Please config src/Ethernet.v Line 200 and use your own IP address
- Config RGB16 in config.h (Default RGB24)
⚠️ Remember to config HOST and PORT in src/Ethernet.v file for your FPGA IP address
- python/send_video.py: send mp4 file (slower)
- python/send_stream.py: send streaming image (HLS protocol) from provided URL (slower)
stream only support RGB24
- python/send_rawfile.py: send raw file (generated by video2file) (faster)
- python/video2file.py: convert mp4 into rawfile
- python/send_image.py: send jpg file
⚠️ Remember to config HOST and PORT in each python file for your FPGA IP address
Because we use UDP protocol, please make sure your internet condition is good, otherwise some data might lost and cause display failure. In this case, you may increase the DELAY in python file to ensure the data was sent successfully (but lower FPS).
Config DE2-115/config.h and define Camera
For TRDB-D5M & FPGA usage, we refer to Ahmed Abdelfattah's project. Reference here
- For camera input, follow steps in "Camera Input" and compile through Quartus.
- For ethernet input, make sure you modify IP address for FPGA and compile through Quartus.
We use quartus to compile and generate the output files. Remember to import assignment (fpga.qsf) and add the following files to the path of your project.
-
all files in src/
-
all files in DE2-115/
-
verilog-ethernet/rtl/arp.v
-
verilog-ethernet/rtl/arp_cache.v
-
verilog-ethernet/rtl/arp_eth_rx.v
-
verilog-ethernet/rtl/arp_eth_tx.v
-
verilog-ethernet/rtl/axis_gmii_rx.v
-
verilog-ethernet/rtl/axis_gmii_tx.v
-
verilog-ethernet/rtl/eth_mac_1g_rgmii_fifo.v
-
verilog-ethernet/rtl/eth_mac_1g_rgmii.v
-
verilog-ethernet/rtl/eth_mac_1g.v
-
verilog-ethernet/rtl/eth_axis_rx.v
-
verilog-ethernet/rtl/eth_axis_tx.v
-
verilog-ethernet/rtl/eth_arb_mux.v
-
verilog-ethernet/rtl/eth_mux.v
-
verilog-ethernet/rtl/iddr.v
-
verilog-ethernet/rtl/ip.v
-
verilog-ethernet/rtl/ip_arb_mux.v
-
verilog-ethernet/rtl/ip_complete.v
-
verilog-ethernet/rtl/ip_eth_rx.v
-
verilog-ethernet/rtl/ip_eth_tx.v
-
verilog-ethernet/rtl/ip_mux.v
-
verilog-ethernet/rtl/lfsr.v
-
verilog-ethernet/rtl/oddr.v
-
verilog-ethernet/rtl/rgmii_phy_if.v
-
verilog-ethernet/rtl/ssio_ddr_in.v
-
verilog-ethernet/rtl/ssio_ddr_out.v
-
verilog-ethernet/rtl/udp_complete.v
-
verilog-ethernet/rtl/udp_checksum_gen.v
-
verilog-ethernet/rtl/udp.v
-
verilog-ethernet/rtl/udp_ip_rx.v
-
verilog-ethernet/rtl/udp_ip_tx.v
-
verilog-ethernet/lib/axis/rtl/arbiter.v
-
verilog-ethernet/lib/axis/rtl/axis_fifo.v
-
verilog-ethernet/lib/axis/rtl/axis_async_fifo.v
-
verilog-ethernet/lib/axis/rtl/axis_async_fifo_adapter.v
-
verilog-ethernet/lib/axis/rtl/priority_encoder.v
-
verilog-ethernet/lib/axis/rtl/sync_reset.v
-
fpga.sdc
- Support jpg input format and implement decoder on FPGA
- Support higher resolution and higher FPS
- Solve display failure issue (Maybe use TCP instead or add check module on FPGA)