Hi 👋, This is the repository of M2O

Many-to-One Training Scheme for Event-to-Video Reconstruction

You can try M2O on Google Colab

Once you are in Google Colab, make sure to run all sections. At the end, you can download the resulting video. The script in Google Colab reconstructs only one sequence due to permanent storage limitations. To reconstruct the entire test set, you can download the code and data locally.

The following images show some of our results

Our M2O-E2VID model, trained with additional real event camera noise, is capable of removing hot pixels.

Additionally, the inclusion of the L2 loss reduces the checkerboard pattern.

The training， validation and test set are on Baidu Netdisk

The train set is at this link: train_set pass：1234
Place all training files (*.h5) inside the tr_m2o_data1 folder, as shown in the file tree below

The validation set is at this link: val_set pass：1234
Place all validation files (*.h5) inside the val_data folder, as shown in the file tree below

├── M2O
│ ├── ECD
│ ├── datasets_path
│ ├── imgs
│ ├── models_
│ ├── tr_m2o_data1
│ │ ├── tr_000000000.h5
│ │ ├── tr_000000000.h5
│ │ ...
│ │ └── tr_000005135.h5
│ ├── utils_
│ ├── val_data
│ │ ├── 000000001.h5
│ │ ├── 000000002.h5
│ │ ├── 000000003.h5
│ │ ├── 000000004.h5
│ │ └── 000000005.h5
└── ...

The test set is at test set pass：1234
Place all test files (*.h5) inside the datasets_path folder

To run the code you need:
python <= 3.10
pytorch <= 2.1.0
numpy <= 1.24.4
opencv <= 4.9.0
tqdm <= 4.66.2
argparse <= 1.1