CEFS

Official code for the TPAMI paper "Coded Event Focal Stack for Continuous Refocusing in Dynamic Scene".

CEFS uses event streams and two boundary RGB frames to synthesize continuously refocused images in dynamic scenes. This repository contains the training code, evaluation code, real-data inference path, safe event preprocessing tools, and release documentation.

Status

• Source code is released under Apache License 2.0.
• Model weights released by the authors are Apache-2.0 model assets.
• Datasets are not included in this repository. The CEFS DF-Motion companion dataset page is moonboat/cefs-df-motion.
• Legacy pretrained checkpoints are included under checkpoints/legacy/.

Installation

conda create -n cefs python=3.12
conda activate cefs
pip install -r requirements.txt

The pinned environment was verified with Python 3.12 and CUDA-enabled PyTorch. If your CUDA runtime or platform differs, install a matching torch / torchvision pair before installing the remaining dependencies.

Repository Layout

CEFS/
├── __main__.py        # unified train / evaluation entry point
├── datasets/          # dataset loaders
├── modules/           # model definitions
├── trainer/           # training loops
├── test/              # evaluation and real-data inference
├── tools/             # safe preprocessing / visualization CLIs
├── scripts/           # portable launch-script templates
├── utils/             # options, logging, checkpoint, and shared utilities
├── docs/              # release, data, checkpoint, training, and evaluation notes
└── requirements.txt

Checkpoints

The current evaluation and real-data inference paths use the legacy inference models in modules/legacy/.

Expected public checkpoint layout:

checkpoints/legacy/
├── cefs_legacy_inter_stack64.pth
└── cefs_legacy_focus_stack64.pth

These files are committed to this repository for the first release. See docs/checkpoints.md for architecture compatibility and SHA256 metadata.

Data

Datasets are hosted outside the code repository. The CEFS DF-Motion companion dataset page is:

https://huggingface.co/datasets/moonboat/cefs-df-motion

Hugging Face files:

• test/cefs_df_motion_test.tar.gz: DF-Motion test data
• train/scene_001_events.tar.gz and train/scene_001_aif.tar.gz: partial training assets for scene 1

The full 30-scene training split is not fully uploaded yet. Package it in smaller verified parts before public use.

Download:

huggingface-cli download moonboat/cefs-df-motion \
  --repo-type dataset \
  --local-dir data/cefs_df_motion

The main layouts expected by the current code are:

• real-data inference: IMG/ and event_stacks_test_15/
• real-video inference: IMG/ and event_stacks_video/
• simulation evaluation: scene folders with events/0.npy and gt/
• training: scene folders with event, ground-truth, and optional all-in-focus targets

See docs/data_format.md for complete layout examples.

Usage

Show all command-line options:

python __main__.py --help

Real-data inference:

python __main__.py \
  --mode exp \
  --test_real \
  --StackSize 64 \
  --evs_ratio 5 \
  --test_data_path /path/to/recording \
  --ckpt_inter checkpoints/legacy/cefs_legacy_inter_stack64.pth \
  --ckpt_focus checkpoints/legacy/cefs_legacy_focus_stack64.pth

Synthetic evaluation:

CKPT_INTER=checkpoints/legacy/cefs_legacy_inter_stack64.pth \
CKPT_FOCUS=checkpoints/legacy/cefs_legacy_focus_stack64.pth \
DATA_PATH=/path/to/synthetic \
OUTPUT_PATH=artifacts/output/synthetic \
bash scripts/eval_synthetic.sh

Camera-shake evaluation:

CKPT_INTER=checkpoints/legacy/cefs_legacy_inter_stack64.pth \
CKPT_FOCUS=checkpoints/legacy/cefs_legacy_focus_stack64.pth \
DATA_PATH=/path/to/camera_shake \
OUTPUT_PATH=artifacts/output/camera_shake \
bash scripts/eval_camerashake.sh

Training scripts are provided as templates under scripts/. They use repo-relative defaults and can be configured through environment variables. See docs/training.md for the script-to-dataset mapping.

Event Tools

Generate event stacks from raw event text files:

python tools/make_event_stack.py \
  --input /path/to/recording \
  --height 720 \
  --width 1280 \
  --stack-size 64 \
  --sweep-frames 12 \
  --start 480

Visualize event text or stack files:

python tools/visualize_events.py \
  --input /path/to/events.txt \
  --height 720 \
  --width 1280 \
  --output /path/to/events.png

Use --sweep-frames 12 for 24 fps real-data recordings and --sweep-frames 15 for 30 fps recordings. Generated outputs are not overwritten by default; pass --overwrite explicitly to replace existing files.

Verification

Run the release-surface checks:

python __main__.py --help
bash -n scripts/*.sh
python tools/make_event_stack.py --help
python tools/visualize_events.py --help
python -m py_compile $(find . -path './.git' -prune -o -path './artifacts' -prune -o -name '*.py' -print)

These checks do not require datasets or checkpoints.

Documentation

• docs/checkpoints.md: checkpoint families, metadata, and compatibility notes
• docs/data_format.md: expected event, stack, real-data, simulation, and training layouts
• docs/evaluation.md: evaluation entry points and metric outputs
• docs/huggingface_dataset.md: DF-Motion dataset release and upload notes
• docs/training.md: training scripts and dataset assumptions
• docs/reproduction.md: minimal reproduction notes
• docs/release_audit.md: release-readiness audit

Citation

@article{teng2026coded,
  title = {Coded Event Focal Stack for Continuous Refocusing in Dynamic Scene},
  author = {Teng, Minggui and Xuan, Suhang and Yan, Zhiang and Lou, Hanyue and Li, Boyu and Fan, Bin and Shi, Boxin},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  volume = {48},
  number = {6},
  pages = {7147--7160},
  year = {2026},
  doi = {10.1109/TPAMI.2026.3664082}
}

License

Source code is released under the Apache License 2.0. Author-released model weights are Apache-2.0 model assets. Datasets are distributed separately from the code repository.