Merge branch 'main' into digicam_example

CHANGELOG.rst

@@ -13,6 +13,25 @@ Unreleased
 Added
 ~~~~~
 
+- Nothing
+
+Changed
+~~~~~~~
+
+- Nothing
+
+Bugfix
+~~~~~~
+
+- Nothing
+
+
+1.0.7 - (2024-05-14)
+--------------------
+
+Added
+~~~~~
+
 - Script to upload measured datasets to Hugging Face: ``scripts/data/upload_dataset_huggingface.py``
 - Pytorch support for simulating PSFs of masks.
 - ``lensless.hardware.mask.MultiLensArray`` class for simulating multi-lens arrays.
@@ -21,12 +40,13 @@ Added
 - ``lensless.utils.dataset.simulate_dataset`` for simulating a dataset given a mask/PSF.
 - Support for training/testing with multiple mask patterns in the dataset.
 - Multi-GPU support for training.
-- DigiCam dataset which interfaces with Hugging Face.
+- Dataset which interfaces with Hugging Face (``lensless.utils.dataset.HFDataset``).
 - Scripts for authentication.
 - DigiCam support for Telegram demo.
 - DiffuserCamMirflickr Hugging Face API.
 - Fallback for normalization if data not in 8bit range (``lensless.utils.io.save_image``).
 - Add utilities for fabricating masks with 3D printing (``lensless.hardware.fabrication``).
+- WandB support.
 
 Changed
 ~~~~~~~
@@ -35,6 +55,8 @@ Changed
 - For trainable masks, set trainable parameters inside the child class.
 - ``distance_sensor`` optional for ``lensless.hardware.mask.Mask``, e.g. don't need for fabrication.
 - More intuitive interface for MURA for coded aperture (``lensless.hardware.mask.CodedAperture``), i.e. directly pass prime number.
+- ``is_torch`` to ``use_torch`` in ``lensless.hardware.mask.Mask``
+- ``self.height_map`` as characterization of phase masks (instead of phase pattern which can change for each wavelength)
 
 
 Bugfix
@@ -151,7 +173,7 @@ Added
 -  Option to warm-start reconstruction algorithm with ``initial_est``.
 -  TrainableReconstructionAlgorithm class inherited from ReconstructionAlgorithm and torch.module for use with pytorch autograd and optimizers.
 -  Unrolled version of FISTA and ADMM as TrainableReconstructionAlgorithm with learnable parameters.
-- ``train_unrolled.py`` script for training unrolled algorithms.
+- ``train_learning_based.py`` script for training unrolled algorithms.
 - ``benchmark_recon.py`` script for benchmarking and comparing reconstruction algorithms.
 - Added ``reconstruction_error`` to ``ReconstructionAlgorithm`` .
 - Added support for npy/npz image in load_image.

README.rst

@@ -45,7 +45,7 @@ The toolkit includes:
 * Measurement scripts (`link <https://lensless.readthedocs.io/en/latest/measurement.html>`__).
 * Dataset preparation and loading tools, with `Hugging Face <https://huggingface.co/bezzam>`__ integration (`slides <https://docs.google.com/presentation/d/18h7jTcp20jeoiF8dJIEcc7wHgjpgFgVxZ_bJ04W55lg/edit?usp=sharing>`__ on uploading a dataset to Hugging Face with `this script <https://github.com/LCAV/LenslessPiCam/blob/main/scripts/data/upload_dataset_huggingface.py>`__).
 * `Reconstruction algorithms <https://lensless.readthedocs.io/en/latest/reconstruction.html>`__ (e.g. FISTA, ADMM, unrolled algorithms, trainable inversion, pre- and post-processors).
-* `Training script <https://github.com/LCAV/LenslessPiCam/blob/main/scripts/recon/train_unrolled.py>`__ for learning-based reconstruction.
+* `Training script <https://github.com/LCAV/LenslessPiCam/blob/main/scripts/recon/train_learning_based.py>`__ for learning-based reconstruction.
 * `Pre-trained models <https://github.com/LCAV/LenslessPiCam/blob/main/lensless/recon/model_dict.py>`__ that can be loaded from `Hugging Face <https://huggingface.co/bezzam>`__, for example in `this script <https://github.com/LCAV/LenslessPiCam/blob/main/scripts/recon/diffusercam_mirflickr.py>`__.
 * Mask `design <https://lensless.readthedocs.io/en/latest/mask.html>`__ and `fabrication <https://lensless.readthedocs.io/en/latest/fabrication.html>`__ tools.
 * `Simulation tools <https://lensless.readthedocs.io/en/latest/simulation.html>`__.

configs/benchmark.yaml

@@ -7,15 +7,15 @@ hydra:
     chdir: True
 
 
-dataset: DiffuserCam   # DiffuserCam, DigiCamCelebA, DigiCamHF
+dataset: DiffuserCam   # DiffuserCam, DigiCamCelebA, HFDataset
 seed: 0
 
 huggingface:
   repo: "bezzam/DigiCam-Mirflickr-MultiMask-25K"
   image_res: [900, 1200]  # used during measurement
   rotate: True   # if measurement is upside-down
   alignment:
-    topright: [80, 100]  # height, width
+    top_left: [80, 100]  # height, width
     height: 200
   downsample: 1
 
@@ -88,6 +88,7 @@ simulation:
   scene2mask: 0.25   # [m]
   mask2sensor: 0.002   # [m]     
   # see waveprop.devices
+  use_waveprop: False    # for PSF simulation
   sensor: "rpi_hq"
   snr_db: 10
   # simulate different sensor resolution

configs/fine-tune_PSF.yaml

@@ -1,4 +1,4 @@
-# python scripts/recon/train_unrolled.py -cn fine-tune_PSF
+# python scripts/recon/train_learning_based.py -cn fine-tune_PSF
 defaults:
   - train_unrolledADMM
   - _self_
@@ -12,25 +12,7 @@ trainable_mask:
 
 #Training
 training:
-  save_every: 10
-  epoch: 50
-  crop_preloss: False
+  save_every: 1   # to see how PSF evolves
 
 display:
   gamma: 2.2
-
-reconstruction:
-  method: unrolled_admm
-
-  pre_process:
-    network: UnetRes
-    depth: 2
-  post_process:
-    network: DruNet
-    depth: 4
-
-optimizer:
-  slow_start: 0.01
-
-loss: l2
-lpips: 1.0

configs/train_celeba_digicam_hitl.yaml

@@ -1,7 +1,7 @@
 # Learn mask with HITL training by setting measure configuration (set to null for learning in simulation)
 # 
 # EXAMPLE COMMAND: 
-# python scripts/recon/train_unrolled.py -cn train_celeba_digicam_hitl measure.rpi_username=USERNAME measure.rpi_hostname=HOSTNAME files.vertical_shift=SHIFT
+# python scripts/recon/train_learning_based.py -cn train_celeba_digicam_hitl measure.rpi_username=USERNAME measure.rpi_hostname=HOSTNAME files.vertical_shift=SHIFT
 
 defaults:
   - train_celeba_digicam

configs/train_celeba_digicam_mask.yaml

@@ -1,5 +1,5 @@
 # fine-tune mask for PSF, but don't re-simulate
-# python scripts/recon/train_unrolled.py -cn train_celeba_digicam_mask
+# python scripts/recon/train_learning_based.py -cn train_celeba_digicam_mask
 defaults:
   - train_celeba_digicam
   - _self_

configs/train_coded_aperture.yaml

@@ -1,4 +1,4 @@
-# python scripts/recon/train_unrolled.py -cn train_coded_aperture
+# python scripts/recon/train_learning_based.py -cn train_coded_aperture
 defaults:
   - train_unrolledADMM
   - _self_

configs/train_digicam_celeba.yaml

@@ -1,4 +1,4 @@
-# python scripts/recon/train_unrolled.py -cn train_digicam_singlemask
+# python scripts/recon/train_learning_based.py -cn train_digicam_celeba
 defaults:
   - train_unrolledADMM
   - _self_
@@ -13,6 +13,7 @@ files:
   huggingface_psf: "psf_simulated.png"
   huggingface_dataset: True
   split_seed: 0
+  test_size: 0.15
   downsample: 2
   rotate: True   # if measurement is upside-down
   save_psf: False
@@ -34,14 +35,14 @@ alignment:
     random_vflip: False
     random_hflip: False
     quantize: False
-    # shifting when there is no files.downsample
+    # shifting when there is no files to downsample
     vertical_shift: -117
     horizontal_shift: -25
 
 training:
   batch_size: 4
   epoch: 25
-  eval_batch_size: 4
+  eval_batch_size: 16
   crop_preloss: True
 
 reconstruction:

configs/train_digicam_multimask.yaml

@@ -1,17 +1,18 @@
-# python scripts/recon/train_unrolled.py -cn train_digicam_multimask
+# python scripts/recon/train_learning_based.py -cn train_digicam_multimask
 defaults:
   - train_unrolledADMM
   - _self_
 
-
 torch_device: 'cuda:0'
 device_ids: [0, 1, 2, 3]
 eval_disp_idx: [1, 2, 4, 5, 9]
 
+
 # Dataset
 files:
   dataset: bezzam/DigiCam-Mirflickr-MultiMask-25K
   huggingface_dataset: True
+  huggingface_psf: null
   downsample: 1
   # TODO: these parameters should be in the dataset?
   image_res: [900, 1200]  # used during measurement
@@ -24,13 +25,13 @@ files:
       display_res: [900, 1200]  # used during measurement
       rotate: True   # if measurement is upside-down
       alignment:
-        topright: [80, 100]  # height, width
+        top_left: [80, 100]  # height, width
         height: 200
 
 # TODO: these parameters should be in the dataset?
 alignment:
   # when there is no downsampling
-  topright: [80, 100]  # height, width
+  top_left: [80, 100]  # height, width
   height: 200
 
 training:
@@ -55,4 +56,5 @@ reconstruction:
   post_process: 
     network : UnetRes  # UnetRes or DruNet or null
     depth : 4 # depth of each up/downsampling layer. Ignore if network is DruNet
-    nc: [32,64,116,128]
+    nc: [32,64,116,128]
+

configs/train_digicam_singlemask.yaml

@@ -1,4 +1,4 @@
-# python scripts/recon/train_unrolled.py -cn train_digicam_singlemask
+# python scripts/recon/train_learning_based.py -cn train_digicam_singlemask
 defaults:
   - train_unrolledADMM
   - _self_
@@ -11,24 +11,26 @@ eval_disp_idx: [1, 2, 4, 5, 9]
 files:
   dataset: bezzam/DigiCam-Mirflickr-SingleMask-25K
   huggingface_dataset: True
+  huggingface_psf: null
   downsample: 1
+  # TODO: these parameters should be in the dataset?
   image_res: [900, 1200]  # used during measurement
   rotate: True   # if measurement is upside-down
   save_psf: False
 
-  # extra_eval: null
   extra_eval: 
     multimask: 
       huggingface_repo: bezzam/DigiCam-Mirflickr-MultiMask-25K
       display_res: [900, 1200]  # used during measurement
       rotate: True   # if measurement is upside-down
       alignment:
-        topright: [80, 100]  # height, width
+        top_left: [80, 100]  # height, width
         height: 200
 
+# TODO: these parameters should be in the dataset?
 alignment:
   # when there is no downsampling
-  topright: [80, 100]  # height, width
+  top_left: [80, 100]  # height, width
   height: 200
 
 training:

configs/train_psf_from_scratch.yaml

@@ -1,11 +1,15 @@
-# python scripts/recon/train_unrolled.py -cn train_psf_from_scratch
+# python scripts/recon/train_learning_based.py -cn train_psf_from_scratch
 defaults:
   - train_unrolledADMM
   - _self_
 
 # Train Dataset
 files:
   dataset: mnist   # Simulated : "mnist", "fashion_mnist", "cifar10", "CelebA". Measure :"DiffuserCam"
+  huggingface_dataset: False
+  n_files: 1000
+  test_size: 0.15
+
   celeba_root: /scratch/bezzam
   downsample: 8
 
@@ -24,8 +28,6 @@ simulation:
   object_height: 0.30
 
 training:
-  crop_preloss: False # crop region for computing loss
-  batch_size: 8
-  epoch: 25
+  batch_size: 2
   eval_batch_size: 16
   save_every: 5