Merge branch 'master' of https://github.com/profjsb/deepCR

README.md

@@ -2,24 +2,12 @@
 
 ## deepCR: Deep Learning Based Cosmic Ray Removal for Astronomical Images
 
-Apply a learned convolutional neural net (CNN) model to a 2d `numpy` array to identify and remove cosmic rays, on multi-core CPUs or GPUs.
+Apply a learned convolutional neural net (CNN) model to a 2d `numpy` array to identify and remove cosmic ray, on multi-core CPUs or GPUs.
 
 <img src="https://raw.githubusercontent.com/profjsb/deepCR/master/imgs/postage-sm.jpg" wdith="90%">
 
 
-This is the installable package which implements the methods described in the paper: Zhang & Bloom (2019), submitted. All figures and speed/scoring benchmarks relative to existing solutions can be found in the paper Github repo: https://github.com/kmzzhang/deepCR-paper
-
-### Quick Start
-
-With Python >=3.5:
-
-```python
-from deepCR import model
-mdl = model.deepCR(mask="ACS-WFC-F606W-2-4",
-	           inpaint="ACS-WFC-F606W-2-32",
-                   device="GPU")
-mask, cleaned_image = mdl.clean(image)
-```
+This is the installable package which implements the methods described in the paper: Zhang & Bloom (2019), submitted. Code to benchmark the model and to generate figures and tables in the paper can be found in the deepCR-paper Github repo: https://github.com/kmzzhang/deepCR-paper
 
 ### Installation
 
@@ -32,19 +20,47 @@ Or you can install from source:
 ```bash
 git clone https://github.com/profjsb/deepCR.git
 cd deepCR/
-pip install .
+pip install
 ```
 
-### Limitations and Caveats
+### Quick Start
 
-In the current release, the included models have been built and tested only on ACS Hubble Space Telescope (HST) images in the F606W filter. Application to native image resolution (ie. not drizzled), electron counts, flattened images from ACS/F606W (`*_flt.fits`) should work well. Use of these prepackaged models in other observing modes with HST, ground-based images, or spectroscopy is not encouraged. However, we expect the method implemented herein to work on such data with sufficient training data.
+With Python >=3.5:
 
-You will need to run your model on arrays which have the same bite ordering as the native machine. Some FITS readers do not switch bit order automatically. In this case, you can ask `numpy` to switch. For instance,
+```python
+from deepCR import deepCR
+from astropy.io import fits
+
+image = fits.getdata("*********_flc.fits")
+mdl = deepCR(mask="ACS-WFC-F606W-2-32",
+	     inpaint="ACS-WFC-F606W-2-32",
+             device="GPU")
+mask, cleaned_image = mdl.clean(image, threshold = 0.5)
+```
+Note:
+Input image must be in units of electrons
 
+To reduce memory consumption (recommended for image larger than 1k x 1k):
 ```python
-out = mdl.clean(my_hst_image.astype(np.float64))
+mask, cleaned_image = mdl.clean(image, threshold = 0.5, seg = 256)
 ```
+which segments the input image into patches of 256*256, seperately perform CR rejection on the patches, before stitching back to original image size.
+
+### Currently available models
+
+mask: ACS-WFC-F606W-2-4
+      ACS-WFC-F606W-2-32(*)
+
+inpaint: ACS-WFC-F606W-2-32
+         ACS-WFC-F606W-3-32(*)
+
+The two numbers following instrument configuration specifies model size, with larger number indicating better performing model at the expense of runtime. Recommanded models are marked in (*). For benchmarking of these models, please refer to the original paper.
+
+### Limitations and Caveats
+
+In the current release, the included models have been built and tested only on Hubble Space Telescope (HST) ACS/WFC images in the F606W filter. Application to native-spatial resolution (ie. not drizzled), calibrated images from ACS/F606W (`*_flc.fits`) is expected to work well. Use of these prepackaged models in other observing modes with HST or spectroscopy is not encouraged. We are planning hosting a "model zoo" that would allow deepCR to be adapted to a wide range of instrument configurations.
 
 ### Contributing
 
-We are very interested in getting bug fixes, new functionality, and new models from the community (built especially on ground-based imaging and spectroscopy). Please fork this repo and issue a PR with your changes. It will be especially helpful if you add some tests for your changes. 
+We are very interested in getting bug fixes, new functionality, and new trained models from the community (especially for ground-based imaging and spectroscopy). Please fork this repo and issue a PR with your changes. It will be especially helpful if you add some tests for your changes.
+

deepCR/model.py

@@ -126,6 +126,9 @@ def clean_(self, img0, threshold=0.5, inpaint=True, binary=True):
         :param binary: return binary mask if True. probabilistic mask otherwise.
         :return: mask or binary mask; or None if internal call
         """
+        # data proprocessing
+        img0 = img0.astype(np.float32) / 100
+
         shape = img0.shape
         pad_x = 4 - shape[0] % 4
         pad_y = 4 - shape[1] % 4
@@ -160,10 +163,10 @@ def clean_(self, img0, threshold=0.5, inpaint=True, binary=True):
 
 
             if binary:
-                return binary_mask[pad_x:, pad_y:], inpainted[pad_x:, pad_y:]
+                return binary_mask[pad_x:, pad_y:], inpainted[pad_x:, pad_y:] * 100
             else:
                 mask = mask.detach().cpu().view(shape[0], shape[1]).numpy()
-                return mask[pad_x:, pad_y:], inpainted[pad_x:, pad_y:]
+                return mask[pad_x:, pad_y:], inpainted[pad_x:, pad_y:] * 100
 
         else:
             if binary:

learned_models/__init__.py

@@ -5,12 +5,9 @@
 
 mask_dict = {'ACS-WFC-F606W-2-32': [UNet2Sigmoid, (1, 1, 32)],
              'ACS-WFC-F606W-2-4': [UNet2Sigmoid, (1, 1, 4)],
-             'ACS-WFC-F606W-2-32-old': [UNet2Sigmoid, (1, 1, 32)],
-             'ACS-WFC-F606W-2-4-old': [UNet2Sigmoid, (1, 1, 4)]}
+             'example_model': [UNet2Sigmoid, (1, 1, 32)]}
 
 inpaint_dict = {'ACS-WFC-F606W-3-32': [UNet3, (2, 1, 32)],
-                'ACS-WFC-F606W-2-32': [UNet2, (2, 1, 32)],
-                'ACS-WFC-F606W-3-32-old': [UNet3, (2, 1, 32)],
-                'ACS-WFC-F606W-2-32-old': [UNet2, (2, 1, 32)]}
+                'ACS-WFC-F606W-2-32': [UNet2, (2, 1, 32)]}
 
 default_model_path = path.join(path.dirname(__file__))