Focus stacking framework

Usage example with jupyter notebook:

from focus_stack import *
job = StackJob("job", "E:/Focus stacking/My image directory/", input_path="source")
job.add_action(AlignLayers("align"))
job.add_action(BalanceLayersLumi("balance", mask_size=0.5, i_min=10, i_max=255))
job.add_action(FocusStackBunch("batches", PyramidStack(), frames=10, overlap=2, denoise=0.8))
job.add_action(FocusStack("stack", PyramidStack(), postfix='_py', denoise=0.8))
job.add_action(FocusStack("stack", DepthMapStack(), input_path='batches', postfix='_dm', denoise=0.8))
job.run()

Required python modules:

• open cv
• numpy
• scipy
• matplotlib
• termcolor
• tqdm
• PIL

Documentation

Schedule the desired actions in a job, then run the job.

Job creation

job = StackJob(name, working_directory)

arguments are:

• working_directory: the directory that contains input and output images, organized in subdirectories as specified by each action
• name: the name of the job, used for printout
• input_path (optional): the subdirectory within working_directory that contains input images for subsequent action. If not specified, at least the first action must specify an input_path.

Image registration: scale, tanslation and rotation correction, or full perspective correction

job.add_action(AlignLayers(name, *options))

arguments are:

• name: the name of the action, used for printout, and possibly for output path
• input_path (optional): the subdirectory within working_directory that contains input images to be aligned. If not specified, the last output path is used, or, if this is the first action, the input_path specified with the StackJob construction is used. If not specified the StackJob specifies no input_path, at least the first action must specify an input_path.
• output_path (optional): the subdirectory within working_directory where aligned images are written. If not specified, it is equal to name.
• working_directory: the directory that contains input and output image subdirectories. If not specified, it is the same as job.working_directory.
• ref_idx (optional): the index of the image used as reference. Images are numbered starting from zero. If not specified, it is the index of the middle image.
• step_align (optional): if equal to True (default), each image is aligned with respect to the previous or next image, depending if it is after or befor the reference image.
• transform (optional): the transformation applied to register images. Possible values are:
  • ALIGN_RIGID (default): allow scale, tanslation and rotation correction. This should be used for image acquired with tripode or microscope.
  • ALIGN_HOMOGRAPHY: allow full perspective correction. This should be used for images taken with hand camera.
• detector (optional): the feature detector is used to find matches. See Feature Detection and Description for more details. Possible values are:
  • SIFT (default): Scale-Invariant Feature Transform]
  • ORB: Oriented FAST and Rotated BRIEF
  • SURF: Speeded-Up Robust Features
  • AKAZE: AKAZE local features matching
• descriptor (optional): the feature descriptor is used to find matches. Possible values are:
  • SIFT (default)
  • ORB
  • AKAZE
• match_method (optional): the method used to find matches. See Feature Matching for more details. Possible values are:
  • KNN (default): Feature Matching with FLANN
  • NORM_HAMMING:
• flann_idx_kdtree (optional, default: 2): parameter used by the FLANN matching algorithm.
• flann_tree (optional, default: 5): parameter used by the FLANN matching algorithm.
• flann_checks (optional, default: 50): parameter used by the FLANN matching algorithm.
• match_threshold (optional, default: 0.75): parameter used to select good matches. See Feature Matching for more details.
• rans_threshold (optional, default: 5.0): parameter used if ALIGN_HOMOGRAPHY is choosen as tansformation, see Feature Matching + Homography to find Objects for more details.
• border_mode (optional, default: cv2.BORDER_REPLICATE): border mode. See Adding borders to your images for more details.
• border_value (optional, default: (0, 0, 0, 0)): border value. See Adding borders to your images for more details.
• plot_matches (optional, default: False): if True, for each image the matches identified with respect to the reference image are plotted. May be useful for inspection and debugging.

Luminosity and color balance

There are four possible luminosity and color balance methods:

• BalanceLayersLumi: balance luminosity equally for R, G and B channels. It should be fine for most of the cases.
• BalanceLayersRGB: balance luminosity separately for R, G and B channels. It may be needed if some but not all of the images have a undesired color dominance.
• BalanceLayersLumiSV: balance saturation a luminosity value in the HSV (Hue, Saturation, brightness Value) representation. It may be needed in cases of extreme luminosity variation that affects saturation.
• BalanceLayersLS: balance saturation a luminosity value in the HLS (Hue, Lightness, Saturation) representation. It may be needed in cases of extreme luminosity variation that affects saturation.

job.add_action(BalanceLayersLumi(working_directory, name, *options))

arguments are:

• name: the name of the action, used for printout, and possibly for output path
• input_path (optional): the subdirectory within working_directory that contains input images to be aligned. If not specified, the last output path is used, or, if this is the first action, the input_path specified with the StackJob construction is used. If not specified the StackJob specifies no input_path, at least the first action must specify an input_path.* output_path (optional): the subdirectory within working_directory where aligned images are written. If not specified, it is equal to name.
• working_directory: the directory that contains input and output image subdirectories. If not specified, it is the same as job.working_directory.
• ref_idx (optional): the index of the image used as reference. Images are numbered starting from zero. If not specified, it is the index of the middle image.
• mask_size (optional): if specified, luminosity and color balance is only applied to pixels within a circle of radius equal to the minimum between the image width and height times mask_size, i.e: 0.8 means 80% of a portrait image height. It may beuseful for images with vignetting, in order to remove the outer darker pixels.
• i_min (optional): if specifies, only pixels with content greater pr equal tham i_min are used. It may be useful to remove black areas.
• i_max (optional): if specifies, only pixels with content less pr equal tham i_max are used. It may be useful to remove white areas.
• plot_histograms (optional, default: False): if True, for each image and for the reference image histograms with pixel content are plotted. May be useful for inspection and debugging.

Focus Stacking

job.add_action(FocusStack(working_directory, name, stacker, input_path, *options))

arguments are:

• name: the name of the action, used for printout, and possibly for output path
• stacker: an object defining the focus stacking algorithm. Can be PyramidStack or DepthMapStack, see below for possible algorithms.
• input_path (optional): the subdirectory within working_directory that contains input images to be aligned. If not specified, the last output path is used, or, if this is the first action, the input_path specified with the StackJob construction is used. If not specified the StackJob specifies no input_path, at least the first action must specify an input_path.* output_path (optional): the subdirectory within working_directory where aligned images are written. If not specified, it is equal to name.
• working_directory: the directory that contains input and output image subdirectories. If not specified, it is the same as job.working_directory.
• exif_path (optional): if specified, EXIF data are copied to the output file from file in the specified directory. If not specified, it is the source directory used as input for the first action. If set equal to '' no EXIF data is saved.
• postfix (optional): if specified, the specified string is appended to the file name. May be useful if more algorithms are ran, and different file names are used for the output of different algorithms.
• denoise (optoinal): if specified, a denois algorithm is applied. A value of 0.75 to 1.00 does not reduce details in an appreciable way, and is suitable for modest noise reduction. See Image Denoising for more details

Bunch Focus Stacking

job.add_action(FocusStackBunch(working_directory, name,  stacker, input_path, *options))

arguments are:

• name: the name of the action, used for printout, and possibly for output path
• stacker: an object defining the focus stacking algorithm. Can be PyramidStack or DepthMapStack, see below for possible algorithms.
• input_path (optional): the subdirectory within working_directory that contains input images to be aligned. If not specified, the last output path is used, or, if this is the first action, the input_path specified with the StackJob construction is used. If not specified the StackJob specifies no input_path, at least the first action must specify an input_path.* output_path (optional): the subdirectory within working_directory where aligned images are written. If not specified, it is equal to name.
• working_directory: the directory that contains input and output image subdirectories. If not specified, it is the same as job.working_directory.
• exif_path (optional): if specified, EXIF data are copied to the output file from file in the specified directory. If not specified, it is the source directory used as * frames (optional, default: 10): the number of frames in each bunch that are stacked together.
• overlap (optional, default: 0): the number of overlapping frames between a bunch and the following one.
• postfix (optional): if specified, the specified string is appended to the file name. May be useful if more algorithms are ran, and different file names are used for the output of different algorithms.
• denoise (optoinal): if specified, a denois algorithm is applied. A value of 0.75 to 1.00 does not reduce details in an appreciable way, and is suitable for modest noise reduction. See Image Denoising for more details

Stack algorithms

• PyramidStack, based on Laplacian pyramids method implementation by Sami Jawhar. Arguments are:
  • pyramid_min_size (optiohnal, default: 32)
  • kernel_size (optional, default: 5)
• DepthMapStack, based on Laplacian pyramids method implementation by Sami Jawhar. Arguments are:
  • map_type (optional), possible values are MAP_MAX (default) and MAP_AVERAGE. MAP_MAX select for wach pixel the layer which has the best focus. MAP_AVERAGE performs for each pixel an average of all layers weighted by the quality of focus.
  • energy (optional), possible values are ENERGY_LAPLACIAN (default) and ENERGY_SOBEL.
  • kernel_size (optional, default: 5)
  • blur_size (optional, default: 5)
  • smooth_size (optional, default: 32)

Credits:

based on Laplacian pyramids method implementation by Sami Jawhar. The original code was used under permission of the author.

Resources:

• Pyramid Methods in Image Processing, E. H. Adelson, C. H. Anderson, J. R. Bergen, P. J. Burt, J. M. Ogden, RCA Engineer, 29-6, Nov/Dec 1984 Pyramid methods in image processing
• A Multi-focus Image Fusion Method Based on Laplacian Pyramid, Wencheng Wang, Faliang Chang, Journal of Computers 6 (12), 2559, December 2011
• Another original implementation on GitHub by Zongnan Bao

Issues

The support of 16 bits TIFF images is still partial. In particular, with 16 bits images:

• Focus stacking modules crashes if denoise is different from zero due to an assertion failure in the Open CV library. This is similar to a known issue on stackoverflow.
• Focus stacking modules may crashes if exif_path is provided and if the directory contains TIFF images, depending on the content of EXIF data. A couple of problematic keys in EXIF data have been identified, namely 33723, 34665, that cause a crash in the PIL library. Those keys are dropped from EXIF data for TIFF files. Moreover, some of the EXIF data may be missing.

PNG files have not been tested so far.

License

The software is provided as is under the GNU Lesser General Public License v3.0.