Rendering algorithms

Rendering algorithms

• Rendering algorithms: Linearization

  Gamma, backlight correction, scanner exposure and dark point compensation

Rendering pipeline for processes with regular screens

Scan of original negative	Infrared scan of original transparency	RGB scan of original transparency
Linearization and inverting negative to positive	Linearization	Linearization and channel mixing
Linear grayscale transparency
Viewing screen registration
linear RGB image with viewing screen (see viewing screen superposition)
Determining luminosity of each color patch (optional)
Blocky linear RGB image (similar to RAW from digital camera)
Demosaicing using bicubic interpolation (optional)
Smooth but unsharp linear RGB image
Combining predicted and original data (optional)
linear RGB image
Simulation dyes of viewing screen and conversion to output color profile
Digital color rendering

## Inverting negative to positive If the original image is a negative, then we first need to simulate process of contact-copying negative to a transparency. Basic invertion to negative may be done by setting `gray_range` in a way minimal value is greater than maximal.

Future plans

Realistic simulation of what happens when negative is copied to a transparency in laboratory involves simulating sensitivity curve of the black and white emulsion used. At the moment this step is not implemented in a satisfactory way and we rely on other tools (such as vuescan) to do this job for us. We yet need to understand how typical sensitivity curves were and what kind of user-interface we want to implement here. Ideally we should do this based on scans of existing negative-transparency pairs.

Channel mixing

When only RGB scan of a color transparency is available, it is possible to estimate the original black and white transparency by mixing RGB channels to grayscale with a certain weights.

In theory it is possible to recover the infrared channel of the scan using the RGB data.
During digitization light passes from the light source through the filters and the emulsion (which is assumed to be close to neutral density filter) and color filters used on the top of the scanner CCD. Knowing RGB values of the scanner's response to the red, green and blue filters of the color screen it is possible possible to produce 3 equations which makes it possible to compute from scanned RGB value the density of the emulsion. mix_weights.red, mix_weights.green and mix_weights.blue are solution to the equations. Value is further compensated by mix_dark, which should ideally be (0,0,0) but it is practical to offset it in some situation (such as when the value is computed using the actual infrared channel).

The simulated infrared channel is computed as:

ir=(r-mix_dark.red)mix_weights.red+(g-mix_dark.green)mix_weights.green+(b-mix_dark.blue)mix_weights.blue.

Here r, g and b are linearized RGB values of the pixel inspected.

Meaningful of mix_dark are values are in range (-1,1). mix_red, mix_green and mix_blue can be both positive and negative.

Viewing screen registration

Screen-coordinates of Paget or Finlay screen.

In this step the screen-coordinates is mapped to the scan-coordinates. This is done by several parameters:

• Affine transformation
  • screen_shift x y specifies that screen-coordinate (0,0) is (x,y) scan-coordinate.
  • coordinate_x xx xy is the x coordinate vector of the basis of linear transformation. Screen-coordinate (1,0) maps to (x+xx,y+xy) scan-coordinate.
  • coordinate_y yx yy is the y coordinate vector of the basis of linear transformation. Screen-coordinate (0,1) maps to the coordinate (x+yx,y+yy) scan-coordinate.
• Perspective correction
  • tilt_x txx txy specifies tilt of the scanned image in the horizontal scan-axis. The rotation is in degrees if the image was scanned screen-distance 1 of the surface of image (thus not very meaningul)
  • tilt_y tyx tyy specifies tild of the scanned image in the vertical scan-axis. For scanners with fixed lens this should always be same as tilt_x. However for scanners with moving lens only one of tilt_x and tilt_y should be non-zero and it should correspond to the axis in which the sensor is fixed.
• Lens distortion correction
  • k1 p specifies the lens correction parameter for the Brown-Conrady lens distortion model. Other parameters are not implemented (yet?) since it seems that quality macro lenses suitable for digitization of additive color screen processes typically have relatively mild (but sometimes still important) geometric distortions.

Future plans

Parameters are specified by hand using the GUI application. This gets laborious since they needs to be specified very precisely. For this reasons we plan to implement image analysis which will detect the dot pattern in the scanned image and help to (partly) automate this process.

Legacy scanners with moving sensor suffers from additional errors caused by inprecisions of the stepping motors. This is visible in final rendering and thus we need to implement compensation for this problem. Main question is how to set an user-interface which makes it possible to specify such corrections. We hope that after implementing image analysis we will be able to make this more automatic, too.

Viewing screen superposition

Once mapping of viewing screen and scan coordinates is made, it is possible to simulate the superposition of viewing screen and transparency. This means that for every pixel of the input image the color of viewing screen is determined in RGB values that represents intensities of each color dye and these RGB values are multiplied by the luminosity of the pixel.

Since scans nor viewing screens are perfectly sharp, parameter screen_blur_radius determine how much the simulated ideal screen should be blurred for this process.

Future plans

For later algorithms it is very desirable to make screen blurring to match the blur caused by the combination of photographic emulsion and scanner, so it precisely match the blur of original viewing screen. Instead of simple screen_blur_radius we may implement bluring based on fourier transformation that takes an MTF curve of the system as parameter.

Determining luminosity of each color patch

Precise data collection

This step is done by considering every pixel of the given color patch and computing weighted average where weights are given by screen blurring. Only pixels where given channel exceeds collection_threshold parameter are considered to be part of the patch. The algorithm uses screen_blur to estimate loss of saturation caused by the reduction of contrast of individual tiles and compensates for it. With correct screen_blur setting the value 0 of collection_threshold should give same results as slightly higher values (until the threshold is so large that no data is collected when the algorithm turns to fast mode). However since with lower values bigger increase of saturation is necessary, it yields to very visible artifacts around edges. Too large values of collection_threshold on the other hand may make image noisy.

Fast data collection

Center of every color patch is identified and the luminosity is given using the bicubic interpolation. This is useful especially for very low resolution scans where there are too few pixels corresponding to a given patch.

Rendering modes

The image can be rendered in mutiple ways

original

This mode passes data from the original scan to the output. It may correct geometrical errors and it can also turn original scan into HD data with scanner profile which can then be viewed in general image manipulation programs with HDR support. The output is either monochrome or color. For color scans the monochrome output is eitehr the infrared channel (if available and enabled) or simulated infrared channel based on channel mixing. Sharpenning is applied to monochrome data only.

scanner_red, scanner_green, scanner_blur parameters can be used to specify matrix profile representing scanner reaction to a given process.

interpolated original (requires registered regular screen)

This mode renders image in original color but performs demosaicing based on the knowledge of the color screen. This is useful when it is desired to reproduce the photograph as it appears physically but the output device is not capable of reproducing faithfully the color pattern.

Scanner profile can be applied.

profiled original

interpolated profiled original (requires registered regular screen)

This output mode works same way as original and interpolated original however it makes use of the matrix profile specified by profiled_red, profiled_green and profiled_blue which can be used to translate scanner's RGB data to estimated separations of the original process. Solver is available to determine this profile based on known spots of the scan.

interpolated diff (requires registered regular screen)

Compares output of interpolated profiled original with the interpolated rendering. One makes use of the color profile while other takes into account the spartial information about the color screen. for quality scans, if the color screen is faded uniformly and if profiling is done correctly, the outputs should match.

preview grid (requires registered regular screen)

Shows a grid representing placement of the screen. Useful for manual fine-tuning of the screen alignment.

realistic (requires registered regular screen)

Simulate digital screen on the top of the scan. This is faithful to the actual physical object, however not very practical in its digital form. Because scans are not sharp, realistic renderings are always somewhat desaturated and the regular pattern of the screen tends to interfere with digital processing

interpolated (requires registered regular screen)

Use demosaicing algoirthm to eliminate the color noise.

interpolated predictive (requires registered regular screen)

Demosaic the image but try to preserve details smaller than one patch.

combined (requires registered regular screen)

Demosaic the image to obtain color channel and then combine it with the black and white channel keeping the regular grid. This mode is not particularly useful and may be removed in future releases

fast (requires registered regular screen)

Render one pixel per color screen cycle. This is useful for fast previews, but it is blocky and not very acurate.

extra

Optional off-tree rendering algorithm

adjusted color (requires color scan)

For automatic screen discovery typical scanner response to red, green and blue patch is determined. This mode turns those color to sRGB red, green and blue making it easy to see if the colors are set correctly

normalized color (requires color scan)

Normalize all colors to the same desnity. If scan is correctly analysed, this should make the photograph disappear and keep only the color screen

screen color (requires color scan)

Shows result of analysis which is trying to classify every pixel to one of 4 categories (red, green, blue and unknown)

realistic with screen detection (requires color scan)

Equivalent of the realistic rendering described above but making use of the pixel color analysis.

screen detection nearest

Use pixel color analysis and very rough demosaic (which always use the density of the closest patch of given color).

screen detection nearest scaled

Same as above but scale by relative patch sizes

screen detection with relaxation

Try to perform demosaicing which works without knowledge of the screen (and can be applied to Autochrome, for example).

Color models

Result of some rendering modes (such as interpoalted) is an RGB image in the color space of the color process used to take the photogrpah. In order to display it correctly, it is necessary to convert it to modern color space (such as sRGB, ProphotoRGB or XYZ). Color models define how colors are converted. The following are available

none

Perform no color transformations. Useful for debugging

red, green, blue

Shown red, green or blue separation

Wall max separation

Use colors that are best separating human vision according to book: Wall EJ. Practical color photography. American Photographic Publishing Company; 1922.

"We may, therefore, say that there are three fundamental or primary colors in the spectrum, by the admixture of which we can form all intermediate colors and white. The positions of these fundamental colors are shown by the heavy vertical lines R. G. B,, in Fig. 3, and the curves show how the intermediate colors are formed. It will be seen from the diagram that the fundamental red is at about 6700, and this excites but the one sensation; but the green at 5180 not only excites the green-sensitive nerve fibril but also the red and the while the blue blue,at 4600 excites the green and red also, though but slightly."

This is interpreted as xy values

• red: 0.7319933,0.2680067 (670nm)
• green: 0.059325533,0.829425776 (518nm)
• blue: 0.143960396, 0.02970297 (460nm)

paget

RGB values which looked good for early set of scans of paget plates available. This setting will be replaced by better one once paged color dyes are understood

This is interpreted as xy values

• red: 0.674, 0.325
• green: 0.059325533,0.829425776
• blue: 0.143960396, 0.02970297

Miethe_Goerz_reconstructed_by_Wagner

Colors used in Goertz projector (designed by Miethe) as reconstructed in Wagner, J. (2006) Die additive Dreifarbenfotografie nach Adolf Miethe. Master thesis. Technische Universität München.

This is interpreted as xy values

• red: 0.674, 0.325
• green: 0.182, 0.747
• blue: 0.151, 0.041

Miethe_Goerz_mesured_by_Wagner

Colors of aged filters used in Goertz projector (designed by Miethe) as measured in Wagner, J. (2006) Die additive Dreifarbenfotografie nach Adolf Miethe. Master thesis. Technische Universität München.

This is interpreted as xy values

• red: 0.620, 0.315
• green: 0.620, 0.315
• blue: 0.182, 0.135

Wratten_25_58_47_xyz

Those are colors of Wratten filters 25, 58 and 47 which were used as standard filters for three-color photography. Based on MacAdam, D. L. Colorimetric specifications of Wratten light filters. Journal of the Optical Society of America 35, 10 (1945), 670–675.

These are XYZ values

• red: 0.3038, 0.1425, 0.0001
• green: 0.0891, 0.2451, 0.0219
• blue: 0.0862, 0.0240, 0.4730

Wratten_25_58_47_spectra

Colors based on spectrum of Wratten filters as published by Kodak "Wratten light color filters, 1945 edition.". Those are tables XYZ values above are based on. Advantage of having full spectrum is that colors can be viewed with different backlights.

Digitized data based on published table

Data can be exported using colorscreen lab save-dyes wratten-red.dat wratten-blue.dat wratten-green.dat wratten_25_58_47_kodak_1945.

dufaycolor_reseau_by_dufaycolor_manual

Spectra based on charts in the Dufaycolor manual. Those are clearly not realistic, but represents the intended sensitivity of the film.

Published chart

Digitized data

dufaycolor_reseau_by_color_cinematography_xyY

xyY specification of Dufaycolor reseau filter in Cornwell-Clyne, Adrian (1951) Colour Cinematography. Third. Champman & Hall. Available at:https://archive.org/details/dli.ministry.11295.

Table is (in xyY):

• red: 0.633, 0.365, 0.177, Dominating wavelength 601.7
• green: 0.233, 0.647, 0.43, Dominating wavelength 549.6
• blue: 0.140, 0.089, 0.037, Dominating wavelength 466.0

We believe the table is faulty since the dominating values does not correspond to the xyY values specified. Also it makes the overall reseau filter to look green.

dufaycolor_reseau_by_color_cinematography_xyY

Our attempt to correct the table (in xyY):

• red: 0.633, 0.365, 0.177
• green: 0.233, 0.647, 0.43
• blue: 0.164, 0.089, 0.087

dufaycolor_reseau_by_color_cinematography_wavelength

Colors based on dominating wavelength data in the table discussed above

dufaycolor_reseau_by_color_cinematography_spectra

Published chart

Digitized data

Spectrum published in Cornwell-Clyne, Adrian (1951) Colour Cinematography. Third. Champman & Hall. Available at:https://archive.org/details/dli.ministry.11295.