masc4ii edited this page Jan 23, 2019 · 37 revisions

Welcome to the MLV-App wiki!

Here we will try to write down a kind of user manual and how we realized the key features of MLV App.

1. Definitions

Session

A session consists of a set of MLV clips. For each clip one receipt is saved.

Receipt

A receipt contains all settings from edit area for one clip.

2. Menu

2.1 File

New Session

Create a new empty session. If you are in a session, this session will be closed.

Open Session

Opens an existing session from disk.

Save Session

Saves the opened session to disk.

Create MAPP Files

A MAPP file contains video frame index (absolute offsets to all video frames start in MLV). MAPP also includes audio frame index and all needed block header copies. Hence there is no need to open and scan MLV file second time at all. Audio index simplifies audio data access when this data is to be loaded. WAV file is not saved separately. So when MAPP is active and MLV does not contain audio, the second loading is lightning fast.

Import MLV

This action imports (a) MLV file(s) to the session.

FCPXML Import Assistant

This opens a dialog, which helps opening all used clips from a FCPXML project file. The assistant trys to find a MLV equivalent to the clips from the FCPXML project.

Example: as first step for a project all recorded MLV files were opened and all clips were exported as proxy. As second step, the project was cut in a NLE. As third step, all used clips shall be graded and exported with maximal quality. Here, this assistant helps opening only the used clips to the session.

bildschirmfoto 2018-08-11 um 16 05 49

FCPXML Selection Assistant

This opens a dialog, which helps selecting all used clips from a FCPXML project file. The assistant trys to find the equivalent clips in the session and selects them. The selection can be inverted.

Example: as first step for a project all recorded MLV files were opened, some adjustments were made and all clips were exported as proxy. As second step, the project was cut in a NLE. As third step, all used clips shall be graded and exported with maximal quality. Here, this assistant helps removing the unused clips from the session. So the adjustments from step one don't get lost.

Clip Information

Open a dialog, which shows the metadata from the MLV file.

Export Selected Clips

This action exports the clips, which are selected in the Session List area, using the selected codec from Export Settings.

Export Actual Frame

Export the frame shown in the viewer as PNG file.

Export Settings

Setup the export settings. They consist of the codec, codec option, debayer (demosaic algorithm), resolution (ffmpeg codecs only), framerate override (ffmpeg only & AVFoundation), Smooth aliasing (ffmpeg only), audio, post export script (OSX only).

CinemaDNG export only contains RAW Correction parameters and CutIn&CutOut applied. All other parameters from edit area don't affect the exported file.

MLV export only contains CutIn&CutOut applied. All other parameters from edit area don't affect the exported file.

Smooth aliasing smooths aliasing artifacts. This works best for clips with a very slight movement.

Note: Smooth aliasing runs when progressbar is at 100% and needs a long time to render.

2.2 Edit

Copy Receipt

First, this opens a dialog where the user is able to select the settings only, which shall be copied to another/other clip(s) from the session. The selected settings from the opened clip will be copied to clipboard.

Paste Receipt

The receipt settings copied to clipboard will be pasted to all selected clips in the session list, or to the opened clip.

Export Receipt

This is the same like Copy Receipt, but the receipt is not copied to clipboard - it is saved to a file on disk.

Import Receipt

This pastes a receipt which was previously saved to disk to the opened clip.

Reset Receipt

This resets all parameters from the receipt (in the edit area) to their default. Note: a single parameter can be reset by doubleclicking on the slider handle.

2.3 Playback

Enable Audio Output

This enables/disables the audio output on playback.

Remember Playback Position

This saves the last playback position for each clip loaded in the session. If a clip in the session is selected again, this last playback position is reloaded.

Drop Frame Mode

There are two different playback modes: non drop frame mode, and drop frame mode. The first one shows all frames from the clip, when playing the clip. On slower computers this will look like slowmotion. In order to preview the clip in realtime (here: synced to real world time and synced to audio (if availlable)), drop frame mode can be used (all frames which can't be rendered and shown in time will be ignored).

Debayer for Preview

This changes the demosaicing algorithm used for the viewer. The demosaicing algorithm is one of the most time consuming parts in the processing pipeline, depending on the chosen algorithm. On the other side preview quality also depends the chosen algorithm. The fastest algorithm is "Simple" and the (mostly) best algorithm is "AMaZE". In order to be able to see the clip as fast as possible with best quality, there is the option "AMaZE cached". If this option is chosen, the user has to wait, if status bar lable tells "Caching: active". If status bar lable switches again to "Caching: idle" the clip is loaded to RAM (depending on the length of the clip, the clip may be larger as the reserved memory) and can be watched.

3. Demosaicing Algorithms

Demosaicing algorithm demonstation

None

No debayering will run. This just copies monochrome color information from RAW to shown picture. Depending on chosen WhiteBalance, the picture could be colored to one single color.

Simple

This is a very fast but stupid demosaicing algorithm. It interprets a 2x2 pixel array (RGGB) to a color. This looks not very fine, consists of copying actions mostly. No interpolation is done at all.

Bilinear

Is a very fast algorithm, and interpolates colors between the pixels. Looks better than "Simple".

LMMSE

Zhang-Wu LMMSE Image Demosaicking - Color demosaicking via directional linear minimum mean square-error estimation.

IGV

Bayer CFA Demosaicing using Integrated Gaussian Vector on Color Differences. Looks best on high ISO clips.

AMaZE

Aliasing Minimization and Zipper Elimination. Looks best mostly.

AHD

Adaptive Homogeneity-Directed interpolation is based on the work of Keigo Hirakawa, Thomas Parks, and Paul Lee.

4. Edit Area

4.1 RAW Correction

Enable RAW Correction

Switch the whole RAW Correction block on/off.

Darkframe Subtraction

Use this to remove static noise from the clip.

  • Record with identical settings and identical equipment a clip with lens cap mounted. Some seconds are enough.
  • Import MLV into MLVApp.
  • Export this file as MLV, Averaged Frame. Now you have your darkframe.
  • Import your clip.
  • Go to RAW Corrections, press darkframe folder icon and select your darkframe. Static noise will be removed now.

Fix Focus Dots

Cameras like EOS 100D, 700D,... have wrong colored focus dots in the image. These dots can be removed using this option.

bildschirmfoto 2018-09-23 um 13 38 08

Fix Bad Pixels

Use this option to remove dead pixels from the clip.

bildschirmfoto 2018-09-23 um 13 40 08

Chroma Smooth

Smoothes chroma on RAW data.

Vertical Stripes

Some cameras record vertical stripes. Use this option to remove them.

bildschirmfoto 2018-09-23 um 13 41 59

Dual ISO

Use this "On" to switch Dual ISO processing on. Note: the calculation is very time consuming! "Preview" mode is by far faster, but only works on 14bit clips.

Pattern Noise

This is the pattern noise reduction algorithm by a1ex.

RAW Black Level

Adjust the RAW Black Level if it was not correct in the metadata (mostly it is). If this parameter is wrong, picture shadows get pink or green.

RAW White Level

Adjust the RAW White Level if it was not correct in the metadata. If "Highlight Reconstruction" does not work, mostly RAW White Level is set wrong and has to be corrected (lowered).

4.2 Cut In & Cut Out

This range of the clip will be played back and exported only.

4.3 Processing

Exposure & Contrast

Correct the exposure in EV and adjust the contrast.

White Balance (Temperature and Tint)

The white balance can be adjusted manually using the two sliders, or using the picker. When using the picker, use the button in the right of the pipette button, to toggle between "grey" and "skin". Picking on grey adjusts the white balance on uncolored areas, picking on skin adjusts white balance on human skin.

Clarity

The Clarity slider is useful, when it comes to give your images extra punch and impact, or to give it a dreamy look.

Saturation

Adjust the saturation intensity of all colors.

Vibrance

Adjust the saturation in dependency, how saturated colors are already.

Curve (Dark/Light Strength/Range)

Adjust "how black black" and "how white white is".

Lighten

Lightens the picture, specially dark parts.

Highlights

Analyses and adjusts the highlights using a blurred mask.

Shadows

Analyses and adjusts the shadows using a blurred mask.

Gradation Curves

Adjust gradation curves independently for Y, R, G, B.

Highlight Reconstruction

Remove pink highlights.

Note 1: if it does not work, mostly RAW White Level is set wrong.

Note 2: for dual iso clips, pink highlights can not always be detected.

Use Camera Matrix

If enabled, scientific whitebalance calculation is applied.

Profile

This adjusts the gamma to a defined profile. For some profiles some parameters are disabled, to be able to export correctly.

4.4 Details

Chroma Separation

This enables also Chroma Blur Radius. If disabled, "Sharpen" will sharpen all channels. If enabled, "Sharpen" will sharpen luminance channel only.

Sharpen

Sharpen the image. Note: see also Chroma Separation.

Chroma Blur Radius

Blur the chroma channels using the given pixel radius. This is useful to remove color noise, false color artifacts, ...

Denoiser (2D median)

This is a very simple 2D median denoiser. Window size and Strength can be adjusted.

4.5 HSL (Hue Saturation Luminance)

Hue vs. Hue

Adjust the color in relation to color. For example: make yellow more green (+) or red (-).

Hue vs. Saturation

Adjust saturation in relation to color. For example: make blue -> grey (-), but red -> red (+).

Hue vs. Luminance

Adjust luminance in relation to color. For example: make blue darker (-), but red brighter (+).

4.6 Linear Gradient

This set of parameters offers a kind of graduated ND filter. Additionally to exposure, contrast can be adjusted as well.

The position and length of the filter can be drawn on the picture using the pen button. Hovering on the lines on the picture, marks the lines yellow. Clicking an the marked lines makes it possible to adjust the position of the filter.

bildschirmfoto 2018-08-29 um 12 45 19

Hide the lines by collapsing the "Linear Gradient" group box.

4.7 LUT

Enable and select a 1D/3D .cube LookUpTable.

4.8 Filter

Enable, select, and adjust Strength of a predefined picture look.

4.9 Transformation

Width Stretch

When using anamorph lenses, the stretch factor can be adjusted here.

Height Stretch

Wrong height ratio can be corrected here, when using pixel binning and skipping. Note: if metadata exists, MLV App adjusts this parameter automatically.

Upside Down

This option turns the picture by 180°.

5. Processing pipeline

Almost everything in processing is done through 65535-long lookup tables for more speed than using actual calculations (and quite badly organized in my opinion, all the arrays[65536] in the processing object are look up tables.

This use of 16 bit integers only causes a huge amount of difficulties with not clipping things, and is why developing new features like highlights and shadows got difficult (also why I did it wrong - did not want to clip stuff - more on this later). However all the processing features I created never ever clipped anything [1], so it's the same result as if it was done in floats... just more difficult to program new features.

  1. well, almost - information could still be lost if no tonemapping enabled, exposure increased followed by saturation reduction. There are probably a few other minor ways to lose some information.

The intention of these methods was to make it run quick, and I remember at the very start (back in early 2017), I could get one frame processed in 40ms on one thread. Not sure if it was ever that fast by the time it got a GUI.

Processing a frame

When changing the parameters exposure, temperature, dark/light strength/range, lighten and profile, in background some matrices were created and precalculated. So rendering the frame can be faster, because only the matrices have to be applied.

start of processing

applyProcessingObject()

  1. shadows and highlights... (just remembered there is actually a big problem in the implementation) Done on debayered raw image before black/white level correction and exposure (this is the problem)

applyProcessingObject() calls apply_processing_object() split in to threads

  1. Black and white level correction (look up table)
  2. Matrix (does white balance, no camera matrix stuff) - there are some variables to realize contrast and clarity. This work exactly like highlights/shadows but:
  • contrast: combines shadows(-)/highlights(+) with no blurred picture
  • clarity: combines contrast(+) and shadows(+)/highlights(-)
  1. Gamma through lookup table, gamma look up table also has exposure and tonemap function applied in it all in one go (so highlight data won't get clipped before tonemapping)
  2. Highlight reconstruction done same time as the matrix. That works by reconstructing the green channel at the point where it was clipped. For dual iso clips, the information of highest_green (input for reconstruction) gets lost on dual iso stitching, and is different in each frame. In the end it seems like there is a gaussian curve around a peak (which may be there, may be not, sometimes it is small, sometimes it is high). That is why an algorithm was implemented, which trys to find this peak, if there is one. Starting from full 16bit (65535) and searching all local maxima, a maxima which is bigger than a limit is searched. If the maxima become bigger than a smaller limit, green signal was found (no peak found -> no clipped green). This algorithm has to run for each frame...
  3. For the linear gradient a second layer is calculated, using a modified copy of the matrices. Only the visible part is calculated, for speed reasons. Using a mask the gradient layer is applied on the main picture.
  4. Saturation using some weird algorithm Ilia came up with that seemed quick, also using strange lookup tables. Vibrance works exactly like saturation, but calculates RGB to (H)S(V) (S channel only) to be able to set saturation in dependency to saturation already being there. The less saturation there was before, the more saturation will be after applying vibrance.
  5. Contrast look up table
  6. if enabled, conversion to ycbcr through some lookup tables
  7. Sharpening, on all channels if rgb, or on just Y channel if ycbcr
  8. if chroma blur enabled and in ycbcr, cb and cr channels are blurred
  9. convert back to rgb from ycbcr
  10. LUT applied
  11. neural network filter applied
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.