Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Welcome to the MLV-App wiki!
Here we will try to write down a kind of user manual and how we realized the key features of MLV App.
A session consists of a set of MLV clips. For each clip one receipt is saved.
A receipt contains all settings from edit area for one clip.
Create a new empty session. If you are in a session, this session will be closed.
Opens an existing session from disk.
Saves the opened session to disk.
Create MAPP Files
A MAPP file contains video frame index (absolute offsets to all video frames start in MLV). MAPP also includes audio frame index and all needed block header copies. Hence there is no need to open and scan MLV file second time at all. Audio index simplifies audio data access when this data is to be loaded. WAV file is not saved separately. So when MAPP is active and MLV does not contain audio, the second loading is lightning fast.
This action imports (a) MLV file(s) to the session.
FCPXML Import Assistant
This opens a dialog, which helps opening all used clips from a FCPXML project file. The assistant trys to find a MLV equivalent to the clips from the FCPXML project.
Example: as first step for a project all recorded MLV files were opened and all clips were exported as proxy. As second step, the project was cut in a NLE. As third step, all used clips shall be graded and exported with maximal quality. Here, this assistant helps opening only the used clips to the session.
FCPXML Selection Assistant
This opens a dialog, which helps selecting all used clips from a FCPXML project file. The assistant trys to find the equivalent clips in the session and selects them. The selection can be inverted.
Example: as first step for a project all recorded MLV files were opened, some adjustments were made and all clips were exported as proxy. As second step, the project was cut in a NLE. As third step, all used clips shall be graded and exported with maximal quality. Here, this assistant helps removing the unused clips from the session. So the adjustments from step one don't get lost.
Open a dialog, which shows the metadata from the MLV file.
Export Selected Clips
This action exports the clips, which are selected in the Session List area, using the selected codec from Export Settings.
Export Actual Frame
Export the frame shown in the viewer as PNG file.
Setup the export settings. They consist of the codec, codec option, debayer (demosaic algorithm), resolution (ffmpeg codecs only), framerate override (ffmpeg only & AVFoundation), Smooth aliasing (ffmpeg only), audio, post export script (OSX only).
CinemaDNG export only contains RAW Correction parameters and CutIn&CutOut applied. All other parameters from edit area don't affect the exported file.
MLV export only contains CutIn&CutOut applied. All other parameters from edit area don't affect the exported file.
Smooth aliasing smooths aliasing artifacts. This works best for clips with a very slight movement.
Note: Smooth aliasing runs when progressbar is at 100% and needs a long time to render.
First, this opens a dialog where the user is able to select the settings only, which shall be copied to another/other clip(s) from the session. The selected settings from the opened clip will be copied to clipboard.
The receipt settings copied to clipboard will be pasted to all selected clips in the session list, or to the opened clip.
This is the same like Copy Receipt, but the receipt is not copied to clipboard - it is saved to a file on disk.
This pastes a receipt which was previously saved to disk to the opened clip.
This resets all parameters from the receipt (in the edit area) to their default. Note: a single parameter can be reset by doubleclicking on the slider handle.
Enable Audio Output
This enables/disables the audio output on playback.
Remember Playback Position
This saves the last playback position for each clip loaded in the session. If a clip in the session is selected again, this last playback position is reloaded.
Drop Frame Mode
There are two different playback modes: non drop frame mode, and drop frame mode. The first one shows all frames from the clip, when playing the clip. On slower computers this will look like slowmotion. In order to preview the clip in realtime (here: synced to real world time and synced to audio (if availlable)), drop frame mode can be used (all frames which can't be rendered and shown in time will be ignored).
Debayer for Preview
This changes the demosaicing algorithm used for the viewer. The demosaicing algorithm is one of the most time consuming parts in the processing pipeline, depending on the chosen algorithm. On the other side preview quality also depends the chosen algorithm. The fastest algorithm is "Simple" and the (mostly) best algorithm is "AMaZE". In order to be able to see the clip as fast as possible with best quality, there is the option "AMaZE cached". If this option is chosen, the user has to wait, if status bar lable tells "Caching: active". If status bar lable switches again to "Caching: idle" the clip is loaded to RAM (depending on the length of the clip, the clip may be larger as the reserved memory) and can be watched.
3. Demosaicing Algorithms
No debayering will run. This just copies monochrome color information from RAW to shown picture. Depending on chosen WhiteBalance, the picture could be colored to one single color.
This is a very fast but stupid demosaicing algorithm. It interprets a 2x2 pixel array (RGGB) to a color. This looks not very fine, consists of copying actions mostly. No interpolation is done at all.
Is a very fast algorithm, and interpolates colors between the pixels. Looks better than "Simple".
Zhang-Wu LMMSE Image Demosaicking - Color demosaicking via directional linear minimum mean square-error estimation.
Bayer CFA Demosaicing using Integrated Gaussian Vector on Color Differences. Looks best on high ISO clips.
Aliasing Minimization and Zipper Elimination. Looks best mostly.
Adaptive Homogeneity-Directed interpolation is based on the work of Keigo Hirakawa, Thomas Parks, and Paul Lee.
4. Edit Area
4.1 RAW Correction
Enable RAW Correction
Switch the whole RAW Correction block on/off.
Use this to remove static noise from the clip.
- Record with identical settings and identical equipment a clip with lens cap mounted. Some seconds are enough.
- Import MLV into MLVApp.
- Export this file as MLV, Averaged Frame. Now you have your darkframe.
- Import your clip.
- Go to RAW Corrections, press darkframe folder icon and select your darkframe. Static noise will be removed now.
Fix Focus Dots
Cameras like EOS 100D, 700D,... have wrong colored focus dots in the image. These dots can be removed using this option.
Fix Bad Pixels
Use this option to remove dead pixels from the clip.
Smoothes chroma on RAW data.
Some cameras record vertical stripes. Use this option to remove them.
Use this "On" to switch Dual ISO processing on. Note: the calculation is very time consuming! "Preview" mode is by far faster, but only works on 14bit clips.
This is the pattern noise reduction algorithm by a1ex.
RAW Black Level
Adjust the RAW Black Level if it was not correct in the metadata (mostly it is). If this parameter is wrong, picture shadows get pink or green.
RAW White Level
Adjust the RAW White Level if it was not correct in the metadata. If "Highlight Reconstruction" does not work, mostly RAW White Level is set wrong and has to be corrected (lowered).
4.2 Cut In & Cut Out
This range of the clip will be played back and exported only.
Exposure & Contrast
Correct the exposure in EV and adjust the contrast.
White Balance (Temperature and Tint)
The white balance can be adjusted manually using the two sliders, or using the picker. When using the picker, use the button in the right of the pipette button, to toggle between "grey" and "skin". Picking on grey adjusts the white balance on uncolored areas, picking on skin adjusts white balance on human skin.
The Clarity slider is useful, when it comes to give your images extra punch and impact, or to give it a dreamy look.
Adjust the saturation intensity of all colors.
Adjust the saturation in dependency, how saturated colors are already.
Curve (Dark/Light Strength/Range)
Adjust "how black black" and "how white white is".
Lightens the picture, specially dark parts.
Analyses and adjusts the highlights using a blurred mask.
Analyses and adjusts the shadows using a blurred mask.
Adjust gradation curves independently for Y, R, G, B.
Remove pink highlights.
Note 1: if it does not work, mostly RAW White Level is set wrong.
Note 2: for dual iso clips, pink highlights can not always be detected.
Use Camera Matrix
If enabled, scientific whitebalance calculation is applied.
This adjusts the gamma to a defined profile. For some profiles some parameters are disabled, to be able to export correctly.
This enables also Chroma Blur Radius. If disabled, "Sharpen" will sharpen all channels. If enabled, "Sharpen" will sharpen luminance channel only.
Sharpen the image. Note: see also Chroma Separation.
Chroma Blur Radius
Blur the chroma channels using the given pixel radius. This is useful to remove color noise, false color artifacts, ...
Denoiser (2D median)
This is a very simple 2D median denoiser. Window size and Strength can be adjusted.
4.5 HSL (Hue Saturation Luminance)
Hue vs. Hue
Adjust the color in relation to color. For example: make yellow more green (+) or red (-).
Hue vs. Saturation
Adjust saturation in relation to color. For example: make blue -> grey (-), but red -> red (+).
Hue vs. Luminance
Adjust luminance in relation to color. For example: make blue darker (-), but red brighter (+).
4.6 Linear Gradient
This set of parameters offers a kind of graduated ND filter. Additionally to exposure, contrast can be adjusted as well.
The position and length of the filter can be drawn on the picture using the pen button. Hovering on the lines on the picture, marks the lines yellow. Clicking an the marked lines makes it possible to adjust the position of the filter.
Hide the lines by collapsing the "Linear Gradient" group box.
Enable and select a 1D/3D .cube LookUpTable.
Enable, select, and adjust Strength of a predefined picture look.
When using anamorph lenses, the stretch factor can be adjusted here.
Wrong height ratio can be corrected here, when using pixel binning and skipping. Note: if metadata exists, MLV App adjusts this parameter automatically.
This option turns the picture by 180°.
5. Processing pipeline
Almost everything in processing is done through 65535-long lookup tables for more speed than using actual calculations (and quite badly organized in my opinion, all the arrays in the processing object are look up tables.
This use of 16 bit integers only causes a huge amount of difficulties with not clipping things, and is why developing new features like highlights and shadows got difficult (also why I did it wrong - did not want to clip stuff - more on this later). However all the processing features I created never ever clipped anything , so it's the same result as if it was done in floats... just more difficult to program new features.
- well, almost - information could still be lost if no tonemapping enabled, exposure increased followed by saturation reduction. There are probably a few other minor ways to lose some information.
The intention of these methods was to make it run quick, and I remember at the very start (back in early 2017), I could get one frame processed in 40ms on one thread. Not sure if it was ever that fast by the time it got a GUI.
Processing a frame
When changing the parameters exposure, temperature, dark/light strength/range, lighten and profile, in background some matrices were created and precalculated. So rendering the frame can be faster, because only the matrices have to be applied.
start of processing
- shadows and highlights... (just remembered there is actually a big problem in the implementation) Done on debayered raw image before black/white level correction and exposure (this is the problem)
apply_processing_object() split in to threads
- Black and white level correction (look up table)
- Matrix (does white balance, no camera matrix stuff) - there are some variables to realize contrast and clarity. This work exactly like highlights/shadows but:
- contrast: combines shadows(-)/highlights(+) with no blurred picture
- clarity: combines contrast(+) and shadows(+)/highlights(-)
- Gamma through lookup table, gamma look up table also has exposure and tonemap function applied in it all in one go (so highlight data won't get clipped before tonemapping)
- Highlight reconstruction done same time as the matrix. That works by reconstructing the green channel at the point where it was clipped. For dual iso clips, the information of highest_green (input for reconstruction) gets lost on dual iso stitching, and is different in each frame. In the end it seems like there is a gaussian curve around a peak (which may be there, may be not, sometimes it is small, sometimes it is high). That is why an algorithm was implemented, which trys to find this peak, if there is one. Starting from full 16bit (65535) and searching all local maxima, a maxima which is bigger than a limit is searched. If the maxima become bigger than a smaller limit, green signal was found (no peak found -> no clipped green). This algorithm has to run for each frame...
- For the linear gradient a second layer is calculated, using a modified copy of the matrices. Only the visible part is calculated, for speed reasons. Using a mask the gradient layer is applied on the main picture.
- Saturation using some weird algorithm Ilia came up with that seemed quick, also using strange lookup tables. Vibrance works exactly like saturation, but calculates RGB to (H)S(V) (S channel only) to be able to set saturation in dependency to saturation already being there. The less saturation there was before, the more saturation will be after applying vibrance.
- Contrast look up table
- if enabled, conversion to ycbcr through some lookup tables
- Sharpening, on all channels if rgb, or on just Y channel if ycbcr
- if chroma blur enabled and in ycbcr, cb and cr channels are blurred
- convert back to rgb from ycbcr
- LUT applied
- neural network filter applied