Image Pipe Daemon 1.0

fischer edited this page Jul 31, 2011 · 6 revisions
Clone this wiki locally

ipiped version 1.0 documentation is available for historic reasons. There is a newer version of ipiped available. Version 1.0 obsolete and is not recommend for new designs.

Table of Contents

Image capture settings

Exposure time

Exposure time is the effective length of time that your camera sensor integrates the scene's light. Exposure time is the length of time that your image sensor ’sees’ the scene you’re attempting to capture.

The longer the exposure time, the more light reaches the camera's sensor. If the camera sensor doesn't get enough light you end up with a photo which is dark (underexposed). Too much light and your photo comes out bright (overexposed).

It must be noted that large values of exposure time can reduce the video frame rate.


Gain is the signal amplification applied to each pixel in the image. The camera sensors and the video image processors often offers hardware's controls capable of adjust the gain value. Generally, the gain can be specified independently for each color channel (RGB) or as a global gain applied to all the color channels. When your image colors looks wrong you can use gain to compensate the effect of the scene illumination. Also the the global gain can be increased to improve the image luminance when capturing images under low light conditions. Increasing the gain also increases the noise.


Functions for dm365

Set previewer mode
The previewer is used for accessing the image tuning capabilities of the hardware. The previewer includes several hardware modules. The previewer operates in 2 modes: continuous and single shot. In the continuous mode the modules are chained in the capture data path and perform image tuning on the fly. In the single shot mode the modules are configured to tune an already captured image stored in the SDRAM.
ipiped sets the previewer mode of operation and initialize the hardware. If you want to use any of the dm365 processing functions available on the ipiped you must set the previewer mode first.

ipipe-client's command to set the previewer mode is set-previewer-mode.

Set Bayer pattern
When you capture raw image data, the previewer needs to know the specific order for the Bayer pattern. The default pattern is GrRBGb. If your sensor produces images with a different Bayer pattern you color data order, you can configure the order using the set Bayer pattern feature. The change to the input color pattern is not updated during the previewer operation.

ipipe-client's command to define the Bayer pattern is set-bayer-pattern.

Set/Get gain
The dm365 image pipe provides a hardware module that executes white balance to each color component. This module includes a gain and offset adjustment. In the white balance gain adjuster, the raw data is multiplied by a selected gain on a per color basis. In the offset adjuster, the raw data is added by the selected offset corresponding to each color. The ipiped offers a function to control the gain adjuster, with the offset values kept at zero. Each gain component can range from 0 to 15.998 in steps in steps of 1/512.
ipipe-client's commands to control the dm365's gain are: set-digital-gain and get-digital-gain
The contrast of the image can be controlled by setting the luminance. Higher luminance will make the picture more bright. The side effect of higher brightness is the picture looks foggy. Higher contrast will make the picture sharp. Both adjustments are applied to the luminance component(Y) in the YUV color space. These adjustments are described by the equation Yctr_ brt = (Y x C) + Br, where C is the contrast factor and Br is the brightness factor. Br is an integer value that can range from 0 to 255 and C is float value that can range from 0 to 15.94.
ipipe-client's command to control the luminance adjustment are: set-luminance and get-luminance

Functions for mt9p031/mt9011

Set/Get gain
This sensor supports adjustments for both analog and digital gain. By default the sensor will use analog gain up to its maximum before applying digital gain to the pixels. ipiped offers functions to set the gain for each color channel and to get the current gain values. Each gain component can range from 0 to 128 in steps of 0.125 if gain is between 1 and 4, 0.250 if gain is between 4.25 and 8, and 1.000 if gain is between 8 and 128.
ipipe-client's commands to control sensor's gains are: set-sensor-gain and get-sensor-gain.
Set/Get exposure time
ipiped supports setting the exposure time and get the current exposure time. The unit used for the exposure time is micro-seconds (us). You must take into account that RidgeRun's mt9p031/mt9011
driver has a maximum limit for the exposure time to maintain a constant frame rate. This limit depends on image size.

ipipe-client's commands to control these functions are: set-exposure and get-exposure.

Flip image vertically and horizontally
mt9p031 and mt9011 sensors have the capability of flipping the image's columns and rows. Actually these sensors do a image mirror, reversing the readout order. This is going to affect the Bayer pattern order and the resulting colors. If you are using the dm365 video processor, ipiped offers a function to correct the pattern, see Functions for dm365.
ipipe-client's command to flip the image vertically is flip-vertical and to flip the image horizontally is flip-horizontal.

Functions for librraew

The ipiped provides 2 commands to get access to the RidgeRun's auto-exposure/auto-white-balance library(librraew)(See librraew 1.0 documentation):

Init aew
Init aew configures librraew and starts the auto exposure and auto white balance iterations. You have to define the image's width and height.

ipipe-client's command is init-aew

Stop aew
Ends the automatic adjustments, free all the memory located for the librraew usage.

ipipe-client's command is stop-aew


Running ipiped

ipiped typically is run in background. If you are using RidgeRun's SDK and enabled ipiped, it will be setup to start automatically when the system boots.

ipiped &

ipiped registers with D-Bus and waits until a client, such as ipipe-client, sends messages.

Ipipe-client: command line application

ipipe-client is a Dbus client that use commands to invoke methods of the ipiped, so ipiped must be running to use ipipe-client. A command can require arguments depending of the command functionality. ipipe-client has two operation modes; single command execution or interactive console to execute a group of commands.

To execute a single command, you can use the following command line syntax

ipipe-client <command> <argument 1> ... <argument n>

To get into the interactive console, you have to run ipipe-client without any command. Then to execute a command you only need to use the command and the required arguments.

ipipe-client$ <command 1> <argument 1> ... <argument n>
ipipe-client$ <command 2> <argument 1> ... <argument n>
To quit the interactive console you can use quit or exit command.

In order to know the commands that are available run:

ipipe-client help
or get into the interactive console and execute help.

This shows a description of each command, as follows:

Command                         Description

help                    Displays the help text for all the possible commands or a specific command.
set-debug               Enable/Disable debug messages.
init-aew                Initialize AEW algorithms.
stop-aew                End AEW algorithm.
shell                   Execute a shell command(shell_cmd) using interactive console.
ping                    Show if ipipe-daemon is alive.
quit                    Quit from the interactive console.
exit                    Exit from the interactive console.
get-video-processor     Show the video processor that is being used.
get-sensor              Show the sensor that is being used.
run-config-script       Execute a group of ipipe-client commands.
set-previewer-mode      Configure previewer on continuous or one-shot mode.
set-bayer-pattern       Sets R/Gr/Gb/B color pattern to the previewer.
set-digital-gain        Sets red (R), green (G) and blue gains (G) on the ipipe.
get-digital-gain        Returns the gain value for each color component(RGB).
set-luminance           Brightness(Br) and contrast(C) adjustment.
get-luminance           Returns the value of the Brightness(Br) and contrast(C) adjustment.
flip-vertical           Flips the image vertically(on the sensor).
flip-horizontal         Flips the image horizontally (on the sensor).
set-exposure            Sets the effective shutter time  of the sensor for the light integration.
get-exposure            Gets the exposure time of the sensor in us.
set-sensor-gain         Sets red(R), green(G) and blue(B) gain directly on the sensor.
get-sensor-gain         Gets sensor red(R), green(G) and blue(B).

If you want more detailed information about a command execute:

ipipe-client help <command> 

Controlling librraew with ipipe

Auto exposure and auto white balance adjustments can be started with an ipipe-client's command called init-aew. Init-aew requires some arguments to define the algorithms and other parameters. To see the arguments required you can request for help that show you the list as follows:

Command: init-aew
Syntax: init-aew <WB> <AE> <G> <EM> <T[us]> <fps> <seg> <width> <height>
Description: Initialize AEW algorithms                                  
        WB: white balance algorithm, the options are:
                G -for gray world algorithm
                W -for retinex algorithm
                W2 -for variant of retinex algorithm
                N -for none
        AE: auto exposure algorithm, the options are
                EC -for electronic centric
                N -for none
        G: gain type, the options are:
                S -to use the sensor gain
                D -to use the digital
        EM: exposure metering method, the options are:
                P -for partial metering that take into account the light
                information of a portion in the center and the rest of
                the frame is ignored. The size  of the center depends of
                of the parameter center_percentage
                C -for center weighted metering that take into account
                the light information coming from the entire frame with
                emphasis placed on the center area
                A -for average metering that take into account the light
                information from the entire frame without weighting
                SG -for segmented metering that divides the frame
                on 6 pieces and weighting them to avoid backlighting
        T: wait time in us, specifies the time between
                algorithm adjustments, max value=1s=1000000us
        fps: minimum frame rate
        seg: frame segmentation factor, each frame is segmented into
                regions, this factor represents the percentage of the
                maximum number of possible regions
        width: captured video/image horizontal size
        height: captured video/image vertical size
        center_percentage: defines the percentage of the image width
                and height to be used as the center size

Also you can stop automatic adjustments with the command stop-aew

Some of the init-aew arguments need to be explained in more detail:

  • T: the time between interactions defines how fast the algorithm can adjust the scene parameters. If you don't need fast changes you can use a greater time to get less CPU usage.
  • seg: this factor is related with the amount of CPU usage and the auto-adjustments precision. If you use a high segmentation percentage you will have greater CPU usage but you will get more precision on the adjustments.
Example: Here will be shown a basic example of use of the ipiped and the librraew library Once your board has started run the following commands:
ipipe-client run-config-script dm365_mt9p031_config
ipipe-client init-aew G EC S C 200000 30 50 640 480 50

The first command will chain the ipiped using an existing script. The second one will start the auto-white balance and the auto-exposure algorithms for an image size of 640x480 pixels and a minimum frame rate of 30fps.

Once you have the ipiped and the AEW algorithms running you can run any image/video capture GStreamer pipeline to test it.