Skip to content

compositor

jeanlf edited this page Dec 6, 2019 · 17 revisions

HOME » Filters » Compositor

Compositor

Register name used to load filter: compositor

The GPAC compositor allows mixing audio, video, text and graphics in a timed fashion.
The filter is explicit only and is never loaded during link resolution.
The compositor operates either in media-client or filter-only mode.

In media-client mode, the compositor acts as a pseudo-sink for the video side and creates its own output window.
The video frames are dispatched to the output video pid in the form of frame pointers requiring later GPU read if used.
The audio part acts as a regular filter, potentially mixing and resampling the audio inputs to generate its output.

In filter-only mode, the compositor acts as a regular filter generating frames based on the loaded scene.
It will generate its outputs based on the input video frames.
If no input video frames (e.g. pure BIFS / SVG / VRML), the filter will generate frames based on the fps, at constant or variable frame rate.
It will stop generating frames as soon as all input streams are done, unless extended/reduced by dur.
If audio streams are loaded, an audio output pid is created.
In filter-only mode, the special URL gpid:// is used to locate PIDs in the scene description, in order to design scenes independently from source media.
When such a pid is associated to a Background2D node in BIFS (no SVG mapping yet), the compositor operates in passthrough mode.
In this mode, only new input frames on the passthrough pid will generate new frames, and the scene clock matches the input packet time.
The output size and pixel format will be set to the input size and pixel format, unless specified otherwise in the filter options.

If only 2D graphics are used and display driver is not forced, 2D rasterizer will happen in the output pixel format (including YUV pixel formats).
In this case, inplace processing (rasterizing over the input frame data) will be used whenever allowed by input data.

If 3D graphics are used or display driver is forced, OpenGL will be used on offscreen surface and the output packet will be an OpenGL texture.

Specific URL syntaxes

The compositor accepts any URL type supported by GPAC. It also accepts the following syntaxes for URLs:

  • views: creates an auto-stereo scene of N views from views://v1:.:vN. vN can be any type of URL supported by GPAC.
  • mosaic: creates a mosaic of N views from mosaic://v1:.:vN. vN can be any type of URL supported by GPAC.

Options

aa (enum, default: all, updatable): set anti-aliasing mode for raster graphics - whether the setting is applied or not depends on the graphics module / graphic card.

  • none: no anti-aliasing
  • text: anti-aliasing for text only
  • all: complete anti-aliasing

hlfill (unsigned int, default: 0x0, updatable): set highlight fill color (ARGB)
hlline (unsigned int, default: 0xFF000000, updatable): set highlight stroke color (ARGB)
hllinew (float, default: 1.0, updatable): set highlight stroke width
sz (boolean, default: true, updatable): enable scalable zoom. When scalable zoom is enabled, resizing the output window will also recompute all vectorial objects. Otherwise only the final buffer is stretched.
bc (unsigned int, default: 0, updatable): default background color to use when displaying transparent images or video with no scene composition instructions
yuvhw (boolean, default: true, updatable): enable YUV hardware for 2D blits
blitp (boolean, default: true, updatable): partial hardware blits (if not set, will force more redraw)
softblt (boolean, default: true): enable software blit/stretch in 2D. If disabled, vector graphics rasterizer will always be used
stress (boolean, default: false, updatable): enable stress mode of compositor (rebuild all vector graphics and texture states at each frame)
fast (boolean, default: false, updatable): enable speed optimization - whether the setting is applied or not depends on the graphics module / graphic card
bvol (enum, default: no, updatable): draw bounding volume of objects.

  • no: disable bounding box
  • box: draws a rectangle (2D) or box (3D mode)
  • aabb: draws axis-aligned bounding-box tree (3D only)

textxt (enum, default: default, updatable): specify whether text shall be drawn to a texture and then rendered or directly rendered. Using textured text can improve text rendering in 3D and also improve text-on-video like content

  • default: use texturing for OpenGL rendering, no texture for 2D rasterizer
  • never: never uses text textures
  • always: always render text to texture before drawing

out8b (boolean, default: false, updatable): convert 10-bit video to 8 bit texture before GPU upload.
drop (boolean, default: false, updatable): drop late frame when drawing. By default frames are not droped until a heavy desync of 1 sec is observed
sclock (boolean, default: false, updatable): force synchronizing all streams on a single clock
sgaze (boolean, default: false, updatable): simulate gaze events through mouse
ckey (unsigned int, default: 0, updatable): color key to use in windowless mode (0xFFRRGGBB). GPAC currently does not support true alpha blitting to desktop due to limitations in most windowing toolkit, it therefore uses color keying mechanism. The alpha part of the key is used for global transparency of the output, if supported.
timeout (unsigned int, default: 10000, updatable): timeout in ms after which a source is considered dead
fps (fraction, default: 30/1, updatable): simulation frame rate when animation-only sources are played (ignored when video is present).
timescale (unsigned int, default: 0, updatable): timescale used for output packets when no input video pid. A value of 0 means fps numerator.
autofps (boolean, default: true): use video input fps for output. If no video or not set, uses fps. Ignored in player mode
vfr (boolean, default: false): only emit frames when changes are detected. Always true in player mode
dur (number, default: 0, updatable): duration of generation. Mostly used when no video input is present. Negative values mean number of frames, positive values duration in second, 0 stops as soon as all streams are done.
fsize (boolean, default: false, updatable): force the scene to resize to the biggest bitmap available if no size info is given in the BIFS configuration
mode2d (enum, default: defer, updatable): specify whether immediate drawing should be used or not

  • immediate: the screen is completely redrawn at each frame (always on if passthrough mode is detected).
  • defer: object positioning is tracked from frame to frame and dirty rectangles info is collected in order to redraw the minimal amount of the screen buffer.
  • debug: only renders changed areas, reseting other areas
    Whether the setting is applied or not depends on the graphics module and player mode.

amc (boolean, default: true): audio multichannel support; if disabled always downmix to stereo. Useful if the multichannel output does not work properly
asr (unsigned int, default: 0): force output sample rate - 0 for auto
ach (unsigned int, default: 0): force output channels - 0 for auto
alayout (unsigned int, default: 0): force output channel layout - 0 for auto
afmt (audio format, default: s16, minmax: none,u8,s16,s24,s32,flt,dbl,u8p,s16p,s24p,s32p,fltp,dblp): force output channel format - 0 for auto
asize (unsigned int, default: 1024): audio output packet size in samples
abuf (unsigned int, default: 100): audio output buffer duration in ms - the audio renderer fills the output pid up to this value. A too low value will lower latency but can have real-time playback issues
avol (unsigned int, default: 100, updatable): audio volume in percent
apan (unsigned int, default: 50, updatable): audio pan in percent, 50 is no pan
async (boolean, default: true, updatable): audio resynchronization; if disabled, audio data is never dropped but may get out of sync
buf (unsigned int, default: 3000, updatable): playout buffer in ms. Overriden by BufferLenth property of input pid
rbuf (unsigned int, default: 1000, updatable): rebuffer trigger in ms. Overriden by RebufferLenth property of input pid
mbuf (unsigned int, default: 3000, updatable): max buffer in ms (must be greater than playout buffer). Overriden by BufferMaxOccupancy property of input pid
nojs (boolean, default: false): disable javascript
ogl (enum, default: auto, updatable): specify 2D rendering mode

  • auto: automatically decides betwwen on, off and hybrid based on content.
  • off: disables OpenGL - 3D will not be rendered.
  • on: uses OpenGL for all graphics - this will involve polygon tesselation and 2D graphics will not look as nice as 2D mode.
  • hybrid: the compositor performs software drawing of 2D graphics with no textures (better quality) and uses OpenGL for all 2D objects with textures and 3D objects.

pbo (boolean, default: false, updatable): enable PixelBufferObjects to push YUV textures to GPU in OpenGL Mode. This may slightly increase the performances of the playback.
nav (enum, default: none, updatable): override the default navigation mode of MPEG-4/VRML (Walk) and X3D (Examine)

  • none: disables navigation
  • walk: 3D world walk
  • fly: 3D world fly (no ground detection)
  • pan: 2D/3D world zomm/pan
  • game: 3D world game (mouse gives walk direction)
  • slide: 2D/3D world slide
  • exam: 2D/3D object examine
  • orbit: 3D object orbit
  • vr: 3D world VR (yaw/pitch/roll)

linegl (boolean, default: false, updatable): indicate that outlining shall be done through OpenGL pen width rather than vectorial outlining
epow2 (boolean, default: true, updatable): emulate power-of-2 textures for openGL (old hardware). Ignored if OpenGL rectangular texture extension is enabled

  • yes: video texture is not resized but emulated with padding. This usually speeds up video mapping on shapes but disables texture transformations
  • no: video is resized to a power of 2 texture when mapping to a shape
    paa (boolean, default: false, updatable): indicate whether polygon antialiasing should be used in full antialiasing mode. If not set, only lines and points antialiasing are used.
    bcull (enum, default: on, updatable): indicate whether backface culling shall be disable or not
  • on: enables backface culling
  • off: disables backface culling
  • alpha: only enables backface culling for transparent meshes

wire (enum, default: none, updatable): wireframe mode.

  • none: objects are drawn as solid
  • only: objects are drawn as wireframe only
  • solid: objects are drawn as solid and wireframe is then drawn

norms (enum, default: none, updatable): normal vector drawing for debug

  • none: no normals drawn
  • face: one normal per face drawn
  • vertex: one normal per vertex drawn

glus (boolean, default: true, updatable): use GLU scale, which may be slower but nicer than GPAC software stretch routines.
rext (boolean, default: true, updatable): use non power of two (rectangular) texture GL extension
cull (boolean, default: true, updatable): use aabb culling: large objects are rendered in multiple calls when not fully in viewport
yuvgl (boolean, default: true, updatable): enable YUV open GL pixel format support if hardware supports it. If not supported and shaders are available, YUV to RGB will be done on GPU
depth_gl_scale (float, default: 100, updatable): set depth scaler
depth_gl_type (enum, default: none, updatable): set geometry type used to draw depth video

  • none: no geometric conversion
  • point: compute point cloud from pixel+depth
  • strip: same as point but thins point set

nbviews (unsigned int, default: 0, updatable): number of views to use in stereo mode
stereo (enum, default: none, updatable): stereo output type. If your graphic card does not support OpenGL shaders, only top and side modes will be available.

  • side: images are displayed side by side from left to right.
  • top: images are displayed from top (laft view) to bottom (right view).
  • hmd: same as side except that view aspect ratio is not changed.
  • ana: standard color anaglyph (red for left view, green and blue for right view) is used (forces views=2).
  • cols: images are interleaved by columns, left view on even columns and left view on odd columns (forces views=2).
  • rows: images are interleaved by columns, left view on even rows and left view on odd rows (forces views=2).
  • spv5: images are interleaved by for SpatialView 5 views display, fullscreen mode (forces views=5).
  • alio8: images are interleaved by for Alioscopy 8 views displays, fullscreen mode (forces views=8).
  • custom: images are interleaved according to the shader file indicated in mvshader. The shader is exposed each view as uniform sampler2D gfViewX, where X is the view number starting from the left

mvshader (string, updatable): file path to the custom multiview interleaving shader
fpack (enum, default: none, updatable): default frame packing of input video

  • none: no frame packing
  • top: top bottom frame packing
  • side: side by side packing

camlay (enum, default: offaxis, updatable): camera layout in multiview modes

  • straight: camera is moved along a straight line, no rotation
  • offaxis: off-axis projection is used
  • linear: camera is moved along a straight line with rotation
  • circular: camera is moved along a circle with rotation

iod (float, default: 6.4, updatable): inter-occular distance (eye separation) in cm (distance between the cameras).
rview (boolean, default: false, updatable): reverse view order
tvtn (unsigned int, default: 30, updatable): number of point sampling for tile visibility algo
tvtt (unsigned int, default: 0, updatable): number of points above which the tile is considered visible
tvtd (boolean, default: false, updatable): disable the tile having full coverage of the SRD, only displaying partial tiles
tvtf (boolean, default: false, updatable): force all tiles to be considered visible, regardless of viewpoint
fov (float, default: 1.570796326794897, updatable): default field of view for VR
vertshader (string): path to vertex shader file
fragshader (string): path to fragment shader file
autocal (boolean, default: false, updatable): auto callibration of znear/zfar in depth rendering mode
dispdepth (int, default: -1, updatable): display depth, negative value uses default screen height
dispdist (float, default: 50, updatable): distance in cm between the camera and the zero-disparity plane. There is currently no automatic calibration of depth in GPAC
focdist (float, default: 0, updatable): distance of focus point
osize (vec2d int, default: 0x0, updatable): force output size. If not set, size is derived from inputs
dpi (vec2d int, default: 96x96, updatable): default dpi if not indicated by video output
dbgpvr (float, default: 0, updatable): debug scene used by PVR addon
player (boolean, default: false): set compositor in player mode, see filter help
opfmt (pixel format, default: none): pixel format to use for output. Ignored in player mode
drv (enum, default: auto): indicate if graphics driver should be used. Ignored in player mode

  • no: never loads a graphics driver (software blitting used, no 3D possible)
  • yes: always loads a graphics driver. Output pixel format will be RGB
  • auto: decides based on the loaded content

HOME » Filters

GPAC Configuration

Configuration File
Global Options
Logging

Filters Overview

General Filters Concepts
gpac General Usage
Built-in Properties

Filters Help

Inspect packets (inspect)
Probe source (probe)
Compositor (compositor)
ISOBMFF/QT demuxer (mp4dmx)
MPEG-4 BIFS decoder (bifsdec)
MPEG-4 OD decoder (odfdec)
File input (fin)
BT/XMT/X3D loader (btplay)
HTTP input (httpin)
SVG loader (svgplay)
JPG/J2K/PNG/BMP reframer (rfimg)
PNG/JPG decoder (imgdec)
ADTS reframer (rfadts)
LATM reframer (rflatm)
MP3 reframer (rfmp3)
FAAD decoder (faad)
MAD decoder (maddec)
XVid decoder (xviddec)
OpenJPEG2000 decoder (j2kdec)
AC3 reframer (rfac3)
A52 decoder (a52dec)
AMR/EVRC reframer (rfamr)
OGG demuxer (oggdmx)
Vorbis decoder (vorbisdec)
Theora decoder (theoradec)
MPEG-2 TS demuxer (m2tsdmx)
UDP/TCP input (sockin)
OpenSVC decoder (osvcdec)
VideoToolBox decoder (vtbdec)
MPEG-4 LASeR decoder (lsrdec)
SAF demuxer (safdmx)
MPEG-DASH and HLS client (dashin)
CENC decryptor (cdcrypt)
CENC encryptor (cecrypt)
ISOBMFF/QT muxer (mp4mx)
QCP reframer (rfqcp)
H263 reframer (rfh263)
M1V/M2V/M4V reframer (rfmpgvid)
NHNT reader (nhntr)
NHML parser (nhmlr)
AVC/HEVC reframer (rfnalu)
MPEG PS demuxer (m2psdmx)
AVI demuxer (avidmx)
Subtitle loader (txtin)
TTXT/TX3G decoder (ttxtdec)
WebVTT decoder (vttdec)
RTP/RTSP/SDP input (rtpin)
File output (fout)
Raw AAC to LATM writer (uflatm)
ADTS writer (ufadts)
Media Reframer (reframer)
Stream to file (writegen)
AVC/HEVC to AnnexB writer (ufnalu)
QCP writer (writeqcp)
WebVTT writer (writevtt)
NHNT writer (nhntw)
NHML writer (nhmlw)
VobSub demuxer (vobsubdmx)
AVI muxer (avimx)
Audio output (aout)
M4V writer (ufm4v)
Audio resampler (resample)
Video output (vout)
Video crop (vcrop)
Video flip (vflip)
RAW video reframer (rfrawvid)
PCM reframer (rfpcm)
JPG encoder (jpgenc)
PNG encoder (pngenc)
Audio/Video rewinder (rewind)
Sources concatenator (flist)
MPEG-2 TS muxer (m2tsmx)
DASH and HLS segmenter (dasher)
HEVC tile aggregator (tileagg)
pipe input (pin)
pipe output (pout)
GSF Muxer (gsfmx)
GSF Demuxer (gsfdmx)
UDP/TCP output (sockout)
AV1/IVF/VP9 reframer (rfav1)
IVF/OBU/annexB writer (ufobu)
ATSC input (atscin)
RTP Streamer (rtpout)
RTSP Server (rtspout)
HTTP Server (httpout)
HEVC tile spliter (hevcsplit)
HEVC Tile merger (hevcmerge)
FLAC reframer (rfflac)
ProRes reframer (rfprores)
MPEG Transport Stream splitter (tssplit)
FFMPEG demuxer (ffdmx)
FFMPEG decoder (ffdec)
FFMPEG AV Capture (ffavin)
FFMPEG video rescaler (ffsws)
FFMPEG encoder (ffenc)
FFMPEG muxer (ffmx)
FFMPEG AVFilter (ffavf)
JavaScript filter (jsf)
OpenHEVC decoder (ohevcdec)

Clone this wiki locally
You can’t perform that action at this time.