Version 0.11.0

Signed-off-by: Carlos Rafael Giani <dv@pseudoterminal.org>

ChangeLog

@@ -1,3 +1,36 @@
+==== version 0.11.0 (2015-07-23) ====
+
+* new common codebase for rectangular regions and video canvases
+
+* new blitter-based video sink with full support for vsync to eliminate tearing
+
+* complete rewrite of blitter base code to better support aspect ratio correction,
+  rotation, empty region filling, and compositing
+
+* new compositor elements, API compatible with the upstream GStreamer compositor
+  These new elements allow for combining N input video streams into one output
+  stream with alpha blending, scaling, rotation, color space conversions etc.
+  and hardware accelerated via G2D, IPU, PxP
+
+* vpu: decoder: Memory leaks fixed
+  Decoder is now drained when stopping
+  Fixed incorrect gst_allocator_free() call
+  GstImxPhysMemory slices weren't being freed properly
+  VPU allocator instances are now instantiated per-decoder
+
+* vpu: encoder: Bitrate can now be set on the fly
+
+* eglvivsink: In Wayland, window x/y coordinates are now supported
+
+* several v4l2src fixes:
+  Removed experimental and deprectated WITH_CHIP_IDENTIFICATION ioctl
+  Use pixelformat reported by the driver instead of hardcoding I420
+  The format fourcc is now read properly (not just as a string)
+  Call VIDIOC_QUERYBUF twice to get a physical address (important for zerocopy)
+  Add interlace-mode to src caps to allow for deinterlacing camera feeds
+  If possible, the actual frame rate is retrieved
+
+
 ==== version 0.10.2 (2015-06-02) ====
 
 * uniaudio: Create correct config header even if no codecs were found

README.md

@@ -18,6 +18,130 @@ License
 These plugins are licensed under the LGPL v2.
 
 
+Features
+--------
+
+* Zero-copy support
+  "Zero-copy" refers to techniques to avoid unnecessary buffer copies, or at least unnecessary CPU-based
+  copies (and copying via DMA instead). In gstreamer-imx, video elements try to avoid CPU-based frame copies
+  as much as possible. To that end, special DMA buffer allocators are used, and outgoing buffers are
+  extended with physical memory metadata. gstreamer-imx video elements only make CPU-based frame copies if
+  the incoming frames have no such metadata attached. The VPU decoder element, the IPU/G2D/PxP compositor
+  and video transform elements, and the imxv4l2videosrc element all produce frames with physically contiguous
+  memory (also called "DMA memory"), and attach the physical memory metadata to the frames. The VPU encoder
+  elements, the IPU/G2D/PxP compositor, transform, and sink elements all check incoming frames for this
+  metadata. As a result, it is possible to use playback and transcoding pipelines which run with very little
+  CPU usage, since the bulk of the transfers are done via DMA.
+* Fully compatible with upstream GStreamer demuxers and parsers
+  gstreamer-imx has been tested and used with the upstream demuxers and parsers. (Currently, there are no
+  plans to add elements to gstreamer-imx which wrap Freescale parsers and demuxers.)
+* Video transform elements for color space conversion and scaling
+  The PxP, IPU, and G2D units in the imx SoC can perform these operations in one step. This is exposed
+  via the pxp/ipu/g2d video transform elements. Not only are conversions much faster this way, they are also
+  compatible with the zerocopy feature explained above.
+* Deinterlacing via the imxipusink and imxpuvideotransform elements
+* G2D/IPU/PxP compositor, compatible with the upstream compositor
+  The imxipu/pxp/g2dcompositor elements mimic the properties of the compositor in GStreamer 1.5 and above.
+  Command lines which use that compositor can make use of hardware-accelerated compositing simply by
+  replacing "compositor" with "imxg2dcompositor", for example. Furthermore, the imx compositor elements
+  also allow for 90-degree step rotations, alpha blending, aspect ratio correction, and empty region filling.
+* Video4Linux source element with i.MX specific enhancements for zerocopy
+* G2D/IPU/PxP video sinks support tearing-free vsync output via page flipping
+
+
+Compositing
+-----------
+
+The compositor is a new feature in gstreamer-imx 0.11.0. Just like with the compositor from gst-plugins-base 1.5.1
+and newer, compositor elements support an arbitrary number of request sink pads, and one srcpad.
+
+Example call:
+
+    gst-launch-1.0   \
+      imxg2dcompositor name=c background-color=0x223344 \
+          sink_0::xpos=0 sink_0::ypos=90 sink_0::width=160 sink_0::height=110 sink_0::zorder=55 sink_0::fill_color=0xff00ff00 sink_0::alpha=0.39 sink_0::rotation=0 \
+          sink_1::xpos=0 sink_1::ypos=20 sink_1::width=620 sink_1::height=380 sink_1::fill_color=0x44441133 ! \
+        queue2 ! "video/x-raw, width=800, height=600" ! imxg2dvideosink \
+      videotestsrc pattern=0 ! "video/x-raw, framerate=30/1" ! c.sink_0 \
+      videotestsrc pattern=18 ! "video/x-raw, framerate=30/1" ! c.sink_1
+
+This creates the following frame:
+
+![compositor frame](compositor-example.png)
+
+The compositor properties are accessible as usual by calling gst-inspect-1.0, like: `gst-inspect-1.0 imxg2dcompositor`
+
+For the sinkpad properties are equal to that of the upstream compositor 
+
+Most of the sink pad properties are the same as that of GstCompositorPad:
+
+* `xpos`: The x-coordinate position of the top-left corner of the picture (gint)
+* `ypos`: The y-coordinate position of the top-left corner of the picture (gint)
+* `width`: The width of the picture; the input will be scaled if necessary (gint)
+* `height`: The height of the picture; the input will be scaled if necessary (gint)
+* `alpha`: The transparency of the picture; between 0.0 and 1.0. The blending is a simple copy when fully-transparent (0.0) and fully-opaque (1.0). (gdouble)
+* `zorder`: The z-order position of the picture in the composition (guint)
+
+In addition, the imx compositor pads have these properties:
+
+* `left-margin` : Left margin in pixels, defining an empty space at the left side between the border of the outer frame and the actual inner video frame
+* `top-margin` : Top margin in pixels, defining an empty space at the top side between the border of the outer frame and the actual inner video frame
+* `right-margin` : Right margin in pixels, defining an empty space at the right side between the border of the outer frame and the actual inner video frame
+* `bottom-margin` : Bottom margin in pixels, defining an empty space at the bottom side between the border of the outer frame and the actual inner video frame
+* `rotation` : 90-degree step rotation mode for the inner video frame
+* `keep-aspect-ratio` : If true, the aspect ratio of the inner video frame is maintained, potentially creating empty regions
+* `fill-color` : What color to use to fill the aforementioned empty regions, specified as a 32-bit ABGR color value
+
+The compositors have the notion of "inner" and "outer" frames. The "inner" frame is the actual video frame,
+for example a movie. The "outer" frame is a superset of the inner one and also of any empty spaces. If for
+example the outer frame is 1600x900 (16:9), and the inner frame is 1280x960 (4:3), and `keep-aspect-ratio`
+is set to true, then the inner frame will be scaled to fit in the middle of the outer frame, and the leftover
+spaces to the left and right are the "empty spaces". These get filled with the `fill-color`. If any of the
+margin values are nonzero, then the empty spaces also include the margin regions. If `keep-aspect-ratio`
+is false, no empty regions exist unless at least one the margins is nonzero.
+
+Current limitations:
+* The G2D compositor is the preferred one. The IPU compositor suffers from IPU peculiarities like "jumps"
+  in the frame positioning. Also, the IPU compositor currently does not support deinterlacing.
+* There is no PxP compositor at the moment, since the PxP engine always fills the entire output frames with
+  black pixels, even if only a subset is drawn to.
+
+
+Avoiding "tearing" via vsync
+----------------------------
+
+The sink elements all support vsync-based output to avoid tearing. The `imxeglvivsink` can achieve this by
+setting the `FB_MULTI_BUFFER` environment variable to 2 prior to starting the pipeline.
+
+The PxP/IPU/G2D sinks perform it by reconfiguring the framebuffer (they do not use the environment variable).
+Page flipping is done in these sinks by simply scrolling inside the virtual framebuffer by one full physical
+height unit. In other words, if the actual framebuffer resolution is 1280x800, the virtual framebuffer is
+reconfigured to 12800x1600, and page flipping is performed by scrolling to the 0th and 800th row.
+
+
+VPU decoder notes
+-----------------
+
+The VPU has an important limitation which needs to be respected when creating complex pipelines with lots of
+branching. The VPU *cannot* reallocate additional buffers for decoded output frames on the fly. When video
+decoding begins, the VPU requests a number of DMA buffers from the caller. The caller (in this case, `imxvpudec`)
+has to allocate at least this many DMA buffers. Once done, these DMA buffers are registered with the VPU
+decoder instance. From that moment on, decoding continues, but it is *not* possible to register additional
+DMA buffers afterwards. This means that the `imxvpudec` decoder element has to use a fixed-size pool. If all
+of the pool's buffers are queued up somewhere, and do not get released, a deadlock might occur, since then,
+the VPU decode has nowhere to decode to (all registered DMA buffers are occupied). This is a fundamental
+problem of incompatible approaches: the VPU's fixed size buffer pool on one side, and the much more flexible
+and dynamic nature of complex GStreamer pipelines on the other. For regular video watching, this is not a
+problem, but it may be if lots of branching and queuing is used in custom designed complex pipelines.
+
+
+VPU encoder notes
+-----------------
+
+The h.264 encoder only supports the baseline profile. Also, the encoded output always uses the h.264
+Annex B byte stream format. It can optionally insert access unit delimiter NALUs.
+
+
 Available plugins
 -----------------
 
@@ -28,8 +152,10 @@ Available plugins
 * `imxvpuenc_mjpeg` : Motion JPEG encoder
 * `imxipuvideosink` : video sink using the IPU to output to Framebuffer (may not work well if X11 or Wayland are running)
 * `imxipuvideotransform` : video transform element using the IPU, capable of scaling, deinterlacing, rotating (in 90 degree steps), flipping frames, and converting between color spaces
+* `imxipucompositor` : video compositor element using the IPU for combining multiple input video streams into one output video stream
 * `imxg2dvideosink` : video sink using the GPU's 2D core (through the G2D API) to output to Framebuffer (may not work well if X11 or Wayland are running)
 * `imxg2dvideotransform` : video transform element using the GPU's 2D core (through the G2D API), capable of scaling, rotating (in 90 degree steps), flipping frames, and converting between color spaces
+* `imxg2dcompositor` : video compositor element using the IPU for combining multiple input video streams into one output video stream
 * `imxpxpvideosink` : video sink using the PxP engine to output to Framebuffer (may not work well if X11 or Wayland are running)
 * `imxpxpvideotransform` : video transform element using the PxP engine, capable of scaling, rotating (in 90 degree steps), flipping frames, and converting between color spaces
 * `imxeglvivsink` : custom OpenGL ES 2.x based video sink; using the Vivante direct textures, which allow for smooth playback
@@ -42,6 +168,9 @@ Dependencies
 ------------
 
 You'll need a GStreamer 1.2 installation, and Freescale's VPU wrapper library (at least version 1.0.45).
+Also, the `videoparsersbad` plugin from the `gst-plugins-bad` package in GStreamer is needed, since
+this plugin contains video parsers like `h264parse`, `mpegvideoparse` (for MPEG1 and MPEG2), and
+`mpeg4videoparse` (for MPEG4).
 
 
 Building and installing

wscript

@@ -6,7 +6,7 @@ from waflib.Build import BuildContext, CleanContext, InstallContext, UninstallCo
 top = '.'
 out = 'build'
 
-gstimx_version = "0.10.2"
+gstimx_version = "0.11.0"
 
 # the code inside fragment deliberately does an unsafe implicit cast float->char to trigger a
 # compiler warning; sometimes, gcc does not tell about an unsupported parameter *unless* the