ImageOverlayFilter didn't work

[Napoleon-Black]

KMS Version:

Kurento Media Server version: 6.9.0
Found modules:
	'core' version 6.9.0
	'elements' version 6.9.0
	'filters' version 6.9.0

Other libraries versions:

ii  gstreamer1.5-libav:amd64                      1.8.2.1.xenial~20181023163456.96.493eee4      amd64        libav plugin for GStreamer
ii  gstreamer1.5-nice:amd64                       0.1.15-1kurento2                              amd64        ICE library (GStreamer 1.5 plugin)
ii  gstreamer1.5-plugins-bad:amd64                1.8.1.1.xenial~20181023161445.100.3db37b1     amd64        GStreamer plugins from the "bad" set
ii  gstreamer1.5-plugins-base:amd64               1.8.1.1.xenial~20181023154838.55.7b19cfd      amd64        GStreamer plugins from the "base" set
ii  gstreamer1.5-plugins-good:amd64               1.8.1.1.xenial~20181023160032.112.9ee4248     amd64        GStreamer plugins from the "good" set
ii  gstreamer1.5-plugins-ugly:amd64               1.8.1.1.xenial~20181023162927.89.2685b0f      amd64        GStreamer plugins from the "ugly" set
ii  gstreamer1.5-pulseaudio:amd64                 1.8.1.1.xenial~20181023160032.112.9ee4248     amd64        GStreamer plugin for PulseAudio
ii  gstreamer1.5-x:amd64                          1.8.1.1.xenial~20181023154838.55.7b19cfd      amd64        GStreamer plugins for X11 and Pango
ii  joxi                                          3.0.13                                        amd64        Joxi - nice screenshoter
ii  kms-core                                      6.9.0.xenial.20181210200440.bee54d3           amd64        Kurento Core module
ii  kms-elements                                  6.9.0.xenial.20181210201219.6d6f745           amd64        Kurento Elements module
ii  kms-filters                                   6.9.0.xenial.20181210201843.08e687c           amd64        Kurento Filters module
ii  kms-jsonrpc                                   6.9.0.xenial.20181210195903.71b45ec           amd64        Kurento JSON-RPC library
ii  kmsjsoncpp                                    1.6.3.xenial.20181023152331.d78deb7           amd64        Kurento jsoncpp library
ii  kurento-media-server                          6.9.0.xenial.20181210202449.f940867           amd64        Kurento Media Server
ii  libgstreamer-plugins-bad1.5-0:amd64           1.8.1.1.xenial~20181023161445.100.3db37b1     amd64        GStreamer development files for libraries from the "bad" set
ii  libgstreamer-plugins-base1.5-0:amd64          1.8.1.1.xenial~20181023154838.55.7b19cfd      amd64        GStreamer libraries from the "base" set
ii  libgstreamer1.5-0:amd64                       1.8.1.1.xenial~20181023153550.170.0d6031b     amd64        Core GStreamer libraries and elements
ii  libnice10:amd64                               0.1.15-1kurento2                              amd64        ICE library (shared library)
ii  libreoffice-avmedia-backend-gstreamer         1:5.1.6~rc2-0ubuntu1~xenial6                  amd64        GStreamer backend for LibreOffice
ii  openh264-gst-plugins-bad-1.5:amd64            1.8.1.1.xenial~20181023161445.100.3db37b1     amd64        GStreamer plugins from openh264

Client libraries

• Language: JavaScript
• Version: 6.9.0

Browsers tested

• Chrome: FAIL
• Firefox: FAIL

What steps will reproduce the problem?

1. Clone Magic Mirror tutorial from https://github.com/Kurento/kurento-tutorial-js/tree/master/kurento-magic-mirror
2. Change FaceOverlayFilter to ImageOverlayFilter, and change setOverlayedImage to addImage with new arguments.
3. Start tutorial

What is the expected result?
ImageOverlayFilter works.

What happens instead?
ImageOverlayFilter didn't work at all. In Chrome I receive error:

Error: 'operationParams' is required
    at kurento-client.js:362
    at Object.dispatchCallback [as callback] (kurento-client.js:36499)
    at processResponse (kurento-client.js:36630)
    at RpcBuilder.decode (kurento-client.js:36691)
    at Stream.transportMessage (kurento-client.js:36110)
    at emitOne (kurento-client.js:19833)
    at Stream.emit (kurento-client.js:19911)
    at drain (kurento-client.js:49942)
    at Stream.stream.queue.stream.push (kurento-client.js:49951)
    at WebsocketStream.onMessage (kurento-client.js:51304)

Does it happen with one of the tutorials?
Yes. Magic Mirror.

Please provide any additional information below.
Also I was change:

client.connect(webRtcEp, filter, webRtcEp, function(error) {
  if (error) return onError(error);

  console.log("WebRtcEndpoint --> filter --> WebRtcEndpoint");
});

to:

client.connect(webRtcEp, filter, function(error) {
  if (error) return onError(error);


  client.connect(filter, webRtcEp, function(error) {
    if (error) return onError(error);

    console.log("WebRtcEndpoint --> filter --> WebRtcEndpoint");
  });

});

because first one didn't work for some reason.

[delia-m]

I have the same issue on 6.10.0.
Is this fixed on 6.11.0 ?

+) I updated to 6.11.0 today and it didn't work - same issue

[j1elo]

Hi,
this is a known issue that is caused by wanting to port to Ubuntu 18.04 Bionic but not having enough hands to separate implementations of OpenCV filters that need to load external HAAR training data sets. All this broke because in Ubuntu 18.04 the old method used by Kurento filters doesn't work any more: https://doc-kurento.readthedocs.io/en/6.11.0/project/relnotes/v6_10_0.html#hello-ubuntu-bionic

Expected fix date is either when the project priorities allow for reviewing all OpenCV code in Kurento filters and update to newer API versions, when some amazing community user contributes the required changes, or when some interested party(ies) contract commercial support.

[delia-m]

Not sure it's resolved on v6.12....??

[micaelgallego]

Nop.

[maitrungduc1410]

I'm now not able to run the magic-mirror example, the error is shown as below

Error: Unexpected error while processing method: /build/opencv-L2vuMj/opencv-3.2.0+dfsg/modules/core/src/persistence.cpp:6628: error: (-2) The node does not represent a user object (unknown type?) in function cvRead

Try searching on Google and it seems because openCV implementation in Kurento doesn't work with Ubuntu 18.04

Seems like this bug hasn't been fixed yet: https://github.com/orgs/Kurento/projects/1

[j1elo]

Yeah, with the migration from Ubuntu 14.04 to 16.04, all things still worked. But then moving from 16.04 to 18.04 requires some more work that nobody has been able to do or contribute yet, to move code and make it work with newer OpenCV versions.

[micaelgallego]

@j1elo just fix the problems with filters.

Test with 6.14 to see if your problems are solved.

Regards

[j1elo]

Hi, some updates regarding Kurento filters, that had been broken for a while:

• All Kurento extra modules can now be installed again from Apt package repositories: kms-chroma, kms-crowddetector, kms-platedetector, kms-pointerdetector.

• All plugins using HTTP are working again. This includes the ImageOverlayFilter, which is used by FaceOverlayFilter, both available from kms-filters (a demo of the FaceOverlayFilter can be seen with the WebRTC Magic Mirror tutorial).

These had been removed since Kurento 6.10, because it was difficult to make them build in Ubuntu 16.04 and 18.04 at the same time. But the issues have been sorted out, and they are now available again.

You will have to use Nightly Builds of Kurento (and its client SDKs) to get them, until they get released as part of Kurento 6.15 (no scheduled date yet). To update a KMS installation from Release to Nightly, follow the instructions given in the Local Upgrade doc section.

Remember that these "extra" modules are provided exclusively for demonstration purposes.

Regarding the OpenCV issues in Ubuntu 18.04:

One BIG problem we have to upgrade from Ubuntu 16.04 to 18.04 is that our OpenCV plugin code needs a good revision and update. As far as I understand, Kurento was written against an old OpenCV C API, which has now been deprecated (but not removed) in favor of a more modern C++ API. Fortunately, our code still builds, even against the OpenCV 3.2 version that comes with Ubuntu 18.04. But some plugins are broken.

For example the FaceDetector plugin (in kms-filters) uses some machine learning training data (Haar Cascades), which since OpenCV 3.x has to be loaded with the newer C++ API, not the older C one. So Kurento is not able to load it and detect faces on Ubuntu 18.04, but it works fine with the training data that comes with OpenCV 2.x in Ubuntu 16.04.

I imagine with 20.04 the situation only gets worse. It comes with OpenCV 4.2, so we're bound to yet more breakage, possibly.

Bottom line is: if you want to see the OpenCV demonstration plugins in action, please run them under Ubuntu 16.04. Pull Requests welcome to make them work also in Ubuntu 18.04 (OpenCV 3.x)! :-)

[j1elo]

This was worth a public announcement in the Kurento forums so I've copied it there: https://groups.google.com/d/topic/kurento/1VOK0c2O4b4/discussion

[delia-m]

Hi guys, thanks for updating. I am glad to hear this is fixed.
I just tested FaceOverlayFilter with kurento 6.14 (via docker) with my local node server and it's working 🎉 Yeah!!
But sadly it still throws error when I change FaceOverlayFilter to ImageOverlayFilter

Trace: Error: 'operationParams' is required

I have seen this for a long time and it doesn't seem to fixed on kurento-client 6.14 (at least for node client)

This is the code for FaceOverlayFilter which is working well

const _filter = await pipeline.create('FaceOverlayFilter');
_filter.setOverlayedImage(imageUrl, -0.35, -1.2, 1.6, 1.6);

and I change like this

const _filter = await pipeline.create('ImageOverlayFilter');
_filter.addImage('imageId', imageUrl, -0.35, -1.2, 1.6, 1.6, true, true, (error) => console.log('---here error throws: ', error));

Actually I wanted to add image and move them to specific position but not sure I can change the image's position without remove and add it again...

Let me know if I miss something. Thanks!

Regards

[j1elo]

Follow this: https://doc-kurento.readthedocs.io/en/latest/user/troubleshooting.html#error-operationparams-is-required

and if it works please let me know so I can confirm that a) the suggested solution in the troubleshooting guide actually works, and b) that this issue is then fixed and can be closed (or otherwise something more has to be worked on)

[delia-m]

@j1elo Awesome! Thank you so much. It totally works!
And I could removed image with

_filter.removeImage(null);

I added an image couple of times with different position and it makes image moves.
I am so glad finally I got this! I think this issue can be closed 👍

[j1elo]

Happy to hear that it works!