play gstreamer videosink in flutter

[KevinJeulink1]

Hi Fengjiongmax,

Again thank you for all the help. I have been trying to use your project and adapt it in order to play a video with gstreamer videosink.
However, I am not able to get it to work. I have replaced the audio sink with a video sink and the audio test src, with the video test src, I am able to compile and startup the application. However the problem is, the video sink needs to be linked to a widget in order to play and show the video. I have tried to look into existing projects, where they suggest e.g., using a Texture widget and link the texture id, however I don't get how to do this.

There is also a tutorial on the gstreamer website how to create a gst video in c code, but this is not with flutter, I have diffeculties how to translate this tutorial and use it in a flutter project with ffi plugins https://gstreamer.freedesktop.org/documentation/tutorials/android/video.html?gi-language=c#a-video-surface-on-android-c-code

My question, do you have any tips in which direction I should look to link the video sink with a widget in flutter, such that it plays the video?

Hope you could help me out, maybe this would also be an interesting follow up video tutorial on YT, when we have a solution.

[fengjiongmax]

From my research, the video displaying can be achieved using platform-view, and it's platform-specific, and the flutter team has documented this, for example on Android: https://docs.flutter.dev/platform-integration/android/platform-views
And I will find some time to look into this.

[fengjiongmax]

Progress update:
To make this work we need another flutter plugin that uses platform-native code ( like Kotlin on Android ) to create native views,
and by far my research was on Android, so I'm going to start this with Android.

So what we need to do on Android:

• create a native view ( Kotlin )
• let videosink render to native view ( C )
• let flutter render native view ( Dart )

[KevinJeulink1]

I will have a look at it this weekend. Will keep you updated.

[fengjiongmax]

Before this issue, what this repo has done is set up communication between Dart and C ( we don't have C to Dart yet):

graph
  a[Dart] --> b[C]

With a new plugin native_view, we now have this project that lets Dart use Kotlin code (which creates a native_window for videosink to render to):

graph
  langA[Dart] --> langB[C]
  langA --> langC[Kotlin]

What we need is to let Kotlin communicate with C, so that we can get the address of native_window:

graph
  langA[Dart] --> langB[C]
  langA --> langC[Kotlin]
  langC --> langB

Now I think this is doable, and I just pushed it to branch videodisplay, haven't tried anything further yet, but this looks promising.

[KevinJeulink1]

Awesome! I am sorry I am very busy with work. Will look more into it when I find the time.

[fengjiongmax]

I got it working!
pipeline is:

gst-launch-1.0 -v videotestsrc ! autovideosink

If you want to check what changes were made, please compare the branch videodisplay with the master branch, feel free to ask questions, though the code is messy.

And I will keep trying to make it work on all other platforms, and by then, I will make videos on them.

[fengjiongmax]

After some investigation, making it work on other platforms seems more difficult than I thought, so I did a bit of clean-up, that should get you started, and if you want to have an easier time developing and debugging, you can open the android folder of the app project with Android Studio, so that you get functions and macros hint from android studio.

[yinyong02]

awesome！ it worked for me.