there is no fallback for vo

[lcksk]

Currently, the MPV core does not allow for VO fallback, right? From what I see in the current reinit_video_chain_src code, it always prioritizes creating a VO, which almost certainly succeeds most of the time. Then it constructs the vo filter chain, followed by creating the decoder. When creating the decoder, it currently allows falling back from hardware decoder to software decoder. So, if you specifies a VO that is strongly related to a hardware decoder, then when the decoder falls back, the whole chain cannot work properly, For example, on the Android platform, if you want to use :
--vo=mediacodec_embed,
--hwdec=mediacodec,
--gpu-context=android,
--hwdec=mediacodec,
and if the hardware decoding fails, the mpv core will fall back to software decoding, but the entire chain still won't work because mediacodec_embed doesn't support software decoding formats (like yuv420p). Even worse, if you don't specify a VO, the default choice during autoprobe will also be video_out_mediacodec_embed because it's always at the top in video_out_drivers. So the current situation is, you must specify a versatile vo (either vo=gpu or vo=gpu-next) that supports both software and hardware decoding; otherwise, fallback will fail due to the VO not supporting certain formats.

If VO could also support fallback, like specifying vo=mediacodec_embed, gpu, then when the decoder falls back, it would automatically switch to vo=gpu, ensuring the whole chain works properly.

[guidocella]

--vo is already a list option that accepts multiple values.

[kasper93]

vo list always uses the first successful

Exactly, testing all combinations of gpu / vo / gpu-context / hwdec, would be unfeasible, simply because number of combinations quickly raise. Patches welcome if you have better ideas to handle it. If I understand your usecase, you should be creating specific vo / hwdec and if it fails select different one.

One fix I could see is to not fallback to swdec for things that will not support it or add hardware upload. Either way patches welcome.

[Dudemanguy]

One fix I could see is to not fallback to swdec for things that will not support it

This option already exists actually --vd-lavc-software-fallback. If it had an auto value it would fix OPs usecase. As it is, it's actually already pretty close.

[lcksk]

If I understand your usecase, you should be creating specific vo / hwdec and if it >fails select different one.

Yes, that's the goal.

One fix I could see is to not fallback to swdec for things that will not support it or >add hardware upload. Either way patches welcome.

THB, I'm not entirely sure what I'm doing either, as my understanding of mpv is still quite limited, I don't quite understand it yet.

I'll try to explain it to see if I can make it clear, using the following use cases as examples:

--hwdec=mediacodec,auto
--gpu-context=android
--hwdec-codecs=h264,hevc,mpeg4,vp8,vp9
--vo=mediacodec_embed,gpu,

Since my main goal is to prioritize displaying video using SurfaceView, I have to place mediacodec_embed first. However, if the format is outside the ones specified by hwdec-codecs, such as mpeg2video, then obviously mediacodec_embed will cause an error. At this point, gpu is clearly the correct choice.

So, my first naive idea is to pass the codec information from video.c to vo.c.

Subject: [PATCH] vo:passing codec information to vo

---
 player/video.c | 1 +
 video/out/vo.c | 1 +
 video/out/vo.h | 1 +
 3 files changed, 3 insertions(+)

diff --git a/player/video.c b/player/video.c
index f0372b69a5..53282af0fc 100644
--- a/player/video.c
+++ b/player/video.c
@@ -231,6 +231,7 @@ void reinit_video_chain_src(struct MPContext *mpctx, struct track *track)
             .encode_lavc_ctx = mpctx->encode_lavc_ctx,
             .wakeup_cb = mp_wakeup_core_cb,
             .wakeup_ctx = mpctx,
+            .stream = track->stream,
         };
         mpctx->video_out = init_best_video_out(mpctx->global, &ex);
         if (!mpctx->video_out) {
diff --git a/video/out/vo.c b/video/out/vo.c
index db29690950..7f42ea2089 100644
--- a/video/out/vo.c
+++ b/video/out/vo.c
@@ -45,6 +45,7 @@
 #include "sub/osd.h"
 #include "osdep/io.h"
 #include "osdep/threads.h"
+#include "demux/stheader.h"
 
 extern const struct vo_driver video_out_mediacodec_embed;
 extern const struct vo_driver video_out_x11;
diff --git a/video/out/vo.h b/video/out/vo.h
index 313c08aafc..9d5139ed34 100644
--- a/video/out/vo.h
+++ b/video/out/vo.h
@@ -210,6 +210,7 @@ struct vo_extra {
     struct encode_lavc_context *encode_lavc_ctx;
     void (*wakeup_cb)(void *ctx);
     void *wakeup_ctx;
+    struct sh_stream *stream;
 };
 
 struct vo_frame {
-- 
2.33.1

Then, logic similar to the following could be implemented for simple filtering, allowing me the opportunity to choose gpu instead of mediacodec_embed. Of course, this is just pseudo-code, and there are many details to consider:

static struct vo *vo_create(bool probing, struct mpv_global *global,
                            struct vo_extra *ex, char *name)
{
    assert(ex->wakeup_cb);

    struct mp_log *log = mp_log_new(NULL, global->log, "vo");
    struct m_obj_desc desc;
    if (!m_obj_list_find(&desc, &vo_obj_list, bstr0(name))) {
        mp_msg(log, MSGL_ERR, "Video output %s not found!\n", name);
        talloc_free(log);
        return NULL;
    };

#ifdef __ANDROID__
    //Fixme: this hack "hwdec-codecs"
    char * hwdec_codecs = "h264,hevc,mpeg4,vp8,vp9";
    bool hwdec_codec_allowed = false;
    char *codec = ex->stream->codec->codec;

    bstr s = bstr0(hwdec_codecs);
    while (s.len) {
        bstr item;
        bstr_split_tok(s, ",", &item, &s);
        if (bstr_equals0(item, "all") || bstr_equals0(item, codec))
            hwdec_codec_allowed = true;
    }

    if(!hwdec_codec_allowed && !strcmp(name,"mediacodec_embed")) {
        mp_msg(log, MSGL_ERR, "Video output %s not used for without hwdec_codecs!\n", name);
        talloc_free(log);
        return NULL;
    }
#endif 
...

}

Although this is far from a real fallback, it can initially solve the problem: if it's clear that hardware decoding won't be used, then there's still an opportunity for the VO list to switch to the second-priority vo (gpu). However, it cannot handle hwdec fallback scenarios .

[lcksk]

Honestly, I'm not happy with the above approach at all,feel like on the wrong way.(>﹏<)

[lcksk]

Pre-selecting an "optimal" vo seems to be the foundation of the entire framework. However, if a fallback occurs in hwdec , which typically happens when hardware decoding fails and is often accompanied by a change in imgfmt , could we reconstruct the vo based on the decoder’s imgfmt? This would be beneficial for highly optimized configurations that can safely achieve fallback without worrying about compromising the whole things due to prioritizing specially optimized vos. For example, in mpv-android with hw+ (hwdec=mediacodec, vo=mediacodec-embed,gpu), this approach could potentially be very useful.