How to enhance rendering of Bitmap TTML subtitles ?

[xirac]

[REQUIRED] Searched documentation and issues

Searched in Exoplayer documentation, current and past issues #4657 and look had a look into SubtitleView.java

[REQUIRED] Question

I have lot of Dash streams with TTML tracks where subtitles are Bitmap, not text.

The subtitles are displayed as expected but the Bitmap rendering could be enhanced:

Is there a hook or a way to process the Cue containing those Bitmap before rendering those on screen?
I had a look into SubtitleView.onCues(List cues) method but I don't see how extend/override it when building the player:

player = SimpleExoPlayer.Builder(context, renderersFactory).setTrackSelector(trackSelector!!).build().apply {
                addListener(PlayerEventListener())
                playWhenReady = true
                addAnalyticsListener(EventLogger(trackSelector))
                val style = CaptionStyleCompat(defaultSubtitleColor,
                    Color.TRANSPARENT, Color.TRANSPARENT,
                    CaptionStyleCompat.EDGE_TYPE_DROP_SHADOW,
                    outlineColor, null)
                subtitleView?.setStyle(style) // Change text subtitles style
}

I'm using exoplayer 2.11.1

[icbaker]

What do you mean by 'enhanced'? It looks like maybe the subtitles in your media are just low resolution and you're seeing the resultant pixelation.

There's no easy hook like you've described to mutate the cues just before they get displayed. I'm not sure whether it's something that would be generally useful, so I think we'd be hesitant to add it for this use-case specifically.

A possible solution:
You could create your own ImageEnhancingTtmlDecoder that delegates to the existing TtmlDecoder and wraps the Subtitle objects coming from decode() in a new Subtitle implementation that mutates the image cues as they're returned from getCues().

You can register this custom decoder by creating a new implementation of SubtitleDecoderFactory and then overriding ExoPlayerFactory#createTextRenderer and passing this factory into the TextRenderer constructor.

class ImageEnhancingTtmlDecoder extends SimpleSubtitleDecoder {

  TtmlDecoder delegate;

  @Override
  Subtitle decode(byte[] bytes, int length, boolean reset)
      throws SubtitleDecoderException {
    Subtitle subtitle = delegate.decde(bytes, length, reset);
    return new ImageEnhancingTtmlSubtitle(subtitle);
  }
}

class ImageEnhancingTtmlSubtitle implements Subtitle {

  Subtitle delegate;

  @Override 
  List<Cue> getCues(long timeUs) {
    List<Cue> cues = delegate.getCues(timeUs);
    return enhanceImageCues(cues);
  }
}

[xirac]

The subtitles are low resolution in some source I get, you're right.
Thanks for pointing to this possible solution, I'll give a try.

[xirac]

The change was actually simple.

In library/ui/src/main/java/com/google/android/exoplayer2/ui/SubtitlePainter.java
by changing function drawBitmapLayout like this:

@RequiresNonNull({"cueBitmap", "bitmapRect"})
private void drawBitmapLayout(Canvas canvas) {
-    canvas.drawBitmap(cueBitmap, /* src= */ null, bitmapRect, /* paint= */ null);
+    Paint paint = new Paint();
+    paint.setFilterBitmap(true);
+    canvas.drawBitmap(cueBitmap, /* src= */ null, bitmapRect, /* paint= */ paint);
}

The rendering looks pettier.
Before

After

Of course the enhancement is less visible on higher resolution bitmap subtitles:

Do you think it could be proposed as a PR?

[icbaker]

Ah interesting, thanks for pointing this out.

I tried to reproduce this locally but it doesn't seem to make any difference to the bitmap subtitles I have access to.

Could you provide the bitmap subtitles you're using to test this, so I can confirm the improvement?

[xirac]

Stream URL provided via private message.

[icbaker]

Thanks for the test stream, I can see a significant improvement. I'll submit a change to enable this.

[icbaker]

This is now available in dev-v2