Adapter for Jetpack Compose Desktop

[FlorianMarsch]

Hi could you maybe consider to make an adapter so vlcj can be used with Jetpack Compose for Desktop?
Unfortunetally this is build on Swing and as AWT is not Heavyweight on Mac anymore (since ever? :D) the Surface Panel stays black and only Audio is playing.

I know that you have already done some Adapters for Java FX or Libgdx but one for Jetpack Compose would be superb. It would be great if you could draw the video on the Canvas which uses Skia under the hood? Please let me know if this is feasible or if you have any advice to do it by my own

[caprica]

I have no idea what Jetpack Compose is.

[caprica]

Ha, I see Jetbrains themselves use vlcj in at least their experimental repo:

https://github.com/JetBrains/compose-jb/blob/master/experimental/components/VideoPlayer/library/src/desktopMain/kotlin/org/jetbrains/compose/videoplayer/DesktopVideoPlayer.kt

[FlorianMarsch]

Uff gosh - why did I have not seen this. thx for your support! i will try this out asap

[FlorianMarsch]

THX worked for me (Kotlin 1.5.31 + Compose 1.0.0)
Case closed. :D

[DrewCarlson]

The JetBrains example has a number of issues, but most importantly it uses the Swing interop. This makes it essentially useless for video playback in a Compose application as you cannot layer it within your Compose UI.

A better approach is to use a custom VideoSurface. This implementation offers similar performance and resource usage to the JavaFX ImageView approach with Compose.

public class SkiaBitmapVideoSurface : VideoSurface(VideoSurfaceAdapters.getVideoSurfaceAdapter()) {

    private val videoSurface = SkiaBitmapVideoSurface()
    private lateinit var imageInfo: ImageInfo
    private lateinit var frameBytes: ByteArray
    private val skiaBitmap: Bitmap = Bitmap()
    private val composeBitmap = mutableStateOf<ImageBitmap?>(null)

    public val bitmap: State<ImageBitmap?> = composeBitmap

    override fun attach(mediaPlayer: MediaPlayer) {
        videoSurface.attach(mediaPlayer)
    }

    private inner class SkiaBitmapBufferFormatCallback : BufferFormatCallback {
        private var sourceWidth: Int = 0
        private var sourceHeight: Int = 0

        override fun getBufferFormat(sourceWidth: Int, sourceHeight: Int): BufferFormat {
            this.sourceWidth = sourceWidth
            this.sourceHeight = sourceHeight
            return RV32BufferFormat(sourceWidth, sourceHeight)
        }

        override fun allocatedBuffers(buffers: Array<ByteBuffer>) {
            frameBytes = buffers[0].run { ByteArray(remaining()).also(::get) }
            imageInfo = ImageInfo(
                sourceWidth,
                sourceHeight,
                ColorType.BGRA_8888,
                ColorAlphaType.PREMUL,
            )
        }
    }

    private inner class SkiaBitmapRenderCallback : RenderCallback {
        override fun display(
            mediaPlayer: MediaPlayer,
            nativeBuffers: Array<ByteBuffer>,
            bufferFormat: BufferFormat,
        ) {
            SwingUtilities.invokeLater {
                nativeBuffers[0].rewind()
                nativeBuffers[0].get(frameBytes)
                skiaBitmap.installPixels(imageInfo, frameBytes, bufferFormat.width * 4)
                composeBitmap.value = skiaBitmap.asComposeImageBitmap()
            }
        }
    }

    private inner class SkiaBitmapVideoSurface : CallbackVideoSurface(
        SkiaBitmapBufferFormatCallback(),
        SkiaBitmapRenderCallback(),
        true,
        videoSurfaceAdapter,
    )
}

    val mediaPlayerComponent = remember { EmbeddedMediaPlayerComponent() }
    val mediaPlayer = remember { mediaPlayerComponent.mediaPlayer() }
    val surface = remember {
        SkiaBitmapVideoSurface().also {
            mediaPlayer.videoSurface().set(it)
        }
    }
    DisposableEffect(Unit) { onDispose(mediaPlayer::release) }
    Box(modifier = modifier) {
        surface.bitmap.value?.let { bitmap ->
            Image(
                bitmap,
                modifier = Modifier
                    .background(Color.Black)
                    .fillMaxSize(),
                contentDescription = null,
                contentScale = ContentScale.Fit,
                alignment = Alignment.Center,
            )
        }
    }

[caprica]

Very helpful, thanks for posting that.

[mahozad]

Hello. This may be related: JetBrains/compose-multiplatform#3336