Configurable media quality

[ara4n]

Currently our gUM constraints are hardcoded and very basic (640x360 video; no audio quality controls at all): https://github.com/matrix-org/matrix-js-sdk/blob/96ba061732bc3b5e748719072356d562e52097ba/src/webrtc/mediaHandler.ts#L296

We should let the user pick at least the video resolution & bitrate they want to send (presumably by some sensible defaults).

Could also be fun to let them try to select audio quality (e.g. stereo or not)

Also, leaving the bitrate entirely up to the browser feels risky in the extreme.

[w3bb]

Framerate is important for gamers, currently there isn't any alternatives to Discord that have framerates high enough to be worthwhile. An option for 30 and 60 FPS would be awesome.

[martininsulander]

Playing music with good quality over matrix would be awesome. I'm currently thinking of two scenarios.
Firstly I thought of play something for a friend, a group or even having a concert, focusing on good quality sound.
Secondly, if it would be possible to Jam together (would require very low latency).

[toniocat]

Currently our gUM constraints are hardcoded and very basic (640x360 video; no audio quality controls at all): https://github.com/matrix-org/matrix-js-sdk/blob/96ba061732bc3b5e748719072356d562e52097ba/src/webrtc/mediaHandler.ts#L296

If element-call is going to be available via Element Web with users not installing their own Synapse client, hard-coding limits are preferred. Perhaps we should have this ability enabled in only certain environments.

[terrytw]

I think it is sensible to at least make default resolution and bitrate configurable for homeservers.

[darkralts]

Framerate is important for gamers, currently there isn't any alternatives to Discord that have framerates high enough to be worthwhile. An option for 30 and 60 FPS would be awesome.

i agree, 60fps is very important for me too, i need to watch and screen share movies at 60fps, as lots of media i have is 60fps vs 30, looks choppy on the others end.

[hazzuk]

Wanted to express, I strongly agree with this issue. This is one of the few issues with Element Call I still have. Coming from Discord, the quality of the audio is subpar, and this is noticeable to myself and other users moving over to Matrix.

Additionally, we have no control over the resolution or framerate of share screen. We normally used Discord share screen for gameplay. Discord defaults to a solid 720p 30fps. Unfortunately Element Call is currently very choppy and unusable for this purpose.

Ideally controls for audio bitrate, any video resolution and video frame rate should be exposed to the user or homeserver owner. And once set should be remembered.

[unknowntrojan]

The low quality for both audio and video is holding Element back from having more Discord users switch over. I'm not experienced with browser development, but I tried to tinker around with the code and see if I can force the browser to use a higher bitrate and achieve something better than 2kbps dishwater HD. So far I've not managed, but I imagine it can't be that complex, and I'm just missing something. Of course it's not only about the bitrate, being able to choose resolution & framerate is also very important. I'll keep trying, I guess.

[a11cf0]

Are there any plans to implement this?

[Moresteck]

The low quality for both audio and video is holding Element back from having more Discord users switch over. I'm not experienced with browser development, but I tried to tinker around with the code and see if I can force the browser to use a higher bitrate and achieve something better than 2kbps dishwater HD. So far I've not managed, but I imagine it can't be that complex, and I'm just missing something. Of course it's not only about the bitrate, being able to choose resolution & framerate is also very important. I'll keep trying, I guess.

Hi, have you got anywhere with this?

[darkralts]

yeah like, discord is browser based electron, so it must be possible

…

On Thu, Oct 10, 2024, 08:32 Moresteck ***@***.***> wrote: The low quality for both audio and video is holding Element back from having more Discord users switch over. I'm not experienced with browser development, but I tried to tinker around with the code and see if I can force the browser to use a higher bitrate and achieve something better than 2kbps dishwater HD. So far I've not managed, but I imagine it can't be that complex, and I'm just missing something. Of course it's not only about the bitrate, being able to choose resolution & framerate is also very important. I'll keep trying, I guess. Hi, have you got anywhere with this? — Reply to this email directly, view it on GitHub <#249 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD43UG3JUQ73KKWVBQ375ETZ2ZXWDAVCNFSM6AAAAABPWVYZZKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBUHE3DGOBQGY> . You are receiving this because you commented.Message ID: ***@***.***>

[Pirosuki]

This seems to work for screen share. No idea how you would make it easily configurable however.

Resolution and framerate

diff --git a/src/room/InCallView.tsx b/src/room/InCallView.tsx
index d50be3c9..26515b3c 100644
--- a/src/room/InCallView.tsx
+++ b/src/room/InCallView.tsx
@@ -475,6 +475,11 @@ export const InCallView: FC<InCallViewProps> = ({
         selfBrowserSurface: "include",
         surfaceSwitching: "include",
         systemAudio: "include",
+        resolution: {
+          width: 1920,
+          height: 1080,
+          frameRate: 30,
+        },
       })
       .catch(logger.error);

Bitrate

diff --git a/src/livekit/options.ts b/src/livekit/options.ts
index f7fe03ff..c99ae98b 100644
--- a/src/livekit/options.ts
+++ b/src/livekit/options.ts
@@ -24,7 +24,10 @@ const defaultLiveKitPublishOptions: TrackPublishDefaults = {  
   forceStereo: false,
   simulcast: true,
   videoSimulcastLayers: [VideoPresets.h180, VideoPresets.h360] as VideoPreset[],
-  screenShareEncoding: ScreenSharePresets.h1080fps30.encoding,
+  screenShareEncoding: {
+    maxBitrate: 5_000_000,
+    maxFramerate: 30,
+  },
   stopMicTrackOnMute: false,
   videoCodec: "vp8",
   videoEncoding: VideoPresets.h720.encoding,