Skip to content

Set is_screencast on NativeVideoSource creation - #552

Closed
iparaskev wants to merge 1 commit into
livekit:mainfrom
gethopp:properly_set_is_screencast
Closed

Set is_screencast on NativeVideoSource creation#552
iparaskev wants to merge 1 commit into
livekit:mainfrom
gethopp:properly_set_is_screencast

Conversation

@iparaskev

Copy link
Copy Markdown
Contributor

Makes InternalSource's is_screencast configurable when creating a NativeVideoSource. This improves the stream's latency when vp9 is used for sharing screen content.

@ilo-nanpa

ilo-nanpa Bot commented Jan 19, 2025

Copy link
Copy Markdown
Contributor

it seems like you haven't added any nanpa changeset files to this PR.

if this pull request includes changes to code, make sure to add a changeset, by writing a file to .nanpa/<unique-name>.kdl:

minor type="added" "Introduce frobnication algorithm"

refer to the manpage for more information.

@iparaskev
iparaskev force-pushed the properly_set_is_screencast branch from eef7d59 to d276130 Compare January 19, 2025 19:20

@typester typester left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like we already have a screenshare flag:

https://github.com/iparaskev/rust-sdks/blob/d276130dbd0205520f272861da718ce84b4bef80/livekit/src/room/options.rs#L76-L88

#[derive(Clone, Debug)]
pub struct TrackPublishOptions {
    // If the encodings aren't set, LiveKit will compute the most appropriate ones
    pub video_encoding: Option<VideoEncoding>,
    pub audio_encoding: Option<AudioEncoding>,
    pub video_codec: VideoCodec,
    pub dtx: bool,
    pub red: bool,
    pub simulcast: bool,
    // pub name: String,
    pub source: TrackSource,
    pub stream: String,
}

https://github.com/iparaskev/rust-sdks/blob/d276130dbd0205520f272861da718ce84b4bef80/livekit/src/room/track/mod.rs#L61-L68

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum TrackSource {
    Unknown,
    Camera,
    Microphone,
    Screenshare,
    ScreenshareAudio,
}

Would it be sufficient to reuse this flag?

cc: @theomonnom

@iparaskev

Copy link
Copy Markdown
Contributor Author

Would it be sufficient to reuse this flag?

I didn't think of this TBH. It could be done by exposing a setter in VideoTrackSource and then set the is_screencast_ member in publish_track. If you are OK with this, I am happy to go with it.

Side note: The only "problem" I see with this, is that NativeVideoSource will require a kind of two stage initialization, which could be error prone. A way to mitigate it would be to create the NativeVideoSource object in publish_track, but I am not sure if this is acceptable.

@iparaskev
iparaskev force-pushed the properly_set_is_screencast branch from 7ea5f04 to 6d49d4c Compare January 24, 2025 09:18
@iparaskev

Copy link
Copy Markdown
Contributor Author

Something that I had forgotten to mention when I created this, is that when is_screencast is set, libwebrtc limits the FPS to 5. I had to add spatial_layers.back().maxFramerate = video_codec.maxFramerate; here as a workaround.

Makes InternalSource's is_screencast configurable. Setting it improves
the latency when sharing screen content.
@iparaskev
iparaskev force-pushed the properly_set_is_screencast branch from 6d49d4c to b04977c Compare November 22, 2025 10:37
@iparaskev

Copy link
Copy Markdown
Contributor Author

I tested it with m137 and the libwebrtc WA is not needed any more, f126e8af824c45ba2343225229f051d3b3418923 fixed it. Setting the screensharing flag doesn't affect the FPS now.

@iparaskev iparaskev closed this Feb 27, 2026
@iparaskev

Copy link
Copy Markdown
Contributor Author

Closing as it was implemented it in a different PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants