-
Notifications
You must be signed in to change notification settings - Fork 136
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Timestamps and time domains #8
Comments
In particular, when it comes to connecting WebCodec objects to |
@padenot can you give toy example for how the problem manifests? |
Here is a typical setup:
For video, simply repeating the frame is the approach usually taken (not always). For audio, it is absolutely essential to do adaptive resampling, or else problems will appear: either latency buildups if the drift goes in one direction, or gaps (translating into very problematic audio glitches) if the drift goes in the other direction. |
Most of the concern about playout via MediaStreamTrack is now out of scope for WebCodecs, and might instead go to https://github.com/w3c/mediacapture-transform/issues. Issue w3c/mediacapture-transform#35 might be a good a starting point. For sync generally, I think resolution here is the same as discussion in #39 (comment). In short, apps can overcome skew concerns by letting audio drive the clock (as a function of its output samples and sample rate), and then paint video frames to match wherever audio is at. Closing as I don't think there's anything else in scope for WC here, but please reopen if I've overlooked anything. |
This is a tracking issue for describing how timestamps work with WebCodecs and integrate with the video and audio playout time domains.
The text was updated successfully, but these errors were encountered: