Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow importExternalTexture from VideoFrame #4063

Merged
merged 3 commits into from
Apr 26, 2023
Merged

Allow importExternalTexture from VideoFrame #4063

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
eb7b00e
Allow importExternalTexture from VideoFrame
kainino0x Apr 21, 2023
387828f
Don't try to exhaustively list places a VideoFrame can be closed
kainino0x Apr 26, 2023
29ea450
Link to "Close VideoFrame" directly
kainino0x Apr 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
30 changes: 25 additions & 5 deletions spec/index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,9 @@ spec: HTML; urlPrefix: https://html.spec.whatwg.org/multipage/
for: video
text: intrinsic width; url: media.html#concept-video-intrinsic-width
text: intrinsic height; url: media.html#concept-video-intrinsic-height
spec: WebCodecs; urlPrefix: https://www.w3.org/TR/webcodecs/#
type: dfn
text: Close VideoFrame; url: close-videoframe
spec: WEBGL-1; urlPrefix: https://www.khronos.org/registry/webgl/specs/latest/1.0/#
type: interface
text: WebGLRenderingContext; url: WEBGLRENDERINGCONTEXT
Expand Down Expand Up @@ -5016,10 +5019,21 @@ GPUExternalTexture includes GPUObjectBase;
An external texture is created from an external video object
using {{GPUDevice/importExternalTexture()}}.

An external texture created from an {{HTMLVideoElement}} is destroyed automatically in a task after
it is imported, instead of manually or upon garbage collection like other resources.
An external texture created from an {{HTMLVideoElement}} expires (is destroyed) automatically in a
task after it is imported, instead of manually or upon garbage collection like other resources.
When an external texture expires, its {{GPUExternalTexture/[[expired]]}} slot changes to `true`.

An external texture created from a {{VideoFrame}} expires (is destroyed) when, and only when,
the source {{VideoFrame}} is [=Close VideoFrame|closed=],
either explicitly by {{VideoFrame/close()}}, or by other means.

Note: As noted in {{VideoDecoder/decode()}}, authors **should** call
{{VideoFrame/close()}} on output {{VideoFrame}}s to avoid decoder stalls.
If an imported {{VideoFrame}} is dropped without being closed, the imported
{{GPUExternalTexture}} object will keep it alive until it is also dropped.
The {{VideoFrame}} cannot be garbage collected until both objects are dropped.
Garbage collection is unpredictable, so this may still stall the video decoder.

Once the {{GPUExternalTexture}} expires, {{GPUDevice/importExternalTexture()}} must be called again.
However, the user agent may un-expire and return the same {{GPUExternalTexture}} again, instead of
creating a new one. This will commonly happen unless the execution of the application is scheduled
Expand All @@ -5030,7 +5044,7 @@ If the same object is returned again, it will compare equal, and {{GPUBindGroup}
<script type=idl>
dictionary GPUExternalTextureDescriptor
: GPUObjectDescriptorBase {
required HTMLVideoElement source;
required (HTMLVideoElement or VideoFrame) source;
PredefinedColorSpace colorSpace = "srgb";
};
</script>
Expand Down Expand Up @@ -5096,20 +5110,26 @@ dictionary GPUExternalTextureDescriptor

1. Let |result| be a new {{GPUExternalTexture}} object wrapping |data|.

1. [$queue an automatic expiry task$] with device |this| and the following steps:
1. If |source| is an {{HTMLVideoElement}},
[$queue an automatic expiry task$] with device |this| and the following steps:

<div data-timeline=content>
1. Set |result|.{{GPUExternalTexture/[[expired]]}} to `true`,
releasing ownership of the underlying resource.
</div>

Note:
An external video texture should be imported in the same task that samples the texture
An {{HTMLVideoElement}} should be imported in the same task that samples the texture
(which should generally be scheduled using `requestVideoFrameCallback` or
{{AnimationFrameProvider/requestAnimationFrame()}} depending on the application).
Otherwise, a texture could get destroyed by these steps before the
application is finished using it.
1. If |source| is a {{VideoFrame}}, then when |source| is
[=Close VideoFrame|closed=], run the following steps:

<div data-timeline=content>
1. Set |result|.{{GPUExternalTexture/[[expired]]}} to `true`.
</div>
1. Set |result|.{{GPUObjectBase/label}} to |descriptor|.{{GPUObjectDescriptorBase/label}}.
1. Return |result|.
</div>
Expand Down