Does capturing a stream from a canvas make sense?

[alvestrand]

Looking at the canvas capture case:

• It can only make one video stream
• It can't make any sound
  So why are we capturing a MediaStream subclass from the canvas?

I think it would be simpler if the spec said that you generate a track from a canvas, not a stream.

[martinthomson]

Having an API that is --isomorphic--ducktyped to the one on HTMLMediaElement was the idea.

[alvestrand]

... but it isn't, since you have the RequestFrame method on it. RequestFrame on a MediaStream that has had another track added to it doesn't make very much sense.

I'd rather have the RequestFrame method on a track.

[martinthomson]

.requestFrame() is an extension that you don't have to use. The API works perfectly well without it. I appreciate that the point of attachment is unfortunate. That's a separate ergonomic issue with its own merits in both directions.

[alvestrand]

OK, .requestFrame is a different question from whether we capture streams or tracks. But what's the merit for leaving it on the MediaStream?
In particular, what's the implication for:

stream = canvas.CreateMediaStream()
stream.addTrack(track)
stream.requestFrame()

?

[martinthomson]

I'm not sure what you are suggesting exactly.

I have no objection to one of the changes you are implicitly suggesting, namely:

var stream = canvas.captureStream();
stream.videoTracks[0].requestFrame();

But I'm less happy with the idea that this might not be consistent with HTMLMediaElement. The packaging is arguably redundant, but it's not redundant in a way that is dangerous. As a high-level construct, streams are more accessible, even if we've (over)rotated away from them WebRTC land. (I still think that we should restore addStream() for instance.)

[martinthomson]

Note to @Pehrsons: since you are doing all the moving of features from MediaStream to MediaStreamTrack, would moving requestFrame() be a problem implementation-wise?

[alvestrand]

Seems to me we have concluded that requestFrame() and the canvas element reference belong on a subclass of MediaStreamTrack, not on a subclass of MediaStream - but that we will continue to let canvas.captureStream() return a MediaStream.

Seems good to me.

[Pehrsons]

Some boilerplate shuffling is involved but I don't see any issues in moving requestFrame() to a track class.

[foolip]

This is still CanvasCaptureMediaStream in Gecko, but there's an Intent to Ship: Media Capture from Canvas on blink-dev which would use the new CanvasCaptureMediaStreamTrack. Will Gecko get this change soonish?