Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change addStream to addTrack; add RTCRtpSender/RTCRtpReceiver #3

Closed
wants to merge 5 commits into from

Conversation

Projects
None yet
6 participants
@juberti
Copy link
Contributor

commented Oct 15, 2014

Initial version of doohickeys. @alvestrand @stefhak @pthatcherg PTAL and see if you think this is going in the right direction. If so I will deal with the few formatting and link nits remaining.

Details:

  • addStream/remoteStream/getSenders/getReceivers and associated APIs have now been moved to a new document section (#5). This allows section #4 to focus on the basics of the PC API, and #5 to extend it with media functionality and #6 to extend it with data channel functionality.
  • addStream has become addTrack(MediaStreamTrack track, MediaStream... streams), as discussed in DC. addTrack returns a RTCRtpSender.
  • removeStream has become removeTrack; removeTrack takes a RTCRtpSender.
  • onaddstream has become onaddtrack; onremovestream has been removed (now, .onended is fired).
  • getLocalStreams/getRemoteStreams -> getSenders/getReceivers.

@juberti juberti changed the title Add RTCRtpSender, RTCRtpSender, and friends Add RTCRtpSender, RTCRtpReceiver, and friends Oct 15, 2014

@alvestrand

This comment has been minimized.

Copy link
Contributor

commented Oct 15, 2014

Good to have this!

Needs a better cover note - the core part in this is the change from
AddStream to AddTrack.

I miss a paragraph on how the track's membership in streams is
signalled in the onaddtrack handler - there doesn't seem to be a
procedure specification for the firing of onaddtrack.

Something like this:

When a PeerConnection object detects that a track has been added by
the remote peer, it performs
the following steps:

- Let TRACK be a new MediaStreamTrack object
- Let STREAMS be the set of stream objects that the TRACK is a member of
- For each stream in STREAMS, construct an AddtrackEvent where 

(member variable listing)
- Fire the events at the PeerConnection

Section 5 of draft-ietf-mmusic-msid-07 needs to be consistent with this
document.

On 10/15/2014 08:23 AM, Justin Uberti wrote:

Initial version of doohickeys. @alvestrand
https://github.com/alvestrand @stefhak https://github.com/stefhak
@pthatcherg https://github.com/pthatcherg PTAL and see if you think
this is going in the right direction. If so I will deal with the few
formatting and link nits remaining.


    You can merge this Pull Request by running

git pull https://github.com/juberti/webrtc-pc master

Or view, comment on, or merge it at:

#3

    Commit Summary


Reply to this email directly or view it on GitHub
#3.

webrtc.html Outdated
@@ -518,7 +509,7 @@
"component".</p>

<p>When a user agent has reached the point where a
<code><a>MediaStream</a></code> can be created to represent incoming
<code><a>MediaStreamTrack</a></code> can be created to represent incoming
components, the user agent MUST run the following steps:</p>

This comment has been minimized.

Copy link
@pthatcherg

pthatcherg Oct 15, 2014

Contributor

Does "components" (plural) still make sense?

This comment has been minimized.

Copy link
@juberti

juberti Oct 17, 2014

Author Contributor

Fixed.

</dd>

<dt>RTCRtpSender addTrack (MediaStreamTrack track, MediaStream... streams)</dt>

This comment has been minimized.

Copy link
@pthatcherg

pthatcherg Oct 15, 2014

Contributor

Should there be some text about the effect putting streams here will have (that the onaddtrack even on the remote side will fire with these streams)?

This comment has been minimized.

Copy link
@juberti

juberti Oct 17, 2014

Author Contributor

I think that is covered in the section that talks about firing onaddtrack. @HTA, wdyt?

This comment has been minimized.

Copy link
@stefhak

stefhak Oct 21, 2014

Contributor

In the discussion about changeTrack we had a while back I think we concluded that one parameter here should be trackId. Perhaps we should not add that now, but in conjunction with introducing changeTrack later.

This comment has been minimized.

Copy link
@juberti

juberti Oct 21, 2014

Author Contributor

Agreed - this PR has enough stuff in it already - would like to stick to the basics for now.

webrtc.html Outdated

<dt>readonly attribute RTCRtpReceiver receiver</dt>
<dt>readonly attribute MediaStreamTrack track</dt>
<dt>readonly attribute MediaStream stream</dt>

This comment has been minimized.

Copy link
@pthatcherg

pthatcherg Oct 15, 2014

Contributor

If addTrack is called with more than one stream (such as pc.addTrack(track, stream1, stream2), which stream is in the event that gets fired? Or does more than one event get fired?

This comment has been minimized.

Copy link
@juberti

juberti Oct 17, 2014

Author Contributor

Changed to a sequence of MediaStreams. Blarg.

objects called RTCRtpSenders. Similarly, the reception and decoding of
MediaStreamTracks is managed through objects called RTCRtpReceivers.</p>

<p>A <code><a>RTCPeerConnection</a></code> object contains a

This comment has been minimized.

Copy link
@pthatcherg

pthatcherg Oct 15, 2014

Contributor

"A" or "An"?

This comment has been minimized.

Copy link
@juberti

juberti Oct 17, 2014

Author Contributor

Fixed. (although other instances of 'A' exist in datachannels)

@pthatcherg

This comment has been minimized.

Copy link
Contributor

commented Oct 15, 2014

I left a few comments, some grammar. One was about what effect the "...
streams" variable has.

On Tue, Oct 14, 2014 at 11:23 PM, Justin Uberti notifications@github.com
wrote:

Initial version of doohickeys. @alvestrand https://github.com/alvestrand
@stefhak https://github.com/stefhak @pthatcherg
https://github.com/pthatcherg PTAL and see if you think this is going
in the right direction. If so I will deal with the few formatting and link

nits remaining.

You can merge this Pull Request by running

git pull https://github.com/juberti/webrtc-pc master

Or view, comment on, or merge it at:

#3
Commit Summary

  • Add RTCRtpSender, RTCRtpSender, and friends

File Changes

Patch Links:


Reply to this email directly or view it on GitHub
#3.

@pthatcherg

This comment has been minimized.

Copy link
Contributor

commented Oct 15, 2014

Overall, it looks very good, by the way.

On Wed, Oct 15, 2014 at 9:08 AM, Peter Thatcher pthatcher@google.com
wrote:

I left a few comments, some grammar. One was about what effect the "...
streams" variable has.

On Tue, Oct 14, 2014 at 11:23 PM, Justin Uberti notifications@github.com
wrote:

Initial version of doohickeys. @alvestrand
https://github.com/alvestrand @stefhak https://github.com/stefhak
@pthatcherg https://github.com/pthatcherg PTAL and see if you think
this is going in the right direction. If so I will deal with the few

formatting and link nits remaining.

You can merge this Pull Request by running

git pull https://github.com/juberti/webrtc-pc master

Or view, comment on, or merge it at:

#3
Commit Summary

  • Add RTCRtpSender, RTCRtpSender, and friends

File Changes

Patch Links:


Reply to this email directly or view it on GitHub
#3.

@juberti juberti changed the title Add RTCRtpSender, RTCRtpReceiver, and friends Change addStream to addTrack; add RTCRtpSender/RTCRtpReceiver Oct 15, 2014

@stefhak

This comment has been minimized.

Copy link
Contributor

commented Oct 16, 2014

Following up on Harald's part about steps when a PeerConnection object detects that a new track has been added by the remote peer: did we not conclude that the PC would not do this, but since the stream association is part of the addtrack event the application can add the track to the stream (and create the stream if it does not exist yet)?

And as a follow up: did we not say that the stream attribute on the addtrack event should be a list (for the case where a track i member of several MS's on the sending side at the time of doing pc.addTrack(track))?

(I may be remembering completely wrong here)

@juberti

This comment has been minimized.

Copy link
Contributor Author

commented Oct 16, 2014

  • Only tracks added via pc.addTrack are communicated to the remote side.
  • The remote side can do whatever it wants regarding moving tracks around to different streams.
  • I think we do create a default stream if none is specified in signaling.
  • I think the stream attribute has to be a list (I will make the change, although I wonder if the complexity here is worth it - remind me why having tracks in multiple streams is useful?)
@juberti

This comment has been minimized.

Copy link
Contributor Author

commented Oct 17, 2014

Fixed AddTrackEvent to handle multiple MediaStreams, and other CR comments. PTAL.

juberti added some commits Oct 18, 2014

AddTrackEvent -> RTCAddTrackEvent
Fix the parameter descriptions for RTCAddTrackEvent
@juberti

This comment has been minimized.

Copy link
Contributor Author

commented Oct 18, 2014

Merged in a link fix from @dontcallmedom and also cleaned up AddTrackEvent.

@juberti

This comment has been minimized.

Copy link
Contributor Author

commented Oct 21, 2014

Any update on this PR? Would like to land this ahead of next week's meeting.

Also, I have this PR queued up to change updateIce -> setConfiguration.
juberti#2

@stefhak

This comment has been minimized.

Copy link
Contributor

commented Oct 21, 2014

The editor's are going to have it reviewed by Thu this week, hopefully it can be merged shortly after that.

webrtc.html Outdated
referenced by the SDP
negotiation, creating a new ones if they do not exist. If no
MediaStreams are indicated in the SDP negotiation, a default MediaStream
is used.</p>

This comment has been minimized.

Copy link
@adam-be

adam-be Oct 23, 2014

Member

This section needs some work. The algorithm talks about several incoming components. That used to be the case when we were creating a MediaStream with several tracks; now we only have one. Step 3 (not in the diff), should be moved earlier, since it creates the MediaStreamTrack.

This comment has been minimized.

Copy link
@juberti

juberti Oct 27, 2014

Author Contributor

OK. I think I addressed these issues.

webrtc.html Outdated
<code title="event-addtrack"><a href=
"#event-addtrack">addtrack</a></code> with
<var>receiver</var>, <var>track</var>, and <var>streams</var>
at the <var title="">connection</var> object.</p>

This comment has been minimized.

Copy link
@adam-be

adam-be Oct 23, 2014

Member

We should really call the event fired on RTCPeerConnection when a remote track is received something else than "addtrack". That event name is used when a track is added to an existing MediaStream. Perhaps we could have a "remotetrack" event.

This comment has been minimized.

Copy link
@juberti

juberti Oct 27, 2014

Author Contributor

I tend to agree. I think the best name would actually be just ontrack, so that the pattern for a new data channel (ondatachannel) is the same as a new track (ontrack).

I've made this edit in my branch.

webrtc.html Outdated
<code><a>MediaStream</a></code> object, the user agent MUST associate
the component with that <code><a>MediaStream</a></code> object.</p>
<code><a>MediaStreamTrack</a></code> object, the user agent MUST associate
the component with that <code><a>MediaStreamTrack</a></code> object.</p>

This comment has been minimized.

Copy link
@adam-be

adam-be Oct 23, 2014

Member

My understanding is that the above note talked about adding media (tracks) to a MediaStream already dispatched by the RTCPeerConnection object. It doesn't apply anymore.

This comment has been minimized.

Copy link
@juberti

juberti Oct 27, 2014

Author Contributor

I read this as saying that if signaling had indicated that a MST should be created, and then media actually arrived, that media must be associated with the existing MST. But if that doesn't make sense, I am OK with removing this.

webrtc.html Outdated
being removed.</p>
</li>

<li>
<p>Let <var>stream</var> be the <code><a>MediaStream</a></code>
<p>Let <var>track</var> be the <code><a>MediaStreamTrack</a></code>
object that represents the media stream being removed, if any. If

This comment has been minimized.

Copy link
@adam-be

adam-be Oct 23, 2014

Member

s/media stream/track/

This comment has been minimized.

Copy link
@juberti

juberti Oct 27, 2014

Author Contributor

done

webrtc.html Outdated
<p>Remove <var>stream</var> from <var>connection</var>'s
<a href="#remote-streams-set">remote streams set</a>.</p>
<p>Remove the RTCRtpSender associated with <var>track</var> from
<var>connection</var>'s <a href="#receivers-set">set of receivers</a>.</p>

This comment has been minimized.

Copy link
@adam-be

adam-be Oct 23, 2014

Member

s/RTCRtpSender/RTCRtpReceiver/

This comment has been minimized.

Copy link
@juberti

juberti Oct 27, 2014

Author Contributor

done

webrtc.html Outdated
<p><a href="#fire-a-track-event">Fire a track event</a> named
<code title="event-MediaStream-ended"><a href=
"#event-mediastream-ended">ended</a></code>
at the <var>track</var>

This comment has been minimized.

Copy link
@adam-be

adam-be Oct 23, 2014

Member

I think we could just say that the track should end here. The media-capture spec defines what should happen in that case. And if the PeerConnection was closed already, and the algorithm aborts before this step, a task that ends the track should already be queued.

http://w3c.github.io/mediacapture-main/#track-ended

This comment has been minimized.

Copy link
@juberti

juberti Oct 27, 2014

Author Contributor

Done. Let me know if you prefer another formulation, e.g. "transition the track to the ended state."

<p>All other tracks that are not accessible to the application
MUST NOT be sent to the peer, with silence (audio), black
frames (video) or equivalently absent content being sent in
place of track content.</p>

This comment has been minimized.

Copy link
@adam-be

adam-be Oct 23, 2014

Member

Is this a note rather than a step in the algorithm?

This comment has been minimized.

Copy link
@juberti

juberti Oct 27, 2014

Author Contributor

I don't think this is new.

webrtc.html Outdated
</ol>
</dd>

<dt>attribute EventHandler onaddtrack</dt>

This comment has been minimized.

Copy link
@adam-be

adam-be Oct 23, 2014

Member

Could this one be called "onremotetrack" (see earlier comment)?

This comment has been minimized.

Copy link
@juberti

juberti Oct 27, 2014

Author Contributor

Changed on ontrack

webrtc.html Outdated
<code>setRemoteDescription</code>. onaddtrack happens as early as
possible after the <code>setRemoteDescription</code>. This callback
does not wait for a given media track to be accepted or rejected via
SDP negotiation.</dd>

This comment has been minimized.

Copy link
@adam-be

adam-be Oct 23, 2014

Member

We should not have any text in the event handler description saying when the event handler is called. The handler is only called as a result of an event being dispatched. The second part of the above text should describe when the event is fired.

Edit: I shouldn't blame you here since it was already in the description for the onaddstream event handler. :)

This comment has been minimized.

Copy link
@juberti

juberti Oct 27, 2014

Author Contributor

OK. Leaving as-is for now.

webrtc.html Outdated

<dl class="idl" title=
"interface RTCRtpReceiver : EventTarget">
<dt>readonly attribute MediaStreamTrack track</dt>

This comment has been minimized.

Copy link
@adam-be

adam-be Oct 23, 2014

Member

Should we wait with extending EventTarget until we actually use it?

This comment has been minimized.

Copy link
@juberti

juberti Oct 27, 2014

Author Contributor

Agreed. Done.

webrtc.html Outdated
</section>

<section>
<h3>RTCAddTrackEvent</h3>

This comment has been minimized.

Copy link
@adam-be

adam-be Oct 23, 2014

Member

This one could be called RTCRemoteTrackEvent

This comment has been minimized.

Copy link
@juberti

juberti Oct 27, 2014

Author Contributor

Changed to RTCTrackEvent

webrtc.html Outdated
</dd>

<dt>readonly attribute sequence&lt;MediaStream&gt; streams</dt>

This comment has been minimized.

Copy link
@adam-be

adam-be Oct 23, 2014

Member

This must be a getter function since a sequence is passed by value.

This comment has been minimized.

Copy link
@juberti

juberti Oct 27, 2014

Author Contributor

Copied what you did in your branch.

webrtc.html Outdated
pc.onaddtrack = function (evt) {
if (evt.track.kind === "video") {
remoteView.srcObject = evt.streams[0];
}

This comment has been minimized.

Copy link
@adam-be

adam-be Oct 23, 2014

Member

Perhaps remove the braces to be consistent with the rest of the example style.

This comment has been minimized.

Copy link
@juberti

juberti Oct 27, 2014

Author Contributor

Done.

@adam-be

This comment has been minimized.

Copy link
Member

commented Oct 25, 2014

The stats-example uses pc.getRemoteStreams(). It needs to be updated as well.

@adam-be

This comment has been minimized.

Copy link
Member

commented Oct 25, 2014

I got a task to isolate the new section so it could be reviewed separately. I did some fixes as well that can be found here: https://github.com/adam-be/webrtc-pc/commits/juberti-rtpsr

@juberti

This comment has been minimized.

Copy link
Contributor Author

commented Oct 27, 2014

Addressed all comments. PTAL.

@juberti

This comment has been minimized.

Copy link
Contributor Author

commented Oct 29, 2014

@alvestrand @adam-be what are the next steps on this?

@stefhak

This comment has been minimized.

Copy link
Contributor

commented Oct 29, 2014

I know Adam has prepared a version of the spec with these parts included; I think the plan was to send it to the WG before the meetings this week.

<code>receiver</code>.</p>
</dd>

<dt>sequence&lt;MediaStream&gt; getStreams()</dt>

This comment has been minimized.

Copy link
@dontcallmedom

dontcallmedom Oct 30, 2014

Member

I know the getter comes from a change suggested by @adam-be , but it sounds to me that using a simple attribute with an array rather than a sequence would work just as well (i.e. readonly attribute MediaStream[] streams).

@alvestrand

This comment has been minimized.

Copy link
Contributor

commented Jan 15, 2015

Obsolete - PR is now #175

@alvestrand alvestrand closed this Jan 15, 2015

aboba added a commit that referenced this pull request Oct 22, 2015

Merge pull request #3 from aboba/outbund-typo
Yet another typo (outbund)

aboba pushed a commit that referenced this pull request Mar 16, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.