Not able to get stream after accepting the call

[savaliyap]

Expected Behavior:

Get remote stream as a user who initiated the call after remote side accepts the call

Observed Behavior:

call is being answered but track event is not firing

Steps to reproduce the issue:

const socket = io('http://localhost:9000');

const VideoCallAppUsingSocket = () => {
  const [localStream, setLocalStream] = useState<MediaStream>();
  const [remoteStream, setRemoteStream] = useState<MediaStream>();
  const otherSocketId = useRef(null);
  let remoteRTCMessage = useRef(null);


  // const [peerConnection, setPeerConnection] = useState<RTCPeerConnection>();
  const [incomingOffer, setIncomingOffer] = useState<any>();
  const peerConnection = useRef(
    new RTCPeerConnection({
      iceServers: [
        {
          urls: 'stun:stun.l.google.com:19302',
        },
        {
          urls: 'stun:stun1.l.google.com:19302',
        },
        {
          urls: 'stun:stun2.l.google.com:19302',
        },
      ],
    }),
  );

  const initializeMediaStream = async () => {
    const isFront = false;
    const sourceInfos: any = await mediaDevices.enumerateDevices();
    let videoSourceId;
    for (let i = 0; i < sourceInfos.length; i++) {
      const sourceInfo = sourceInfos[i];
      if (
        sourceInfo.kind == 'videoinput' &&
        sourceInfo.facing == (isFront ? 'user' : 'environment')
      ) {
        videoSourceId = sourceInfo.deviceId;
      }
    }

    const constraints = {
      audio: true,
      video: {
        mandatory: {
          minFrameRate: 30,
        },
        facingMode: isFront ? 'user' : 'environment'
      },
    }
    const stream = await mediaDevices.getUserMedia(constraints);
    socket.emit('user-connect', currentUserId);
    stream.getTracks().forEach((track) => {
      peerConnection.current.addTrack(track);
    });
    setLocalStream(stream);
    peerConnection.current.addEventListener('track', (e) => {
      console.log('Received remote track:', e.track);
      if (!remoteStream) {
        setRemoteStream(new MediaStream([e.track]));
      } else {
        remoteStream.addTrack(e.track);
      }
    });
  };

  const createPeerConnection = async () => {
    const pc = new RTCPeerConnection();
  
    peerConnection.current.addEventListener('icecandidate', (e) => {
      if (e.candidate) {
        socket.emit('ice-candidate', { candidate: e.candidate });
      }
    });
  
    peerConnection.current.addEventListener('track', (e) => {
      console.log('Received remote track:', e.track);
      if (!remoteStream) {
        setRemoteStream(new MediaStream([e.track]));
      } else {
        remoteStream.addTrack(e.track);
      }
    });
    return pc;
  };

  useEffect(() => {
    initializeMediaStream();

    socket.on('newCall', async (offer) => {
      setIncomingOffer(offer);
      remoteRTCMessage.current = offer.offer;
      otherSocketId.current = offer.senderId;
    });

    socket.on('callAnswered', async (answer) => {
      console.log(answer, "ANSWERFROMREMOTE"); // this is working but I am unable to get remote stream, 'track event is not firing'**
      peerConnection.current.setRemoteDescription(
        new RTCSessionDescription(remoteRTCMessage.current),
      );
      peerConnection.current.addEventListener('track', e => {
        console.log(e, 'ANSWERFROMREMOTE TRACK');
        
      })
    });

    socket.on('ice-candidate', async (candidate) => {
      console.log(candidate);
      
      await peerConnection.current.addIceCandidate(new RTCIceCandidate({
        candidate: candidate.candidate,
        sdpMid: candidate.id,
        sdpMLineIndex: candidate.label,
      }),);
    });

    return () => {
      if (localStream) {
        localStream.getTracks().forEach(track => track.stop());
      }
    };
  }, [peerConnection]);

  const handleCallButtonPress = async () => {
    console.log(peerConnection.current, localStream, 'PEERCONNECTION');
    const offer = await peerConnection.current.createOffer(null);
    await peerConnection.current.setLocalDescription(offer);

    socket.emit('offer', { recipientId: remoteUserId, offer });
  };

  const setupMedia = async () => {
    
    
  }

  const handleAcceptCall = async () => {
    if (remoteRTCMessage.current) {
      
      try {
        await peerConnection.current.setRemoteDescription(incomingOffer.offer);
        console.log('\n\nHERE\n\n');
        
        peerConnection.current.addEventListener('track', (e) => {
          setRemoteStream(e.streams[0]);
        });
        const answer = await peerConnection.current.createAnswer();
        
        await peerConnection.current.setLocalDescription(answer);
        
        socket.emit('answer', { senderId: remoteUserId, answer });
      } catch (error) {
        console.error('Error handling accept call:', error);
      }
    } else {
      console.error('Incoming offer is undefined.');
    }
  };

  const handleRejectCall = () => {
    socket.emit('reject', { recipientId: currentUserId });
    setIncomingOffer(null);
  };

  return (
    <View style={styles.container}>
      <View style={styles.videoContainer}>
        {localStream && <RTCView streamURL={localStream.toURL()} style={styles.localVideo} />}
        {remoteStream && <RTCView streamURL={remoteStream.toURL()} style={styles.remoteVideo} />}
      </View>
      {!incomingOffer && <Button title="Call" onPress={handleCallButtonPress} />}
      {incomingOffer  && (
        <View style={styles.callButtonsContainer}>
          <Button title="Accept" onPress={handleAcceptCall} />
          <Button title="Reject" onPress={handleRejectCall} />
        </View>
      )}
    </View>
  );
};

Socket server file

import * as http from 'http';
import { Server } from 'socket.io';

const server = http.createServer();
const io = new Server(server);

interface Users {
  [userId: string]: string;
}

const users: Users = {}; // Store connected users and their socket IDs

io.on('connection', (socket) => {
  console.log('A user connected:', socket.id);

  // Store user's socket ID
  socket.on('user-connect', (userId: string) => {
    users[userId] = socket.id;
    console.log(`User ${userId} connected with socket ID ${socket.id}`, users);
  });

  // Forward signaling messages
  socket.on('offer', (data: { recipientId: string, offer: any }) => {
    console.log(data, 'OFFER');
    
    const { recipientId, offer } = data;
    const recipientSocketId = users[recipientId];
    if (recipientSocketId) {
      io.to(recipientSocketId).emit('newCall', { senderId: socket.id, offer });
    } else {
      console.error(`Recipient ${recipientId} not found.`);
    }
  });

  socket.on('answer', (data: { senderId: string, answer: any }) => {
    const { senderId, answer } = data;
    const senderSocketId = users[senderId];
    if (senderSocketId) {
      io.to(senderSocketId).emit('callAnswered', { recipientId: socket.id, answer });
    } else {
      console.error(`Sender ${senderId} not found.`);
    }
  });

  socket.on('ice-candidate', (data: { recipientId: string, candidate: any }) => {
    const { recipientId, candidate } = data;
    const recipientSocketId = users[recipientId];
    if (recipientSocketId) {
      io.to(recipientSocketId).emit('ice-candidate', { senderId: socket.id, candidate });
    } else {
      console.error(`Recipient ${recipientId} not found.`);
    }
  });

  // Handle disconnection
  socket.on('disconnect', () => {
    console.log('User disconnected:', socket.id);
    // Remove disconnected user from users object
    const userId = Object.keys(users).find(key => users[key] === socket.id);
    if (userId) {
      delete users[userId];
      console.log(`User ${userId} disconnected.`);
    }
  });
});

const port = 9000;
server.listen(port, () => {
  console.log(`Signaling server is running on port ${port}`);
});

Platform Information

• React Native Version:
• 0.73.6
• WebRTC Module Version:
• 118.0.4
• Platform OS + Version:
• Expo 50.0.14

[saghul]

You need to add the track event before you call setRemoteDescription.

[savaliyap]

You need to add the track event before you call setRemoteDescription.

In call answered event listners I had partial success(remote stream is still blank) with this, I don't know if it is correct approach

  const receivers: RTCRtpReceiver[] = peerConnection.current.getReceivers();
  console.log('\n\n\n\n\n', receivers, 'RECEIVERS');

  const stream = new MediaStream();
  receivers.forEach(receiver => {
    stream.addTrack(receiver.track);
  })
  setRemoteStream(stream)

[saghul]

That is not correct. You need have 2 video track per stream since an RTcView is only able to render a single video track.

Handle the track event, and create a stream there, if you don't have one.

You will need to attach to the track event early, so you don't miss it.

[savaliyap]

That is not correct. You need have 2 video track per stream since an RTcView is only able to render a single video track.

Handle the track event, and create a stream there, if you don't have one.

You will need to attach to the track event early, so you don't miss it.

That actually worked but the main issue still persists. remoteStream view is just plain blank

[saghul]

You'll need to go through your code. Ink Lu pointed out the obvious mistakes, but there may be more.

[savaliyap]

You'll need to go through your code. Ink Lu pointed out the obvious mistakes, but there may be more.

I don't see the comment on this issue, can you link it here please?

[savaliyap]

Nevermind, it was a small issue with icecandidate. You can close this one now. Thanks for all the help

@saghul