[BUG] - Sending command after dataChannelOpen does not work

[ibreathebsb]

UE Version:
5.2.1

Frontend Version:
"@epicgames-ps/lib-pixelstreamingfrontend-ue5.2": "^0.5.3",

Problem component

maybe UE plugin

Description
sending command after dataChannelOpen does not work

const stream = new PixelStreaming(config);
stream.addEventListener("dataChannelOpen", () => {
    stream.emitCommand({ Resolution: { Width: 1000, Height: 500 } });
});

Expected behavior

1. resize correctly

[Belchy06]

Hey @ibreathebsb,

I've tried this just today and it seems to be working as expected.

Just to check, is your Unreal application running in windowed mode? If it's not running windowed, the resolution command will have no effect.

Cheers

[ibreathebsb]

Hey @ibreathebsb,

I've tried this just today and it seems to be working as expected.

Just to check, is your Unreal application running in windowed mode? If it's not running windowed, the resolution command will have no effect.

Cheers

I did some research and logging and found the message was not receviced by ue:

UE LOG:

[2023-12-15T03:30:01.003Z]LogPixelStreaming: OnSignalingChange (Stable)
[2023-12-15T03:30:01.007Z]LogPixelStreaming: OnIceConnectionChange (IceConnectionChecking)
[2023-12-15T03:30:01.008Z]LogPixelStreaming: OnIceGatheringChange (IceGatheringComplete)
[2023-12-15T03:30:01.008Z]LogPixelStreaming: OnIceConnectionChange (IceConnectionConnected)
[2023-12-15T03:30:01.008Z]LogPixelStreaming: OnIceConnectionChange (IceConnectionCompleted)
[2023-12-15T03:30:01.009Z]LogPixelStreaming: OnIceConnectionChange (IceConnectionConnected)
[2023-12-15T03:30:01.022Z]LogPixelStreaming: Player 11 has requested quality control through the data channel.
[2023-12-15T03:30:01.023Z]LogPixelStreaming: Player 11 has requested quality control through the data channel.
[2023-12-15T03:30:01.033Z]LogPixelStreamingInputHandler: Verbose: Command: {"Encoder.MinQP":0}
[2023-12-15T03:30:01.033Z]LogPixelStreamingInputHandler: Verbose: Command: {"Encoder.MaxQP":51}
[2023-12-15T03:30:01.033Z]LogPixelStreamingInputHandler: Verbose: Command: {"WebRTC.MinBitrate":100000}
[2023-12-15T03:30:01.033Z]LogPixelStreamingInputHandler: Verbose: Command: {"WebRTC.MaxBitrate":100000000}
[2023-12-15T03:30:01.033Z]LogPixelStreamingInputHandler: Verbose: Command: {"WebRTC.Fps":60}
[2023-12-15T03:30:01.033Z]LogPixelStreamingInputHandler: Verbose: Command: {"WebRTC.MaxFps":60}
[2023-12-15T03:30:01.049Z]LogPixelStreamingInputHandler: Verbose: Command: {"Encoder.MinQP":0}
[2023-12-15T03:30:01.049Z]LogPixelStreamingInputHandler: Verbose: Command: {"Encoder.MaxQP":51}
[2023-12-15T03:30:01.050Z]LogPixelStreamingInputHandler: Verbose: Command: {"WebRTC.MinBitrate":100000}
[2023-12-15T03:30:01.051Z]LogPixelStreamingInputHandler: Verbose: Command: {"WebRTC.MaxBitrate":100000000}
[2023-12-15T03:30:01.051Z]LogPixelStreamingInputHandler: Verbose: Command: {"WebRTC.Fps":60}
[2023-12-15T03:30:01.052Z]LogPixelStreamingInputHandler: Verbose: Command: {"WebRTC.MaxFps":60}
[2023-12-15T03:30:01.052Z]LogPixelStreamingInputHandler: Verbose: Command: {"Encoder.MinQP":0}
[2023-12-15T03:30:01.052Z]LogPixelStreamingInputHandler: Verbose: Command: {"Encoder.MaxQP":51}
[2023-12-15T03:30:01.052Z]LogPixelStreamingInputHandler: Verbose: Command: {"WebRTC.MinBitrate":100000}
[2023-12-15T03:30:01.053Z]LogPixelStreamingInputHandler: Verbose: Command: {"WebRTC.MaxBitrate":100000000}
[2023-12-15T03:30:01.053Z]LogPixelStreamingInputHandler: Verbose: Command: {"WebRTC.Fps":60}
[2023-12-15T03:30:01.054Z]LogPixelStreamingInputHandler: Verbose: Command: {"WebRTC.MaxFps":60}
[2023-12-15T03:30:16.212Z]LogPixelStreaming: OnIceConnectionChange (IceConnectionCompleted)

FrontEnd code :

   _dataChannelOpen = () => {
    console.log("dataChannelOpen");
    this.dataChannelOpen = true;
    const descriptor = {
      Resolution: {
        Width: 500,
        Height: 500,
      },
    };

    const ret = this.stream.emitCommand(descriptor);
   // ret is false!!!
    console.log(ret, new Date().toISOString());
  };

so I checked the source code

    public emitCommand(descriptor: object) {
        if (!this._webRtcController.videoPlayer.isVideoReady()) {
           // this is is the reason why we get  a false
            return false;
        }
        if (!this.allowConsoleCommands && 'ConsoleCommand' in descriptor) {
            return false;
        }
        this._webRtcController.emitCommand(descriptor);
        return true;
    }

so I made some change to my frontend code

  _dataChannelOpen = () => {
    console.log("dataChannelOpen");
    this.dataChannelOpen = true;
    const descriptor = {
      Resolution: {
        Width: 500,
        Height: 500,
      },
    };

    const ret = this.stream.emitCommand(descriptor);
    console.log(ret, new Date().toISOString());
    if (!this.stream._webRtcController.videoPlayer.isVideoReady()) {
      // we got a hit!!!!
      debugger;
      return false;
    }
  };

so the reason is quite clear, the video player is not ready yet,but does it matter? even if the video is not ready,we should be able to send some control command,is blocking command before video ready expected behavior?

[ibreathebsb]

I could make a PR if this is not an expected behavior 😁 @Belchy06

[github-actions]

Issues go stale after 30 days of inactivity. Please comment or re-open the issue if you are still interested in getting this issue fixed.