Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

UE5 Server MAC build crash with DolbyIOModule error #415

Closed
AntonIovbakDev opened this issue Sep 20, 2023 · 22 comments
Closed

UE5 Server MAC build crash with DolbyIOModule error #415

AntonIovbakDev opened this issue Sep 20, 2023 · 22 comments
Assignees
@FabienLavocat @GaryMakinMe
Labels
bug Something isn't working

Comments

@AntonIovbakDev
Copy link

Describe the bug

After packaging UE5 Server on MAC crash with DolbyIOModule error

Expected Behavior

[Describe the expected behavior of the feature.]

Specifications

  • OS: macOS 13.5.1 (M2)
  • Version:
    • dolby 1.2.1 last release version

Screen Shot 2023-09-20 at 10 11 05 AM
@AntonIovbakDev AntonIovbakDev added the bug Something isn't working label Sep 20, 2023
@djova-dolby
Copy link
Contributor

djova-dolby commented Oct 4, 2023 •
edited

It looks like it is failing here to load the first library it tried to find. I found one recent release note that indicates a change to library path creation.
Not sure if this is the issue though so I have some question:

  • Did this work in previous versions and stopped working in the latest ones?
  • For instance 1.1.X did not have this issue or has packaging a game never been possible on macOS?
  • Did it work with previous versions with Windows?

The reason I ask is because I see there is another issue reported for 1.2.1 for Windows that seems similar about regarding game packaging, which I assume was working before, but could be wrong.
If it did work before it is possible that something like this change will fix it and here are the artifacts for this attempt.

@djova-dolby djova-dolby mentioned this issue Oct 4, 2023
Closed
@AntonIovbakDev
Copy link
Author

AntonIovbakDev commented Oct 5, 2023 via email

@djova-dolby
Copy link
Contributor

djova-dolby commented Oct 5, 2023 •
edited

We've made fix and it is work: [cid:82347bd2-2d39-44a9-a780-553b3f12a82c]

what does this mean? What fix have you made and what was it for (packaging or the crashes with the texture); is it in the Unreal SDK? Could you share/submit it to some branch?

When was the last time you packaged a game and it didn't crash on mac? Or does it always crash?

@AntonIovbakDev
Copy link
Author

AntonIovbakDev commented Oct 5, 2023 via email

@djova-dolby
Copy link
Contributor

djova-dolby commented Oct 5, 2023 •
edited

Ok so it just for rendering, not packaging. Since this project is completely open source can you upstream the fix so we can merge it?

And then to reiterate the previous question about plugin packaging; it is using the DolbyIOModule.cpp from CoSocietyUnreal/Plugins... folder but looking for the library inside CoSociety/Plugins.... is this how plugins are normally loaded or should it be in the same folder? Like could this be the bug that the DolbyIOModule.cpp code is not setting proper path or something?

@djova-dolby
Copy link
Contributor

djova-dolby commented Oct 5, 2023 •
edited

Can you try with the artifacts from this run? https://github.com/DolbyIO/comms-sdk-unreal/actions/runs/6429286687
Also what is UE5 Server on MAC is this like a different way of packaging of Unreal or something like this? Google has not been helpful here. @Krysed was able to package game on macOS from what I understand with UE5.

@djova-dolby
Copy link
Contributor

djova-dolby commented Oct 9, 2023 •
edited

@Krysed and I have been able to use the 1.2.2-beta.1 plugin from the latest artifacts and package the internal repo application on x86/arm macOS platforms.
We were able to run the resulting application on both platforms. So not really sure where the issue is on yours? Does this UE5 server concept change things, or is this irrelevant?

@AntonIovbakDev
Copy link
Author

AntonIovbakDev commented Oct 9, 2023 via email

@djova-dolby
Copy link
Contributor

djova-dolby commented Oct 9, 2023 •
edited

Sorry I added the wrong link. It should be [internal repo]

@djova-dolby
Copy link
Contributor

Note that we were also able to package/run using the 1.2.1 version of the plugin on macOS, so I am not sure that 1.2.2 changes will affect you. Likely you will be running into the same issue you had before.
So to re-iterate something that we may not be understanding; what does it exactly mean UE5 Server MAC build? Is this a different way to package the game that we need to try? Or does it mean something else? Like from what I understand you are still using the plugin as intended for client application, so clarifying this part could be the part we are missing.

@AntonIovbakDev
Copy link
Author

AntonIovbakDev commented Oct 9, 2023 via email

@AntonIovbakDev
Copy link
Author

AntonIovbakDev commented Oct 9, 2023 via email

@djova-dolby
Copy link
Contributor

djova-dolby commented Oct 9, 2023 •
edited

Yes my bad, these are all internal. I thought this second one was a public repository. But I see now it is all internal repositories that were never opened.

@AntonIovbakDev
Copy link
Author

AntonIovbakDev commented Oct 9, 2023 via email

@djova-dolby
Copy link
Contributor

djova-dolby commented Oct 9, 2023 •
edited

No I cannot share the sample project. It has not been open sourced.

@AntonIovbakDev
Copy link
Author

AntonIovbakDev commented Oct 13, 2023 via email

@djova-dolby
Copy link
Contributor

Yes you are correct it seems that the default permissions for the repo is to create PRs/Issues but can't push directly into repo. So in order to make the PR will in fact need to do the fork/PR workflow.

@AntonIovbakDev
Copy link
Author

AntonIovbakDev commented Oct 16, 2023 via email

@djova-dolby djova-dolby closed this as not planned Won't fix, can't repro, duplicate, stale Nov 3, 2023
@AntonIovbakDev
Copy link
Author

AntonIovbakDev commented Nov 9, 2023 via email

@AntonIovbakDev
Copy link
Author

AntonIovbakDev commented Nov 10, 2023 via email

@AntonIovbakDev
Copy link
Author

AntonIovbakDev commented Nov 13, 2023 via email

@djova-dolby
Copy link
Contributor

djova-dolby commented Nov 13, 2023 •
edited

Hi, so we do not have a frame-rate control mechanism in the underlying c++sdk and unfortunately we will not be adding it as a new feature to the sdk in the near future. In general webrtc decides based on BWE the available bitrate what it can encode and at what frequency. The video processor plugin does not control the frame rate, it is actually using a Dolby Video Processing library to segment and blur the background of the capture image. There is no mechanism exposed from the c++sdk to just set desired frames, and thus nothing in the Unreal SDK.
However you can use similar concept that video processor is using and you can implement a frame rate control mechanism rather simply yourself. The C++SDK provides the following start_video method which is accepting a device and video_frame_handler. This video frame handler can be implemented by any application, so it can receive frames from the sdk, process and inject them back, or just render them. Using this frame handler is essentially how Unreal SDK screen share local preview is done. So to implement frame rate control you can create a custom video frame handler (it must be done in the layer of Unreal SDK which interacts with c++sdk, something along the lines of the video processing handler).
This video frame handler you create and provide to the start_video call will receive each frame from the SDK captured by the camera, this is done at 30 fps. Then you can just drop whatever frames you do not want to achieve your desired frame rate, so if you want 5fps, you will only provide back to the SDK every 6th frame. A basic c++ class example to achieve this would look something like:

#include <dolbyio/comms/media_engine/media_engine.h>
#include <memory>
#include <mutex>

class dummy_frame_dropper final
    : public dolbyio::comms::video_source,
      public dolbyio::comms::video_sink,
      public dolbyio::comms::video_frame_handler,
      public std::enable_shared_from_this<dummy_frame_dropper> {
 public:
  static constexpr int frame_rate_ = 30;
  explicit dummy_frame_dropper(int fps) : desired_frame_rate_(fps) {
    assert(desired_frame_rate_);
    frame_to_pass_back_ = frame_rate_/desired_frame_rate_;
  }
  ~dummy_frame_dropper() = default;
  void handle_frame(const dolbyio::comms::video_frame& frame) override {
    std::lock_guard<std::mutex> lock(sink_lock_);
    if (sdk_video_sink_) {
      // Only provide frame back to SDK if it the desired fps frame
      if (++frame_counter_ >= frame_to_pass_back_) {
        sdk_video_sink_->handle_frame(frame);
        frame_counter_ = 0;
      }
    }    
  }
  void set_sink(const std::shared_ptr<video_sink>& sink,
                const config&) override {
    std::lock_guard<std::mutex> lock(sink_lock_);
    sdk_video_sink_ = sink;
  }
  std::shared_ptr<video_sink> sink() override { return shared_from_this(); }
  std::shared_ptr<video_source> source() override { return shared_from_this(); }
 private:
  std::mutex sink_lock_;
  std::shared_ptr<dolbyio::comms::video_sink> sdk_video_sink_{};
  int desired_frame_rate_;
  int frame_to_pass_back_;
  int frame_counter_{0};
};

Or if you were to just make a copy of/edit the DolbyIOVideoProcessingFrameHandler and modify it a bit. You will need to pass it the desired frame rate that you would like. Then the handle frame function would look something like:

void handle_frame(const dolbyio::comms::video_frame& VideoFrame) override {
  if (SdkSink) {
    // Only pass frams to the SDK sink for desired frame rate
    if (++frame_counter_ >= frame_to_pass_back_) {
      SdkSink->handle_frame(VideoFrame);
      frame_counter_ = 0;
    }
    // Preview the frames at 30fps
    PreviewSink->handle_frame(VideoFrame);
  }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@FabienLavocat FabienLavocat

@GaryMakinMe GaryMakinMe

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@FabienLavocat @GaryMakinMe @djova-dolby @AntonIovbakDev