Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Epic: libmuxr to replace current runtime #30

Closed
sr229 opened this issue Mar 16, 2022 · 2 comments
Closed

Epic: libmuxr to replace current runtime #30

sr229 opened this issue Mar 16, 2022 · 2 comments
Labels
area:pinvoke The issue is about PInvoke enhancement New feature or request help wanted Extra attention is needed priority:high

Comments

@sr229
Copy link
Member

sr229 commented Mar 16, 2022

Due to the myriad of issues we have encountered with our current wrapper, it's sufficient to say that we cannot trust the wrapper we derived from homuler due to testability, reproducibility of issues, and everything is just a black hole to us. Therefore, the next major task is re-architecting the wrapper to a new implementation.

Introduction to MUXR: The MediaPipe Universal eXtension Runtime

libmuxr, or simply MUXR, is our answer to a lot of issues we've encountered during the creation of MediaPipe.NET and porting MediaPipeUnity as Akihabara. MUXR aims to do the following:

  • MUXR should handle every pointer ownership by MediaPipe. Any implementing API on top of it should not be able to touch the MediaPipe pointers as much as possible.
  • Produce a Facade approach to the API, which is, make a C-compatible and convenient library as much as possible: which opens MUXR to be integrated by other languages if we wish to which only supports C ABIs.
  • Invoking MUXR should be as easy as instantiating a new MediaPipe context in the Browser.

Of course a lot of the APIs we use like custom resources should still be supported, but we will have to re-architect everything including the wrapper to accomodate this new architecture.

@sr229 sr229 added enhancement New feature or request help wanted Extra attention is needed priority:high area:pinvoke The issue is about PInvoke labels Mar 16, 2022
@sr229
Copy link
Member Author

sr229 commented Mar 16, 2022

Related Literatures:

#9
#18
#19
#29

@Speykious
Copy link
Contributor

Basically: the problem with the current MediaPipe runtime is that we directly expose internal MediaPipe components. This is fine in C++, but not in any other language, as even Google provides a facade for them, such as Java (for Android), Objective C (for iOS) and Python.

(Javascript is different as they quite literally reimplemented their calculators in Typescript.)

We want to take the same approach and rewrite the C++ runtime so that it exposes C functions that will be at the level of MediaPipe.NET's current Calculators instead of CalculatorGraphs, OutputStreamPollers and Packets.

@sr229 sr229 closed this as not planned Won't fix, can't repro, duplicate, stale Feb 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:pinvoke The issue is about PInvoke enhancement New feature or request help wanted Extra attention is needed priority:high
Projects
None yet
Development

No branches or pull requests

2 participants