iOS pip implementation

[said15904]

I want to implement a pip in ios but i can't get a solution
i use capacitor-jitsi-meet v2.2.1
and SDK 4.1.0
Any help

[calvinckho]

Hi @said15904,

Picture-in-Picture Mode for iOS has been developed as a premium feature. Because PIP is not yet supported by the Jitsi iOS SDK, the non-native PIP is implemented, which involves adding gesture detection and translating the gesture events into js touch and click events to be sent to the iOS WKWebView. Because the Jitsi PIP UIView lies on top of the WKWebView, where the hybrid app is rendered, my original thought was to try to capture the gestures and send it to the underlying WKWebView via the Response Chain. Unfortunate Apple’s design doesn't allow it. As a result, the best solution is to capture all the gestures, and translate them into javascript events or swift UIScrollView commands. In some cases, the swift/javascript translation is very complicated. For example, when trying to move the cursor in ion-textarea and ion-input, I have to create a separate module to translate touch coordinates into text selection index.

The big picture development goal is to create something that is a transitional solution, knowing that Jitsi is working on jitsi/jitsi-meet#9437, although we do not know the timeline yet, and have no control over it. So I still believe my approach is a good one, as PIP is a very powerful feature which allows users to access the app while on a video or audio call.

Here is how the PiP works in a real iOS device: https://ds.ivr.solutions/media/pip_demo.mp4. Please note that unlike the native iOS PIP implementation, where the PIP window is active when the app is moved to the background, the non-native PIP window will be hidden when the app is moved to the background.

iOS gestures already implemented and should be working: tap, double tap, pan, pinch, drag, vertical and horizontal scroll. Please report anything that is not working/compatible with your app/framework.

Known limitations that I tried to solve but will need more time to work on. I am listing the ones that I am aware of but there may be more:

• moving of cursor inside div with contenteditable
• highlighting of text in a div
• copying, pasting, long press to open context menu in ion-textarea and ion-input
• ion-content scroll can use a more natural velocity curve

If you are interested in accessing the premium repo which comes with a 1-year customer support included, DM me here to discuss more.