Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perform async rendering in the main thread on Mac & iOS #3166

Merged
merged 2 commits into from
Jul 7, 2022

Conversation

sauwming
Copy link
Member

@sauwming sauwming commented Jun 29, 2022

Currently, worker thread will dispatch the rendering task synchronously to the main thread (while holding vid conf mutex lock), i.e.:

3   Foundation -[NSObject(NSThreadPerformAdditions) performSelector:onThread:withObject:waitUntilDone:modes:] + 784 (NSThread.m:1266)
4   Foundation -[NSObject(NSThreadPerformAdditions) performSelectorOnMainThread:withObject:waitUntilDone:] + 136 (NSThread.m:1284)
5   ios_sdk    iosgl_stream_put_frame + 2730932 (ios_opengl_dev.m:488)
6   ios_sdk    pjmedia_vid_dev_stream_put_frame + 2661692 (videodev.c:684)
7   ios_sdk    vid_pasv_port_put_frame + 2583192 (vid_port.c:1349)
8   ios_sdk    pjmedia_port_put_frame + 2398176 (port.c:117)
9   ios_sdk    on_clock_tick + 2611880 (vid_conf.c:803)

So if the main thread is trying to acquire vid conf mutex lock, such as by calling pjsua_call_hangup() or calling pjsua_vid_conf_*() APIs, a deadlock can occur.

The proposed solution in this PR is to perform the rendering asynchronously. A few notes:

  • The renderer now requires its own frame buffer. Since the rendering is now async, the frame buffer (which was owned by the vid port) may have already changed, or in the process of being changed, when the rendering actually happens. Without its own frame buffer, the async rendering will cause flickers and artefacts.
  • For efficiency, the async rendering task posted is limited to one. Since we only have one buffer, it's useless to post more than one tasks to draw the same buffer.
  • In some places, dispatch_sync() is replaced by performSelectorOnMainThread(). According to the doc, performSelectorOnMainThread() queues the message on the run loop of the main thread and then later dequeues the message and invokes the desired method (And multiple calls to this method from the same thread cause the corresponding selectors to be queued and performed in the same same order in which the calls were made.). While for dispatch_sync(), the doc states that as a performance optimization, the function can execute blocks on the current thread whenever possible. So here we use the method queuing property of performSelectorOnMainThread() to make sure that the async rendering has completed when stopping the renderer.

@sauwming sauwming merged commit 7231146 into master Jul 7, 2022
@sauwming sauwming deleted the vid-mainthread branch July 7, 2022 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants