Can not record multiple sessions at the same time? #33
Comments
|
If you wish to make multiple recordings at the same time, simply spin up more instances/VMs running the jibri service. Jicofo will choose from the available pool of jibri recorders and assign them when the user enters their stream ID.
…-Aaron
On Mar 27, 2017, at 12:39, Alfredo Guzman ***@***.***> wrote:
in the doc:
"It is intended to be run on a separate machine (or a VM), with no other applications using the display or audio devices. Only one recording at a time is supported on a single jibri."
Is it definitely impossible to record more than one at a time?
If I wanted to record or trasminitr to youtube many sessions at the same time what should I do?
Thank you
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub <#33>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC8JeB5rnp98mrflp50jRvDWljLr8Wu2ks5rp_RBgaJpZM4MqlvE>.
|
|
pool of jibri recorders? Where is that configured? Thanks! |
|
Hi, any info on this ? Where can we add servers to the pool ? |
|
You just need to install multiple jibri machines the same way, the all join a muc room which is monitored by jicofo and it chooses the next free one when recording is requested. But Aaron already mentioned all these. |
|
Do we need to change any usernames etc in config files ? |
|
Actually what happens is that they both start recording and it is fine , then i stop one and start it again , still fine. I want to stop second one but instead of asking me to stop recording it says "no recorders available" , then i stop the first one and press again on second one and it wants to start recording again even when it is still recording. |
|
You only need to have different nicknames on the different jibris, user and password are the same. You can do this by adding: "nick": "jibri123456" to the config.json. |
|
I just tried it and it still does not add "Recording" floating notification to second recording session , but it is recording and it can't stop. Also "nick" variable is not used , i see that every server uses some random jibri-randomseed nick unique to its session. |
|
This is from jicofo log: |
|
The nick is not for the session it is for the jibri users that enter the muc control room, where jicofo is checking for available recorders. |
|
Ok from the jibri log nick is not used form the config file , but even if it is used i still have same issue:
|
|
Hi , can you help with this ? |
|
Hi I made cleaner test where in warnings you can see the whole process , please respond. `##### 2 sessions Opening roomJicofo 2018-03-02 15:26:48.770 INFO: [69] org.jitsi.jicofo.recording.jibri.JibriRecorder.log() Publish new JIBRI status: in: 58@conference.external.jibri.zimbra-aaa.de Jicofo 2018-03-02 15:27:25.038 INFO: [53] org.jitsi.jicofo.recording.jibri.JibriRecorder.log() Publish new JIBRI status: in: 58-2@conference.external.jibri.zimbra-aaa.de START of recordingJicofo 2018-03-02 15:28:30.442 INFO: [98] org.jitsi.jicofo.recording.jibri.JibriRecorder.log() Publish new JIBRI status: in: 58@conference.external.jibri.zimbra-aaa.de Jicofo 2018-03-02 15:29:22.561 INFO: [99] org.jitsi.jicofo.recording.jibri.JibriRecorder.log() Publish new JIBRI status: in: 58-2@conference.external.jibri.zimbra-aaa.de Jicofo 2018-03-02 15:30:52.562 SEVERE: [105] org.jitsi.jicofo.recording.jibri.JibriSession.log() Jibri pending timeout! 58-2@conference.external.jibri.zimbra-aaa.de .................. STOP recording of session 1Jicofo 2018-03-02 15:32:11.541 INFO: [109] org.jitsi.jicofo.recording.jibri.JibriRecorder.log() Publish new JIBRI status: in: 58@conference.external.jibri.zimbra-aaa.de CLOSING session 2 tabJicofo 2018-03-02 15:33:51.039 INFO: [38] org.jitsi.protocol.xmpp.AbstractOperationSetJingle.sendRemoveSourceIQ().558 Notify remove SSRC 58-2@conference.external.jibri.zimbra-aaa.de/recorder-ade5f0 SID: 99ggjtgsfmcmc Sources{ audio: [ssrc=2034517262 ] video: [ssrc=1669193102 ssrc=205460022 ssrc=3853850348 ssrc=51791073 ssrc=180095571 ssrc=1126842249 ] }@595975689 source_Groups{ video:[ SourceGroup(FID)[ ssrc=1669193102 ssrc=205460022 ]SourceGroup(FID)[ ssrc=3853850348 ssrc=180095571 ]SourceGroup(FID)[ ssrc=51791073 ssrc=1126842249 ]SourceGroup(SIM)[ ssrc=1669193102 ssrc=3853850348 ssrc=51791073 ] ] }@890407653 ` So the second recording start starts on jibri server but it is not communicated and recording seems off from jitsi side , only way to stop recording is to close chatroom and let the timeout on jibri happen. |
|
Here are the full logs from jibri servers and jicofo from xmpp server: jicofo_full_log_91-1_2.txt Please respond and continue conversation to find resolution for multiple jibri recording sessions. |
|
It may well be that Jicofo doesn't properly handle the case of multiple jibri recorders in a single room. This wasn't ever a use-case we considered, beyond a primary and backup streamer instance. The main use case that's been tested for multiple Jibri instances is to have them recording in separate rooms at the same time. Thank you for the detailed logs, we will examine them and consider what the best next step might be. It's possible that the solution will be block the starting of a second jibri if one is already started. |
|
@aaronkvanmeerten Thank you for response but in our scenario those are 2 separate rooms (2 for testing , later we will add much more jibri servers per need. The recorder1-91-2.txt should be named recorder2 actually , just a typo in log file name when uploading and 91-1 ,91-2 are rooms. |
|
Hi @aaronkvanmeerten @damencho @alfredogt . This is very urgent for us , are you willing to get in touch and that we arrange some sort of business cooperation to resolve multiple recording sessions without errors? The issue here is that status update goes to first room "91-1@" instead of the second "91-2@".
|
This. One recording session per jibri JID is supported. If you try to have more than one, the requests get routed to the wrong place. :( |
|
Hi @bgrozev even the 'nick' is added to config but it is not used. |
|
Oh, I see! Looking into it further, we use the occupant JIDs of the jibri's and we compare them as bare JIDs. And the bare JIDs of the two jibri's are the same, because they are in the same MUC. To confirm that this is the problem can you please try this patch? If that works I'll make a ticket to implement a more proper fix. |
|
so far this works ! I will make detailed tests tomorrow but for now it all seems ok. |
|
Thanks for the report! I've opened #264 to get this merged into master. |
|
This should be fixed in master. Please re-open if you find that it isn't. |
|
Can someone clarify if a single machine can or cannot run multiple jibri processes? Right now, it seems that jibri process has an internal jetty listening to ports 2222 and 3333. It prevents another jibri process started. If this restriction is removed somehow, any other components (e.g. device modules, chrome...) may cause the limitation? |
|
@vincentwlau that's not a configuration we've tested as jibri's services can be pretty sensitive to contention, so we've found it easier to just spin up multiple. |
|
@bbaldino Thanks for the response. But single jibri per machine is not a viable solution since our customer requires 1000+ concurrent recording sessions. Can you elaborate where the contention is? If jibri is not a right solution, do you have any suggestions? |
|
My team is willing to look into this contention issue. If we can solve it, we would like to contribute the solution to the community. Would anyone like to share any insights where the contention is, or point us to the direction where we can explore ourselves? |
|
couple things i can think of off the bat:
that's all that came to me immediately, i'll see if i come up with anything else. |
|
@bbaldino Thank you very much for the information; I will try out #2 and #3. For #1, using command-line args is a good idea. Any thought on how launch.sh and shutdown.sh (and graceful_shutown.sh) handle multiple jibri processes on a single machine? I have an idea. The config.json can have two optional new elements "internal_http_api_port" (default is 3333) and "external_http_api_port" (default is 2222.) The launch.sh and shutdown.sh can look for config[0-9]*.json files to start and stop all additional instances. To be backward compatible, launch.sh will always look for config.json and shutdown.sh will always do curl on port 3333. For #4, we will change the logging handler patterns from "xxx.%g.txt" to "xxx.%g.%u.txt" where xxx is "log", ffmpeg", "pjsua" and "browser". |
|
for your questions on #1 i think @aaronkvanmeerten would have some good thoughts, but putting the ports in config.json instead of command line args could work well (i don't think we need to pass them as args if they're in config.json) one concern about your idea for #4 is: how will we know which jibri is logging to which file? i guess maybe we don't care. i'd have to think about whether we do or not. also, i'm not sure the exact semantics of |
|
From FileHandler javadoc: |
|
The main issue with using multiple jibris on a single instance is actually the distinct number of X desktops and ALSA loopback devices configured on the instance. As long as there are distinct desktops and distinct devices, and jibri is re-written to be configured to use a specific device per instance, I believe this is possible. There was an excellent fork of the python version that provided for this functionality, although we didn't end up merging it as we ran into trouble testing it and it then it got stale/out of date. |
|
When you refer a distinct desktop, does it mean a unique display number? By any chance, do you know how to configure more than one ALSA loopback device? That is a good start for us to explore. @bbaldino For the concurrent logging issue, will it be better to have config.json containing an array of the elements? Then in the arg line, there will be an option of instance identifier (0, 1...) One thing that I dislike this scheme is breaking the compatibility of config.json. It is just a thought. |
|
Yes, I do mean a unique display number. By default the jibri service is configured to start up a supporting X11 server on :0 As far as multiple ALSA loopback devices, I believe you can do it with the enable= option, as referenced here: in /etc/modprobe.d/sound.conf add the following to enable for example 5 loopback devices Then you'd have to set up the .asoundrc in the jibri homedir to reference all the devices like we've done for the first one. |
Yeah, maybe we can think about this more. I'd say for now don't worry about the logging issue (the other problems will likely be more difficult anyway) and we can figure that one out down the line. |
|
Thank you both for the valuable input. I really appreciate your time helping me. |
|
Does anyone have a recommended (minimum) hardware requirement (# of cores, CPU speed, RAM, swap space) for Jibri server? We had a single core, 2.3GHz, 4GB RAM, 0B swap-space 64-bit Ubuntu cloud server and it hung after 20 seconds of recording with two participants due to out of swap space. |
|
We are currently using the c5.xlarge AWS instance for recordings. This is a 4 vCPU, 8GB of RAM instance. We do not configure any swap either. |
|
@aaronkvanmeerten Thank you very much for information. Just FYI. We were able to run Jibri with 2 vCPU (2.3GHz), 8GB RAM, 0 swap and its memory usage never dipped below 6GB. So, we think it should be able run with 4GB RAM as well. |
|
Dears, Can you share more details for configuring Jibri in one instance and be able to record many sessions simultaneously, Thanks |
|
@arts111188 Although we modified the source in our private experiment to allow ports 2222 and 3333 configurable via the config.json, we stopped pursuing further to support different the x11 display and audio loopback for ffmpeg because the CPU consumption for a recording session is very high and it won't be scaled well for our requirement. We are trying to take a different approach to "record" multiple sessions without x11 and audio loopback from a single server. Unlike Jibri's real-time recording, we are merely archiving the media streams for offline mpeg encoding. It is sort of a hybrid of Jibri and Jirecon. We are still experimenting with this idea, so it is too soon to share. |
|
Hello everyone, |
|
@alfredogt , |
|
Hello, @andaralex
|
|
Hello, @alfredogt Can you explain how you have made modifications to associate a specific Greetings. |
|
Hello, I answer to myself : simply use HTTP API instead of XMPP API ! Greetings. |
|
Hi, I need to run 3 jibri instances in a single VM. Could anyone please help me or guide me how to configure that, or what modification is needed? I have installed a single instance in a VM and it is working fine. Thanks for your help. |
|
This isn't really possible with Jibri in its current state. We recommend using separate VMs for this type of functionality. |
@aaronkvanmeerten Thanks for a quick response. But a single jibri instance can be used to record a single meeting at a time. So if I need to record 50 meetings at a same time I need to install 50 separate jibri instances in 50 separate VMs? It is not really feasible. Isn't there any other way so that we can install at least 5 instances in a single VM? Please help on this regards. |
|
Yes that's accurate. Jibri uses a kernel module (Alsa loopback) for doing the audio capture, and this is set up per kernel. So it MAY be possible to set up a single VM with as many alsa loopback devices as you want per instance, and then run the docker container of jibri with the proper devices mapped, we don't suggest it as the CPU and memory constraints of jibri means if you have many recordings on a single VM at one time you'll end up with bad recordings (stuttering video, dropped audio, etc). So in general recommend running jibri in an autoscaling setup if you wish to do multiple recordings at the same time. |
|
Maybe client recording is a good solution for recording multiple replies at the same time.link |
|
Thank you so much @aaronkvanmeerten for guiding me, need a help from you. As you told that we can run jibri in an auto-scaling setup to do multiple recording at the same time. I have installed jibri in a VM of my office. So how can I do the auto scale there, please be noted it is not an AWS server and I am using ubuntu 18.04. Can you please help me how can I do autoscaling there as I am beginner on this kind of set up. Thanks in advance. |
|
Hi @daxiondi : I have installed your client side recording solution. It is working fine. Just one quick question the recording is restricted for moderator only? Or all the participants of a conference can record the session. Thanks, |
in the doc:
"It is intended to be run on a separate machine (or a VM), with no other applications using the display or audio devices. Only one recording at a time is supported on a single jibri."
Is it definitely impossible to record more than one at a time?
If I wanted to record or trasminitr to youtube many sessions at the same time what should I do?
Thank you
The text was updated successfully, but these errors were encountered: