How to make it work behind Apache reverse proxy?

[senatoregiorgio]

Hi,

I'm interested in your attempt to dockerize BigBlueButton, then I tried to follow the installation guide on this repository, but I'm not an expert so I'd like to ask for some clarification.

Apache is running on my server, so in order to use the HTTPS proxy container I had to modify the file "docker-compose.https.yml" to map ports 80 and 443 in the container to other ports (9010 and 9011) on the host, since 80 and 443 are already used by Apache.

Then I created a virtual server on Apache whose server name was the same of the domain name I specified during the setup of your BigBlueButton dockerized version. For this virtual server I added some reverse proxy directives to map incoming requests to port 443 of the HTTPS proxy container (port 9011 of the host). After many attempts, I managed to make it work thanks to these directives:

SSLProxyEngine on
SSLProxyVerify none 
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off
ProxyPreserveHost on
ProxyRequests off

ProxyPass / https://10.7.7.1:9011/
ProxyPassReverse / https://10.7.7.1:9011/ 

RewriteEngine on
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
RewriteCond %{HTTP:CONNECTION} ^Upgrade$ [NC]
RewriteRule .* wss://10.7.7.1:9011%{REQUEST_URI} [P]

I had to use the rewrite rule (last line) because the websocket connection failed when entering a BigBlueButton room without it (so there were no audio nor video).

Now BigBlueButton works (I can connect in listen mode and activate the webcam), except that when I try to enable the microphone I get the error ICE 1007. I checked in the browser console and I found that BigBlueButton tries to connect to the local turn server on port 465 (using the public domain name), but I don't know if it works and I don't even know if the problem is the turn server of something else.

All the docker containers are running properly.

How would you suggest to use this dockerized version when ports 80 and 443 are already occupied by a websever? I would be grateful for your help.

Thank you in advance.

[alangecker]

first of all the docker-compose.https.yml exists only for SSL/TLS termination and is not needed if you have another reveres proxy (in your case apache) in front.
rather set ENABLE_HTTPS_PROXY=false in your .env and point apache directly to http://10.7.7.1:8080

maybe the removed nginx layer solved the problem already?

if not, make sure that apache sets the header X-Real-IP and X-Forwarded-For to the client IP. (I don't know anything about apache, so you unfortunately you need to find out by yourself how you can achieve that^^)

Some background

the listen only mode works via kurento, the microphone mode via freeswitch. apparently freeswitch somehow needs to know the clients real IP to start the WebRTC connection, which it doesn't get by the additional apache layer. (the nginx by docker-compose.https.yml sets the headers, but only sees the requests coming from localhost)

If the WebRTC connection fails, it falls back to the TURN Server via port 465. Right know I can't understand why it doesn't work. Do you maybe have an mail server running on the same server? In this case the port might be used for SMTP.

you can use ss -tlpen | grep :465 to see what processes are listening on that port.

[senatoregiorgio]

Thank you so much for your answer.

Can I install the turn server without running the HTTPS proxy container? I read in the .env that the HTTPS proxy container is required in order to enable the turn server container. I tried to set ENABLE_COTURN=true, TURN_SERVER= ... and TURN_SECRET= ... in the .env, but I don't know if I can do it.

Then, I reinstalled the whole thing without the HTTPS proxy container and set the reverse proxy to point to http://10.7.7.1:8080, as you can see here below:

# I tried to add these two lines and I enabled "remoteip" (Apache module), but I'm not sure it's enough
RemoteIPHeader X-Forwarded-For
RemoteIPHeader X-Real-IP

SSLProxyEngine on
ProxyPreserveHost on

# The virtual server now points to port 8080
ProxyPass / http://10.7.7.1:8080/
ProxyPassReverse / http://10.7.7.1:8080/ 

# If there's a WebSocket request, I redirect it to port 7443
RewriteEngine on
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
RewriteCond %{HTTP:CONNECTION} ^Upgrade$ [NC]
RewriteRule .* wss://10.7.7.1:7443%{REQUEST_URI} [P]

I had to set the rewrite rule in order to redirect WebSocket requests to port 7443, otherwise it doesn't work.

The browser console shows me a request to the turn server address I specified in the .env, so maybe it works.

The strange thing is that now the microphone mode works well (I can communicate via the audio channel), but I cannot enable the webcam, since I get the error 1020. This is what the console shows me:

[15:23:21:0031] ERROR: clientLogger: Camera SHARER has not succeeded in 15000 for w_mv5ddopzunco

If we can figure out how to make it work properly when there's a webserver already installed, I think it would be nice to add some instructions to the documentation, because one of the major problems of BigBlueButton, in my opinion, is that it requires a "clean" server, so if you want to use it on a server where other web applications or a webserver are already installed, you can only search for some working docker images, and it's hard to find one. This one works well.

Thanks again for your support.

P.S.: I'm not running a mail server, but when I check what processes are using port 465, two entries are shown: is it correct, or maybe the second entry is about something else using that port?

LISTEN0      100                                0.0.0.0:465        0.0.0.0:*     users:(("master",pid=1502,fd=111)) ino:22332 sk:bf <->                         
LISTEN0      100                                   [::]:465           [::]:*     users:(("master",pid=1502,fd=112)) ino:22333 sk:d9 v6only:1 <->

[alangecker]

Can I install the turn server without running the HTTPS proxy container?

ah yeah, you are right! you can't. the integrated turn server requires certificates created by the https proxy container. So if you want to use a TURN server, you need to install & configure coturn by yourself.

In an normal, non-restricted network you everything should work even without the turn server. so maybe first try the installation without turn and add it later as soon as everything else works?

Disabling TURN

• ENABLE_COTURN=false
• comment out TURN_SERVER and TURN_SECRET
• set an external STUN server like the public stun.freeswitch.org one:

  STUN_IP=216.93.246.18
  STUN_PORT=3478
  

Is it possible, that it currently still tries your own server with non-working coturn as a STUN server?

---

Regarding port 465: I think the master process using the port belongs to postfix. maybe there is a mail server running you don't know about?^^

I think it would be nice to add some instructions to the documentation.

definitely! so far I was not in the situation to deal with that, especially with apache. But maybe you could share what you have learned in a small guide "How to integrate bbb-docker into an existing apache setup?" :)

[senatoregiorgio]

definitely! so far I was not in the situation to deal with that, especially with apache. But maybe you could share what you have learned in a small guide "How to integrate bbb-docker into an existing apache setup?" :)

When I understand what are the correct Apache directives to use, I will do it with pleasure. 😊

In an normal, non-restricted network you everything should work even without the turn server. so maybe first try the installation without turn and add it later as soon as everything else works?

I agree with you, it's more important to make it work, even without TURN, before thinking about the TURN server, so at the moment I took your advice about disabling TURN in the .env file.

---

From what I understand, the reverse proxy requires some specific directives to handle wss requests. In fact, adding the following directives allows Apache to redirect wss requests like wss://bbb.example.com/html5client ... or wss://bbb.example.com/ws to port 7443, which is used by FreeSwitch for microphone mode (and FreeSwitch only listens on 10.7.7.1, as I realised):

RewriteEngine on
RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
RewriteCond %{HTTP:CONNECTION} ^Upgrade$ [NC]
RewriteRule /html5client(.*) wss://10.7.7.1:7443/html5client$1 [P]
RewriteRule /ws wss://10.7.7.1:7443/ws [P]

Now, microphone mode works, but I'm not able to share my webcam and I get some Kurento errors in the browser console. This makes me think that it's necessary to add some Apache directives that redirect wss requests like wss://bbb.example.com/bbb-webrtc-sfu?sessionToken= ... to ports used by Kurento and WebRTC, as I did for FreeSwitch. This should solve the problem, I think.

For that reason, I ask you: what ports do you think I should deal with? What ports are used by Kurento and WebRTC for wss requests that are made when I enable the webcam in a BigBlueButton room?

[senatoregiorgio]

I managed to make it work. Some correct Apache directives were enough. I'll propose changes to README.md adding some instructions to install bbb-docker into an existing Apache environment.

[alangecker]

sorry, have been busy the last days!
nice, that you managed to make it work already!

---

following line doesn't make any sense, since it forwards the meteor connection to freeswitch

RewriteRule /html5client(.*) wss://10.7.7.1:7443/html5client$1 [P]

bigbluebutton uses following websocket paths:

RewriteRule /ws wss://10.7.7.1:7443/ws [P]
RewriteRule /html5client(.*) ws://10.7.7.11:3000/html5client$1 [P]
RewriteRule /bbb-webrtc-sfu(.*) ws://10.7.7.13:80/bbb-webrtc-sfu$1 [P]

in theory the last one could also get directly redirected, in this case to webrtc-sfu (at 10.7.7.10), but apparently nginx takes care about the authorization, so this proxys it to nginx first.

[senatoregiorgio]

Yes, you were right: that line was completely wrong. Anyway, the new directives I wrote don't forward any single kind of websocket request to a specific port: they only forward them to port 8080. You will see these working directives in the little guide I've added to README.md. Thank you very much for your time! 😊