Firebase integration with self-hosted server

[echo-slam-jam]

Description

I installed typesense in my

ubuntu 20.04 apache server

and it is successfully running but I can't succeed in trying to sync my firestore data using the firebase extension. When I input the public IP of my server in the typesense host portion, it fails to upsert the data.

Steps to reproduce

firebase extension

and this is my config file for typesense server

I'm guessing that it needs SSL certification but where do I get my SSL certificates to put in the configs?
Can self-signed work? Can certbot work?

I'm new to this so have mercy on me

""

Expected Behavior

Actual Behavior

Metadata

Typsense Version:

OS:

[britisharmy]

I made it to work with letsencrypt, email me i cann share my config,it encrypts but you shall have to switch to localhost for inserts and curl only for upserts and reading remotely.

[echo-slam-jam]

Thanks!
I emailed you at vectorofficeautomation@gmail.com

[jasonbosco]

The extension expects Typesense to be running on port 443 with https enabled.

So you need to start the Typesense server with the ssl key, ssl cert and api port (443) specified in the configs. More info about these params here: https://typesense.org/docs/overview/benchmarks.html

You can definitely use certbot / LetsEncrypt SSL certs. Self-signed certs don’t work.

[jasonbosco]

@britisharmy Curious why you had to do this:

but you shall have to switch to localhost for inserts and curl only for upserts and reading remotely.

[echo-slam-jam]

The extension expects Typesense to be running on port 443 with https enabled.

So you need to start the Typesense server with the ssl key, ssl cert and api port (443) specified in the configs. More info about these params here: https://typesense.org/docs/overview/benchmarks.html

You can definitely use certbot / LetsEncrypt SSL certs. Self-signed certs don’t work.

I see. Now my problem is that I have a virtual host in my server which has certbot SSL installed and I tried to use it as my API address with its certificates

api-address = domain.com
api-port = 8108
data-dir = /var/lib/typesense
api-key = value here
log-dir = /var/log/typesense
enable-cors = true
ssl-certificate = /etc/letsencrypt/live/domain.com/fullchain.pem
ssl-certificate-key = /etc/letsencrypt/live/domain.com/privkey.pem

I can now curl with https in port 8108

Now when I change the port to 443, my typesense server fails to start when I restart it :(

I am guessing that I'm doing something wrong in setting up my SSL but I don't know how exactly to set it up in this case

[britisharmy]

@britisharmy Curious why you had to do this:

but you shall have to switch to localhost for inserts and curl only for upserts and reading remotely.

I am almost done, updating the multiple catalogs i have and then i shall revisit the ssl issue and email you. I don't recall why, but i made some notes somewhere, once the entire catalog is up, i am going to shoot you an email.

[jasonbosco]

@echo-slam-jam You want to use an IP address for api-adddress, not domain name.

Could you also post the logs from /var/log/typesense/typesense.log?

[echo-slam-jam]

Do I need to post the whole log file?

[jasonbosco]

Just since the last restart. You should see a line saying Starting Typesense when you restart Typesense.

[echo-slam-jam]

[echo-slam-jam]

@echo-slam-jam You want to use an IP address for api-adddress, not domain name.

Could you also post the logs from /var/log/typesense/typesense.log?

OHH so that's one thing I needed to know haha

Actually I tried to input the IP address on the host at the firebase extension and the error was about not being recognized by the certificate

ext-firestore-typesense-search-indexToTypesenseOnFirestoreWrite

Request #1628516055899: Request to Node 0 failed due to "ERR_TLS_CERT_ALTNAME_INVALID Hostname/IP does not match certificate's altnames: IP: my public IP is not in the cert's list: "

[echo-slam-jam]

Will continue to reply tomorrow since it's midnight here in the Philippines

I guess the problem I have now is setting up SSL for this case?

[jasonbosco]

Request #1628516055899: Request to Node 0 failed due to "ERR_TLS_CERT_ALTNAME_INVALID Hostname/IP does not match certificate's altnames: IP: my public IP is not in the cert's list: "

It sounds like you've generated your certificate using a domain name, but you've used your IP address when configuring the Firebase extension. Instead, you want to use the domain name in the Firebase extension - the same domain name you used when generating the SSL cert.

Separately, those logs seem like they're from before you had changed the port to 443. If the above still doesn't work, could you change the port, restart Typesense and then capture the logs from that point on?

[echo-slam-jam]

LOG file

Starting the service

My config
; Typesense Configuration

[server]

api-address = xxx.xx.xxx.xxx
api-port = 443
data-dir = /var/lib/typesense
api-key = value here
log-dir = /var/log/typesense
enable-cors = true
ssl-certificate = /etc/letsencrypt/live/domain/cert.pem
ssl-certificate-key = /etc/letsencrypt/live/domain/privkey.pem

[jasonbosco]

May I know which cloud provider you’re running this on? It looks like the public IP address is not directly accessible on the instance. May be try using the private IP of the instance for api-address?

[echo-slam-jam]

May I know which cloud provider you’re running this on? It looks like the public IP address is not directly accessible on the instance. May be try using the private IP of the instance for api-address?

The virtual machine is actually owned by the university where I am studying at.
They are also the ones responsible for giving me the public DNS which I used.

I tried the private IP which also didn't work

What might be the cause of this accessibility problem? I can try to bring this up to the manager of the server.

[echo-slam-jam]

The server is actually hosted by the university where I study at.

[jasonbosco]

Notice how the log here says "Cannot assign requested IP". That's the underlying issue.

• When you run ifconfig does that show you the public IP?
• Does ifconfig show you an address like 10.x.x.x or 192.168.x.x?

[echo-slam-jam]

[jasonbosco]

Could you post the logs, once you update the IP address in the Typesense configs to 10.207.9.46 and restart the Typesense process?

[echo-slam-jam]

Ohhhh I can see that it says that the address is already in use

[jasonbosco]

Yup, so some other process is already using port 443. You can try lsof -i :443 to see which process that is.

[echo-slam-jam]

Hmmm the directory is the same as the directory where I hosted the file storage that I use for serving the files from this server
So does this mean that I cannot host this service anymore due to that?

[jasonbosco]

Ah, if you're already using 443 for Apache, another thing you could do is run Typesense with http protocol on say port 8108 (the default). Then setup Apache to handle HTTPS on port 443, and then have Apache reverse proxy to Typesense on port 8108.

[echo-slam-jam]

Ohh nice. This is what @britisharmy also suggested.

Are these steps for elasticsearch applicable?
https://devdocs.magento.com/guides/v2.4/install-gde/prereq/es-config-apache.html

[jasonbosco]

That link doesn't seem to cover https. This looks more recent and talks about https setup as well: https://www.digitalocean.com/community/tutorials/how-to-use-apache-http-server-as-reverse-proxy-using-mod_proxy-extension

The key thing to change in that guide is:

ProxyPass / http://0.0.0.0:8080/
ProxyPassReverse / http://0.0.0.0:8080/

becomes

ProxyPassReverse / http://0.0.0.0:8108/

[echo-slam-jam]

I see.

Do I need to follow the whole steps starting from Modifying The Default Configuration

Or just Enabling SSL Reverse-Proxy Support

[jasonbosco]

Everything except "Enabling Load-Balancing"

[echo-slam-jam]

ok I'll get back to you after I do it

[echo-slam-jam]

That link doesn't seem to cover https. This looks more recent and talks about https setup as well: https://www.digitalocean.com/community/tutorials/how-to-use-apache-http-server-as-reverse-proxy-using-mod_proxy-extension

The key thing to change in that guide is:

ProxyPass / http://0.0.0.0:8080/
ProxyPassReverse / http://0.0.0.0:8080/

becomes

ProxyPassReverse / http://0.0.0.0:8108/

This is my config for the 000-default
<VirtualHost :>
ProxyPreserveHost On

# Servers to proxy the connection, or;
# List of application servers:
# Usage:
# ProxyPass / http://[IP Addr.]:[port]/
# ProxyPassReverse / http://[IP Addr.]:[port]/
# Example:
ProxyPass / http://10.207.9.46:8108/
ProxyPassReverse / http://10.207.9.46:8108/

ServerName localhost

Listen 443

NameVirtualHost *:443
<VirtualHost *:443>

SSLEngine On

# Set the path to SSL certificate
# Usage: SSLCertificateFile /path/to/cert.pem
SSLCertificateFile /etc/letsencrypt/live/my domain/cert.pem


# Servers to proxy the connection, or;
# List of application servers:
# Usage:
# ProxyPass / http://[IP Addr.]:[port]/
# ProxyPassReverse / http://[IP Addr.]:[port]/
# Example:
ProxyPass / http://10.207.9.46:8108/
ProxyPassReverse / http://10.207.9.46:8108/

# Or, balance the load:
# ProxyPass / balancer://balancer_cluster_name

I cannot restart apache2 because of the error

Line 16 is Listen 443

[britisharmy]

The certificate cant work on a naked ip. You shall need a sub domain or a tld.

[echo-slam-jam]

Hmmm

So I managed to successfully restart apache2.

• I removed Listen 443
• I removed *NameVirtualHost :443

This is now my config for 000-default
<VirtualHost :>
ProxyPreserveHost On

# Servers to proxy the connection, or;
# List of application servers:
# Usage:
# ProxyPass / http://[IP Addr.]:[port]/
# ProxyPassReverse / http://[IP Addr.]:[port]/
# Example:
ProxyPass / http://10.207.9.46:8108/
ProxyPassReverse / http://10.207.9.46:8108/

ServerName localhost

SSLEngine On

# Set the path to SSL certificate
# Usage: SSLCertificateFile /path/to/cert.pem
SSLCertificateFile /etc/letsencrypt/live/my-domain/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/my-domain/privkey.pem

# Servers to proxy the connection, or;
# List of application servers:
# Usage:
# ProxyPass / http://[IP Addr.]:[port]/
# ProxyPassReverse / http://[IP Addr.]:[port]/
# Example:
ProxyPass / http://10.207.9.46:8108/
ProxyPassReverse / http://10.207.9.46:8108/

# Or, balance the load:
# ProxyPass / balancer://balancer_cluster_name

And here is my config for typesense

GNU nano 4.8 /etc/typesense/typesense-server.ini ; Typesense Configuration

[server]

api-address = 10.207.9.46
api-port = 8108
data-dir = /var/lib/typesense
api-key = value here
log-dir = /var/log/typesense
enable-cors = true

Now both services are running

Are these configs correct?
I can now curl "HTTP://10.207.9.46:8108/health"

I can't curl https version

What should I be able to curl?
How do I verify if my reverse proxy works?

[jasonbosco]

You only need

ProxyPassReverse / http://10.207.9.46:8108/

and NOT

ProxyPass / http://10.207.9.46:8108/.

Essentially, the idea is that you're running multiple web-servers behind Apache, one is your existing site, and the other is Typesense. Depending on what hostname shows up in the HTTP headers, Apache will either send the request to your existing site or to Typesense. HTTPS/SSL is handled by Apache and the connection to Typesense uses regular http on port 8108.

In this setup, the idea is that if you've configured say typesense.domain.com to reverse proxy to http://10.207.9.46:8108, then, you should be able to curl https://typesense.domain.com/health and Apache will send that request to Typesense internally.

[echo-slam-jam]

You only need

ProxyPassReverse / http://10.207.9.46:8108/

and NOT

ProxyPass / http://10.207.9.46:8108/.

Essentially, the idea is that you're running multiple web-servers behind Apache, one is your existing site, and the other is Typesense. Depending on what hostname shows up in the HTTP headers, Apache will either send the request to your existing site or to Typesense. HTTPS/SSL is handled by Apache and the connection to Typesense uses regular http on port 8108.

In this setup, the idea is that if you've configured say typesense.domain.com to reverse proxy to http://10.207.9.46:8108, then, you should be able to curl https://typesense.domain.com/health and Apache will send that request to Typesense internally.

If I understand correctly, do I need to create a subdomain for my domain which I will call, say, "typesense.domain" which I will reverse proxy to http://10.207.9.46:8108
or can I just use 000-default like what digital ocean did?

[echo-slam-jam]

Or can I also do it like this

I will just assign the specific /typesense path to reverse proxy to port 8108

[jasonbosco]

Yeah, either of those should work. You might have to do some URL re-writing so Typesense doesn't see the /typesense/ portion of the URL... Not sure if that's how Apache reverse-proxying works out of the box, or if more configuration is needed.

[echo-slam-jam]

Ok
So I have now successfully configured my reverse proxy and curl "https://mydomain/typesense/health" now returns ok:true
I can also see it through web browser so I assume that this is already working

Now I went to the firebase extension and

typesense host: mydomain/typesense

still returns HTTP error 404

so I changed it to
typesense host: mydomain/typesense/collections
because that is where my collection is

but it still returns an error

What should be the correct host?
is there still something missing from my server setup?
This is my reverse proxy setup and I think it works fine

[jasonbosco]

Oh hang on, my bad. While Typesense itself can be hosted under a path, the Firebase extension requires Typesense to be hosted under the root domain (and not in a sub-path). So you'd need to create a new virtual host in Apache and move the reverse proxy config under that, and setup a different sub-domain just for Typesense.

[echo-slam-jam]

Hmmm let me clarify.

Currently, the process goes like this

extension -> my SSL domain/typesense subpath (is this what you were talking about?) -> reverse proxy to 10.207.9.46:8108
which does not work

Can you illustrate in terms of this process chain?

[jasonbosco]

You'd have to set it up this way:

Extension -> https://typesense-sub.domain (setup in Apache as a separate Virtual Host with its own SSL cert) -> reverse proxy to 10.207.9.46:8108

[echo-slam-jam]

Hmm can I still use my ssl domain like this?

extension -> https://mydomain/typesense -> reverse proxy to newly created https://typesense -> reverse proxy to 10.207.9.46:8108

So in this way I wont have to request for another public DNS from the university

[jasonbosco]

That won’t work with the Firebase extension. So you need a new domain/sub-domain just for Typesense, but it can still run on the same server.

[echo-slam-jam]

Ok I just checked with the server handler and I can fully use the root of mydomain

So now I don't need the subpath anymore and my process now is this:
extension -> my SSL domain (no more subpath) -> reverse proxy to 10.207.9.46:8108

[echo-slam-jam]

NICE it is finally working !!!!!
After 4 days of troubleshooting

THANKS @jasonbosco and @britisharmy for your assistance :)

Also I noticed that it fails to write empty arrays
Is this as intended?

[echo-slam-jam]

NICE it is finally working !!!!!
After 4 days of troubleshooting

THANKS @jasonbosco and @britisharmy for your assistance :)

Also I noticed that it fails to write empty arrays
Is this as intended?

Idk why but now it accepts empty arrays haha

[jasonbosco]

Amazing! Kudos to you for your persistence! 🙌 🙌

It would be awesome if you're able to share the final Apache configs that worked for you, for the benefit of other folks who might have a need to reverse proxy through Apache in the future.

[echo-slam-jam]

Sure!

So the process goes like this for me:

Firebase extension -> my SSL (by certbot) "domain.xxx" -> Reverse proxy to my VM local IP address

My reverse proxy settings on my domain's xxx-le-ssl.conf file

ProxyPreserveHost On
ProxyPass "/" "http://local IP:8108/"
ProxyPassReverse "/" "http://local IP:8108/"
AllowEncodedSlashes On

The config for my typesense

api-address: local IP
port: 8108
api-key: value here
enable-cors: true

Then on the firebase extension, use the ssl domain name like "sampledomain.xxx"

Make sure to enable the necessary modules in apache for reverse proxying
https://www.digitalocean.com/community/tutorials/how-to-use-apache-http-server-as-reverse-proxy-using-mod_proxy-extension

Apache docs about reverse proxying that might help understand this stuff
https://httpd.apache.org/docs/2.4/howto/reverse_proxy.html

Also, if you are using the domain for other subdirectories like domain.xxx/otherpath
You want to make an exception for that path to not be included in reverse proxy requests

ex.
ProxyPass /sample/path/ !
https://stackoverflow.com/questions/39666686/proxypass-but-exclude-certain-sub-directory/39666874

[braincomb]

Is there a reason why Typesense extension doesn't allow to specify a custom port? Is this a Firebase requirement?

[jasonbosco]

@braincomb It's not a Firebase requirement, but a security best practice: #22 (comment)