How do you run this through a proxy?

[Okozzie]

I'm not really familiar with Go, but I've run go build -ldflags="-X main.version=$(git log -n1 --format='%h_%cI')" ./cmd/goatcounter and it was able to download the files.

It seems the next step is to run goatcounter serve but that just returns a goatcounter not found

[arp242]

It looks like you need to use ./goatcounter instead of just goatcounter? You can use goatcounter if you use go install, assuming that $GOBIN is in your shell's $PATH.

[Okozzie]

Thanks for the quick reply, that seems to work!

Is there anything I need to change with NGINX config files, can't seem to figure out how to get it to work?

I receive this:
zhttp.Serve: listen tcp :443: bind: address already in use

After running ./goatcounter serve, even after killing all the ports.

If I run ./goatcounter serve -db sqlite3://db/goatcounter.sqlite3 -listen :5000 -tls none, it will show the default Thank you for using GoatCounter message but I can't access the site on port 5000

[arp242]

You'll need to use a different port if you're already running something on port 443, e.g. goatcounter serve -listen :8080.

Since I assume you will use nginx as a proxy, you will probably also want to disable HTTPS support with tls=none and let nginx take care of that.

[arp242]

Saw your edit just after posting 😅

I don't know why you can't access the site; maybe there's something in the firewall of the server? What kind of message do you get if you try to access the site?

If you want to directly access the site on port 5000, you will also want to set -port=5000 (this is because we can't know if goatcounter is running in front of a proxy).

[Okozzie]

https://rgth.co/blog/replacing-google-analytics-with-goatcounter/

I found that tutorial online, will give it a try with Caddy instead of NGINX 👍

[arp242]

What's the easiest way to set this up?

The easiest way is to not use nginx at all; that way you don't need to worry about proxying and the like which makes things a lot more complex. GoatCounter will take care of most things like https certificates and the like, and is designed to be runnable "out of the box".

For example, for my own site I run it just as:

$ ./goatcounter serve

And that's it really 😅

If you want something more complex then it might be helpful to describe a bit what your setup is like. Is the nginx server a web server that's actually running? Or are you just using it as a proxy? Do you want to be able to access goatcounter on port 80/443, or on port 5000?

[arp242]

I haven't used Apache in ten years 😅 Maybe longer.

At any rate, you want to run goatcounter like:

$ goatcounter serve -listen=localhost:5000 -tls=none

And then set up nginx (or caddy, apache, doesn't really matter) to forward requests to that; you can do this based on the hostname ("stats.example.com") or path ("example.com/stats").

Here is something I came up before when this question came up, for nginx:

user root;
daemon off;

events {
        worker_connections  1024;
}

http {
        server {
                listen       80;
                server_name  goat.mydomain.com;
                root         /;

                location / {
                        proxy_redirect off;
                        proxy_set_header Host $http_host;
                        proxy_set_header X-Forwarded-For $remote_addr;
                        proxy_set_header X-Forwarded-Proto $scheme;
                        proxy_set_header Connection "";
                        proxy_http_version 1.1;
                        proxy_pass http://127.0.0.1:5000;
                }
        }
}

I should probably add a "goatcounter help proxy" page for this in addition to the "goatcounter help listen".

[Okozzie]

It partially works now 😂

If I run ./goatcounter create -domain <IP ADDRESS> -email email@example.com

Followed by ./goatcounter serve -listen=<IP ADDRESS>:5000 -tls=none,

I am able to access IP-ADDRESS:5000 in the browser, but this blocks every port for some reason so I can't even access any domains pointed - also the fact that it's an IP and not an actual domain.

Is there a way for someone to upload this to the Digitalocean Marketplace?

Also tried getting it to work without NGINX, and it's the same - doesn't allow for domains

[arp242]

Is there a way for someone to upload this to the Digitalocean Marketplace?

Last time I tried I couldn't create an account as DO doesn't like my credit card 🤷 It seems to work now and I signed up as a "vendor"; I'll probably look at it at some point, but it's not going to be today or tomorrow 😅

I'm not sure what you mean with "blocks every port"? I never actually tested using an IP address, but I think this should work. I recommend using localhost or some such instead if you're testing locally.

[Okozzie]

It works now, just set up a new instance and started from scratch - probably something wrong with the old config 😅

Is there a way to access the new sites created? If I add an additional site, it doesn't say what the new site code is.

Also clicking on a site redirects you to the domain of that site, is there a way to view data of that additional site specifically?

How do I keep it running btw, after a ./goatcounter serve, it only stays running whilst the terminal is open

Thanks!

[arp242]

If I add an additional site, it doesn't say what the new site code is.

Not sure what you mean? You access it on the domain you fill in.

Also clicking on a site redirects you to the domain of that site, is there a way to view data of that additional site specifically?

You can view the data on the site's domain; I'm not sure what you mean?

How do I keep it running btw, after a ./goatcounter serve, it only stays running whilst the terminal is open

Run it in tmux or screen, or use the service manager of your system (e.g. systemd, openrc, runit, etc.)

[Okozzie]

For this page, I thought it means you can host multiple sites from the same account?

On Fathom/Plausible, the dashboard lets you switch between your child sites and view the stats individually (under the same account). The site code is the same, but you change the data-domain attribute.

I assume I'd have to change the data-goatcounter attribute to match the child site domain, but then how do I access this data from the dashboard?

The dashboard shows the default domain where I'm running the instance from, and at the top there's just links to my other domains (which don't have a separate GoatCounter instance installed).

[arp242]

The same GoatCounter is accessed on different domains: e.g. stats.example.com, other.mysite.com, etc. You need to access the correct domain to access the correct site.

[Okozzie]

Wouldn't this mean we would have to install a brand new GoatCounter instance for each domain, even though it comes under additional sites?  For example, let's say I set up a droplet and install GoatCounter for `mysite.com`. I also have 5 other domains all pointed to Github Pages.  If I want to start tracking anothersite.org (hosted on Github Pages), it seems like I would have to spin up another Droplet for `stats.anothersite.org` that runs GoatCounter or proxy that to work under `mysite.com`. As opposed to just going to mysite.com/settings, add an additional "child site" and paste the snippet code (data-domain attribute) in anothersite.org - which is how Plausible/Umami allow for the multiple domains to work under the same instance.  Is this possible?  Thanks

[ForestJohnson]

You can track as many different sites as you want on one Goatcounter instance. If you want each site to have its own "analytics home page" then I think that's what the "child sites" feature is for.

When martin said:

The same GoatCounter is accessed on different domains: e.g. stats.example.com, other.mysite.com, etc. You need to access the correct domain to access the correct site.

They were kinda assuming that you already understand how DNS / subdomains / virtual hosts work. The jist is, domain names and digitalocean droplets are two completely separate things. Also, not all domain names are the same. There are two different kinds.

First of all there are the "2nd level domain" names like mysite.com. They are called second level because they have two levels, the top level domain (TLD) like .com and the second level mysite, the part you register. These are domains you have to pay for, they are essentially just line-items in the database of a public trust institution called a DNS registrar, similar to a Bank.

Then there are the "subdomains" like goatcounter.mysite.com, and foosite.goatcounter.mysite.com. Do you see where we're going with this? You can create as many subdomains as you want for free once you have a 2nd level domain registered. They can be nested as deep as you want too, like a.b.c.d.mysite.com. Typically you do this by logging into your DNS provider's web management portal and navigating to the "manage DNS records" section (sometimes they hide it under "Advanced" 😫 ). Then you would create A Records with names like goatcounter and foosite.goatcounter and for the value, you would put the IP address of your digitialocean droplet. Those A records will create the subdomains goatcounter.mysite.com, and foosite.goatcounter.mysite.com. Then you can create the corresponding "child sites" in goatcounter with the same domain names. You may also have to configure each child site subdomain in your reverse proxy & obtain a TLS certificate for each one.

Another option which I decided to use, you can put the domain name (or a shortened alias for the domain name) into the "path" property of the events you send to Goatcounter. One single goatcounter site is not designed to be used on multiple domains yet but you can use a bit of a "hack" to make it work by prepending the domain name onto the URI. (By default goatcounter use the URI as the aggregation key).

There is an example of how to do this one the "site code" page of your goatcounter instance:

Multiple domains

GoatCounter doesn’t store the domain a pageview belongs to; if you add GoatCounter to several (sub)domain then there’s no way to distinguish between requests to a.example.com/path and b.example.com/path as they’re both recorded as /path.

This might be improved at some point in the future; the options right now are:

Create a new “additional site” for every domain; this is a completely separate site which inherits the user, login, plan, etc. You will need to use a different site code for every (sub)domain.

If you want everything in a single overview then you can add the domain to the path, instead of just sending the path:

 <script>
     window.goatcounter = {
         path: function(p) { return location.host + p }
     }
 </script>
 <script data-goatcounter="https://goatcounter.sequentialread.com/count"
         async src="//goatcounter.sequentialread.com/count.js"></script>

This is what I opted to use because on my site, I have a lot of subdomains that are all technically part of the same "site".

So I used a short alias to make the domain part take up less space, instead of return location.host + p I used something more like return replace(location.host, "sequentialread.com", "sqr") + p

[Okozzie]

Thanks for the explanation, though I already understand how DNS/Virtual hosts work.

My point wasn't "how do you connect a domain to a droplet?" 😅

I saw that this project was started as Fathom decided to drop support for the open sourced version of their platform. With the other open-source analytic tools out there, you have one account in an instance and it can support unlimited "domains".

The <script> tag would be something like this:

<script src="https://mysite.com/counter.js" data-domain="example.com">
<script src="https://mysite.com/counter.js" data-domain="anothersite.org">
<script src="https://mysite.com/counter.js" data-domain="hello.co">

We can see that they are all hosted under the same mysite.com instance, but are differentiated via the domain attribute.

In your dashboard, you would be able to view the stats of each of the sites (which are all done under one instance, but tracked via the domain attribute).

I see that GoatCounter doesn't support this the same way - I was just confused by the "Additional Sites" tab which states that you can have multiple websites and that they are treated separately, though under the same account - I just assumed it was similar to Fathom/Plausible/Umami.

Thanks anyway though 👍

[arp242]

Wouldn't this mean we would have to install a brand new GoatCounter instance for each domain, even though it comes under additional sites?

No, a single GoatCounter installation can serve any number of domains – as many as you want. goatcounter.com has thousands on a single goatcounter instance.

It was a bit late last night, and I'm leaving for a small holiday in a bit (still need to pack too 😅), so I don't have too much time atm, but the way it works is basically that GoatCounter can have any number of sites, and looks at the domain to determine which site to use.

So if you add a site with stats.example.com and then access the goatcounter instance on that domain, it will serve the dashboard for that domain, and if you have a site mine.domain.com then it will serve the dashboard for that domain.

This is basically the same as domain-based "virtual hosting" in Apache, nginx, etc.

Right now this is the only way to have multiple sites. You can't have multiple sites and not do domain-based virtual hosting. I'm not sure if we really need that either, because the assumption is that if you have a site you're also going to have a domain, which is probably a reasonable assumption?

So you can serve everything from your single droplet, you just need to create five DNS records (stats.mysite.com, stats.anothersite.org, etc.) all to the same IP.

This is how goatcounter.com works as well: for example https://goatcounter.goatcounter.com/ and https://stats.zgo.at both have the same IP (172.104.182.174) and GoatCounter knows which site to use based just on that.

In the script you use the correct domain too, with different data-goatcounter endpoints.

[arp242]

You don't need to use stats.mysite.org and stats.othersite.org btw; you can also use mysite.statsdomain.com or othersite.statsdomain.com or something like that. Anything you want really.

[Okozzie]

Thanks for clarifying! Reason for asking was because most analytic tools operate this way and it does not seem practical to set up records for each child site (for certain use cases).

As for @ForestJohnson, although they appear somewhat confused, maybe someone will benefit from their explanation of how DNS works 😂

[anarcat]

just for the record, i had trouble since the (2.x?) upgrade with the websocket connexion, which apache is notorious for happily mangling.

i ended up with the following:

<VirtualHost *:443>
        ServerName analytics.anarc.at
        Use common-letsencrypt-ssl analytics.anarc.at
        DocumentRoot /var/www/html/

        RewriteEngine on
        RewriteCond %{HTTP:Upgrade} websocket [NC]
        RewriteCond %{HTTP:Connection} upgrade [NC]
        RewriteRule ^/?(.*) "ws://127.0.0.1:8081/$1" [P,L]

        ProxyPass /.well-known/ !
        RewriteCond %{HTTP:Upgrade} !=websocket [NC]
        RewriteRule ^/?(.*) http://localhost:8081/$1 [P,L]
        ProxyPassReverse / http://localhost:8081/

        ProxyPreserveHost on
</VirtualHost>

the important part is to properly throw the right traffic to the right backend. by default, apache, with a ProxyPass, will happily convert websocket requests into HTTP requests. the RewriteCond %{HTTP:Upgrade} !=websocket [NC] is the bit that keeps it from doing that for websocket requests, while the bit above that is the one that properly sends those requests as websocket in the backend.

a symptom of the problem this solves is a 400 Bad request error that the client receives, and this error from goatcounter:

Feb 17 15:46:05 ERROR: websocket: the client is not using the websocket protocol: 'upgrade' token not found in 'Connection' header {code="1vq8w9p" http_form="" http_headers="Accept-Encoding: gzip, deflate, br · Accept-Language: en-US,en;q=0.5 · Accept: */* · Cache-Control: no-cache · Connection: Keep-Alive [...]

the Connection: Keep-Alive bit is the problematic part.

I finally figured it out by bumping the LogLevel to trace2 as well:

LogLevel trace2

That generates a lot of output in the logs, so be careful before enabling this on a production server. It does show how apache resolves requests in the backend and allowed me to stop guessing what it was doing.

[Okozzie]

You can track as many different sites as you want on one Goatcounter instance. If you want each site to have its own "analytics home page" then I think that's what the "child sites" feature is for.

When martin said:

The same GoatCounter is accessed on different domains: e.g. stats.example.com, other.mysite.com, etc. You need to access the correct domain to access the correct site.

They were kinda assuming that you already understand how DNS / subdomains / virtual hosts work. The jist is, domain names and digitalocean droplets are two completely separate things. Also, not all domain names are the same. There are two different kinds.

First of all there are the "2nd level domain" names like mysite.com. They are called second level because they have two levels, the top level domain (TLD) like .com and the second level mysite, the part you register. These are domains you have to pay for, they are essentially just line-items in the database of a public trust institution called a DNS registrar, similar to a Bank.

Then there are the "subdomains" like goatcounter.mysite.com, and foosite.goatcounter.mysite.com. Do you see where we're going with this? You can create as many subdomains as you want for free once you have a 2nd level domain registered. They can be nested as deep as you want too, like a.b.c.d.mysite.com. Typically you do this by logging into your DNS provider's web management portal and navigating to the "manage DNS records" section (sometimes they hide it under "Advanced" 😫 ). Then you would create A Records with names like goatcounter and foosite.goatcounter and for the value, you would put the IP address of your digitialocean droplet. Those A records will create the subdomains goatcounter.mysite.com, and foosite.goatcounter.mysite.com. Then you can create the corresponding "child sites" in goatcounter with the same domain names. You may also have to configure each child site subdomain in your reverse proxy & obtain a TLS certificate for each one.

Another option which I decided to use, you can put the domain name (or a shortened alias for the domain name) into the "path" property of the events you send to Goatcounter. One single goatcounter site is not designed to be used on multiple domains yet but you can use a bit of a "hack" to make it work by prepending the domain name onto the URI. (By default goatcounter use the URI as the aggregation key).

There is an example of how to do this one the "site code" page of your goatcounter instance:

Multiple domains

GoatCounter doesn’t store the domain a pageview belongs to; if you add GoatCounter to several (sub)domain then there’s no way to distinguish between requests to a.example.com/path and b.example.com/path as they’re both recorded as /path.
This might be improved at some point in the future; the options right now are:
Create a new “additional site” for every domain; this is a completely separate site which inherits the user, login, plan, etc. You will need to use a different site code for every (sub)domain.
If you want everything in a single overview then you can add the domain to the path, instead of just sending the path:

 <script>
     window.goatcounter = {
         path: function(p) { return location.host + p }
     }
 </script>
 <script data-goatcounter="https://goatcounter.sequentialread.com/count"
         async src="//goatcounter.sequentialread.com/count.js"></script>

This is what I opted to use because on my site, I have a lot of subdomains that are all technically part of the same "site".

So I used a short alias to make the domain part take up less space, instead of return location.host + p I used something more like return replace(location.host, "sequentialread.com", "sqr") + p

Had to come back to this one years later, I still don't understand how @ForestJohnson was so confused 😂 English appears to be his first language and he struggled completely here