Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to get IP geolocation working #8

Closed
birjj opened this issue Oct 28, 2020 · 16 comments
Closed

Unable to get IP geolocation working #8

birjj opened this issue Oct 28, 2020 · 16 comments

Comments

@birjj
Copy link
Contributor

birjj commented Oct 28, 2020

I'm having trouble getting IP geolocation to work. I've done a bit of debugging, and found that the geoip volume isn't mounted to the plausible container by default (as has someone else, it seems #7). After fixing this however, it still doesn't appear to work - no data is shown on the map, nor is there a tooltip when hovering.

So far I've tried:

  1. Update geoip/docker-compose.geoip.yml so the geoip volume is mounted at /geoip for the plausible service.
    index 239577c..04ebf28 100644
    --- a/geoip/docker-compose.geoip.yml
    +++ b/geoip/docker-compose.geoip.yml
    @@ -3,6 +3,8 @@ services:
       plausible:
         depends_on:
           - geoip
    +    volumes:
    +      - geoip:/geoip:ro
         environment:
           - GEOLITE2_COUNTRY_DB=/geoip/GeoLite2-Country.mmdb

    Then reload using docker-compose -f docker-compose.yml -f geoip/docker-compose.geoip.yml up -d.

  2. Check that the file actually exists within the plausible container.
    $ docker exec -it plausible_plausible_1 /bin/bash
    # ls -l /geoip/
    total 3836
    -rw-r--r-- 1 root root 3926614 Oct 28 00:13 GeoLite2-Country.mmdb
    # echo $GEOLITE2_COUNTRY_DB
    /geoip/GeoLite2-Country.mmdb
    
  3. Try completely from fresh
    $ docker-compose -f docker-compose.yml -f geoip/docker-compose.geoip.yml down -v
    Stopping plausible_plausible_1           ... done
    Stopping plausible_plausible_events_db_1 ... done
    Stopping plausible_mail_1                ... done
    Stopping plausible_plausible_db_1        ... done
    Stopping plausible_geoip_1               ... done
    Removing plausible_plausible_1           ... done
    Removing plausible_plausible_events_db_1 ... done
    Removing plausible_mail_1                ... done
    Removing plausible_plausible_db_1        ... done
    Removing plausible_geoip_1               ... done
    Removing network plausible_default
    Removing volume plausible_db-data
    Removing volume plausible_event-data
    Removing volume plausible_geoip
    $ docker-compose -f docker-compose.yml -f geoip/docker-compose.geoip.yml up -d
    Creating network "plausible_default" with the default driver
    Creating volume "plausible_db-data" with local driver
    Creating volume "plausible_event-data" with local driver
    Creating volume "plausible_geoip" with local driver
    Creating plausible_plausible_db_1        ... done
    Creating plausible_mail_1                ... done
    Creating plausible_geoip_1               ... done
    Creating plausible_plausible_events_db_1 ... done
    Creating plausible_plausible_1           ... done
    
  4. Check logs
    $ docker logs plausible_plausible_1
    Loading plausible..
    Starting dependencies..
    Starting repos..
    create Plausible.Repo database if it doesn't exist
    00:55:06.992 [error] Postgrex.Protocol (#PID<0.238.0>) failed to connect: ** (Postgrex.Error) FATAL 3D000 (invalid_catalog_name) database "plausible_db" does not exist
    00:55:06.992 [error] Postgrex.Protocol (#PID<0.237.0>) failed to connect: ** (Postgrex.Error) FATAL 3D000 (invalid_catalog_name) database "plausible_db" does not exist
    00:55:07.126 [error] Clickhousex.Protocol (#PID<0.244.0>) failed to connect: ** (ErlangError) Erlang error: "Code: 81, e.displayText() = DB::Exception: Database plausible_events_db doesn't exist (version 20.10.2.20 (official build))\n"
    00:55:07.126 [error] Clickhousex.Protocol (#PID<0.245.0>) failed to connect: ** (ErlangError) Erlang error: "Code: 81, e.displayText() = DB::Exception: Database plausible_events_db doesn't exist (version 20.10.2.20 (official build))\n"
    create Plausible.ClickhouseRepo database if it doesn't exist
    Creation of Db successful!
    Loading plausible..
    Starting dependencies..
    Starting repos..
    Running migrations for Elixir.Plausible.Repo
    Running migrations for Elixir.Plausible.ClickhouseRepo
    Migrations successful!
    Loading plausible..
    Starting dependencies..
    Starting repos..
    Admin user created successful!
    01:07:01.181 [error] Task #PID<0.4448.0> started from #PID<0.4449.0> terminating
    ** (CaseClauseError) no case clause matching: []
        (hackney 1.15.2) /app/deps/hackney/src/hackney_url.erl:224: :hackney_url.parse_netloc/2
        (hackney 1.15.2) /app/deps/hackney/src/hackney.erl:349: :hackney.request/5
        (httpoison 1.6.2) lib/httpoison/base.ex:796: HTTPoison.Base.request/6
        (httpoison 1.6.2) lib/httpoison.ex:128: HTTPoison.request!/5
        (elixir 1.10.3) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2
        (stdlib 3.12.1) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
    Function: #Function<0.124948693/0 in Plausible.Slack.notify/1>
        Args: []
    

At this point I feel like more in-depth debugging is needed. I've verified that the file exists where Plausible is told it is, yet no data is returned from the /api/stats/:domain/countries endpoint.

Other than that, Plausible is amazing so far; major props for putting a self-hosted version out there.

@birjj
Copy link
Contributor Author

birjj commented Oct 28, 2020

If it helps, my Plausible instance is public at https://plausible.jfagerberg.me/jfagerberg.me

@ukutaht
Copy link
Contributor

ukutaht commented Oct 28, 2020

Hey, thanks for the report.

I maintain a testing instance for the self-hosted configuration over at testing.plausible.io (our prod setup is a bit different). So far I didn't have geoip configured so I thought I would follow the same steps to see what works.

After updating the geoip/docker-compose.geoip.yml with the volume and the geoip/geoip.conf with my credentials, I rebooted the system with docker-compose down and then docker-compose -f docker-compose.yml -f geoip/docker-compose.geoip.yml up -d.

As you can see, countries are being captured now: https://testing.plausible.io/plausible.io

Unfortunately without being able to reproduce the issue, it's difficult to debug. My first thought is that maybe you're sitting behind a proxy that doesn't implement X-Forwarded-For correctly? This would throw off the Plausible server, not just for countries but also for unique user counting. Let me know if you think that's a possibility.

Otherwise we can think about adding some logging around the country lookup and you could deploy that image with log_level: :debug to get some more intel.

@birjj
Copy link
Contributor Author

birjj commented Oct 28, 2020

@ukutaht Looks like that was it! Thanks for the pointer; I hadn't even considered my reverse proxy being the problem.

Would there be interest in a PR adding documentation for how to setup NGINX in front of Plausible in the self-hosting docs? I see that the docs mention there being instructions "below", but I couldn't find them. Alternatively (or additionally) a pre-made NGINX config could be added to this repo if there is interest.

@birjj birjj closed this as completed Oct 28, 2020
@ukutaht
Copy link
Contributor

ukutaht commented Oct 29, 2020

Yeah I think we'll get a PR with configuration for Traefik soon: https://plausible.discourse.group/t/self-hosting-https-how-to/87/5

I'd also love to have read-made configs for Nginx and Caddy.

Just out of curiosity, how was your nginx configured? What change made it work?

@birjj
Copy link
Contributor Author

birjj commented Oct 29, 2020

@ukutaht My NGINX config is super simple. The part that isn't managed by Certbot is:

server {
	server_name plausible.jfagerberg.me;

	location / {
		proxy_pass http://127.0.0.1:7654;

		# added line below to fix issue
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	}
}

(where 7654 is the port I'm binding Plausible to)


I'll hopefully get around to throwing together a PR for the self-hosting docs sometimes tomorrow. I'm assuming a new page under the self-hosting category would be preferable over adding it as a subsection of the Getting started page?

@ukutaht
Copy link
Contributor

ukutaht commented Oct 29, 2020

Looks good! I wish nginx automatically sent X-Forwarded-For when using proxy_pass

Would be best to add a new docker-compose override in addition to adding documentation. I think runnable code is the best way to document something.

@birjj
Copy link
Contributor Author

birjj commented Oct 29, 2020

If we're adding a docker-compose override, then I'm assuming that the goal is to be as plug-and-play as possible. Since it doesn't really make much sense to offer multiple different reverse proxy images, it'd probably be best to choose between caddy-gen for its simplicity, nginx-proxy for being semi-simple and well-maintained, or Traefik for its maturity (and then hope that anyone who's running an existing reverse proxy know how to configure it ;) )

I'll try to get around to it tomorrow.

@ukutaht
Copy link
Contributor

ukutaht commented Oct 29, 2020

I'd be quite happy to offer all 3 as well and let the user pick depending on which docker-compose override file they include

@birjj
Copy link
Contributor Author

birjj commented Oct 29, 2020

It's obviously your decision in the end, but may I ask why? Since running a reverse proxy as a docker container only makes sense if users aren't running a reverse proxy to begin with (otherwise it will clash with the existing instance), there's very little point in letting people choose, assuming all the reverse proxies have feature parity which they do for this case.

I can see the point in providing configs for both Caddy, NGINX and Traefik so users can choose the appropriate one if they're already running a reverse proxy - but bundling the reverse proxy as a docker-compose override would only make sense if they aren't running one to begin with.

@ukutaht
Copy link
Contributor

ukutaht commented Oct 30, 2020

@birjolaxew I think you're right, it's best to choose one to bundle in the docker-compose and provide example configs for different servers for when they run separately.

@casperdcl
Copy link

Hey I'm using plausible with traefik and can't get location working...

docker compose logs plausible
plausible-1  | Loading plausible..
plausible-1  | Starting dependencies..
plausible-1  | Starting repos..
plausible-1  | create Plausible.Repo database if it doesn't exist
plausible-1  | create Plausible.IngestRepo database if it doesn't exist
plausible-1  | Creation of Db successful!
plausible-1  | Loading plausible..
plausible-1  | Starting dependencies..
plausible-1  | Starting repos..
plausible-1  | Running migrations for Elixir.Plausible.Repo
plausible-1  | 14:38:56.439 [info] Migrations already up
plausible-1  | Running migrations for Elixir.Plausible.IngestRepo
plausible-1  | Migrations successful!
plausible-1  | 14:38:56.455 [info] Migrations already up
plausible-1  | 14:38:59.541 [info] Loading 147 CA(s) from :otp store
plausible-1  | 14:38:59.702 [info] No tzdata release files found in custom data dir. Copying release file from tzdata priv dir.

According to https://doc.traefik.io/traefik/getting-started/faq/#what-are-the-forwarded-headers-when-proxying-http-requests I think X-Forwarded-For should be present, but I added entryPoints.websecure.forwardedHeaders.insecure just to be sure (from https://doc.traefik.io/traefik/routing/entrypoints/#forwarded-headers).

Any suggestions? I'm recording events, but no locations (db-ip & MaxMind).

@ruslandoga
Copy link
Contributor

ruslandoga commented Jun 8, 2024

👋 @casperdcl

You can use this snippet to log headers and check if Plausible receives a correct IP address in x-forwarded-for from Traefik

iex> headers = fn {:trace, _pid, :call, {_mod, _fun, [%Plug.Conn{req_headers: headers}]}} -> inspect(headers) end
iex> mod_fun_args = {PlausibleWeb.RemoteIP, :get, 1}
iex> how_many_times = 5
iex> :recon_trace.calls(mod_fun_args, how_many_times, formatter: headers)

More info: plausible/analytics#2767 (reply in thread) plausible/analytics#3248 (comment)

@casperdcl
Copy link

casperdcl commented Jun 8, 2024

hmm I get 192.168.96.1 as the IP address, which is wrong (probably something to do with docker?)...

curl http event

curl -H Host:$BASE_URL localhost:8000/api/event -d '{"name": "pageview","url":"https://dummy.site/baldur","domain":"dummy.site","referrer":null,"screen_width":520}' -H 'content-type: application/json'

[{"accept", "*/*"}, {"accept-encoding", "gzip"}, {"content-length", "111"},
 {"content-type", "application/json"}, {"host", "$BASE_URL"}, {"user-agent", "curl/7.81.0"},
 {"x-forwarded-for", "192.168.96.1"}, {"x-forwarded-host", "$BASE_URL"},
 {"x-forwarded-port", "80"}, {"x-forwarded-proto", "http"},
 {"x-forwarded-server", "17cb132c672d"}, {"x-real-ip", "192.168.96.1"}]
real https event
[{"accept", "*/*"}, {"accept-encoding", "gzip, deflate, br, zstd"},
 {"accept-language", "en-GB,en;q=0.5"}, {"content-length", "72"},
 {"content-type", "text/plain"}, {"dnt", "1"},
 {"forwarded", "for=192.168.96.1;host=$BASE_URL;proto=https"},
 {"host", "$BASE_URL"}, {"origin", "$DATA_DOMAIN"}, {"referer", "$DATA_DOMAIN"},
 {"sec-fetch-dest", "empty"}, {"sec-fetch-mode", "cors"}, {"sec-fetch-site", "cross-site"},
 {"sec-gpc", "1"}, {"te", "trailers"}, {"user-agent", "$UA"}, {"x-forwarded-for", "192.168.96.1"},
 {"x-forwarded-host", "$BASE_URL"}, {"x-forwarded-port", "443"}, {"x-forwarded-proto", "https"},
 {"x-forwarded-server", "86b72bba7a44"}, {"x-real-ip", "192.168.96.1"}]

@ruslandoga
Copy link
Contributor

ruslandoga commented Jun 8, 2024

You seem to be accessing Plausible from the same host as its running on (curl localhost), so it makes sense that the client IP address would be local. I suggest trying routing the request via the public internet and maybe sharing more information about your setup. In previous cases (I linked them above) the problem was always at the reverse-proxy(ies) level.

@casperdcl
Copy link

casperdcl commented Jun 8, 2024

The "real https event" was via public internet (browser pointed at $DATA_DOMAIN) yet still didn't receive my public IP.

My setup is quite minimal (I just cloned the repo, set BASE_URL, SECRET_KEY_BASE, TOTP_VAULT_KEY, ran with -f docker-compose.yml -f reverse-proxy/traefik/docker-compose.traefik.yml)

Of course I had to add an actual traefik service in docker-compose.traefik.yml too:

services:
  plausible:
    labels:
      traefik.http.routers.plausible.rule: Host(`$BASE_URL`)
      traefik.http.services.plausible.loadbalancer.server.port: 8000
      traefik.http.routers.plausible.tls: true
      traefik.http.routers.plausible.tls.certresolver: myresolver
  traefik:
    image: traefik:3.0
    command:
    - --providers.docker
    - --entryPoints.web.address=:80
    - --entryPoints.websecure.address=:443
    - --certificatesresolvers.myresolver.acme.tlschallenge=true
    - --certificatesresolvers.myresolver.acme.email=me@cdcl.ml
    - --certificatesresolvers.myresolver.acme.storage=/etc/traefik/acme/acme.json
    ports: ["80:80", "443:443"]
    volumes: [/var/run/docker.sock:/var/run/docker.sock, ./data/.traefik:/etc/traefik/acme]

@casperdcl
Copy link

casperdcl commented Jun 9, 2024

I just switched to caddy and still have the same problem.

Perhaps it's because I'm using a jump server?

$BASE_URL:~$ ssh -fNTL 4443:localhost:443 private-machine
$BASE_URL:~$ sudo socat TCP-LISTEN:443,fork TCP:localhost:4443

I wonder if forwarding private-machine:443 to $BASE_URL:443 using ssh & socat results in the IP 192.168.96.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants