Setup successfullly, but users not receiving notifications

[whisperdancer]

Nextcloud Server:
Nextcloud version (eg, 20.0.5): 21
Operating system and version (eg, Ubuntu 20.04): Raspbian GNU/Linux 10 (buster)
Apache or nginx version (eg, Apache 2.4.25): nginx/1.14.2
PHP version (eg, 7.4): v7.4.15

Nginx Reverse Proxy ( on a separate physical server ):

Note, I have not added a location block to the nginx reverse proxy as I'm unsure the correct way to do that. I surmise that may be the issue of why it isn't working.

On the nextcloud server:
I setup the notify_push service successfully. As least according to the setup wizard.

push server seems to be functioning correctly
reverse proxy seems to be setup already
reverse proxy seems to be setup correctly
configuration saved.

I tested the service with sudo service notify_push status and it's active and running so all good there

I then tested with 2 users using Nextcloud Talk. User #1 initiates a conversation with User #2. User #2 does not receive a notification. User #1 mentions User #2 and User #2 does not receive a notification.

I checked the logs within nextcloud and no errors reported.

In your docs, you mention something about Metrics as a way to troubleshoot, but do not provide instructions for how to do that. Perhaps that would help troubleshoot. In addition, what do you suggest in terms of troubleshooting the issue.

Metrics
The push server can expose some basic metrics about the number of connected clients and the traffic flowing through the server by setting the METRICS_PORT environment variable.
Once set the metrics are available in a prometheus compatible format at /metrics on the configured port.

Thank you.

[icewind1991]

You can run occ notify_push:self-test to re-do the setup tests to be sure.

You can download the test_client from https://github.com/nextcloud/notify_push/actions/runs/606033251 and run it as

test_client https://cloud.example.com username password

which will print a message every time a push notification is received. (If you have 2fa or a non-standard login you'll have to use an app password)

[whisperdancer]

I'd love to do this, but unsure how. Can you please provide instructions for how to install the test_client on my server?

Also, I ran the test you suggested and all passed

sudo -u www-data php occ notify_push:self-test
✓ redis is configured
✓ push server is receiving redis messages
✓ push server can load mount info from database
✓ push server can connect to the Nextcloud server
✓ push server is a trusted proxy
✓ push server is running the same version as the app

But users still not receiving notifications on Talk test

[szaimen]

The test client instructions were just updated:
https://github.com/nextcloud/notify_push#test-client

[whisperdancer]

Please note: the Test client only works on x86_64 Linux currently.

Unfortunately, I don't have a server that can run the test client. Any other ideas on how I can troubleshoot this issue. Notifications are currently broken for all users.

Thanks

[szaimen]

It doesn't have to be a server it also can be any other PC or VM with x86_64 Linux. The important thing is that it can reach your server via https.

[whisperdancer]

The only Linux servers I have are Rasp Pi's. They are not x86_64. They are Arm7

[szaimen]

Yes, but the test_client doesn't need to get installed on your server!

[whisperdancer]

Would it work on a RaspPi4?

[szaimen]

Would it work on a RaspPi4?

No

So you have not even one PC with x86_64 architecture at hand?
(you could e.g. also spin up a Linux VM on a x86_64 Windows PC)

[whisperdancer]

Yes, I have an intel I7 laptop running Win 10, but it's not Linux. All Linux stuff is done on the Pi's

[szaimen]

Okay, so in this case, just spin up a 64-bit Linux VM on the laptop and run the test_client there!
Or you could also boot up 64-bit Linux from a USB-stick on that laptop and run the binary there.

[whisperdancer]

What do you suggest is the best way to spin up a 64 bit Linux VM on a Win 10 machine?

[alerque]

VirtualBox is an easy open source way. Running a LiveCD from an ISO image is pretty straight forward including a graphical UI. Docker is another way if you happen to use it for something else and have it already set up and keeps things simpler with just a shell interface. If you do this kind of stuff from Windows a lot you probably want to setup WSL, which would let you run this "natively" in your Windows environment.

[whisperdancer]

I currently don't have VirtualBox installed on my laptop, but can install it. How about Windows subsystem for Linux as on option?

[szaimen]

Yes, this should work, too.
Here is an example guide:
https://maurogiusti.medium.com/running-ubuntu-on-windows-10-with-wsl2-c4f06b3c353
(didn't test it myself)

[alerque]

How about Windows subsystem for Linux as on option?

I offered three possible solutions. WSL was one of them.

[whisperdancer]

I have another idea which I didn't initially think of. Can I spin up an Ubuntu 20.04 LTS instance on AWS Lightsail and use that to run the test client?

[szaimen]

That should theoretically work, too. But at least I have no experience in that.

[whisperdancer]

Good news! uname -mrs
Linux 5.4.0-1030-aws x86_64

This should work, correct?

[szaimen]

If sudo uname -p returns x86_64 you are safe to go 👍

[whisperdancer]

Yay! sudo uname -p
x86_64

[whisperdancer]

Ok, instance setup and verified x86_64. I ran into an issue. I tried to download the test client
wget https://github.com/nextcloud/notify_push/suites/2162868561/artifacts/44388618

Resolving github.com (github.com)... 192.30.255.113
Connecting to github.com (github.com)|192.30.255.113|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2021-03-03 20:22:26 ERROR 404: Not Found.

Note, I'm not a rust developer so may need a bit of guidance on how to get this test-client setup and running. Thanks in advance!

[szaimen]

It seems like you need to be logged in to github to be able to download the file. So a simple wget won't work.
Since that seems to be the case, I guess a Linux VM or e.g. WSL with GUI will be the better option.

[whisperdancer]

Can I download it to my windows laptop, then file transfer it to the instance?

[szaimen]

Yes, should also be possible via ssh. But I've myself never done that.

[szaimen]

Alternatively you could upload the file to your nextcloud, share it vial link (without password protection) and download the file using the public download link...

[whisperdancer]

I solved it. I used WinSCP and now have the file on my ubuntu server. Do I just run it like this:
sudo ./test_client https://cloud.example.com username password
or do I need to install any dependencies first?

[szaimen]

or do I need to install any dependencies first?

No

sudo ./test_client https://cloud.example.com username password

Yes, this should work. (though probably you need to use an app-password as password.)

[whisperdancer]

I created a test account on the server for just this purpose. No 2FA for this user. Do I need to first make test_client executable?

[whisperdancer]

BTW: could you please share some more infos about your setup? I have the feeling that something else is broken here.
I need the output of occ config:list system and occ app:list

Do you want me to post that here?

[szaimen]

Yes, since it is needed for debugging purposes.

[szaimen]

Thanks for posting!
Unfortunately there isn't anything obvious which could explain the problems that you are experiencing.
But most likely the other bugs that you've experienced so far with NC21 should get fixed first (apart from if they are setup or NC21 related), before you should continue trying to get this app working.

[whisperdancer]

Thanks for posting!
Unfortunately there isn't anything obvious which could explain the problems that you are experiencing.
But most likely the other bugs that you've experienced so far with NC21 should get fixed first (apart from if they are setup or NC21 related), before you should continue trying to get this app working.

I setup my nextcloud using one of Carsten's tutorials. He really knows his stuff. So I figure my setup is good.

[szaimen]

So we should abandon trying to debug this app for now? And revisit this down the road? Perhaps after the first few maintenance releases?

Yes, that sounds like a good plan 👍

[whisperdancer]

So we should abandon trying to debug this app for now? And revisit this down the road? Perhaps after the first few maintenance releases?

Yes, that sounds like a good plan 👍

Ok, How should I reach out to you again?

[szaimen]

Just write here again. I mean just try again when the maintenance release is out and report back if it's working ;)

[whisperdancer]

Just write here again. I mean just try again when the maintenance release is out and report back if it's working ;)

Ok, will do. Thanks for taking the time to help today!

[szaimen]

It seems like the test client was just updated: #51 (comment)

I've updated the test client a bit to give better info about what is going wrong, it can be found at https://github.com/nextcloud/notify_push/actions/runs/620642603

[rakurtz]

i already used that above new test_client and also updated to the newest talk app but getting the invalid credentials error with the test_client.

[whisperdancer]

Same as rakurtz
I created new test account and test it, logged successfully into nc. Then ran test:
Tried with new test_client and get this error:
Error:
0: capabilities response doesn't contain expect items, credentials are probably invalid

I added this code block to nginx on the reverse proxy:
location /push/ {
proxy_pass http://ip_nc_server:7867/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

and opened port 7867 on the nc server.

What am I doing wrong?

[szaimen]

Unfortunately, I don't know :/

Edit: sry for closing. Wasn't intentional.

[icewind1991]

Try running the test client with LOG=debug

[whisperdancer]

Try running the test client with LOG=debug

How? Can you please provide the full syntax to run it with LOG=debug. Thank you.

[szaimen]

It should be:
LOG=debug ./test_client https://cloud.example.com username password

[whisperdancer]

szaimen, thanks for the tip on how to run the test. Test run with LOG=debug and output below: (note, ran test with my actual domain, substituted fake domain my.ncdomain.com for the paste here)

LOG=debug ./test_client https://my.ncdomain.com username password

DEBUG [rustls::client::hs] No cached session for DNSNameRef("my.ncdomain.com")
DEBUG [rustls::client::hs] Not resuming any session
INFO [ureq::unit] sending request GET https://my.ncdomain.com/ocs/v2.php/cloud/capabilities
DEBUG [ureq::unit] writing prelude: GET /ocs/v2.php/cloud/capabilities HTTP/1.1
Host: my.ncdomain.com
User-Agent: ureq/1.5.4
Authorization: Basic ZGVsZXRlbWU6M1pMZzRidmQ4XmNNKg==
Accept: application/json
OCS-APIREQUEST: true


DEBUG [rustls::client::hs] Using ciphersuite TLS13_CHACHA20_POLY1305_SHA256
DEBUG [rustls::client::tls13] Not resuming
DEBUG [rustls::client::tls13] TLS1.3 encrypted extensions: [ServerNameAck]
DEBUG [rustls::client::hs] ALPN protocol is None
DEBUG [rustls::client::tls13] Ticket saved
DEBUG [rustls::client::tls13] Ticket saved
DEBUG [cookie_store::cookie_store] inserting Set-Cookie 'Cookie { cookie_string: Some("oc_sessionPassphrase=ry%2BuYTG7XeqHAg%2Bm3TbwS2jp0Ip2UpGWRT19tLAdJPIPEvrT6kltYE95aCYLjSv%2FafCDnuIhFeW1Z6n1XpsMBORMSlRLm3YFRW1aTthKh4BKGelmA3rDZqdjzJuWgSH%2B; path=/; secure; HttpOnly; SameSite=Lax"), name: Indexed(0, 20), value: Indexed(21, 157), expires: None, max_age: None, domain: None, path: Some(Indexed(164, 165)), secure: Some(true), http_only: Some(true), same_site: Some(Lax) }'
DEBUG [cookie_store::cookie_store] inserting Set-Cookie 'Cookie { cookie_string: Some("__Host-nc_sameSiteCookielax=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=lax"), name: Indexed(0, 27), value: Indexed(28, 32), expires: Some(OffsetDateTime { utc_datetime: PrimitiveDateTime { date: Date { year: 2100, ordinal: 365 }, time: Time { hour: 23, minute: 59, second: 59, nanosecond: 0 } }, offset: UtcOffset { seconds: 0 } }), max_age: None, domain: None, path: Some(Indexed(39, 40)), secure: Some(true), http_only: Some(true), same_site: Some(Lax) }'
DEBUG [cookie_store::cookie_store] inserting Set-Cookie 'Cookie { cookie_string: Some("__Host-nc_sameSiteCookiestrict=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=strict"), name: Indexed(0, 30), value: Indexed(31, 35), expires: Some(OffsetDateTime { utc_datetime: PrimitiveDateTime { date: Date { year: 2100, ordinal: 365 }, time: Time { hour: 23, minute: 59, second: 59, nanosecond: 0 } }, offset: UtcOffset { seconds: 0 } }), max_age: None, domain: None, path: Some(Indexed(42, 43)), secure: Some(true), http_only: Some(true), same_site: Some(Strict) }'
DEBUG [cookie_store::cookie_store] inserting Set-Cookie 'Cookie { cookie_string: Some("oc4yk3ed0ma4=lmsla2h8jvbig1s429au0cu82q; path=/; secure; HttpOnly; SameSite=Lax"), name: Indexed(0, 12), value: Indexed(13, 39), expires: None, max_age: None, domain: None, path: Some(Indexed(46, 47)), secure: Some(true), http_only: Some(true), same_site: Some(Lax) }'
DEBUG [ureq::unit] response 200 to GET https://my.ncdomain.com/ocs/v2.php/cloud/capabilities
DEBUG [test_client] Capabilities response: {"ocs":{"meta":{"status":"ok","statuscode":200,"message":"OK"},"data":{"version":{"major":21,"minor":0,"micro":0,"string":"21.0.0","edition":"","extendedSupport":false},"capabilities":{"bruteforce":{"delay":400},"spreed":{"features":["audio","video","chat-v2","conversation-v2","guest-signaling","empty-group-room","guest-display-names","multi-room-users","favorites","last-room-activity","no-ping","system-messages","delete-messages","mention-flag","in-call-flags","conversation-call-flags","notification-levels","invite-groups-and-mails","locked-one-to-one-rooms","read-only-rooms","listable-rooms","chat-read-marker","webinary-lobby","start-call-flag","chat-replies","circles-support","force-mute","conversation-v3","sip-support","chat-read-status","phonebook-search","raise-hand","room-description","rich-object-sharing","chat-reference-id"],"config":{"attachments":{"allowed":false},"chat":{"max-length":32000,"read-privacy":0},"conversations":{"can-create":false},"previews":{"max-gif-size":3145728}}}}}}}
INFO [test_client] Supported capabilities: ["bruteforce", "spreed"]
Error:
   0: capabilities response doesn't contain expect items, credentials are probably invalid

Location:
   test_client/src/main.rs:79

[Schmuuu]

I have the same issue and just post my output from the client here as well, hoping it helps:

$ LOG=debug ./test_client https://cloud.mydomain.tld MyName <app-password>
DEBUG [rustls::client::hs] No cached session for DNSNameRef("cloud.mydomain.tld")
DEBUG [rustls::client::hs] Not resuming any session
INFO [ureq::unit] sending request GET https://cloud.mydomain.tld/ocs/v2.php/cloud/capabilities
DEBUG [ureq::unit] writing prelude: GET /ocs/v2.php/cloud/capabilities HTTP/1.1
Host: cloud.mydomain.tld
User-Agent: ureq/1.5.4
Authorization: Basic S3Jpc3RpYW46SDV0d0Itd0JZNzctN0VzNzUtUGpxekUtM0NMSlE=
Accept: application/json
OCS-APIREQUEST: true


DEBUG [rustls::client::hs] Using ciphersuite TLS13_CHACHA20_POLY1305_SHA256
DEBUG [rustls::client::tls13] Not resuming
DEBUG [rustls::client::tls13] TLS1.3 encrypted extensions: [ServerNameAck, Unknown(UnknownExtension { typ: EllipticCurves, payload: Payload([0, 8, 0, 30, 0, 29, 0, 24, 0, 25]) })]
DEBUG [rustls::client::hs] ALPN protocol is None
DEBUG [rustls::client::tls13] Ticket saved
DEBUG [rustls::client::tls13] Ticket saved
DEBUG [cookie_store::cookie_store] inserting Set-Cookie 'Cookie { cookie_string: Some("oc_sessionPassphrase=%2BX1NYeGrsRHrad80AX4%2BYdNEwvSBYkmMGXLJ8hbAu31oBYNTnjFxaXj7nyGXlfesDVmfmKJYarytNFAmlh%2BKtOPkKKb1JATnDeW1lna7YijUngq5ZobtK%2BYezbTRhk9X; path=/; secure; HttpOnly; SameSite=Lax"), name: Indexed(0, 20), value: Indexed(21, 157), expires: None, max_age: None, domain: None, path: Some(Indexed(164, 165)), secure: Some(true), http_only: Some(true), same_site: Some(Lax) }'
DEBUG [cookie_store::cookie_store] inserting Set-Cookie 'Cookie { cookie_string: Some("__Host-nc_sameSiteCookielax=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=lax"), name: Indexed(0, 27), value: Indexed(28, 32), expires: Some(OffsetDateTime { utc_datetime: PrimitiveDateTime { date: Date { year: 2100, ordinal: 365 }, time: Time { hour: 23, minute: 59, second: 59, nanosecond: 0 } }, offset: UtcOffset { seconds: 0 } }), max_age: None, domain: None, path: Some(Indexed(39, 40)), secure: Some(true), http_only: Some(true), same_site: Some(Lax) }'
DEBUG [cookie_store::cookie_store] inserting Set-Cookie 'Cookie { cookie_string: Some("__Host-nc_sameSiteCookiestrict=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=strict"), name: Indexed(0, 30), value: Indexed(31, 35), expires: Some(OffsetDateTime { utc_datetime: PrimitiveDateTime { date: Date { year: 2100, ordinal: 365 }, time: Time { hour: 23, minute: 59, second: 59, nanosecond: 0 } }, offset: UtcOffset { seconds: 0 } }), max_age: None, domain: None, path: Some(Indexed(42, 43)), secure: Some(true), http_only: Some(true), same_site: Some(Strict) }'
DEBUG [cookie_store::cookie_store] inserting Set-Cookie 'Cookie { cookie_string: Some("oc4679w80k55=e3jl3rlt9lpfmuoj3ihsha8uar; path=/; secure; HttpOnly; SameSite=Lax"), name: Indexed(0, 12), value: Indexed(13, 39), expires: None, max_age: None, domain: None, path: Some(Indexed(46, 47)), secure: Some(true), http_only: Some(true), same_site: Some(Lax) }'
DEBUG [ureq::unit] response 200 to GET https://cloud.mydomain.tld/ocs/v2.php/cloud/capabilities
DEBUG [test_client] Capabilities response: {"ocs":{"meta":{"status":"ok","statuscode":200,"message":"OK"},"data":{"version":{"major":21,"minor":0,"micro":0,"string":"21.0.0","edition":"","extendedSupport":false},"capabilities":{"core":{"pollinterval":60,"webdav-root":"remote.php\/webdav"},"bruteforce":{"delay":0},"files":{"bigfilechunking":true,"blacklisted_files":[".htaccess","Thumbs.db","thumbs.db"],"directEditing":{"url":"https:\/\/cloud.mydomain.tld\/ocs\/v2.php\/apps\/files\/api\/v1\/directEditing","etag":"35ea4439bbb54832fead4a0634cc3339"},"comments":true,"undelete":true,"versioning":true},"activity":{"apiv2":["filters","filters-api","previews","rich-strings"]},"ocm":{"enabled":true,"apiVersion":"1.0-proposal1","endPoint":"https:\/\/cloud.mydomain.tld\/ocm","resourceTypes":[{"name":"file","shareTypes":["user","group"],"protocols":{"webdav":"\/public.php\/webdav\/"}}]},"dav":{"chunking":"1.0"},"deck":{"version":"1.3.1","canCreateBoards":true,"apiVersions":["1.0","1.1"]},"external":{"v1":["sites","device","groups","redirect"]},"notes":{"api_version":["0.2","1.1"],"version":"4.0.4"},"notifications":{"ocs-endpoints":["list","get","delete","delete-all","icons","rich-strings","action-web","user-status"],"push":["devices","object-data","delete"],"admin-notifications":["ocs","cli"]},"notify_push":{"type":["files","activities","notifications"],"endpoints":{"websocket":"wss:\/\/cloud.mydomain.tld\/push\/ws","pre_auth":"https:\/\/cloud.mydomain.tld\/apps\/notify_push\/pre_auth"}},"password_policy":{"minLength":12,"enforceNonCommonPassword":true,"enforceNumericCharacters":true,"enforceSpecialCharacters":true,"enforceUpperLowerCase":true,"api":{"generate":"https:\/\/cloud.mydomain.tld\/ocs\/v2.php\/apps\/password_policy\/api\/v1\/generate","validate":"https:\/\/cloud.mydomain.tld\/ocs\/v2.php\/apps\/password_policy\/api\/v1\/validate"}},"files_sharing":{"sharebymail":{"enabled":true,"upload_files_drop":{"enabled":true},"password":{"enabled":true,"enforced":false},"expire_date":{"enabled":true}},"api_enabled":true,"public":{"enabled":true,"password":{"enforced":false,"askForOptionalPassword":false},"expire_date":{"enabled":true,"days":"14","enforced":false},"multiple_links":true,"expire_date_internal":{"enabled":false},"send_mail":true,"upload":true,"upload_files_drop":true},"resharing":false,"user":{"send_mail":false,"expire_date":{"enabled":true}},"group_sharing":true,"group":{"enabled":true,"expire_date":{"enabled":true}},"default_permissions":5,"federation":{"outgoing":true,"incoming":true,"expire_date":{"enabled":true}},"sharee":{"query_lookup_default":false}},"spreed":{"features":["audio","video","chat-v2","conversation-v2","guest-signaling","empty-group-room","guest-display-names","multi-room-users","favorites","last-room-activity","no-ping","system-messages","delete-messages","mention-flag","in-call-flags","conversation-call-flags","notification-levels","invite-groups-and-mails","locked-one-to-one-rooms","read-only-rooms","listable-rooms","chat-read-marker","webinary-lobby","start-call-flag","chat-replies","circles-support","force-mute","conversation-v3","sip-support","chat-read-status","phonebook-search","raise-hand","room-description","rich-object-sharing","chat-reference-id"],"config":{"attachments":{"allowed":true,"folder":"\/Talk"},"chat":{"max-length":32000,"read-privacy":0},"conversations":{"can-create":true},"previews":{"max-gif-size":3145728}}},"theming":{"name":"My Cloudserver","url":"https:\/\/cloud.mydomain.tld","slogan":"Cloud! (v21.0.0)","color":"#0082c9","color-text":"#ffffff","color-element":"#0082c9","color-element-bright":"#0082c9","color-element-dark":"#0082c9","logo":"https:\/\/cloud.mydomain.tld\/apps\/theming\/image\/logo?useSvg=1&v=65","background":"https:\/\/cloud.mydomain.tld\/core\/img\/background.png?v=65","background-plain":false,"background-default":true,"logoheader":"https:\/\/cloud.mydomain.tld\/apps\/theming\/image\/logo?useSvg=1&v=65","favicon":"https:\/\/cloud.mydomain.tld\/apps\/theming\/image\/logo?useSvg=1&v=65"},"user_status":{"enabled":true,"supports_emoji":true},"weather_status":{"enabled":true}}}}}
INFO [test_client] Supported capabilities: ["activity", "bruteforce", "core", "dav", "deck", "external", "files", "files_sharing", "notes", "notifications", "notify_push", "ocm", "password_policy", "spreed", "theming", "user_status", "weather_status"]
Found push server at wss://cloud.mydomain.tld/push/ws
DEBUG [tungstenite::client] Trying to contact wss://cloud.mydomain.tld/push/ws at 84.117.125.19:443...
DEBUG [rustls::client::hs] No cached session for DNSNameRef("cloud.mydomain.tld")
DEBUG [rustls::client::hs] Not resuming any session
DEBUG [rustls::client::hs] Using ciphersuite TLS13_CHACHA20_POLY1305_SHA256
DEBUG [rustls::client::tls13] Not resuming
DEBUG [rustls::client::tls13] TLS1.3 encrypted extensions: [ServerNameAck, Unknown(UnknownExtension { typ: EllipticCurves, payload: Payload([0, 8, 0, 30, 0, 29, 0, 24, 0, 25]) })]
DEBUG [rustls::client::hs] ALPN protocol is None
DEBUG [rustls::client::tls13] Ticket saved
DEBUG [rustls::client::tls13] Ticket saved
DEBUG [tungstenite::handshake::client] Client handshake done.
Received error: Invalid credentials

My server setup:
nginx 1.18.0
PHP 8.0.3
NC 21.0.0

notify_push:self-test is successful:

# sudo -u http php /var/www/nextcloud/occ notify_push:self-test
✓ redis is configured
✓ push server is receiving redis messages
✓ push server can load mount info from database
✓ push server can connect to the Nextcloud server
✓ push server is a trusted proxy
✓ push server is running the same version as the app

[icewind1991]

Is there anything in the log of the notify_push server while you're running the test client.

(You can change the log level while the notify_push server is running by using occ notify_push:log debug)

[whisperdancer]

I assume you mean in the nextcloud log. Over 2000 log entries with notify_push, so to many to manually go through, but it appears to just keep repeating these:

,"level":0,"time":"2021-03-10T12:14:08-08:00","remoteAddr":"xx.xxx.xx.67","user":"user","app":"guests","method":"POST","url":"/apps/notify_push/pre_auth","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0","version":"21.0.0.18"}
,"level":0,"time":"2021-03-10T12:14:08-08:00","remoteAddr":"xx.xxx.xx.67","user":"user","app":"contacts","method":"POST","url":"/apps/notify_push/pre_auth","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0","version":"21.0.0.18"}
,"level":0,"time":"2021-03-10T12:14:08-08:00","remoteAddr":"xx.xxx.xx.67","user":"user","app":"files_external","method":"POST","url":"/apps/notify_push/pre_auth","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0","version":"21.0.0.18"}
,"level":0,"time":"2021-03-10T12:14:08-08:00","remoteAddr":"xx.xxx.xx.67","user":"user","app":"files_sharing","method":"POST","url":"/apps/notify_push/pre_auth","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0","version":"21.0.0.18"}

Also, how do I reset the notify_push log level back to it's default setting using occ? Thank you.

[icewind1991]

I mean the log output of the notify_push binary. (if you followed the default systemd setup you can get it using sudo journalctl -eu notify_push)

You can reset the notify_push log level using occ notify_push:log --restore

[whisperdancer]

Some updates:

Good news. I am able to see the log output of the notify_push binary.

Unfortunately when running the test client, nothing shows up for it in the log and the test_client continues to fail with same error message. However, once I could see the log I was able to test just doing things on the server. I created new folders and files using the nc desktop client and I see the entries reflected properly in the log. I see many entries for the file and folder activity including trashbin and versions. I see additional entries for joplin.

A sample line from log file:
Mar 12 10:36:56 nextcloud notify_push[7561]: [2021-03-12 10:36:56.873132 -08:00] DEBUG [notify_push::connection] src/connection.rs:93: Sending notify_file to dave

I followed that with a Talk test. I have 2 users: tom and dave, each logged in a separate browser on separate networks. tom sent dave a message. dave did not receive a notification. From the log it appears to be working, but no notifications are being received for either tom or dave. I've tried initiating the message from both, but doesn't seem to matter. Yet the log does show the pinging.

We're making progress. All tests pass. Self test passes fine and metrics shows connections. The log shows it appears to be working, sending pings, etc.

The issue is that despite everything appearing to be working, the users continue to not receive notifications. Not on browsers and I've tried with many different browsers. And not on mobile. No notifications for anything file related or Talk related. Note, user dave is also logged into Talk on an android phone and receives no notifications.

Here are some log entries after the users send messages in Talk to one another:

Mar 12 10:21:49 nextcloud notify_push[7561]: [2021-03-12 10:21:49.731996 -08:00] DEBUG [notify_push] src/lib.rs:370: Received pre_auth user tom
Mar 12 10:21:50 nextcloud notify_push[7561]: [2021-03-12 10:21:50.863877 -08:00] DEBUG [notify_push] src/lib.rs:219: new websocket connection from Some(43.29.39.xx)
Mar 12 10:21:54 nextcloud notify_push[7561]: [2021-03-12 10:21:54.777410 -08:00] DEBUG [notify_push] src/lib.rs:370: Received pre_auth user tom
Mar 12 10:21:54 nextcloud notify_push[7561]: [2021-03-12 10:21:54.960455 -08:00] DEBUG [notify_push] src/lib.rs:219: new websocket connection from Some(43.29.39.xx)
Mar 12 10:21:55 nextcloud notify_push[7561]: [2021-03-12 10:21:55.018144 -08:00] DEBUG [notify_push::connection] src/connection.rs:178: Authenticated socket for tom using pre authenticated token
Mar 12 10:21:55 nextcloud notify_push[7561]: [2021-03-12 10:21:55.018198 -08:00] INFO [notify_push::connection] src/connection.rs:66: new websocket authenticated as tom
Mar 12 10:22:18 nextcloud notify_push[7561]: [2021-03-12 10:22:18.414946 -08:00] DEBUG [notify_push::connection] src/connection.rs:106: Sending ping to dave
Mar 12 10:22:18 nextcloud notify_push[7561]: [2021-03-12 10:22:18.415217 -08:00] DEBUG [notify_push::connection] src/connection.rs:106: Sending ping to dave
Mar 12 10:22:18 nextcloud notify_push[7561]: [2021-03-12 10:22:18.415316 -08:00] DEBUG [notify_push::connection] src/connection.rs:106: Sending ping to dave
Mar 12 10:22:25 nextcloud notify_push[7561]: [2021-03-12 10:22:25.018971 -08:00] DEBUG [notify_push::connection] src/connection.rs:106: Sending ping to tom

And tom shared file with dave. This is the log, but dave did not receive a notification when logged into edge browser. (note, gave edge permission to receive notifications)

Mar 13 09:09:18 nextcloud notify_push[7561]: [2021-03-13 09:09:18.583496 -08:00] DEBUG [notify_push] src/lib.rs:370: Received share create notification for user dave
Mar 13 09:09:18 nextcloud notify_push[7561]: [2021-03-13 09:09:18.583574 -08:00] DEBUG [notify_push::connection] src/connection.rs:93: Sending notify_file to dave
Mar 13 09:09:18 nextcloud notify_push[7561]: [2021-03-13 09:09:18.716091 -08:00] DEBUG [notify_push] src/lib.rs:370: Received notification notification for user dave
Mar 13 09:09:18 nextcloud notify_push[7561]: [2021-03-13 09:09:18.716178 -08:00] DEBUG [notify_push::connection] src/connection.rs:93: Sending notify_notification to dave

[ebin-dev]

desktop clients not receiving notifications by notify_push (solved)

I had the same issue: notify_push was successfully startet on the server but the Nextcloud desktop clients did not receive any notifications via notify_push. I was observing this in the context of Nextclouds "High Efficiency Backend for Documents".

The reason for this behaviour of the desktop client was (as observed in the debug logs of the Nextcloud desktop client), that it did not successfully establish a ws connection with notify_push on the server, because the desktop client did not accept a self signed certificate used by the server for the ssl connection. But the same certificate was accepted by the Nextcloud desktop client for the standard way of polling the server and syncing the client with the server.

The obvious solution was to use a valid Let's Encrypt ssl certificate on the Nextcloud server. To implement that on a nextcloud intranet instance was non-trivial as it requires a FQDN as a server URL.

As indicated by "sudo -u www-data /var/www/nextcloud/occ notify_push:metrics", the active connection count is incremented by one if a Nextcloud desktop client is startet and decremented if it is closed. File syncronisation with the desktop client starts immediately upon changes on the Nextcloud server as expected.

Here are some details of the setup:

Nextcloud Server:
Nextcloud version (eg, 23.0.3.2)
Operating system and version (eg, Ubuntu 20.04): Debian GNU/Linux 11 (bullseye)
Apache or nginx version (eg, Apache 2.4.25): nginx/1.18.0
PHP version (eg, 7.4): v7.4.28

Nextcloud Desktop Client
Version 3.4.4 (macOS)

(Note: after the installation of the client push app (0.3.0) on the Nextcloud server the corresponding notify_push binary was missing the executable bit; 'chmod +x notify_push' solved it)

@szaimen @icewind1991 : notify_push is not working as expected with desktop clients if the server uses a self signed certificate, although that same certificate is accepted by the desktop client for polling the server. It would be very helpful if the desktop client and/or notify_push could be modified so that notify_push is also working with self signed certificates out of the box. Would you please consider that ?

[gcpmendez]

Same problem here!

Metrics:

Self-test: