Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add ntfy plugin #524

Merged
merged 7 commits into from
Apr 4, 2022
Merged

add ntfy plugin #524

merged 7 commits into from
Apr 4, 2022

Conversation

particledecay
Copy link
Contributor

@particledecay particledecay commented Jan 25, 2022

Add a plugin for the Ntfy notification service.

Description:

Related issue (if applicable): #520

New Service Completion Status

  • apprise/plugins/NotifyNtfy.py
  • setup.py
    • add new service into the keywords section of the setup() declaration
  • README.md
    • add entry for new service to table (as a quick reference)
  • packaging/redhat/python-apprise.spec
    • add new service into the %global common_description

Checklist

  • The code change is tested and works locally.
  • There is no commented out code in this PR.
  • No lint errors (use flake8)
  • 100% test coverage

README.md Outdated Show resolved Hide resolved
apprise/plugins/NotifyNtfy.py Outdated Show resolved Hide resolved
apprise/plugins/NotifyNtfy.py Outdated Show resolved Hide resolved
apprise/plugins/NotifyNtfy.py Outdated Show resolved Hide resolved
packaging/redhat/python-apprise.spec Outdated Show resolved Hide resolved
@binwiederhier
Copy link

Fantastic!

@caronc
Copy link
Owner

caronc commented Jan 25, 2022

This is a great first start guys! 🚀 I can see you're still fine-tuning your details. I'll pull my half of my promise and write the test cases once you're comfortable with what you've got.

Please feel free to ask any questions you want; if you're stuck on how to do something, it's likely been done in another Notification service (so feel free to steal/copy paste). I can also help guide you to solve anything too if needed.

Since this is web based and allows you to host your own server (with https/http), you might see a lot of commonality in NotifyXMBC.py and NotifyRocketChat.py.

Chris

@particledecay
Copy link
Contributor Author

@caronc thanks for that! I was looking through a few of the other plugins and wondering which ones were probably closest. I'd never heard of RocketChat so I'll definitely take a look in there.

Add a plugin for the [Ntfy](https://ntfy.sh) notification service.
Supports topics, emojis, priorities, and tags.
@particledecay particledecay marked this pull request as ready for review March 15, 2022 03:50
@particledecay
Copy link
Contributor Author

Ok this seems to work with topic, priorities, emojis, and tags. Good enough for a first pass I think, sorry for the delay 😄

@codecov-commenter
Copy link

codecov-commenter commented Mar 15, 2022

Codecov Report

Merging #524 (fb8e1e1) into master (aa3d30f) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##            master      #524    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files          102       107     +5     
  Lines        13419     13905   +486     
  Branches      2465      2600   +135     
==========================================
+ Hits         13419     13905   +486     
Impacted Files Coverage Δ
apprise/plugins/NotifyNtfy.py 100.00% <100.00%> (ø)
apprise/utils.py 100.00% <0.00%> (ø)
apprise/Apprise.py 100.00% <0.00%> (ø)
apprise/plugins/NotifyXML.py 100.00% <0.00%> (ø)
apprise/plugins/NotifyForm.py 100.00% <0.00%> (ø)
apprise/plugins/NotifyJSON.py 100.00% <0.00%> (ø)
apprise/plugins/NotifyEmail.py 100.00% <0.00%> (ø)
apprise/plugins/NotifySlack.py 100.00% <0.00%> (ø)
apprise/plugins/NotifyDapnet.py 100.00% <0.00%> (ø)
apprise/plugins/NotifyGotify.py 100.00% <0.00%> (ø)
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aa3d30f...fb8e1e1. Read the comment docs.

README.md Outdated Show resolved Hide resolved
apprise/plugins/NotifyNtfy.py Show resolved Hide resolved
schema = 'https' if self.secure else 'http'

# Use default host if one is not defined
host = self.default_host

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know where else to comment, but this does not work as a host, but it should: ntfys://--joey, or ntfys://joey--

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What spec are you looking at for that? That would be a host of joey-- with no topic, which doesn't seem valid. In fact, as a shortcut, I currently have that going to the default ntfy.sh host with a topic of joey-- instead.

Either way, as a hostname it would be invalid according to RFC 952.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a hostname. It's a ntfy topic name.

ntfys://joey-- translates to https://ntfy.sh/joey-- which is a valid topic name. The spec is my regex of allowed topic names: https://github.com/binwiederhier/ntfy/blob/main/server/server.go#L58

I think you are right though that apprise is using a URI parser which doesn't recognize this as a valid host and doesn't allow it, since joey-- in the previous example would be parsed as a hostname.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. I felt like it would be a nice shortcut, but it inadvertently restricts topic names to conform to hostname standards. I'll see if I can fix it, otherwise we may have to be explicit about host 😕

@binwiederhier
Copy link

@caronc Hey I just tested this and it works well for the basic use case. How do we get this rolling and merged? What else do we need to do? :-D

Add the ability to specify a 'click' option, 'delay', and 'email' for
forwarding in ntfy plugin. All three are done via URL params.
@particledecay
Copy link
Contributor Author

@binwiederhier I added X-Click, X-Delay, and X-Email, since they were about as straightforward as X-Tags and I could add them now. However, I left out X-Attach for the moment since I don't know enough about attachments in apprise yet.

@binwiederhier
Copy link

@particledecay X-Attach and X-Filename used for the "attach by URL" case (https://ntfy.sh/docs/publish/#attach-file-from-a-url), so you can add them here too without having to mess with uploads.

For "upload this file" kind of attachments (https://ntfy.sh/docs/publish/#attach-local-file), we can skip that for now I think and tackle that in round two.

@caronc
Copy link
Owner

caronc commented Mar 16, 2022

You guys are really doing a great job; this is fantastic.. If you want to manipulate the headers, you could make it custom (or does the upstream plugin require it?)... For example for the json:// plugin (see here) it's done by adding support for the + (addition) character that can be placed in the URL... for example (stolen from the same link and massaged to fit this thread):

# Below would set the header:
#    X-Delay: value
#
# Assuming our {hostname} is localhost
# Assuming our {port} is 8080
apprise -vv -t "Test Message Title" -b "Test Message Body" \
   "ntfy://localhost:8080/path/?+X-Delay=value"

# Multiple headers just require more entries defined:
# Below would set the headers:
#    X-Delay: abcdefg
#    X-Tags: awesome,sauce
#
# Assuming our {hostname} is localhost
# Assuming our {port} is 8080
apprise -vv -t "Test Message Title" -b "Test Message Body" \
   "ntfy://localhost:8080/path/?+X-Delay=abcdefg&+X-Tags=awesome,sauce"

Food for thought anyway... i can help you implement this as well while i add testing.

Otherwise if you want to add support for something like --attach to which it needs to create the X-Attach headers etc, that's a bit different (but also still very do-able).

Add the ability to attach files via URL to ntfy messages.
@particledecay
Copy link
Contributor Author

@binwiederhier added the X-Attach stuff just now, and verified that it works.

@caronc is your offer to build tests still on the table? I think this is ready for testing.

@caronc
Copy link
Owner

caronc commented Mar 17, 2022

Of course, i wouldn't go back on my word. Is the foundation of what you're building what you want?

@caronc
Copy link
Owner

caronc commented Mar 20, 2022

@particledecay , i added some unit-tests (not done yet) and also re-factored your code a little bit.

  • supports multiple topics as part of the Apprise URL now; you can also use the to= on the parameter path now as well. This is useful for YAML configuration files (makes it easier to read if you choose to).
  • Added a mode (private or cloud) to which it's attempted to be detected if not otherwise specified. This eliminates the ambiguity of trying to decide if it's upstream or private just by whether or not a path was specified or not (becoming your target)
  • This may or may not break some things you may have had previously since you were using the fullpath before - not sure here. Have a look and see what I've done anyhow.
  • I also added support for https://ntfy.sh/topic so that it can be passed directly into Apprise as that. Apprise can go ahead and initialize a cloud mode
  • I also added support for user/pass configuration; some people who host their own servers like to at least add authentication if it's to be exposed to the internet (their choice). If no user/pass is specified, then it still works as you had it before. At that same point, no user/pass will be applied to the cloud mode. Same with the port ranges. I don't think you want Apprise acting as a port scanner against your upstream service, so we'll fix it to https:// only for that (port 443).

Anyway; i still have a lot of test cases to add, but you can see a basis of what i've done. I'd be curious on your feedback.

Chris

@binwiederhier
Copy link

That looks fantastic. Thank you both for working on this. :-D

One question:

Added a mode (private or cloud) to which it's attempted to be detected if not otherwise specified. This eliminates the ambiguity of trying to decide if it's upstream or private just by whether or not a path was specified or not (becoming your target)

Could you explain to me where the ambiguity is? I thought ntfy://topic and ntfy://myhost.com/topic would be unambiguous.

@caronc
Copy link
Owner

caronc commented Mar 21, 2022

To address your ambiguity question:

  • basically topic has become topics since my changes. This flows with all of the other notifications Apprise supports.
  • this means https://topic1/topic2/topicN and https://localhost/topic1/topic2 can't easily be deciphered. Simply adding ?mode=cloud or mode=private can allow you to handle this case. The same check you had before, i left in place and the mode is automatically detected if possible. Hence https://topic would be detected as cloud
  • providing the user the ability to force the mode they want just grants the user full power over the actions they expect.

@caronc
Copy link
Owner

caronc commented Mar 21, 2022

One question i have actually is that the code parses the JSON returned from a post to the ntfy server, but doesn't do anything with what it gets, was this intended?

@binwiederhier
Copy link

I can only comment on UX, not on the Python. I'll defer to @particledecay for the Python pieces.

Multiple topics

In ntfy itself I avoided having one message posted to multiple topics, because it messages with the rate limiting, but I think the way you implemented it (multiple POSTs) is fine. It seems to work nicely too.

bin/apprise -D -b "body of the message" -t "title" "ntfy://localhost/abc/def?priority=max&tags=smile,de"
2022-03-21 14:40:23,628 - DEBUG - Loaded ntfy URL: ntfy://localhost/abc/def?format=text&verify=yes&rto=4.0&tags=de%2Csmile&priority=max&mode=private&overflow=upstream&cto=4.0
2022-03-21 14:40:23,629 - DEBUG - Ntfy POST URL: http://localhost/def (cert_verify=True)
2022-03-21 14:40:23,629 - DEBUG - Ntfy Payload: body of the message
2022-03-21 14:40:23,641 - INFO - Sent Ntfy notification to 'http://localhost/def'.
2022-03-21 14:40:23,641 - DEBUG - Ntfy POST URL: http://localhost/abc (cert_verify=True)
2022-03-21 14:40:23,641 - DEBUG - Ntfy Payload: body of the message
2022-03-21 14:40:23,641 - DEBUG - Throttling for 5.487428s...
2022-03-21 14:40:29,159 - INFO - Sent Ntfy notification to 'http://localhost/abc'.

In ntfy, it is possible to SUBSCRIBE (but not publish) to multiple topics using the comma syntax: https://ntfy.sh/mytopic1,mytopic2/json would subscribe to two topics. Is the slash syntax common in apprise? If so, definitely keep it. If not, maybe go with the comma thing instead to match it with ntfy? This would avoid the awkward mode parameter too (sorry, I find it awkward 😬) :-D

Topic names must match ^[-_A-Za-z0-9]{1,64}$ (see server.go)

I know this is unusual, but topic names can contain dashes and underscores at the beginning and the end, so ---joey and phil--- are valid ntfy topics. Currently the code doesn't allow that:

bin/apprise -D -b "body of the message" -t "title" "ntfys://abc---?priority=max&tags=smile,de"
2022-03-21 14:35:56,628 - WARNING - No Ntfy topics were identified to be notified
2022-03-21 14:35:56,628 - DEBUG - Loaded ntfy URL: ntfys://?format=text&verify=yes&rto=4.0&tags=de%2Csmile&priority=max&mode=cloud&overflow=upstream&cto=4.0
2022-03-21 14:35:56,628 - WARNING - There are no Ntfy topics to notify

"code parses the JSON returned from a POST"

Typically it is not necessary to parse the JSON response, unless it's a non-2xx return. If that is the case, it can likely be removed.

@binwiederhier
Copy link

Hey @particledecay, I'm very eager to get apprise support out, but I've been backing off intentionally to let you take the reigns. It has been very quiet though lately, so I was wondering if you wanted to let me work with @caronc to finish things up?

My python fu is not strong, but I am pretty sure I'll manage, with the countless examples that apprise has.

@caronc
Copy link
Owner

caronc commented Apr 3, 2022

@binwiederhier Got to 100% test coverage today; i think we're ready for a merge once you approve that it works okay for you.

In ntfy, it is possible to SUBSCRIBE (but not publish) to multiple topics using the comma syntax: https://ntfy.sh/mytopic1,mytopic2/json would subscribe to two topics. Is the slash syntax common in apprise?

Yup, done and fully supported in this last few chunks of code i've pushed. So even a mix such as https://ntfy.sh/topic1,topic2/oreven-topic3/topic4,topic5 will work.

.. avoid the awkward mode parameter too (sorry, I find it awkward grimacing) :-D

Honestly don't worry about the mode. I was careful to leave the parsing exactly as you had it before. You don't need to set the mode at all. The entire purpose of it is just to remove ambiguity if necessary, but it doesn't have to be part of your Apprise URL constructions at all. Before logic was just performed if ntfy.sh then else. This was everywhere in the code. During the parsing stage of the URL i perform the exact same check and just set a cloud or private flag to make the code use this instead. the only thing i did in addition to this is just allow someone to force it's value if they ever needed to (it's just added configuration). It also can be used to force unambiguity such that one could (if they wanted to) type something like:
ntfy://topic1/topic2/topic3/?mode=cloud. By the old logic, topic1 would have become the hostname.

Give it a try and let me know if you still have concerns; if not we can merge this into the master branch.

great work all!

Copy link

@binwiederhier binwiederhier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love it ❤️

'attach': {
'name': _('Attach'),
'type': 'string',
},

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're missing Filename, which is the displayname/filename; it's fine if it's missing for now, but it should be easy to add.

See https://ntfy.sh/docs/publish/#attach-file-from-a-url and https://ntfy.sh/docs/publish/#list-of-all-parameters

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

X-Filename has been added in the last check for Attachments. but for local file attachments as identified here. This still needs to be done.

I would advise maybe reviewing what is there and see if the rest is okay, and then we can start adding the --attach support to Apprise to push uploaded content next.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds great.

apprise/plugins/NotifyNtfy.py Outdated Show resolved Hide resolved
@binwiederhier
Copy link

I read the code and tried everything out. Looks great. Apart from the one nitpick and the missing Filename param, it looks great. Thank you so much!

@caronc
Copy link
Owner

caronc commented Apr 3, 2022

I made a small change to checkdone.sh; a local file i test with; but didn't realize i accidentally pushed it up (it's already in the master now). I just rolled that back and did 1 force-push (i know this is frowned upon).

Same source though. Glad you're happy, I'll merge what we have and we can isolate any new bugs you find and fix them separately. A least you have a core foundation to work with now.

The --attach should be pretty trivial to add support for.

the only part of the code source as it is now that doesn't make sense is that after a post is successful to your API, a JSON response is checked, but nothing is done with the results (in Apprise). Can we drop this code completely? Or should we take action on specific return values? See here

@binwiederhier
Copy link

The --attach should be pretty trivial to add support for.

Excuse my ignorance, but would this be to attach local files (https://ntfy.sh/docs/publish/#attach-local-file)? Like -T <file> in curl? e.g. for

curl \
    -T flower.jpg \
    -H "Filename: flower.jpg" \
    ntfy.sh/flowers

the only part of the code source as it is now that doesn't make sense is that after a post is successful to your API, a JSON response is checked, but nothing is done with the results (in Apprise). Can we drop this code completely? Or should we take action on specific return values?

Yes you can remove that entirely, no problem. The error message is often useful though, and some errors do return a link to the documentation, so if you have some sort of feedback mechanism to users that could display this, that'd be useful.

Example:

$ curl -d hi "ntfy.sh/abc?prio=invalid"
{"code":40007,"http":400,"error":"invalid priority parameter","link":"https://ntfy.sh/docs/publish/#message-priority"}

@binwiederhier
Copy link

Ship it ;-)

@caronc caronc merged commit b28cd4c into caronc:master Apr 4, 2022
@caronc
Copy link
Owner

caronc commented Apr 4, 2022

Done! 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants