All Rss Feeds are tooted with each call #16
Comments
|
Hmm nothing really jumps out at me. The way the bot decides whether to post content or not is by checking the timestamp on the latest post for the Mastodon account and filtering posts that have a later timestamp. That's handled by the post-items function. You could try adding a |
|
Thanks, but I am more a C programmer and don't know much about js or clojure.. I would appreciate more hints on where to look next, thank you! |
|
It sounds like the dates aren't getting parsed out from the feed correctly. So, next step would be to check what you're getting in the parse-feed function. The following block is the relevant part: (for [{:keys [title isoDate pubDate content link]} (-> % js->edn :items)]
{:created-at (js/Date. (or isoDate pubDate))
:text (str (trim-text title) "\n\n" (strip-utm link))})What that says is that each RSS feed item is a map containing the keys (let [items (-> % js->edn :items)]
(-> items first println)
(for [{:keys [title isoDate pubDate content link]} items]
{:created-at (js/Date. (or isoDate pubDate))
:text (str (trim-text title) "\n\n" (strip-utm link))}))this will print out the first item in the parsed feed, and if you see something that looks like publication date then it's just a matter of adding that key in. |
|
This how a typical item in the RSS feed looks like. The key
Second example from other feed: |
|
The feed seem to be in a proper format, so would need to see why the data isn't being parsed out correctly. |
|
Could you provide the feed URL itself. The Node module works against URLs and I'd need to see what gets parsed out. Also, have you tried printing each item in the loop as I noted above. That will show what the script sees after the items are read from the feed. |
|
Ok this is what I got. |
|
Really not sure what's happening there. You can see the feed item is parsed out correctly. Both (js/Date. "2018-11-05T13:38:42.000Z")
#inst "2018-11-05T13:38:42.000-00:00"
(js/Date. "Mon, 05 Nov 2018 13:38:42 GMT")
#inst "2018-11-05T13:38:42.000-00:00"And that's exactly what happens here Again, if you can give me the URL for the feed I could take a closer look. |
|
Yep I see it. I don't think the problem is feed-specific as I have several config files and all show the same symptoms. |
|
So, I tried the feeds locally and I can't seem to reproduce the issue. I replaced the (defn post-items [last-post-time items]
(doseq [item items]
(println item)))here's what I'm seeing in the output: {:created-at #inst "2019-01-04T12:24:57.000-00:00", :text Tiere suchen neues Zuhause
https://www.hamburg.de/tierheim/}
{:created-at #inst "2019-01-03T14:43:10.000-00:00", :text Darts-Wahnsinn in Hamburg
https://www.hamburg.de/sport/12019718/world-cup-of-darts/}
{:created-at #inst "2018-12-27T09:33:36.000-00:00", :text 11 Geheimtipps für denJanuar
https://www.hamburg.de/geheimtipps/7651522/geheimtipps-januar/}
{:created-at #inst "2018-12-20T11:17:44.000-00:00", :text Tipps für die Weihnachtsferien
https://www.hamburg.de/kinder/3863036/kinder-ferien/}
{:created-at #inst "2018-12-18T12:01:51.000-00:00", :text Machen Sie Hamburg digital!
https://www.hamburg.de/hamburggoesdigital/}
{:created-at #inst "2018-12-17T16:31:32.000-00:00", :text Hannoversche Brücke
https://www.hamburg.de/stoerungsmeldungen/}
{:created-at #inst "2018-12-11T16:41:01.000-00:00", :text blickfang in den Deichtorhallen
https://www.hamburg.de/messe/4041828/designmesse-blickfang/}
{:created-at #inst "2018-12-07T14:52:49.000-00:00", :text Hamburg und der Brexit
https://www.hamburg.de/brexit/}
...The |
|
Well, when I copy your code for the Problem is, it doesn't work with the code I check out from master. |
|
I'm using the master branch as well for testing, and that was the only change I made. The dates are only parsed once, so I'm really not sure why you're not seeing the correct dates in the posts. If the items are parsed out of the feed with the right dates, they should be posted with the same dates. |
|
This behavior does not seem to be specific to a single RSS url. I tested several urls, all with the same outcome. To be sure it is not my system or a wrong nodejs/npm version I tested the script on my debian stretch stable server and my arch linux desktop, again with the same outcome. I even tried different mastodon accounts on different instances, again same outcome. I assume the cause for my symptoms lies somewhere in the
Output: |
|
Oh, I think I misunderstood you originally. I thought the issue was with the dates in the feed, and those appear to be ok. If the issue is with The way the bot works is to query the Mastodon timeline, and check the time of the latest post. This should be the time used for filtering. The result is passed to Could you check what you end up with from your timeline. It's read here: (defn get-mastodon-timeline [callback]
(.then (.get mastodon-client "timelines/home" #js {}) #(-> % .-data js->edn callback)))and the callback is at the end of the script: (get-mastodon-timeline
(fn [timeline]
...So, if you add a print there: (get-mastodon-timeline
(fn [timeline]
(println timeline)We could see what's being parsed. |
|
That's it. It printed: I closed the issue. Though in my opinion this is an issue with mastodon, please post the minimal scopes (it needs read:statuses and write:statuses) your bot needs in mastodon to work properly in your Readme so others don't make my mistake, too. Thanks for your help and your patience! |
|
Ah yeah that would do it. I updated the script to check for the error to make this transparent. Glad we got this figured out in the end. :) |
Every time I call my config with
./mastodon-vot.cljs hamburg_de.ednall posts are tooted, not just the new ones.How can I debug the script and find my issue?
Here is my config:
EDIT: Within the mastodon account the bot is allowed to read:statuses and write:statuses.
The text was updated successfully, but these errors were encountered: