RSS feed support

[digital-dreamer]

You can now subscribe to your twister postboard via RSS feed reader. Incoming direct messages are synchronized too. Feed URL is /rss on your twister client, for example http://127.0.0.1:28332/rss

Parameters: If you have more than one username, you can specify which one you want with the “account” parameter. If you want to change number of items, use the “max” parameter, default is 20. Format is the same as in any other HTTP URL. For example:

http://127.0.0.1:28332/rss?account=johndoe&max=10

Implements feature request #161

[miguelfreitas]

Cool stuff! I guess this is the first true c++ contribution i get :-)
Allow me a little time to review it before merging...

[digital-dreamer]

Thanks :-) By the way, I finished the RPC proxy in Node.js you suggested. I did it about three weeks ago, but I need help setting the defaults for call limits, and I would appreciate some code review before we declare it ready for use. But the Twister messages I sent you were not delivered, they can only be sent to followers. They are too short to discuss this anyway. Can you please give me your email address?

[Erkan-Yilmaz]

one email can be found on page 1 of the preprint: http://twister.net.co/?attachment_id=355

[miguelfreitas]

I'm not sure if we want /rss to bypass authentication, do we?
One might argue that we already bypass authentication for http static web serving, but we should change that sooner or later. Besides, static serving looks less dangerous than rss.

Update: now i see getposts and getdirectmsgs below. both require authentication. so we certainly don't want to bypass authentication here.

[digital-dreamer]

I added the improvements. But I would like to warn that having authenticated RSS doesn't make opening twisterd secure (unless you activate public_server_mode restrictions).

Here is why I think it is insecure: rpcuser and rpcpassword values are currently hardcoded in the twister html client. An attacker can bypass authentication simply by opening normal web interface (home.html) and read the same information (postboard, direct messages) as in RSS. And not just read, he can also post. The only thing preventing this is rpcallowip - as long as it is set to localhost. Also, the credentials are always user:pwd, and users can't change it without manually editing the hardcoded value in UI javascript, because all functionality would break due to not being able to connect.

It would be a nice idea to support custom login credentials in twister-html. It would enable people to do things like run their Twister on a private home server, and access it from a tablet or other device. The credentials shouldn't be sent in plaintext, but you could probably just run it with -rpcssl. So what do you think? Should I add support for custom username and password to twister-html?

[nitmir]

You can change rpcuser and rpcpassword in the .twister/twister.conf file, then twister-html just prompt (in the classic http auth way) for the custom credentials. I added a SSL proxy in front of twister-core for serving static contents and provide encryption and it works like a charm.
More details on the wiki http://wiki.twister.net.co/w/using:howto:twister_on_your_self-host_server#apache_proxy

[digital-dreamer]

Maybe I'm doing something wrong, but Firefox doesn't even prompt for the credentials, it just displays “Error connecting to local twister daemon”. Chromium asks for them, but doesn't use them, the default credentials are used again, and the prompt is displayed again as a result of the 401 / HTTP_UNAUTHORIZED code it always receives.

You will see these results when using Twister directly without the Apache proxy, which can probably get around this problem. I would see this as a bug to be fixed - but I agree it is much better to run Twister behind an Apache proxy with SSL like you do.

[nitmir]

strange, my iceweasel (v30) prompt for credentials and everything work after, even without SSL proxy. Maybe I'm just lucky…
I'll try out tomorow on another computer.

[miguelfreitas]

@digital-dreamer don't worry: we will remove the hardcoded values from twister-html and start requiring authentication for everything.

[thedod]

Glad to hear about auth, @miguelfreitas. Was about to pull-request it instead of hacking it manually each time 😉
Anyway, I'd like to see direct-message RSS as something that you should actively enable at twister.conf (and it should require basic auth): if you use a general purpose RSS reader app, there's no telling how many copies of your darkest secrets are spread all over the disk in plaintext, so I would recommend only to enabele direct message RSS if you're using a very small and privacy-minded piece of code with it (no idea what exactly, just sayin').

Non-direct-message RSS is kinda public domain content anyway (that's what "censorship-free" is all about. No? 👼). Not sure whether this should be password protected (maybe if RSS content depends on who I follow, cache, etc, it could leak some matadata-ish info. IDK), so maybe best is to let users decide about this at twister.conf.

[digital-dreamer]

thedod - good point, I added the commit. So now, direct messages are not synchronized by default, you can manually enable them in twister.conf with rss_dm=1

[mrvdb]

Is there a pressing reason to choose RSS over ATOM ? The latter is a much better format, specifically designed to be the successor of RSS.
Also, some other open microblogging systems (StatusNet, GNU-social and pump.io) use it extensively so integration would be easier I guess.
http://nullprogram.com/blog/2013/09/23/ has a good summary on Atom vs RSS

[Erkan-Yilmaz]

@digital-dreamer @miguelfreitas

doesn't compile, error in make-step:

src/json/json_spirit_value.h:219:5: note: json_spirit::Value_impl::Value_impl(json_spirit::Value_impl::Const_str_ptr) [with Config = json_spirit::Config_vectorstd::basic_string; json_spirit::Value_impl::Const_str_ptr = const char*]
Value_impl< Config >::Value_impl( const Const_str_ptr value )

src/json/json_spirit_value.h:219:5: note: no known conversion for argument 1 from ‘time_t {aka long int}’ to ‘json_spirit::Value_impl<json_spirit::Config_vector<std::basic_string > >::Const_str_ptr {aka const char*}’
src/json/json_spirit_value.h:439:5: error: initializing argument 2 of ‘json_spirit::Pair_impl::Pair_impl(const String_type&, const Value_type&) [with Config = json_spirit::Config_vectorstd::basic_string; json_spirit::Pair_impl::String_type = std::basic_string; json_spirit::Pair_impl::Value_type = json_spirit::Value_impl<json_spirit::Config_vector<std::basic_string > >]’
Pair_impl< Config >::Pair_impl( const String_type& name, const Value_type& value )

make[1]: *** [src/twister_rss.o] Fehler 1
make[1]: Verlasse Verzeichnis '/home/USER/Schreibtisch/twister-core'
make: *** [all-recursive] Fehler 1

more at: http://skilledtests.com/twister/twister-core-pullrequest204.txt

[Erkan-Yilmaz]

stat so far:
2 persons with 32bit OS couldn't compile
1 person on 64bit OS succeeded

[digital-dreamer]

@Erkan-Yilmaz, thank you for reporting this, I'm on a 64bit system and didn't think about the differences. Here is a patch: #208. Can you please try it to see if everything works for you now?

[mrvdb]

Resulting feeds also forget to escape XML chars. For example, the note:

 <item>
    <title>vegos</title>
    <author>vegos</author>
    <description>@erkan_yilmaz @letstwist @mfreitas @twister :basic_string<char> > >}' is ambiguous (3/3)</description>
    <pubDate>Tue, 03 Jun 2014 16:44:20 +0000</pubDate>
  </item>

will throw errors on the <char> (will be interpreted as a XML tag) and the > > there-after.

[Erkan-Yilmaz]

@digital-dreamer THX (just tested and it's compiling again)

[digital-dreamer]

OK, so that's done :-)

@mrvdb - thanks for testing the special XML characters, a patch is here: #209