Looking for a hoster for mnserver (serving matray)

[moson-mo]

Since I'm shutting down my instance of mnserver on manjaro.moson.eu and Manjaro is not willing to host this service, I'm looking for volunteers in the community.

The service is currently being utilized by about 14.000 (concurrent) matray users.

Min. Requirements:

• 1 CPU core (avg load ~ 3%)
• 100 MB memory
• 10 MBit/s connection (serving 14k concurrent users requires about ~1 MBit/s of upstream bandwidth)

The server should be running 24/7 with high availability and a stable internet connection.

Some stats:

• Average requests per second: ~23
• Traffic per day: ~11 GB

Install instructions:

git clone https://github.com/moson-mo/mnserver.git
cd mnserver
go build .
sudo cp mnserver /usr/local/bin/
sudo cp mnserver.service /etc/systemd/system/
sudo systemctl enable --now mnserver

Note that the systemd unit file is set up with User=www-data.
Change this according to your setup... (if you have for example nginx installed, this user account should exist already)

The service listens on port 10111.
You can easily test it with curl:

curl -X POST http://127.0.0.1:10111/news -d '{"MaxItems":15,"Categories":["All"]}' 

[iruizr7]

Hi! As a die-hard manjaro user, I offer myself voluntary in order to host the service.

However I have some questions, after reviewing the architecture of the system and so... AFAIK the mnserver parses the Manjaro RSS feed (And twitter feed??) and serve the news in some form of API to the clients (I havent dived in the code yet).

So, wouldn't be more efficient to let the clients parse the Manjaro RSS feed directly? Or it its because it aggregates the twitter data also?

Anyway, as I said, I am using Manjaro since very long time ago and I feel like contributing a little bit back to the community.

[moson-mo]

Hi @iruizr7

So, wouldn't be more efficient to let the clients parse the Manjaro RSS feed directly? Or it its because it aggregates the twitter data also?

There is two main reasons:

The RSS feed is about 500 KB in size. If you have ~24 requests per second, each one receiving this amount of data, you will pretty quickly bring the forum server to it's knees. Also, there are several feeds that need to be consumed (mainly for ARM) based on what kind of news you want to retrieve (the different categories). In the worst case you'd need suck a couple of megabytes.
mnserver strips this down to the bare minimum so that a typical request is about 5 KB in size.

As you mentioned there can be several sources (RSS, Twitter atm). Having a middle-ware makes it way more flexible to add feeds (even with different formats) in the future since you'd simply implement that into the service. You won't need to change the client (matray) at all

regards,
MO

[y5]

Hi,

I'm currently in the process of setting up the project on a spare server I have. I'll wait for the DNS record to update and then report back here.

Is there anything else I should keep in mind for running this service?

[FallenAngel666]

Hi @moson-mo

is there a possibility and if yes, does it make sense to run multiple smaller services like this one?
That approach would resolve the need for a single 10 Mbit/s server with high availability.

Basically matray would fetch either random one from multiple sources and if that fails, take another one.

regards,

Luzi

[moson-mo]

Basically matray would fetch either random one from multiple sources and if that fails, take another one.

Sure, in theory that's possible. Now, I'm not going to put any additional work into matray myself to implement this, but if someone is willing to do so...
A DNS round-robin scenario could also be an idea and won't even require any changes. Just a domain is needed.

Now 10 MBit/s is not really a lot though. The real bandwidth that is needed is more or less 1 MBit/s to be able serve 14k clients.
So not sure if it is even worth it...

[moson-mo]

Is there anything else I should keep in mind for running this service?

Not really I guess. You might want to proxy it through nginx or so to add rate-limiting or TLS on top. (Mine runs naked though. Haven't had any problems with that...)

[y5]

For some reason Cloudflare isn't a fan of having this stuff work, http://70.34.197.142:10111/ works and responds with the news though.

[moson-mo]

For some reason Cloudflare isn't a fan of having this stuff work, http://70.34.197.142:10111/ works and responds with the news though.

This service doesn't run on TempleOS 🤣 Just kidding.

Looks good to me.
What are the specs of the machine?

[y5]

Should be 4 cores with 2 gig of RAM if I recall correctly, more than enough for this small news server.

[moson-mo]

Should be 4 cores with 2 gig of RAM if I recall correctly, more than enough for this small news server.

Yeah. What about the network connection? 1 GBit/s I'd assume?

I'll run some tests and hammer it a bit if you don't mind.

[y5]

Yes, it should be a 1gbit connection. Feel free to run whatever tests you want - let's hope it stays up :p

[moson-mo]

Feel free to run whatever tests you want - let's hope it stays up :p

Looks good to me. I couldn't kill it 😉

You are willing to run this for some time, right?
(I just want to avoid giving it away and have it die a month later...)

If so, I'd suggest the following:

• We wait until DNS has propagated everywhere (btw, which name would it be?)
• I'll update the matray code to replace the URL in the users config file and release a new version.
• I'll ask at Manjaro to package that new version and push it out to their repositories.

Would be good if you could git pull and rebuild mnserver.
I did a couple of changes to avoid spamming journal with log entries.

[Sommerwiesel]

Hello there,
if it isn't too late already, I would also be willing to provide a server for this.
The more - the merrier :)

4 Cores
8 GiB RAM
1 GBit/s Network
99.997% Uptime in the last 6 months

I have a domain already pointing to the server since it's used for other stuff already.

I could also set up a new standalone server (1C 1GiB RAM) just for this, no problem.

Just give me a holler if you need another server and I'll set it up.

[y5]

You are willing to run this for some time, right? (I just want to avoid giving it away and have it die a month later...)

If so, I'd suggest the following:

* We wait until DNS has propagated everywhere (btw, which name would it be?)

* I'll update the matray code to replace the URL in the users config file and release a new version.

* I'll ask at Manjaro to package that new version and push it out to their repositories.

Would be good if you could git pull and rebuild mnserver. I did a couple of changes to avoid spamming journal with log entries.

Yes, I plan to run this for a long time.

DNS should be fine now although mnserver is not available from the outside yet for some reason (probably something with the firewall? I'll take a look), the server itself is available at https://manjaro.rob.social now.

I'll rebuild mnserver in a few minutes after I figure out why it's accessible via IP but not via the domain.

[moson-mo]

the server itself is available at https://manjaro.rob.social now.

Hmm, that DNS entry points to a couple of different IP-Addresses, but not the one you mentioned earlier.

https://dnschecker.org/#A/manjaro.rob.social

[y5]

the server itself is available at https://manjaro.rob.social now.

Hmm, that DNS entry points to a couple of different IP-Addresses, but not the one you mentioned earlier.

https://dnschecker.org/#A/manjaro.rob.social

They all seem to be Cloudflare-controlled IP addresses. Do you think changing the port is a viable option? Cloudflare only seems to support proxying for certain ports (https://developers.cloudflare.com/fundamentals/get-started/reference/network-ports/) so I would prefer to go for a supported port like 2052.

[moson-mo]

They all seem to be Cloudflare-controlled IP addresses. Do you think changing the port is a viable option?

Ah, ok you have cloudflare running in front of it.
Actually I didn't bother to make it a command line argument for the port, but you could simply change it here and then recompile...

edit: Just added the port number as a flag: 4f42437

btw. no clue how this behaves with caching and cloudflare.

[y5]

After some troubles with the go buildcache everything works as expected now: http://manjaro.rob.social:2082/news returns some news.

[moson-mo]

Actually I just noticed I'm in a bit of dilemma here.

How do I know you have good intentions and don't want to abuse it? You profile picture makes me nervous I must say. 🤣

[y5]

Truth is you can't know, that's how the internet works.

I offered to host the news server on a spare server because me (and a bunch of people at my company) use Manjaro and it would be sad to see a service go down.

It makes no difference to me whether it's me or someone else from this thread hosting the server. Feel free to pick whoever you find most trustworthy - I just care about keeping this service alive and giving something back to the community.

I'm also not sure how you would even abuse this kind of service (except for phishing or fake news?), I haven't looked at the code of the tray application but I would assume that you don't execute anything returned by the server.

Please take your time before making this decision - my server will stay up regardless.

[BadAtIrcBots]

@moson-mo are you going to change the matray config to point to manjaro.rob.social or keep manjaro.moson.eu and point that to y5's server?

Additionally is it normal for it to fetch news on non-standard ports with no TLS? Does matray even support TLS?

I like @FallenAngel666's idea of making this more of a shared responsibility and having more than one server in a rotation. I can donate a VM on my own dedicated hardware with 10 Gbit networking in the US.

Given the extremely low resource requirements it is frankly ridiculous that the Manjaro team isn't helping out.

[y5]

Given the extremely low resource requirements it is frankly ridiculous that the Manjaro team isn't helping out.

Even if you rent a new small VM just for this server (which I did) it'll only run you a couple of dollars per month. Surprising not to see them jump in and host it honestly, I think someone else mentioned this on the forum discussion thread already.

[FallenAngel666]

I'll update the matray code to replace the URL in the users config file and release a new version.

Would be also a good idea to put these two lines in the configs somewhere, so it's not hard coded.

[thexhq]

To some people, it is maybe also concerning to use Cloudflare under EU privacy law and in some open source communities. Especially when this happens automatically. There could be tracking etc.

[Sommerwiesel]

To weight in about multiple servers - I can also provide a zero analytics load balancer on my VPS services that will route to different mnserver servers. This would allow us to use only one domain - I could even order one if needed.
I personally will not use Cloudflare.
I can host servers in EU West and USA.

[BadAtIrcBots]

To some people, it is maybe also concerning to use Cloudflare under EU privacy law and in some open source communities. Especially when this happens automatically. There could be tracking etc.

Given the request is sent over plain HTTP (checked my own matray config to confirm), I think the fact @y5's is proxied by Cloudflare is the least of your worries!

I get the concern and I think one thing that might open up a can of worms is Cloudflare's analytics. I would encourage y5 to not use Cloudflare as I really don't see what benefit it'd provide in this case for such a simple service.

[FallenAngel666]

To weight in about multiple servers - I can also provide a zero analytics load balancer on my VPS services that will route to different mnserver servers. This would allow us to use only one domain - I could even order one if needed. I personally will not use Cloudflare. I can host servers in EU West and USA.

Then we're back to the initial issue (not an issue, just a thought though), we need10Mbit/s Server with high availability.

[FallenAngel666]

To some people, it is maybe also concerning to use Cloudflare under EU privacy law and in some open source communities. Especially when this happens automatically. There could be tracking etc.

Given the request is sent over plain HTTP (checked my own matray config to confirm), I think the fact @y5's is proxied by Cloudflare is the least of your worries!

I get the concern and I think one thing that might open up a can of worms is Cloudflare's analytics. I would encourage y5 to not use Cloudflare as I really don't see what benefit it'd provide in this case for such a simple service.

I totally agree! We also are drifting to offtopic here... sorry.

The goal is: find someone who is willing to host the server as is. Best option without CloudFlare if it wasn'
t in use before. All other improvements and Ideas shall go to the GitHub issue page of that particular repo. Which would be

• using HTTPS
• optional: using multiple hosts as feed source
• • optional: using load balancing if it makes sense

[Sommerwiesel]

To some people, it is maybe also concerning to use Cloudflare under EU privacy law and in some open source communities. Especially when this happens automatically. There could be tracking etc.

Given the request is sent over plain HTTP (checked my own matray config to confirm), I think the fact @y5's is proxied by Cloudflare is the least of your worries!
I get the concern and I think one thing that might open up a can of worms is Cloudflare's analytics. I would encourage y5 to not use Cloudflare as I really don't see what benefit it'd provide in this case for such a simple service.

I totally agree! We also are drifting to offtopic here... sorry.

The goal is: find someone who is willing to host the server as is. Best option without CloudFlare if it wasn' t in use before. All other improvements and Ideas shall go to the GitHub issue page of that particular repo. Which would be

• using HTTPS
• optional: using multiple hosts as feed source
• • optional: using load balancing if it makes sense

Then I'll write again what I said in my first post.
I can provide an ultra high availibility Server (99.997% in the last 6 months) without Cloudflare right now.
DNS is ready.
At this point, moson-mo just has to decide who he wants/trusts with this.

Signing off for today. Its 8pm in Germany.

[y5]

To some people, it is maybe also concerning to use Cloudflare under EU privacy law and in some open source communities. Especially when this happens automatically. There could be tracking etc.

Cloudflare does not proxy requests for my server, it is merely a DNS provider at the moment.

[moson-mo]

@Sommerwiesel Can you send me a PM at the Manjaro forum?

[moson-mo]

are you going to change the matray config to point to manjaro.rob.social or keep manjaro.moson.eu and point that to y5's server?

I'd be changing the URL on the client side. (I'm totally against forcefully changing users config settings, but in this case it makes sense). It (manjaro.moson.eu) still refers to a server that runs a manjaro mirror atm, but I also want to get rid of that subdomain at some point. So the successor should have it's own domain...

Would be also a good idea to put these two lines in the configs somewhere, so it's not hard coded.

Tha's just the appid, not a URL or anything.

Additionally is it normal for it to fetch news on non-standard ports with no TLS? Does matray even support TLS?

No, while it's not a big deal to implement in mnserver itself, I'd simply reverse proxy it with nginx doing the TLS termination...
I agree it should be going with TLS. Just didn't bother back then...

[iruizr7]

The RSS feed is about 500 KB in size. If you have ~24 requests per second, each one receiving this amount of data, you will pretty quickly bring the forum server to it's knees.

Woa! It seems that we have more voluntaries than I initally thought! That's nice! FOSS power!

Anyway, I am curious why you think 24 hits/s with 500K size, summing to 12MB/s, represents a challenge, even for the most ridiculous server. I doubt Manjaro would be hosting the forum in a Raspberry Pi... Heck, even the Rpi3/4 would bear that load without sweating. And that feed would seriously benefit from some basic caching, so, basically serving static content!

Maybe I am missing something or presuming so much about the core Manjaro Team. As I said, I am using Manjaro for a long time, but never had taken the leap to help (Don't know where I could do it best, tbh), maybe the are much greater resource-constrained that I thought.

So, maybe this serves me as a stepping-stone. There are better servers here, offered, than mine, which was simply a little space at my ARM cluster at home. But if you could point me to some other needed work at Manjaro, I could provide some synergies.

Greetings everyone!

[moson-mo]

summing to 12MB/s, represents a challenge, even for the most ridiculous server.

The bandwidth itself is probably not an issue.
I don't know how the discourse software is generating this data internally. In any case it's a total waste of bandwidth and computing on both ends, the server and client...

[BadAtIrcBots]

The RSS feed is about 500 KB in size. If you have ~24 requests per second, each one receiving this amount of data, you will pretty quickly bring the forum server to it's knees.

Anyway, I am curious why you think 24 hits/s with 500K size, summing to 12MB/s, represents a challenge, even for the most ridiculous server. I doubt Manjaro would be hosting the forum in a Raspberry Pi... Heck, even the Rpi3/4 would bear that load without sweating. And that feed would seriously benefit from some basic caching, so, basically serving static content!

That's still about 30 TB per month (I think?) spent on just downloading that RSS feed over and over. It'd be one thing if it was cached on disk, but almost inevitably Discourse is doing some processing with each request and would likely get significantly bogged down regenerating the RSS feed over and over.

Plus the tray app is configured to phone in every 10 minutes and would consume about 72 MB of the user's Internet data per day if it directly polled that RSS feed.

[moson-mo]

Big thanks to @Sommerwiesel. He will take over hosting of the service.
Also thanks to everyone offering help here.

[SergSel2006]

On one update I saw that manjaro has its own RSS feed... I found link, we should investigate how to use it for matray:
Stable update: https://forum.manjaro.org/c/announcements/stable-updates.rss
Unstable update: https://forum.manjaro.org/c/announcements/unstable-updates.rss
Testing update: https://forum.manjaro.org/c/announcements/testing-updates.rss
And some other on just: https://forum.manjaro.org/c/announcements/
Why matray don't use it? Or it had made recently?

[BadAtIrcBots]

On one update I saw that manjaro has its own RSS feed... I found link, we should investigate how to use it for matray: Stable update: https://forum.manjaro.org/c/announcements/stable-updates.rss Unstable update: https://forum.manjaro.org/c/announcements/unstable-updates.rss Testing update: https://forum.manjaro.org/c/announcements/testing-updates.rss And some other on just: https://forum.manjaro.org/c/announcements/ Why matray don't use it? Or it had made recently?

The mnserver fetches these RSS feeds, caches them (for 10 mins), then breaks them down into a significantly simplified JSON payload that only includes exactly what matray needs to display news items.

So matray is using these RSS feeds in a roundabout fashion and mnserver is saving the project and end users a significant amount of bandwidth as the current settings are set to poll for an update every 10 minutes. If matray polled every 24 hours instead, for instance, it could perhaps use the RSS feeds. Doing so would probably require a major re-architecture of the application and given the author has moved on and only committed to maintenance, that ain't happening.