About Manjaro and the AUR, and updates

[KierPrev]

It's important to recognize that any distribution can break, especially in the hands of those who enjoy tinkering with their systems. Damn, even my first Ubuntu installation was destroyed when I was first starting out with desktop Linux.

One of the most common complaints is that Manjaro delays packages for two weeks, but what people fail to mention is that this only applies to specific branches. It's important to take this into consideration before jumping to conclusions. For people that doesn't use the AUR, or just for an old AUR package, Stable branch is great!

The criticisms surrounding the AUR breakage problem have been blown out of proportion and lack context. In fact, problems are typically isolated to a single package, and only occur if you use core system components from AUR.

As you said, Manjaro is a good starting point for beginners who want to explore an Arch based like distro. Even if you, or me, use vanilla Arch or Arch-based.

It's crucial to exercise caution when using any Linux distribution and to avoid jumping to conclusions based on anecdotal evidence.

Thank you, and sorry for diving once again in the topic.

[philmmanjaro]

Pamac is a very popular graphical package manager. This is also known to the Arch Developers as they track usage with the user agent the application provides. So the AUR is a special user maintained database of PKGBUILD scripts of almost all software out there. People even search in the AUR before they decide to install a distro.

DDOS the AUR is such a buzz word. Yes, we managed to get the AUR website many times down. This was due to several reasons. Pamac makes it very simple to activate the AUR by just a slider. People love to search for applications, and in an early version it started to search by every key stroke the user made. Since Manjaro nowadays as over 500.000 users online and we have about 13+ million active installations of the system it is clear that we create a lot of traffic on server infrastructure.

Traffic report last 24 hours on 2023-04-03 09:52 ICT

So how did we fix all of that? Well, we didn't do it alone. We had help and support by developers and even company partners. In such cases you can only do it with partners.

With Pamac 10.5.0 release we even optimized on how the search functions in general, but step by step.

• first we delayed the search and not issue a query when there was a keystroke
• then Arch AUR Developers started to create a database which we downloaded. (Due to our big user base even that created issues)
• we transferred the 8 MB to our own CDN infrastructure which is sponsored by CDN77
• now we pre-load the databases locally so pamac does searches "offline"

It is a process and complex issues won't get solved in a blink of a second.

So to conclude:

• we shipped a new version of pamac to our stable branch that accidentally sent thousands of requests on the 2020-04-26 to the AUR per user. This rendered the AUR offline for all users across every Arch-based distro for a few hours.
• on the 2021-10-14 we shipped a new pamac version to our stable branch, which includes an updated search feature across the application. However it resulted in pamac being blocked again. This may have been the cause for the day’s earlier outage.

It simply showed that our huge user base is not suitable for the provided infrastructure by Arch Linux. Hence we have now our own and deal with our own issues regarding all things related to AUR databases.

It may suck that some IT service is down. Even more if it is a very popular one. It shows however how fragile things can be and with a wrong set of requests send by a large user base those IT infrastructure most likely won't hold stand and goes down. Also remember, why should we even want to DDOS a service we offer to use by our users as well. Those two cases showed us and the AUR Developers on how to improve the service and their infrastructure they host it on.

Since pamac has a user agent per distribution, why not ask the AUR website maintainers if Manjaro even uses their infra at all, after the last DDOS attack from us ...

Further info

https://gitlab.manjaro.org/applications/pamac/-/issues/1017
https://gitlab.manjaro.org/applications/pamac/-/issues/1135
https://gitlab.manjaro.org/applications/pamac/-/issues/1161

[arindas]

Thank you for mitigating the problems and providing such a thorough technical statement. I have updated the document to reflect the above information. Additionally, I have directly linked the messages in this issue.

[KierPrev]

Thanks to you for reflecting on the issue, and do the right thing: telling the plain truth and modifying the statements to better reflect what happened.