Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packaging for the AUR #641

Open
PolarianDev opened this issue May 14, 2023 · 58 comments
Open

Packaging for the AUR #641

PolarianDev opened this issue May 14, 2023 · 58 comments
Labels
distribution Regarding app distribution including in app stores

Comments

@PolarianDev
Copy link

PolarianDev commented May 14, 2023

Hello world,

So I found saber via fdroid because it bumps recently updated, or added apps to the top of the store. I also realise that the app is not contained in any official arch repository, nor the AUR, so I intended to package it.

The reason I am making an issue to inform you of this is because I believe in communication between a package maintainer and upstream to ensure the highest quality builds possible.

I would be submitting saber to the AUR today, however due to dependency issues this can not be done.

Flutter is within the AUR, but the package is disfunctional and refuses to build any flutter application, due to the fact it constantly tries to connect to google servers, and pull stuff down which is not possible. Therefore, I must write up one (or more) patches to flutter before I can even build saber.

I do notice you provide an Appimage for Linux, I can use this to create a -bin package, but these are generally avoided for the sake of security and privacy, as nobody is able to verify the code compiled is not malicious when its in binary form, which is why source builds are preferred.

So for saber-bin I can most likely submit that by the end of the day, the appimage should contain a natively built executable.

If the package is voted, and popular, and a official package maintainer (Formerly TU) is willing to take the package on, it will be moved into the official repository where anyone can pull it directly through the use of pacman.

Have a good day,
Polarian

@PolarianDev
Copy link
Author

Also I left you a £2 donation via liberapay, probably nothing for a university student but its better than nothing, keep up the good work! :)

@adil192
Copy link
Member

adil192 commented May 14, 2023

Hi, I'm thankful for any donation, especially since it lets me publish to the App Store.
Also if it's relevant, I recall that @fybx mentioned packaging for the AUR a while ago in

@PolarianDev
Copy link
Author

hm I can't find it under the AUR though...

@PolarianDev
Copy link
Author

image

I searched again, definitely not there... unless it was released under a different name?

@adil192
Copy link
Member

adil192 commented May 14, 2023

Yeah I don't know how far they got

@PolarianDev
Copy link
Author

I will email them now and ask, thanks for letting me know :)

@adil192 adil192 added the distribution Regarding app distribution including in app stores label May 14, 2023
@Midou36O
Copy link

Any update regarding this? I'd love to use this app but i'm somewhat struggling with manually compiling it myself :)

@PolarianDev
Copy link
Author

@Midou36O Hey,

I have got it building in a clean chroot, but I have not written up the PKGBUILD, I might be able to get it done tomorrow :)

Remember to vote for the package once I push it to the AUR :)

Polarian

@PolarianDev
Copy link
Author

Ok some bad news,

I am currently struggling to get saber building with the package now.

There is currently no standard for packaging flutter software for Arch Linux, and due to issues I brought up about this a package maintainer gromit (formerly known as Trusted User) has drafted a WIP guideline for it:

https://wiki.archlinux.org/title/User:Gromit/Flutter_package_guidelines

However, its not fully working and I will need to speak to gromit (which has a working flutter package) to see if he can help get saber working.

Until then though, the appimage works perfectly on Arch Linux, but a distribution package would be far better and would eliminate dependency duplication which appimages have a lot of issues with.

@Midou36O I will let you know if I make any more progress on this issue.

Polarian

@Midou36O
Copy link

Alright, good luck! And thanks for your contribution. I will patiently wait then.

@PolarianDev
Copy link
Author

I have been working with gromit to get a PKGBUILD draft which works, and I have now got a package which builds on my laptop, bare in mind this has not been tested within a clean chroot, and therefore could still break.

What is working so far is the package builds, however the simlink to /usr/bin is broken, which is a simple fix.

@adil192 Have you got a .desktop file anywhere here? or do I need to write one for the package.

It would help if upstream provides one along with the asset for the icon :)

Polarian

@adil192
Copy link
Member

adil192 commented May 31, 2023

@adil192 Have you got a .desktop file anywhere here? or do I need to write one for the package.

The .desktop file I use is at flatpak/com.adilhanney.saber.desktop:
https://github.com/adil192/saber/blob/97fe2a6809936dd4c0a4babfef9ed093e87c1158/flatpak/com.adilhanney.saber.desktop#L1-L13

It would help if upstream provides one along with the asset for the icon :)

There's various sizes of icons at https://github.com/adil192/saber/tree/main/assets/icon/resized. If it helps, I use this small list of commands to copy them into the right place when I build the Flatpak:
https://github.com/adil192/saber/blob/97fe2a6809936dd4c0a4babfef9ed093e87c1158/flatpak/com.adilhanney.saber.json#L51-L58

@PolarianDev
Copy link
Author

Thats I can pull this from the source, might need to write a patch though to patch the .desktop to arch's liking, but apart from that this saves me a ton of time :)

@adil192
Copy link
Member

adil192 commented May 31, 2023

Thats I can pull this from the source, might need to write a patch though to patch the .desktop to arch's liking, but apart from that this saves me a ton of time :)

Could the patch be applied on this repository, or would that interfere with flatpak?

@PolarianDev
Copy link
Author

I have already patched it, it is applied during the prepare stage, all it does is move the flatpak .desktop file to the base path of the source, and rename it to saber.desktop and renaming the icon file expected to saber, then the icon is copied during the build stage into the base directory of the source called saber.png, both are then installed to their respective paths.

I have pushed the first version to a git repository for testing:

https://git.polarian.dev/AUR/saber

however unfortunately when testing, a library libsuper_native_extensions.so: is not in the build directory along with the rest of the saber binaries, and once the build files are removed you get a loader error:

saber: error while loading shared libraries: libsuper_native_extensions.so: cannot open shared object file: No such file or directory

So I have to find this library and figure out where to stick xD

I guess time to ldd the saber binary.

@PolarianDev
Copy link
Author

They are the plugins, and they are not a part of the bundle, and they are spat out in with the cmake files, now I got to figure out where to stick these libs in the install path, and then I also got to strip out the sources because its unnecessary.

I will speak to gromit tomorrow and see if he has any ideas.

@Midou36O Hopefully tomorrow you can have a test with the package, but tonight I don't think I will be able to get it working tonight, it still has not be successfully built in a clean chroot.

also after the package is pushed to the AUR then I got to document it on the archwiki, along with all the little issues, such as flutter demanding rustup and not cargo which means the toolchain needs to be pulled from upstream and not from the arch repository, which is a massive drain of bandwidth when I have a local copy of the arch repository for fast package testing and building.

So yes, still a lot to do!

@Luciogi
Copy link

Luciogi commented Jun 1, 2023

Hi, I created PKGBUILD for arch linux
https://codeberg.org/Luciogi/saber-pkgbuild
I am not gona upload to AUR, because I don't want to maintain it.
Anyone can take it and maintain it


Edit 1 : lol i did not checked , it is made by PolarianDev
Edit 2: After reviewing https://git.polarian.dev/AUR/saber , my version had minor issue

  • Not adding icon to default arch linux path
  • I doubt about gtk3 runtime dependency. Is is really required?

Speaking of rustup, simple solution would be to stick to it. I guess so it is hardcoded or something.

@PolarianDev
Copy link
Author

@Luciogi Yes gtk3 is required, this was something I discussed with gromit as I am also working on the flutter packaging guidelines.

if you ldd saber you will see it has been dynamically linked to gtk3 libraries, if these do not exist at runtime you will have a loader error, therefore gtk3 is a depend not a makedepend.

I have realised looking at yours that I have missed out some libs specified within README.md in saber's repository.

@adil192 What is the following libs used for (ubuntu packages, I see @Luciogi has used the arch equivalents):

libsecret-1-dev libjsoncpp-dev

Its always better to have dependency issues and then add the ones you need, instead of listing everything and half of it never being used, therefore pulling in tons of packages, but this is my personal belief, I am sure some people out there would lean on the side of caution.

@Luciogi As for rustup, unfortunately you must use it, flutter looks explicitly for the rustup binary, I have tried getting cargo to work, as cargo will pull in all the rust components directly from the Arch Repository which speeds up arch builds, as I got my own arch mirror on the build server and can pull at about 550MB/s (4.4gbps), when having to pull rust binaries this must be done from WAN and is a lot slower. I believe pkgbuild.com also has its own mirror for the same reason, using disk or LAN resources instead of flooding the WAN (and wasting bandwidth).

You should not configure rustup within the PKGBUILD, its outside the scope of the package, unfortunately until cargo or rust (ultimately the same package) works with building flutter apps, this step will need to be done manually, as rustup toolchain install stable will delete the toolchain and reinstall every time you build, which is not what we want.

@adil192 if it is your build scripts explicitly calling rustup, please let me know, I am aware super_clipboard is the library using rust.

Polarian

@Luciogi
Copy link

Luciogi commented Jun 1, 2023

What is the following libs used for (ubuntu packages, I see @Luciogi has used the arch equivalents):
libsecret-1-dev libjsoncpp-dev

libsecret would be used for managing username and password for nextcloud login
linjsoncpp is used , as .sbn is actually json file

Its always better to have dependency issues and then add the ones you need, instead of listing everything and half of it never being used, therefore pulling in tons of packages, but this is my personal belief, I am sure some people out there would lean on the side of caution.

above libs are necesarry for runtime

You should not configure rustup within the PKGBUILD, its outside the scope of the package, unfortunately until cargo or rust (ultimately the same package) works with building flutter apps, this step will need to be done manually, as rustup toolchain install stable will delete the toolchain and reinstall every time you build, which is not what we want.

I did it because rustup and rust both packages are conflict , so choosing rustup should not effect usage of rust.
As for deletion thing, rustup install toolchain at ~/.rustup/toolchains/ which remains installed , until user explicitly delete this directory

@Luciogi
Copy link

Luciogi commented Jun 1, 2023

I noticing weird behavior,
When building package and executing went fine app works
after rebooting system , then running app reports missing of libsuper_native_extensions.so, though other libs are at same location /opt/saber/lib , this single .so file causing problem


Edit: Tried this solution https://aur.archlinux.org/packages/flemozi#comment-909654 but it is also not working
What worked .. is to make symbolic link of /opt/saber/lib -> /usr/lib/

@PolarianDev
Copy link
Author

PolarianDev commented Jun 1, 2023

Hello,

The solution linked there does actually work, I have tested it on my package.

The issue is that flutter doesn't expect the libraries to be in /opt changing the path for libraries works...

I have the solution in progress... bare with me please :)

Polarian

@PolarianDev
Copy link
Author

I am still waiting for Adil to verify both dependencies above are needed, I could ldd saber to see if they are dynamically linked, if so they are needed at runtime.

Apart from that I will probably be able to push the package to the AUR within a few hours.

Polarian

@PolarianDev
Copy link
Author

@Luciogi I would also like to point out that we do not create the .desktop file unless it is not contained within upstreams repository.

A simple patch makes it usable within AUR, as you can see in my repository.

Polarian

@Luciogi
Copy link

Luciogi commented Jun 1, 2023

yes. I not did got idea to see .desktop file in flatpak directory xd


Tried this solution https://aur.archlinux.org/packages/flemozi#comment-909654 but it is also not working

using $LD_LIBRARY_PATH works for me

@PolarianDev
Copy link
Author

yes. I not did got idea to see .desktop file in flatpak directory xd
Well Adil did make a post explaining it :/

using $LD_LIBRARY_PATH works for me
As it should

I will fix the package shortly, then you and @Midou36O can test it, if you are up to it?

@Luciogi
Copy link

Luciogi commented Jun 1, 2023

yup I m with you

@PolarianDev
Copy link
Author

@Luciogi @Midou36O I have got it building, under annoying circumstances, so you must follow the following build steps to work around issues with flutter:

  1. clone the repository
  2. install flutter, and rustup
  3. run: git config --global --add safe.directory /opt/flutter to prevent flutter error, this is a bug within the flutter package.
  4. run: rustup toolchain install stable
  5. run: rustup default stable
  6. finally makepkg and it should build successfully

Let me know if you have any issues, I have installed this to my system and it works fine.

Thanks for the help,
Polarian

@Midou36O
Copy link

Midou36O commented Jun 1, 2023

Looks like it's failing here following your guide.

Error: The Flutter directory is not a clone of the GitHub project.
       The flutter tool requires Git in order to operate properly;
       to install Flutter, see the instructions at:
       https://flutter.dev/get-started

And yes i followed the 3rd step properly, using root (idk if that was a good or bad idea 👀 ) and user.

@PolarianDev
Copy link
Author

Looks like it's failing here following your guide.

Error: The Flutter directory is not a clone of the GitHub project.
       The flutter tool requires Git in order to operate properly;
       to install Flutter, see the instructions at:
       https://flutter.dev/get-started

And yes i followed the 3rd step properly, using root (idk if that was a good or bad idea eyes ) and user.

This is another issue with the flutter aur package.

Also you should never build with root, makepkg should stop you however, so I do not know why it did not stop you here...

to fix this, simply:

mv "/opt/flutter/.git-refs" "/opt/flutter/.git"

This was taken from the flutter repository:

https://aur.archlinux.org/packages/flutter#comment-916623

Good luck,
Polarian

@Midou36O
Copy link

Midou36O commented Jun 1, 2023

Okay now i get this

patching file saber.desktop (already renamed from flatpak/com.adilhanney.saber.desktop)
Analytics reporting disabled.

You may need to restart any open editors for them to read new settings.
Setting "enable-linux-desktop" value to "true".

You may need to restart any open editors for them to read new settings.

And then it fails during the preparation :p

(Note: i didn't run the makepkg command as root, it was the git command.)

@PolarianDev
Copy link
Author

@Midou36O just type y when it promts you.

@Midou36O
Copy link

Midou36O commented Jun 2, 2023

Hmm, those don't really ask to input a command. Anyway, i checked the PKGBUILD file and proceeded to manually run the commands, looks like this specific command is causing issues.

flutter --no-version-check pub get 
[ble: exit 66]

Exit code: 66, searching this on google it says that apparently flutter cannot access some sort of directory due to permissions issues (i sure do hope it's not because of the git command i ran as root).

I deleted .pub-cache and tried doing the same thing, it just took longer before throwing the "66" exit code again.

@PolarianDev
Copy link
Author

Hmm, those don't really ask to input a command. Anyway, i checked the PKGBUILD file and proceeded to manually run the commands, looks like this specific command is causing issues.

flutter --no-version-check pub get 
[ble: exit 66]

Exit code: 66, searching this on google it says that apparently flutter cannot access some sort of directory due to permissions issues (i sure do hope it's not because of the git command i ran as root).

I deleted .pub-cache and tried doing the same thing, it just took longer before throwing the "66" exit code again.

This is when you have not run the following command:

git config --global --add safe.directory /opt/flutter

@Midou36O
Copy link

Midou36O commented Jun 2, 2023

image
I did run this command as you instructed :p

@Luciogi
Copy link

Luciogi commented Jun 2, 2023

run build by appending -v in flutter command to get full logs

@Luciogi
Copy link

Luciogi commented Jun 2, 2023

Build successfully :) with your PKGBUILD


@PolarianDev append --prefixed-errors in flutter build command, it would show errors to console

@PolarianDev
Copy link
Author

Build successfully :) with your PKGBUILD

@PolarianDev append --prefixed-errors in flutter build command, it would show errors to console

sure, I will add that now

@PolarianDev
Copy link
Author

now I got to rebuild in clean chroot 😞

@PolarianDev
Copy link
Author

@Midou36O if you still can't build it please read below!

I have deployed the final version of saber (I am about to push it to aur) to my unofficial arch repository, which you can see on ArchWiki, website contains instructions.

The official repository will allow you to pull saber directly through pacman, my repository is signed so you will need to import my signature, which is on the website link on the archwiki.

I will also document the build steps and troubleshooting issues on archwiki when creating the saber page.

Enjoy!

Polarian

@PolarianDev
Copy link
Author

Link to saber on aur: https://aur.archlinux.org/packages/saber

@PolarianDev
Copy link
Author

Link to ArchWiki page (WIP): https://wiki.archlinux.org/title/Saber

@Midou36O
Copy link

Midou36O commented Jun 2, 2023

@Midou36O if you still can't build it please read below!

I have deployed the final version of saber (I am about to push it to aur) to my unofficial arch repository, which you can see on ArchWiki, website contains instructions.

The official repository will allow you to pull saber directly through pacman, my repository is signed so you will need to import my signature, which is on the website link on the archwiki.

I will also document the build steps and troubleshooting issues on archwiki when creating the saber page.

Enjoy!

Polarian

Yep, the repo worked, although i had to use TrustAll for your repo :S. But saber runs perfectly fine now, thanks a lot!

@PolarianDev
Copy link
Author

@Midou36O if you still can't build it please read below!
I have deployed the final version of saber (I am about to push it to aur) to my unofficial arch repository, which you can see on ArchWiki, website contains instructions.
The official repository will allow you to pull saber directly through pacman, my repository is signed so you will need to import my signature, which is on the website link on the archwiki.
I will also document the build steps and troubleshooting issues on archwiki when creating the saber page.
Enjoy!
Polarian

Yep, the repo worked, although i had to use TrustAll for your repo :S. But saber runs perfectly fine now, thanks a lot!

You shouldn't need to my documentation states you need to run the following commands:

curl https://git.polarian.dev/AUR/polarrepo/raw/branch/master/pkg/keys/polarian.asc | sudo pacman-key --add -

You can substitute sudo with doas, or remove it entirely if ran as root. This command will fetch my key from my repository (gpg keyservers are not the most reliable, and this allows for a one line command).

Once pacman gpg store has my key, you then need to locally sign it trusting that it is valid:

pacman-key --lsign-key 0770E5312238C760

And yes, this is my valid key if you are wondering xD

After that it should be able to verify the authenticity of all files within the repository along with the package database (which is also signed for security)

@Midou36O
Copy link

Midou36O commented Jun 2, 2023

I opted for the easiest road since both locally signing it and using trustall for the specific repo were not recommended hehe. I proceeded to do it properly as you instructed me to do.

And regarding the app:
image
B)

@PolarianDev
Copy link
Author

I would like a link to where it says local signing is not recommended, I know it is not advised because Arch Linux staff will not take responsibility for any abuse by third party repositories, but TrustAll is not recommended because you can not verify the authenticity of the files you are downloading, this is especially important if you use http instead of https (but I assumed you use https like I have listed on my repository, because there is little reason not to).

And I am glad the package works fine :)

@PolarianDev
Copy link
Author

^^^ Feel free to submit any issues with the package to the repository: https://git.polarian.dev/AUR/saber/issues

Unless its a known issue with upstream :)

Also dont forget to vote for the aur package so its bumped up so more people can find the package.

@Midou36O
Copy link

Midou36O commented Jun 2, 2023

I would like a link to where it says local signing is not recommended, I know it is not advised because Arch Linux staff will not take responsibility for any abuse by third party repositories, but TrustAll is not recommended because you can not verify the authenticity of the files you are downloading, this is especially important if you use http instead of https (but I assumed you use https like I have listed on my repository, because there is little reason not to).

And I am glad the package works fine :)

Looks like you're looking for this https://wiki.archlinux.org/title/Pacman/Package_signing#Signature_is_unknown_trust (4th mitigation option.)

@PolarianDev
Copy link
Author

I would like a link to where it says local signing is not recommended, I know it is not advised because Arch Linux staff will not take responsibility for any abuse by third party repositories, but TrustAll is not recommended because you can not verify the authenticity of the files you are downloading, this is especially important if you use http instead of https (but I assumed you use https like I have listed on my repository, because there is little reason not to).
And I am glad the package works fine :)

Looks like you're looking for this https://wiki.archlinux.org/title/Pacman/Package_signing#Signature_is_unknown_trust (4th mitigation option.)

Unfortunately there is no way around this, see:

https://wiki.archlinux.org/title/Pacman/Package_signing#Adding_unofficial_keys

@PolarianDev
Copy link
Author

Ok I have finished the wiki as best as possible for now, I will ask for it to be reviewed to see if I made any minor errors.

AUR package I wlil also ask gromit to review to see if he can find any potential issues, apart from that I don't think there is much more to do here other than maintenance.

@Luciogi
Copy link

Luciogi commented Jun 2, 2023

@PolarianDev close this issue

@PolarianDev
Copy link
Author

@PolarianDev close this issue

Shortly, I am still waiting on the finalisation.

@PolarianDev
Copy link
Author

@adil192 I can not update the package to the version you pushed today because flutter is broken, without any response from the devs or the package maintainers for arch.

Currently when flutter is outdated the tool self destructs, so unfortunately I can't build saber until then.

@PolarianDev
Copy link
Author

Just a quick note, I have not forgot about this package but currently have other priorities.

If you have experience with Arch Linux package maintaining, I am happy to give you co-maintainer so you can also help keep saber up to date.

However as I have explained on the wiki, the saber package is flaky due to the bad flutter support on Arch Linux.

@PolarianDev
Copy link
Author

The latest version pushed today (v0.15.2) has been built successfully, I have pushed it to the AUR.

Enjoy!

@PolarianDev
Copy link
Author

Hello,

It has happened again, yet again flutter has been updated, and the google team messed up the update and overwrote another version meaning that flutter is broken until the next flutter update, I attempted to update saber but I can't compile it because flutter is broken:

https://git.polarian.dev/AUR/saber/pulls/6#issuecomment-217

I will reattempt building saber once flutter is updated.

I apologise for the constant pain of dealing with awful flutter support, I am trying my best.

The flatpak or appimage can be used in the meantime.

Take care,
Polarian

@PolarianDev
Copy link
Author

Latest flutter version has worked within Arch Linux, v.0.15.5 has now been pushed to the AUR and is available prebuilt on polarrepo.

Man dealing with flutter is a right pain, but hey the latest version of saber is working natively again \o/

@PolarianDev
Copy link
Author

Latest version (0.17.0) has been pushed to the AUR and polarrepo.

However when building this I saw the following:

! bson 4.1.0 from git https://github.com/adil192/bson at a176e3 (overridden)
  collection 1.17.2 (1.18.0 available)
  csv 5.0.2 (5.1.0 available)
  dynamic_color 1.6.7 (1.6.8 available)
  file 6.1.4 (7.0.0 available)
  flutter_keyboard_visibility 5.4.3 (retracted)
  flutter_lints 2.0.3 (3.0.0 available)
  flutter_quill 7.4.7 (7.8.0 available)
  go_router 11.1.4 (12.0.0 available)
  irondash_engine_context 0.3.1 (0.4.0 available)
! json2yaml 3.0.1 from git https://github.com/adil192/json2yaml at 618fc6 (overridden)
  lints 2.1.1 (3.0.0 available)
  material_color_utilities 0.5.0 (0.8.0 available)
  meta 1.9.1 (1.11.0 available)
! pdf 3.10.5 from git https://github.com/DavBfr/dart_pdf at 41cbf8 in pdf (overridden)
! perfect_freehand 1.0.4 from git https://github.com/adil192/perfect-freehand-dart at f26bc5 (overridden)
  petitparser 5.4.0 (6.0.1 available)
  platform 3.1.0 (3.1.3 available)
! printing 5.11.1 from git https://github.com/DavBfr/dart_pdf at 41cbf8 in printing (overridden)
  process 4.2.4 (5.0.1 available)
  process_run 0.13.1 (0.13.2 available)
  stack_trace 1.11.0 (1.11.1 available)
  stream_channel 2.1.1 (2.1.2 available)
  super_clipboard 0.6.4 (0.7.0 available)
  super_native_extensions 0.6.4 (0.7.0 available)
  test_api 0.6.0 (0.6.1 available)
  uuid 3.0.7 (4.1.0 available)
  vm_service 11.7.1 (12.0.0 available)
  web 0.1.4-beta (0.3.0 available)
  webdriver 3.0.2 (3.0.3 available)
  xml 6.3.0 (6.4.2 available)

Seems a lot of the dependencies are outdated @adil192 , I doubt this matters too much, just a small heads up.

Take care,
Polarian

@adil192
Copy link
Member

adil192 commented Oct 23, 2023

This is normal don't worry

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
distribution Regarding app distribution including in app stores
Projects
None yet
Development

No branches or pull requests

4 participants