[WIP] New packages: proton-bridge-1.2.7 and dependency

[cinerea0]

I'm trying to package proton-bridge, but I'm running into some issues with the build process. Here's the relevant output from xbps-src:

make: qtdeploy: No such file or directory
make: *** [Makefile:66: cmd/Desktop-Bridge/deploy/linux/proton-bridge-1.2.7-live.1] Error 127
=> ERROR: proton-bridge-1.2.7_1: do_build: '${make_cmd} CC="$CC" CXX="$CXX" LD="$LD" AR="$AR" RANLIB="$RANLIB" CPP="$CPP" AS="$AS" OBJDUMP="$OBJDUMP" CFLAGS="$CFLAGS" LDFLAGS="$LDFLAGS" ${makejobs} ${make_build_args} ${make_build_target}' exited with 2
=> ERROR:   in do_build() at common/build-style/gnu-makefile.sh:9

I know that xlint throws up errors on this template as well, I thought I should fix those after solving the build errors. Any other suggestions for improvement are welcome as well!

As for qtdeploy, the best I can figure is that it's a tool provided by this repository. Unfortunately, the repo has no releases, so the bindings and tools it provides can't be packaged. ericonr on IRC said that it might be possible to build without qtdeploy, but I haven't understood the Makefile well enough yet to figure out how to do that.

[ericonr]

I corrected myself later, it seems like qtdeploy is indeed necessary for it, which is a damn shame. The Makefile calls it directly.

qtdeploy is here: https://github.com/therecipe/qt/blob/master/cmd/qtdeploy . No releases so far.

[cinerea0]

That leaves things in a bit of a bind. I've opened an issue in that repository to request a release, but I have no idea if it will be answered.

[ProjectMoon]

I think the main reason there's no release of qtdeploy is because it's meant to be used for building go applications as a library/module thingy. Then you upload your super cool static binary somewhere for people to download.

But seeing as how the Go QT bindings have been around for quite awhile and don't seem to show any signs of making a proper release as such, wouldn't this be one of the good reasons to package qtdeploy despite it not having a proper version as such?

To be honest I find it weird that they don't at least tag stuff in that repository.

Edit: there are distinct "releases" of the Docker images, although I'm guessing they're just tied to commits.

[cinerea0]

@ProjectMoon To clarify, are you saying that this could be a situation where it would be appropriate to package the bindings or just qtdeploy in spite of the "must be tied to a release" rule?

[ProjectMoon]

Yes. Or use qtdeploy locked to a specific tag in the build for this package.

[ericonr]

If we could use the same commit as the ones tagged in the docker images, that would be great. Then document this in the template. Or we can go wild and just use the version that it downloads, which I dislike.

[cinerea0]

@ericonr I'll get to work packaging those go bindings, then. I'm not familiar with standard procedure in cases like this: Should I close this PR, open a new one for the bindings once I have it finished, and then open a PR for proton-bridge once the bindings have been accepted, or should I add the template for the bindings to this pull request?

[ericonr]

@cinerea0 since it's a tool to use in this build, I'd put it in this same PR.

[cinerea0]

Just wanted to give an update, this is currently blocking on ProtonMail/proton-bridge#52 because proton-bridge fails to build either manually or with xbps-src. I have managed to package qt5-go successfully, though.

[ProjectMoon]

I was able to build the bridge some time ago. But I had the go qt bindings installed via Go.

[Chocimier]

pkg-config: exec: "pkg-config": executable file not found in $PATH

Looks like missing pkg-config.

[cinerea0]

I added pkg-config in "hostmakedepends", and I also added "libsecret" because that error came up afterwards. However, that resulted in the following error:

# pkg-config --cflags  -- libsecret-1
Package libsecret-1 was not found in the pkg-config search path.
Perhaps you should add the directory containing `libsecret-1.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libsecret-1' found
pkg-config: exit status 1

Does this mean I have to do some patching to rename "libsecret" to "libsecret-1"? Also, I've added the most recent version of the templates for review.
I should also note that the compilation error (or at least that's what it looks like) is still happening. Here's the full output as of right now.
proton-bridge-build.log

[Chocimier]

libsecret-1.pc is part of libsecret-devel.

pkg-config, git and other packages installed for executables should go into hostmakedepends. gcc, make are always available during build as part of base-chroot.

Compiler output are warnings, not error.

[cinerea0]

Thank you for the tip about libsecret-devel! That got rid of that error and all the compiler warnings, but I'm now encountering this new one:
make: *** No rule to make target 'install'. Stop.
As a side note, I have a manual post_install function that just does vlicense LICENSE. Is that necessary, or does xbps-src take care of that automatically if the license is just called "LICENSE"?

[Chocimier]

gnu-makefile build style calls make install to install package, but there is no install target defined by authors. You need to define own do_install function, using eg. vbin, vcopy. Look in their documentation for what to install.

[cinerea0]

Upon updating the templates, I'm getting two new errors:
ERROR: proton-bridge-1.2.7_1: failed to install 'go>=1.13' dependency! (error 34)
ERROR: qt5-go-0.1_1: SHA256 hash is not valid: Numerical result out of range
Additionally, how should icons be handled in the do_install function?

[abenson]

Each template should have its own commit, but both should be in this PR

[abenson]

I've gotten it to this point

=> proton-bridge-1.2.7_1: running post-install hook: 06-strip-and-debug-pkgs ...
   Stripped executable: /usr/bin/proton-bridge
=> ERROR: proton-bridge-1.2.7_1: non-PIE executable found in PIE build: /usr/bin/proton-bridge

Which means it's not respecting the flags in the environment.

I tried to build it using the go build_helper but that seems to have its own issues

[cinerea0]

@ericonr I think that would mainly affect qt5-go, but I'm willing to give it a shot. Two problems:

1. I have no idea which Void packages contain the html/doc .index files that are mentioned in the package manager method.
2. The setup requires using a binary that is included in the package. Is that even remotely doable with xbps-src?

[ericonr]

@cinerea0 I think that should fix qt5-go enough that it will work properly when building proton-bridge. Regarding those files, check the qt5-devel and qt5-doc packages. Possibly qt5-quickcontrols2-devel or qt5-quickcontrols and qt5-declarative. Could need pkg-config in hostmakedepends.

Running a binary inside xbps-src should be possible, just tends to make stuff nocross. There's a chance the qtsetup step is for building proton-bridge instead, but I'm not entirely sure.

[cinerea0]

I've attempted to translate the qt5-go installation instructions into an xbps template as best I can, but there's a couple hangups. Focusing on this part of the template:

post_install() {
	$(go env GOPATH)/bin/qtsetup
}

There are two possible errors depending on if that block exists or not. With it, the error looks like this: /void-packages/srcpkgs/qt5-go/template: line 24: /builddir/qt-b56356132618ecc7c0029f4262d4ac986afe1477/_build-qt5-go-xbps/bin/qtsetup: No such file or directory. Without it, the error looks like this: ERROR: qt5-go-0.0.2020.06.04_1: PKGDESTDIR is empty and build_style != meta.

[cinerea0]

Alright, I think I've improved the qt5-go template as much as I can. It turns out invoking qtsetup was actually ruining the package because it was turning it into a local install as opposed to a system-wide install. However, the error with the undefined symbol still exists. At this point, I think the best way forward is for me to focus on patching the build scripts in such a way that the project is tricked into linking against system libraries.

[fosslinux]

Yes, that is precisely the best option here, to patch the build scripts :)

edit, gosh, this project is ugly, happy to help if you need some

[cinerea0]

Looking through the Makefile, there's a couple things I need to get straight before attempting to patch. For one, I have no idea what these lines are doing:

.PHONY: prepare-vendor update-vendor
THERECIPE_QTVER:=$(shell grep "github.com/therecipe/qt " go.mod | sed -r 's;.* v[0-9\.]+-[0-9]+-([a-f0-9]*).*;\1;')

Next, is it possible for me to prevent this block from being executed simply by removing all references to it:

prepare-vendor:
	go install -v -tags=no_env github.com/therecipe/qt/cmd/...
	go mod vendor

Finally, these sections are the ones that deal with pulling in the environment libraries:

# vendor folder will be deleted by gomod hence we cache the big repo
# therecipe/env in order to download it only once
vendor-cache/${THERECIPE_ENV}:
	git clone https://${THERECIPE_ENV}.git vendor-cache/${THERECIPE_ENV}

LINKCMD:=ln -sf ${CURDIR}/vendor-cache/${THERECIPE_ENV} vendor/${THERECIPE_ENV}

# update-vendor is PHONY because we need to make sure that we always have updated vendor
update-vendor: vendor-cache/${THERECIPE_ENV} prepare-vendor
	${LINKCMD}

My plan is to allow these blocks to execute, but afterwards to replace the contents of the environment directory with libraries copied from wherever xbps-src puts them during the build process. Does that make sense?

[cinerea0]

I've made my first attempt at creating the patch files. During the patching process, I'm encountering the following error:

=> proton-bridge-1.2.7_1: patching: init.sh.patch.
=> ERROR: proton-bridge-1.2.7_1: do-patch_00-patches: 'patch -sl ${_args} -i ${_patch} 2> /dev/null' exited with 2
=> ERROR:   in _process_patch() at common/hooks/do-patch/00-patches.sh:34
=> ERROR:   in hook() at common/hooks/do-patch/00-patches.sh:51
=> ERROR:   in run_func() at common/xbps-src/shutils/common.sh:21
=> ERROR:   in run_pkg_hooks() at common/xbps-src/shutils/common.sh:232
=> ERROR:   in run_step() at common/xbps-src/shutils/common.sh:71
=> ERROR:   in main() at common/xbps-src/libexec/xbps-src-dopatch.sh:33

I think I've constructed my patch file for init.sh improperly, but I'm not quite sure how.

[cinerea0]

Updated the patch files with suggestions, now I'm getting a new error:

=> proton-bridge-1.2.7_1: patching: init.sh.patch.
The text leading up to this was:
--------------------------
|--- vendor-cache/github.com/therecipe/env_linux_amd64_513/init.sh	2020-06-05 22:39:59.740997705 -0400
|+++ "init copy.sh"	2020-06-19 21:32:02.000000000 -0400
--------------------------
File to patch:
Skip this patch? [y]
1 out of 1 hunk ignored
=> ERROR: proton-bridge-1.2.7_1: do-patch_00-patches: 'patch -sl ${_args} -i ${_patch} 2> /dev/null' exited with 1

This looks like maybe the path is wrong? To explain what I did, I put the modified init.sh file in the wrksrc level directory, then executed diff -u /path/to/original/init.sh "init copy.sh" > init.sh.patch, then placed that patch file in the patch folder.

[cinerea0]

Okay, I finally realized what was going on. It turns out I'm trying to patch a file that doesn't exist until after the build process starts. Is there any way to force a patch during the middle of the build process?

[ericonr]

Can you manually drive the build process to stop in the middle of that step? Otherwise you'll have to do something like patch the makefile to run your command the moment it needs to be run.

[fosslinux]

Generally, the best idea would be to find out where the file is coming from in the build process, and patch the input. For example, you would patch Makefile.in, not Makefile in a configure-style tarball.

[fosslinux]

Oooh, I see, it's from a pulled remote. I really hate this, Rust and Go both do it and it makes a pain.

Here's how to approach this:

1. Make env_linux_amd64_513 tarball a distfile. Download and extract it.
2. Apply the patch to the extracted tarball.
3. Patch the Makefile for THERECIPE_ENV on line 83 to point to the extracted tarball.

Then go will use your patch.

[cinerea0]

There still seems to be problems with xbps-src finding the init.sh file. Is my directory structure wrong?

EDIT: Hold on, I think I haven't been using create_wrksrc properly.

[cinerea0]

Good news: I figured out the patching. Turns out I just needed to make more judicious use of the wrksrc variable and how it changes.
Bad news: The init.sh script now needs more patching. Turns out the library structure that exists in the xbps-src chroot is a lot different from whatever the dev's environment is.

[cinerea0]

I'm deciding to close this (for now) for three reasons:

1. I realized that the function of this package can be easily replicated by setting up hydroxide as a per-user service.
2. We already have a restricted version of this package that provides the same program.
3. This thing's build process is still a nightmare, and judging by the new releases that isn't changing anytime soon.

I'll still be keeping tabs on new releases, and if the build process improves I'll make another attempt at packaging.

[ProjectMoon]

It should be noted that hydroxide isn't a 100% solution. It can be a bit flaky and doesn't support attachments (at time of writing).