Add tooling for Debian Chromium package conversion

[iskunk]

(Originally posted in this "contrib" repo issue)

I have put together a set of scripts that make it possible to convert a Debian source packaging of Chromium into an equivalent ungoogled-chromium package. The end result is not only very close to what a native packaging of u-c would look like (all that's missing is u-c-specific documentation), it can be installed on the system concurrently with the original Chromium (because all the files that would otherwise conflict have been renamed).

Note that the converted package retains the original upstream source tarball used by Debian. Yet it does not require the ungoogled-chromium tooling to build, because the necessary modification logic is condensed into a script generated by the new make_domsub_script.py utility. This generated script is invoked as an additional patching operation during the package build, and its effects can also be reverted, all in accordance with Debian packaging practice.

By design, the delta between the Debian package and the converted one is as small and easily-reviewable as possible. I've even provided a script (compare.sh) to facilitate this. (This is why I opted against bundling the domain substitutions as a patch. No additional scripts would have been needed, but the patch was over 30 MB in size!)

This PR is about getting in the new scripts, and while it does not change the current workflow of this repo, that is what I hope to make possible. The current process has a few issues that I can see:

• It doesn't produce a proper Debian source package of ungoogled-chromium (i.e. the matched set of .dsc, .debian.tar.xz, and .orig.tar.xz files) that anyone can grab and build locally;

• It takes on the burden of patching Chromium to build on Debian (e.g. lld-14.patch), work that Debian themselves already do. Even if this consists mainly of importing patches from Debian, it is still duplicative work taking up precious developer time;

• It does not produce an ungoogled-chromium package that can build on i386 or PowerPC, as Debian's chromium can, because the necessary patches are not included.

My view is that the u-c-debian process should build on the release-engineering work performed by Debian, without retreading the same ground. Whatever niceties their chromium package has, ungoogled-chromium should inherit, without needing to be aware of the specifics of them.

(One other thing I would point out is that the modifications necessary to build a Debian chromium package on Ubuntu should really be maintained in a separate project. Because Ubuntu no longer ships a proper chromium package, responsibility for adapting Debian's package to Ubuntu falls to the community, and the audience for that is wider than for ungoogled-chromium. I'm working on this problem with another gentleman, however, and hope to have something useful to report along these lines before long.)

But for now, please give this tooling a try, and let me know your feedback. The conversion process is fully automated, and the main point of upkeep is managing conflicts between the Debian patches and ungoogled-chromium's (some due to the latter importing patches from the former). The process is amenable to automation, although as I brought up in the earlier thread, automation should be considered a stopgap measure until ungoogled-chromium can be shipped by Debian proper. That is where I ultimately hope to go with this work.

[networkException]

@ungoogled-software/debian

[jhonny-oliveira]

I managed to successfully build and publish (ungooled-chromium@xtradeb.net) version 116.0.5845.96 for Ubuntu Jammy+ with @iskunk 's conversion script from this PR and yet another PR to the ungoogled-chromium repository (ungoogled-software/ungoogled-chromium#2461).

Here are the steps I followed:

# Retrieve ungoogled chromium debian repo and switch to Daniel's PR.
git clone https://github.com/ungoogled-software/ungoogled-chromium.git
cd ungoogled-chromium/
git fetch origin pull/2461/head:myLocalTest
git checkout myLocalTest
cd ..
git clone https://github.com/ungoogled-software/ungoogled-chromium-debian.git
cd ungoogled-chromium-debian/
git fetch origin pull/328/head:myLocalTest
git checkout myLocalTest

# Adjust Makefile
cd convert
# Adjust VERSION, UNGOOGLED and any other variable in the Makefile to your needs. For instance, DEBFULLNAME and DEBEMAIL

# Put chromium source in place (the same I just released - )
dpkg-source -x <path were I previously downloaded a working .orig. , .dsc and .debian. files>/chromium_116.0.5845.96-1~xtradeb1.dsc

# Demonstrate we have everything in place
ls -liah
Makefile
README.md
chromium-116.0.5845.96
compare.sh
editcontrol.pl
replace-name.pl

# run conversion
make
# if everything went fine, the last lines of the output should be something similar to the ones between below
---
touch stage-6.stamp
ls -Ll ungoogled-chromium_116.0.5845.96*
-rw-rw-r-- 1 xd xd   1741660 Aug 20 13:17 ungoogled-chromium_116.0.5845.96-1~xtradeb1.debian.tar.xz
-rw-rw-r-- 1 xd xd      2962 Aug 20 13:17 ungoogled-chromium_116.0.5845.96-1~xtradeb1.dsc
-rw-rw-r-- 1 xd xd 648561460 Aug 20 13:03 ungoogled-chromium_116.0.5845.96.orig.tar.xz
---

# Build
sbuild -d jammy ungoogled-chromium_116.0.5845.96-1~xtradeb1.dsc


# Re-sign the package to upload to launchpad.net

You can find the source package of both Google Chromium and UnGoogled Chromium (direct conversion of the previous) in my PPA: chromium@xtradebPPA

Great job @iskunk !
Thank you!

[iskunk]

Hi everyone. I've updated this PR to work with the current Debian Chromium source, and also integrate numerous improvements. A summary of the changes:

• Add INPLACE option to allow modifying an existing source tree;

• Update PATCH_DROP_LIST to re-establish compatibility with the Debian source, and break it up into individual append-assignments to allow better documentation of each item;

• Simplify the tree handling (i.e. don't work on a .new tree and then rename it at the end);

• Add more convenience targets;

• Copy patches into debian/patches/ungoogled-full/ instead of debian/patches/ungoogled/, as the latter is now used by Debian;

• Rework the check-patch-drop-list target;

• Move deletion of the output source package files into a clean-more target;

• Minor documentation tweaks.

I've been working with @jhonny-oliveira to develop tooling that will automate the process of converting/uploading the builds of u-c available in his repository, so that it closely tracks Debian's security updates.

[iskunk]

Minor update:

• Use rsync instead of rm -rf + cp -a to copy the working tree, to avoid wasting time when the destination is already (mostly) there;
• Add unpatch convenience target to clear out all applied patches and remove the .pc/ subdirectory;
• Add KEEP_TREE switch to prevent make clean from deleting the working source tree (which, in combination with the aforementioned rsync change, allows for much faster iteration in development);
• Update editcontrol.pl so that new fields added using set_field() are added after the current field, not at the end of all the fields. This keeps the XSBC-Original-Maintainer: field where it should be.

[networkException]

This is really really great work!

I think I have a good overview of everything now. I noted down some thoughts I had while playing around and reading through this, none of these are a blocker though.

[iskunk]

Thank you @networkException for the careful review! Please let me know if you catch anything else.

[Eloston]

Didn't dig too deep into the details but LGTM. Great work and commendable goals 👍

One request: Please don't touch the unportable branch in this repo. It's a standalone branch I (sporadically) maintain as a thin wrapper around Portable Linux, and I'd like to keep it around for now as I personally use it. Thanks :)

[iskunk]

Thank you as well @Eloston. Noted that the unportable branch belongs to you.

I'm still in the midst of developing the automation I have in mind for this, so no immediate changes are forthcoming (after this PR). But I hope to have something ready before long. For sure the new way will be less labor-intensive than the old.

[iskunk]

Heads up, an update to PATCH_DROP_LIST was needed for 119.0.6045.105. I'm hesitant to push it here, as that will invalidate the existing approvals, so I've pushed it to a duplicate branch in my fork.

Please let me know if there are any concerns.

[networkException]

Sorry for the delay again, I haven't had much time lately (still have to work through a backlog of ~500 mails). You should have received an invite to the organization. I set up branch protection rules for unportable, please let me know if I need to change anything else about the repository settings

You are free to merge this yourself now :)

[iskunk]

Thank you @networkException, and everyone! I've merged the PR, and pushed a follow-on commit with the update for 119.0.6045.105. The repo config looks good to me.

I'll see to it that the conversion framework remains functional---by design, it should only need updates sporadically---and will continue developing the new automation that I hope to introduce here soon.