Missing snapshot releases - OpenSUSE Build Service packages

[hpannenb]

Hello,

I discovered for different distros the snapshot releases under http://relax-and-recover.org/download/ are not up-to-date (se example screenshot below). Is this by intention, problem in the build process(es) or did I miss something in the discussion(s)?

Regards,
Holger.

or

[jsmeix]

@hpannenb
as far as I know those snapshots are not done automatically
(in particular no automated update after a git commit at ReaR upstream).

Perhaps @gdha could tell more about how that works?

[jsmeix]

In general regarding using current ReaR GitHub master code see the section
"Testing current ReaR upstream GitHub master code" in
https://en.opensuse.org/SDB:Disaster_Recovery

This is how I do it all the time but I run ReaR only on a few test systems
(mostly QEMU/KVM virtual machines on my homeoffice laptop).

[hpannenb]

@jsmeix Yes. On direct or indirect internet facing compute nodes (servers, VMs, ...) I would do so as well. Unfortunately this is not the case here. Obviously it is possible to download the GIT content and transport it to the affected system, unpack /usr/share/rear/... etc. But a baked/built package is much more efficient and its installation/troubleshooting much more reproducible.

Would be appreciated very much if an automatic workflow creating the packages can be implemented to test the current "bleeding edge" version of ReaR without much manual tasks in a reproducible way.

[gdha]

@jsmeix @hpannenb We used to have a nightly job to build new rpm/deb after git updates, but we re-used this box for other means. Due to lack of sponsoring we stopped this activity on a nightly basis.

[hpannenb]

Not knowing much of the SuSE OBS the issue might be related to rear.spec (as already managed in #2289). According to https://spdx.org/licenses/ the GPL-3.0 seems to be deprecated.

[hpannenb]

Due to lack of sponsoring we stopped this activity on a nightly basis.

I was under the impression this was automagically created via OBS where download links under Snapshot releases from Git are referencing to as well.

Edit: Ok. Seems to be quiet new OBS can be integrated with GitHub/GitLab as mentioned in Continuous Integration with OBS and GitHub/GitLab This has "beta" state but is heading towards the proper direction.

[jsmeix]

I never used OBS source services.
So here my very first attempt:
I did not understand anything in
https://openbuildservice.org/help/manuals/obs-user-guide/cha.obs.source_service.html
so I did it in script-kiddie mode based on the examples in
https://openbuildservice.org/help/manuals/obs-user-guide/cha.obs.best-practices.scm_integration.html

I made now
https://build.opensuse.org/package/show/Archiving:Backup:Rear:Snapshot/rear-github-master
but that builds only for some recent SUSE systems
with Fedora_33 as the only non-SUSE system.

So it seems it could get hard to make OBS source services working also
for all the other systems.

So I tried a simpler and more manual approach based on manual
wget https://github.com/rear/rear/archive/refs/heads/master.zip
which resulted
https://build.opensuse.org/package/show/Archiving:Backup:Rear:Snapshot/rear-snapshot
that builds for all SUSE systems and most Red Hat based systems
but fails for some Red Hat based systems and
fails for all Debian based systems.

I cannot fix build on Debian based systems
because I know nothing about the Debian build system.

[gdha]

@jsmeix @hpannenb It seems to work the OBS workflow after a PR, albeit, with a new name (see picture):

Bottomline is now we have an automated manner to create ReaR snapshots after each PR - thanks @hpannenb for the golden tip

[jsmeix]

@gdha
I fear it may not work as you assume because
I think that new rear-2.6.5-1.el8.x86_64.rpm RPM may come from my above testing
https://build.opensuse.org/package/show/Archiving:Backup:Rear:Snapshot/rear-snapshot
because this is what I get as binary RPM from that source package

# osc getbinaries Archiving:Backup:Rear:Snapshot rear-snapshot CentOS_8 x86_64
...
rear-2.6.5-1.el8.x86_64.rpm

In particular note the special version 2.6.5 which is what I specified in
https://build.opensuse.org/package/view_file/Archiving:Backup:Rear:Snapshot/rear-snapshot/rear.spec?expand=1

# Current snapshots are after ReaR 2.6 and before ReaR 2.7 i.e. between 2.6 and 2.7
Version: 2.6.5

[gdha]

@jsmeix According https://build.opensuse.org/package/show/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2649/rear it did work.

[jsmeix]

https://build.opensuse.org/package/show/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2649/rear
results this binary RPM:

# osc getbinaries home:gdha:Archiving:Backup:Rear:Snapshot:PR-2649 rear CentOS_8 x86_64
...
rear-2.6-146.git.4317.be8b6ed.master.el8.x86_64.rpm

I think the 'unpublished' in

# osc results -v home:gdha:Archiving:Backup:Rear:Snapshot:PR-2649 rear | grep CentOS_8
CentOS_8             x86_64     succeeded(unpublished)

means that this binary RPM is not made available on OBS download servers.

[jsmeix]

I removed my testing Archiving:Backup:Rear:Snapshot rear-snapshot
and added Archiving:Backup:Rear:Snapshot rear-manual-snapshot
which builds for all systems.
The built binary RPMs have rather simple names like

# osc getbinaries Archiving:Backup:Rear:Snapshot rear-manual-snapshot openSUSE_Leap_15.2 x86_64
...
rear-2.6.5-1.x86_64.rpm

where currently only the special version number '2.6.5' indicates
it is a state between the current latest ReaR release 2.6
and the upcoming next ReaR release 2.7.

[jsmeix]

@hpannenb
if you like you may try out if RPMs from
Archiving:Backup:Rear:Snapshot rear-manual-snapshot
i.e. binary RPMs with currently special version number '2.6.5'
work for you.
Be careful - I did not at all test those RPMs.

@gdha
when your automated build works well we could remove that
Archiving:Backup:Rear:Snapshot rear-manual-snapshot
or we could keep it (perhaps with some enhancements)
so that we could - if we like - provide RPMs for a certain fixed snapshot state
instead of a too much moving target that rebuilds for each upstream change?

[gdha]

@jsmeix sure why not - kind of pre-releases

[hpannenb]

Glad to see You managed the interworking between OBS and GitHub. Honestly it was just "by accident" I found this SCM integration hint at OBS. It is rather new.

@hpannenb
if you like you may try out if RPMs from
Archiving:Backup:Rear:Snapshot rear-manual-snapshot
i.e. binary RPMs with currently special version number '2.6.5'
work for you.
Be careful - I did not at all test those RPMs.

I am currently checking the Centos_7 RPM within one of my (K)VMs from here
https://download.opensuse.org/repositories/Archiving:/Backup:/Rear:/Snapshot/CentOS_7/x86_64/
I will let You know once I installed it.

Edit: Installation of the 2.6.5 RPM works perfectly fine.
rear-centos-7-rpm-check.txt

[jsmeix]

@hpannenb
thank you for your prompt feedback!
It is much appreciated.

[gdha]

@jsmeix I think that the PR github workflow towards OBS is user bounded, so if you, merge a PR nothing will happen. To make your OBS account linked to your GibHub account follow the procedure found at https://openbuildservice.org/2021/05/31/scm-integration/

[jsmeix]

I had never before let computers act on their own on as if they were me.

https://openbuildservice.org/2021/05/31/scm-integration/
reads (excerpts):

You have to give OBS permissions
to report the build status to the pull/merge request to the SCM
on your behalf.
For this, you have to create a personal access token.
...
You also have to give your SCM permissions
to trigger a workflow inside OBS
on your behalf.
That’s why you need to create an OBS access token.
...
Tokens are like the keys to (parts of) your house.
You have to keep your token secret
to prevent someone else
from triggering operations
in your name!

This is a contradiction in itself.
Via the tokens
I give the programmers of the SCM (GitHub in my case)
and the programmers of the OBS
permissions
to do things in my name
because what actually happens is not what I myself do
but what programmers of those workflows/operations implement
where I have no control in practice what actually happens
(in practice I cannot verify their source code).

Of course when I log in at any foreign service and manually do things there
I depend on what the programmers implemented for what I do
i.e. I let their programs run on their servers on my behalf, cf.
https://www.gnu.org/philosophy/who-does-that-server-really-serve.html

The difference - as far as I understand it - is that
with access tokens I give them some kind of key/password
while via login at any foreign service and manually do things there
I do not give them some kind of key/password
to let them do arbitrary things as if it was me
(under the assumtion they do not store my clear text login password).

Perhaps it might be acceptable for me if I had a separated robot account
where I could specify limited permissions as I like and then
do such access token things only via the restricted robot account.

Then it could be at least clear what happened via my robot account
versus what I myself did via my normal user account.

[jsmeix]

@gdha
when I made #2658
I noticed that it triggered a rebuild of some rear package in OBS
home:gdha:Archiving:Backup:Rear:Snapshot...

At #2658
click on Show all checks and there the Details links all point to
https://build.opensuse.org/package/show/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear

But the commits in this pull request belong to a branch jsmeix-fix2622
and not to master.

I think only commits to master should trigger builds in OBS
but not commits to (development work in progress) branches.

On the other hand when pull requests create sepatated OBS projects like
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658
the additional builds are separated but I wonder where the built RPMs appear?

On
https://build.opensuse.org/package/show/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear
the link Download package is
https://software.opensuse.org//download.html?project=home%3Agdha%3AArchiving%3ABackup%3ARear%3ASnapshot%3APR-2658&package=rear
but it only results

No data for home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658 / rear

[gdha]

@jsmeix See https://build.opensuse.org/repositories/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear see the reposity rows which contains the deb/rpms to download.

[jsmeix]

@gdha
thank you - now I see and understand!

https://build.opensuse.org/repositories/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear
shows that all "Publish Flag"s are disabled.

But on
https://build.opensuse.org/package/show/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear
I can click in the "Build Results" list on a repository e.g. "CentOS_7" which leads to
https://build.opensuse.org/package/binaries/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear/CentOS_7
and there are download links to the source RPM and the binary RPM
e.g. the binary RPM links is
https://build.opensuse.org/package/binary/download/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear/CentOS_7/x86_64/rear-2.6-147.git.4317.be8b6ed.master.el7.x86_64.rpm

When I am not logged in at OBS and
click on that binary RPM "Download" link
I get a page that shows
Please login to access the requested page.

Also

# wget https://build.opensuse.org/package/binary/download/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear/CentOS_7/x86_64/rear-2.6-147.git.4317.be8b6ed.master.el7.x86_64.rpm
...
2021-07-16 09:04:06 (0.00 B/s) - ‘rear-2.6-147.git.4317.be8b6ed.master.el7.x86_64.rpm’ saved [0/0]

# file rear-2.6-147.git.4317.be8b6ed.master.el7.x86_64.rpm
rear-2.6-147.git.4317.be8b6ed.master.el7.x86_64.rpm: empty

does not provide the actual binary RPM.

In contrast when I am logged in at OBS and
click on that binary RPM "Download" link
I get the actual binary RPM.

Conclusion:
When the "Publish Flag" is disabled it means the built binary RPM
will not appear in a commonly known public accessible repository for binary RPMs
but the built binary RPM is still available via a direct "Download" link
for prople who can log in at OBS.

I don't know if there are certain restrictions for those who can log in at OBS
which binary RPMs they can access via such direct "Download" links.
Because
https://build.opensuse.org/package/users/home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658/rear
lists only
https://build.opensuse.org/users/gdha
as the only user of the OBS project "home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658"
I assume everybody who can log in at OBS can get source RPM and binary RPM
via those direct "Download" links.

I.e. when the "Publish Flag" is disabled it does not mean
the source RPM and the binary RPM are not accessible for others.

[github-actions]

Stale issue message

[hpannenb]

Prevent the bot from auto-closing.

[hpannenb]

@jsmeix and @gdha: Does this topic intentionally stay closed?

[jsmeix]

According to

# osc search rear | grep 'home:gdha:Archiving:Backup:Rear:Snapshot'
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2649  rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2650  rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2655  rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2656  rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2658  rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2659  rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2660  rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2661  rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2662  rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2664  rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2665  rear
home:gdha:Archiving:Backup:Rear:Snapshot:PR-2675  rear

it seems that automatism has shomehow stopped after pull request 2675.
I have no idea why because I do not understand how that works.
I gave up because I could never make it work for me, cf.
#2645 (comment)

As time permits I will update my
Archiving:Backup:Rear:Snapshot rear-manual-snapshot

[jsmeix]

Was much easier than expected because I made comments in
https://build.opensuse.org/package/view_file/Archiving:Backup:Rear:Snapshot/rear-manual-snapshot/rear.spec?expand=1
how to make its Source0

# How to make Source0:
# Download current ReaR upstream GitHub master.zip:
# # wget https://github.com/rear/rear/archive/refs/heads/master.zip
# Unzip it (unzips into rear-master directory):
# # unzip -q master.zip
# Rename the rear-master unzip directory:
# # mv rear-master rear-snapshot
# Tar it (use the above 'Version' value):
# # tar -czf rear-snapshot-2.6.5.tar.gz rear-snapshot
# Remove master.zip and the rear-snapshot directory:
# # rm -r master.zip rear-snapshot
Source0: rear-snapshot-%{version}.tar.gz

so I did that and - voila! - here you are:

https://download.opensuse.org/repositories/Archiving:/Backup:/Rear:/Snapshot/openSUSE_Leap_15.3/x86_64/
contains a new rear-2.6.5-2.x86_64.rpm dated right now 09-Feb-2022 08:53

For other distributions see what there is under
https://download.opensuse.org/repositories/Archiving:/Backup:/Rear:/Snapshot/

The RPM package file must be named rear-2.6.5...
in particular its version 2.6.5 indicates it is from the sources in
https://build.opensuse.org/package/show/Archiving:Backup:Rear:Snapshot/rear-manual-snapshot

[github-actions]

Stale issue message

[hpannenb]

Hmppfff...the bot is bot-hering. :-)

[jsmeix]

At least the bot reminds one from time to time about older issues...

[jsmeix]

...that are still of interest.

[github-actions]

Stale issue message

[pcahyna]

@jsmeix @hpannenb I am not familiar with this build service at all, but maybe the openSUSE build targets in Packit #2823 could provide at least a partial replacement?

[schlomo]

@jsmeix it seems like the snapshot builds again stopped to work, at least the last RPM in https://download.opensuse.org/repositories/Archiving:/Backup:/Rear:/Snapshot/15.4/x86_64/ is from last year and the last DEB in https://download.opensuse.org/repositories/Archiving:/Backup:/Rear:/Snapshot/Debian_11/amd64/ is also from last year.

It would be nice if we can get the snapshot builds to work again.

[jsmeix]

I have no personal interest in automated builds and
in particular not in automated snapshot builds.

My personal reasons:

I do not want to get one single user issue/bug report
when something does not (yet) work properly in his case
because it is under development but a blind automatism
made a useless snapshot build where of course the user
cannot know he got a (partially) inconsistent state
(often users blindly install just the highest version).

Automated (snapshot) builds enforce us at ReaR upstream
to never merge a pull request that requires further work
by subsequent pull requests until all is properly done
which means in particular we can never again merge a
somewhat not yet complete pull request from a user
who kindly contributed something that works for him
but does not yet work sufficiently well in general.

I do not want to be enforced to enforce users who
contributed something to enhance their contribution
until it works sufficiently well in general.
In particular not because in most cases I cannot verify
what a user contributed because I don't have his environment
or I am busy with other (higher priority) things
when a user's pull request appears.

As far as I know automated builds require to let computers
act on their own on as if they were me, see my above
#2645 (comment)

Users who like to test our current ReaR GitHub master code
don't need automated snapshot builds, see the section
"Testing current ReaR upstream GitHub master code" in
https://en.opensuse.org/SDB:Disaster_Recovery
See also
#2923 (comment)
(excerpt)

... everybody can use our current ReaR upstream
GitHub master code via "git clone" and from there
also get any needed state before via "git checkout"

As far as I can imagine only users who like to actually use
our current ReaR GitHub master code ask for snapshot builds
so that they can install a built software package as usual
on their systems and use that normally on their systems
but this is exactly not what they should do with some
GitHub master code from a random (snapshot) point in time.

@schlomo @rear/contributors and ReaR users:
if you personally want automated (snapshot) builds,
feel free to implement it as you think it works best and
then also continuously maintain what you implemented
for some years in the future and/or find other ReaR
upstream maintainers and/or ReaR users who also can
implement and maintain it or at least help with it.

FWIW:
I neither have special openSUSE Build Service knowledge
nor special openSUSE Build Service rights or permissions.
I am only a normal end-user of the openSUSE Build Service.