Using rpm-list-builder for Python 3.7 rebase in Fedora

[hroncok]

I'm slowly rebuilding all our Python packages in Copr: @python/python3.7

I want to prep everything and once it's working, use the yaml file to rebuild stuff in Fedora. As I'm looking at rpm-list-builder, I'm not sure I can use it out of the box, so I'm opening this issue to track things.

What I need to do when I finally run this in Fedora proper.

1. Use fedpkg as source ✔️
2. When rebuilding, bump the release and changelog entry
   • This should be possible with a custom build command and rpmdev-bumpsec && git commit && git push && fedpkg build right?
3. Before rebuilding next package, use koji wait-repo to wait for the previous package to be available in the buildroot
   • This should be appendable to the custom build command
   • I'd need to get the NVR to wait for, this should do: rpm --define 'dist .fc29' -q --qf "%{NAME}-%{VERSION}-%{RELEASE}\n" --specfile *.spec | head -n1
4. When I build something without custom macros, I don't want to have the # Edited by rpmlb commited. Is it possible to disable this edit?
5. When I build something with custom macros and later without, I want the second bump to revert the changes (see example bellow). This is the part when it gets tricky I guess.

    - python-pluggy:
        macros:
          _without_tests: 1
    - pytest
    - python-pluggy

This should first commit the following pseudopatch to python-pluggy:

+# Edited by rpmlb
+%global _without_tests 1
 ...
-Release: 1
+Release: 2
 ...
 %changelog
+* Sun May 13 2018 Me Myself <me@fedoraproject.org> - 1.0-2
+- Bootstrapping Python 3.7
+

And later commit this:

-# Edited by rpmlb
-%global _without_tests 1
 ...
-Release: 2
+Release: 3
 ...
 %changelog
+* Sun May 13 2018 Me Myself <me@fedoraproject.org> - 1.0-3
+- Bootstrapping Python 3.7
+
 * Sun May 13 2018 Me Myself <me@fedoraproject.org> - 1.0-2
 - Bootstrapping Python 3.7

This is something that seems almost impossible currently.

My idea is:

• when doing macro changes, save the diff (prior to running bumpspec)
• when builiding a package, look to a specific directory for saved diffs and revert the last one (prior to running bumpspec)

I'm not quite sure this will even work, but it might. Or am I overcomplicating it?

[junaruga]

Everything you want to do might be achievable in current specification.
I would like to ask you to contribute your custom file to below directory for other peole who want to do it, after you would finish your work, if you like.
https://github.com/sclorg/rpm-list-builder/tree/master/tests/fixtures/custom

1. => Yes, right.

2. => Yes, possible. Though I have not tried the case by myself.

3. The custom file does not get "the previous package". So your commands could be appended at the end of "build" element like this. See below example.

4. Yes, possible to disable the edit text by a hack for custom file by "sed" command.
   The logic to print "# Edited by rpmlb" is in a parent class of builder class.
   I am fine to remove the logic from the parent class.

5. Yes, you can do this.
   Note that the 1st python-pluggy and 2nd python-pluggy are managed separately in rpmlb's working directory based on the sourced package directory.
   As a first build, using fedpkg --scratch build instead of fedpkg build might be safer.

build:
  -|
    # A line for question 4
    sed -i '/^# Edited by rpmlb/d' *.spec &&
    # A line for question 5.
    rpmdev-bumpsec *.spec && \
    git add *.spec && \
    git commit -m "something." && \
    git push && \
    fedpkg build && \
    # lines for question 3
    koji wait-repo && \
    rpm --define 'dist .fc29' -q --qf "%{NAME}-%{VERSION}-%{RELEASE}\n" --specfile *.spec | head -n1

Your command might be like this.

rpmlb --download local or fedpkg or custom
  --build custom \
  ...
  PATH_TO_RECIPE_FILE COLLECTION_ID

[junaruga]

This is something that seems almost impossible currently.

I think it is possible in current specification.

Current logic is

1. Download all the target packages from source directory or repository to working directory. 1st python-pluggy and 2nd python-pluggy are managed separately in the working directory.
2. Build those packages by order of the packages element a recipe file.

rpmlb --download local SOURCE_DIRECTORY would safely save the original packages before editing though you have to download packages manually by fedpkg clone PACKAGE before running rpmlb --download local.

I'm not quite sure this will even work, but it might. Or am I overcomplicating it?

My first impression is overcomplicating. But if my suggested use case has a problem for you, we have to consider the possibility by your way.

[junaruga]

The logic to print "# Edited by rpmlb" is in a parent class of builder class.
I am fine to remove the logic from the parent class.

@khardix @hroncok how do you think about removing the printing logic from the parent class?
Are you fine for that?

[hroncok]

The more I think about it, the more I'm playing with a thought of writing a dirty script for exactly what i need instead :)

[hroncok]

Something like https://github.com/hroncok/rpm-list-builder/blob/python37/dirty-rebuild-script.py

[junaruga]

@hroncok sure. Thanks for the script. As you are a master of Python, it is no wonder for you to write the Python script. Let me see the script tomorrow.
If you like, I try to convert it to a custom file later.

[hroncok]

As you are a master of Python

🎉

it is no wonder for you to write the Python script.

It's mostly a Bash script wrapped in Python.

If you like, I try to convert it to a custom file later.

Long term, that would be helpful. Thank you!

[junaruga]

okay!

[khardix]

how do you think about removing the printing logic from the parent class?

Well, it is important to note which file is the original, but since we are already storing a backup with the .orig extension, I'm fine with removing the comment.