Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support of CentOS Stream 9 #2085

Merged
merged 7 commits into from Sep 20, 2023
Merged

Add support of CentOS Stream 9 #2085

merged 7 commits into from Sep 20, 2023

Conversation

mfocko
Copy link
Member

@mfocko mfocko commented Sep 18, 2023

Based on the changes from the closed #2015

Signed-off-by: Gris Ge fge@redhat.com
(cherry picked from commit 1d74e96)

TODO:

  • Write new tests or update the old ones to cover new functionality.
  • Update doc-strings where appropriate.
  • Update or write new documentation in packit/packit.dev.

RELEASE NOTES BEGIN

Packit now supports the pkg_tool option in the config (at the top-level or with specific packages when using the monorepo syntax). This option can be used for switching between fedpkg or centpkg.

RELEASE NOTES END

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://softwarefactory-project.io/zuul/t/packit-service/buildset/9aae9305fd5344b9aeb245768bb87068

✔️ pre-commit SUCCESS in 7m 27s
packit-tests-rpm NODE_FAILURE Node request 200-0006402277 failed in 0s
✔️ packit-tests-pip-deps SUCCESS in 14m 01s
✔️ packit-tests-git-main SUCCESS in 13m 28s
packit-tests-pip-deps-sess-rec NODE_FAILURE Node request 200-0006402280 failed in 0s
✔️ packit-tests-git-main-sess-rec SUCCESS in 2m 53s
✔️ reverse-dep-packit-service-tests SUCCESS in 3m 38s

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://softwarefactory-project.io/zuul/t/packit-service/buildset/71a7805139fe4af789953e4cd325f7a1

✔️ pre-commit SUCCESS in 1m 38s
packit-tests-rpm FAILURE in 13m 39s
packit-tests-pip-deps FAILURE in 13m 54s
packit-tests-git-main FAILURE in 14m 01s
✔️ packit-tests-pip-deps-sess-rec SUCCESS in 2m 41s
✔️ packit-tests-git-main-sess-rec SUCCESS in 3m 02s
✔️ reverse-dep-packit-service-tests SUCCESS in 3m 52s

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://softwarefactory-project.io/zuul/t/packit-service/buildset/869b5e02015e4a53935a0972c6121b53

✔️ pre-commit SUCCESS in 1m 27s
✔️ packit-tests-rpm SUCCESS in 13m 00s
✔️ packit-tests-pip-deps SUCCESS in 12m 35s
✔️ packit-tests-git-main SUCCESS in 13m 10s
✔️ packit-tests-pip-deps-sess-rec SUCCESS in 2m 39s
✔️ packit-tests-git-main-sess-rec SUCCESS in 2m 54s
reverse-dep-packit-service-tests FAILURE in 3m 25s

@cathay4t
Copy link
Contributor

@mfocko Please let me know when you need to test your patch in field. I have nispor packages ready for ship new version to CentOS stream.

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://softwarefactory-project.io/zuul/t/packit-service/buildset/4556de43845749a89fbdee1f173946cf

✔️ pre-commit SUCCESS in 1m 29s
✔️ packit-tests-rpm SUCCESS in 13m 13s
✔️ packit-tests-pip-deps SUCCESS in 13m 09s
✔️ packit-tests-git-main SUCCESS in 13m 13s
packit-tests-pip-deps-sess-rec FAILURE in 2m 45s
packit-tests-git-main-sess-rec FAILURE in 2m 53s
reverse-dep-packit-service-tests FAILURE in 3m 43s

@mfocko
Copy link
Member Author

mfocko commented Sep 19, 2023

@cathay4t I think you should be able to test… Few things first though:

  1. I'm not sure how did you install Packit before…

    • maybe try it from the Copr from PR, or
    • pip3 install --user git+https://github.com/mfocko/packit@c9s and cleaning it up afterwards.
  2. Regarding the config, I've done some changes to make it a bit more maintainable (e.g. eventual C10(s)), so the deduction of dist-git happens by itself, but you need to specify the pkg_tool; based on the config from your commit message, it should look like:

    ---
    downstream_package_name: nispor
    upstream_project_url: https://github.com/nispor/nispor
    issue_repository: https://github.com/nispor/nispor
    update_release: false
    upstream_package_name: nispor
    upstream_tag_template: v{version}
    copy_upstream_release_description: false
    
    actions:
      post-upstream-clone:
        - bash -c "cp /tmp/nispor.spec ./"
    
    jobs:
      - job: propose_downstream
        trigger: release
        dist_git_branches:
          - fedora
    
      # CentOS jobs
      - job: propose_downstream
        trigger: release
        pkg_tool: centpkg
        dist_git_branches:
          - c9s
  3. I would appreciate if you tried rerunning with --debug, the lookaside check didn't work before, so it tried upload anyways, right now it should do the check and don't touch the cache if not necessary.

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://softwarefactory-project.io/zuul/t/packit-service/buildset/ddd4900ba7c1481c801ab8dd7d79a20f

✔️ pre-commit SUCCESS in 1m 30s
✔️ packit-tests-rpm SUCCESS in 13m 25s
✔️ packit-tests-pip-deps SUCCESS in 13m 07s
✔️ packit-tests-git-main SUCCESS in 13m 31s
✔️ packit-tests-pip-deps-sess-rec SUCCESS in 2m 29s
✔️ packit-tests-git-main-sess-rec SUCCESS in 2m 49s
reverse-dep-packit-service-tests FAILURE in 3m 27s

Copy link
Member

@lachmanfrantisek lachmanfrantisek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Thanks both of you!

Let's try this on stage 🚀

packit/api.py Outdated Show resolved Hide resolved
Comment on lines +175 to +178
@property
def pkg_tool(self) -> str:
"""Returns the packaging tool. Prefers the package-level override."""
return self.package_config.pkg_tool or self.config.pkg_tool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't you think package-level might be an overloaded term here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand :D Do you mean the confusion between package as in packaging tool vs Packit config?

config["lookaside"],
config["lookaside_cgi"],

class LookasideCache:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw. A really nice refactoring!

@cathay4t
Copy link
Contributor

My packit CLI is installed via pip3 install --user --editable . with git commit changed to this PR done by command hub pr checkout 2085.

Using this config:

---
downstream_package_name: nispor
upstream_project_url: https://github.com/nispor/nispor
issue_repository: https://github.com/nispor/nispor
update_release: false
upstream_package_name: nispor
upstream_tag_template: v{version}
copy_upstream_release_description: false

actions:
  post-upstream-clone:
    - bash -c "cp /tmp/nispor.spec ./"

jobs:
  - job: propose_downstream
    trigger: release
    pkg_tool: centpkg
    dist_git_branches:
      - c9s

I invoked packit -d --config ~/packit_ds_nispor.yml propose-downstream in the git clone of https://github.com/nispor/nispor.git

I still get error: gitdb.exc.BadName: Ref 'remotes/origin/c9s' did not resolve to an object as packit is still using fedpkg.

Full debug can be found in https://gist.github.com/cathay4t/a75b04dbcb79ef300bcac0a2351554f3

@lachmanfrantisek
Copy link
Member

@cathay4t thanks for trying. You didn't get a parsing error so you should have the correct Packit installed. I'll wait for Matěj about the issue itself.

Next time, you can also use the RPM builds produced by Packit itself, e.g. https://dashboard.packit.dev/results/copr-builds/1016150

sudo dnf copr enable packit/packit-packit-2085
sudo dnf install -y packit-0.80.0.post45+ga44dc510-1.20230919160946694561.pr2085.45.ga44dc510.fc39.noarch python3-packit-0.80.0.post45+ga44dc510-1.20230919160946694561.pr2085.45.ga44dc510.fc39.noarch

@mfocko
Copy link
Member Author

mfocko commented Sep 20, 2023

Oh, I know! I'm an idiot… it must be taken from the job config!
1st edit: well, that doesn't look fun…

@mfocko mfocko self-assigned this Sep 20, 2023
Introducing new `dist_git_branches` -- `c9s` for CentOS stream 9
downstream build.

To make it work, you need store gitlab token in `~/.config/packit.yml`,
like:

```config
authentication:
    gitlab.com:
        token: glpat-<secrets>
```

You need to run packit in the upstream git repo with command:

```bash
    packit --config ~/packit_ds_nispor.yml  propose-downstream
```

Example packit.yml(since nispor has no spec in upstream tarball, we use
/tmp/nispor.spec for test purpose):

```yml
---
downstream_package_name: nispor
upstream_project_url: https://github.com/nispor/nispor
issue_repository: https://github.com/nispor/nispor
update_release: false
upstream_package_name: nispor
upstream_tag_template: v{version}
copy_upstream_release_description: false

actions:
  post-upstream-clone:
    - bash -c "cp /tmp/nispor.spec ./"

jobs:
  - job: propose_downstream
    trigger: release
    dist_git_branches:
      - c9s
      - fedora
```

Signed-off-by: Gris Ge <fge@redhat.com>
(cherry picked from commit 1d74e96)
@mfocko
Copy link
Member Author

mfocko commented Sep 20, 2023

OK, I've been testing it locally… using the job config from the CLI feels like a no-go for the complexity of sync-release tasks…

---
downstream_package_name: nispor
upstream_project_url: https://github.com/nispor/nispor
issue_repository: https://github.com/nispor/nispor
update_release: false
upstream_package_name: nispor
upstream_tag_template: v{version}
copy_upstream_release_description: false

actions:
  post-upstream-clone:
    - bash -c "cp /tmp/nispor.spec ./"

packages:
  nispor-fedora: {}
  nispor-centos:
    pkg_tool: centpkg

jobs:
  - job: propose_downstream
    trigger: release
    packages: [nispor-fedora]
    dist_git_branches:
      - fedora

  - job: propose_downstream
    trigger: release
    packages: [nispor-centos]
    dist_git_branches:
      - c9s

@cathay4t Could you please try with this? (I got to this; feel free to close and verify that you don't upload the same archive to lookaside cache).

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://softwarefactory-project.io/zuul/t/packit-service/buildset/7d1d1988a10244239f4b97d51d8a98bb

✔️ pre-commit SUCCESS in 9m 39s
✔️ packit-tests-rpm SUCCESS in 13m 47s
✔️ packit-tests-pip-deps SUCCESS in 13m 37s
✔️ packit-tests-git-main SUCCESS in 14m 20s
✔️ packit-tests-pip-deps-sess-rec SUCCESS in 2m 56s
✔️ packit-tests-git-main-sess-rec SUCCESS in 2m 51s
reverse-dep-packit-service-tests FAILURE in 3m 49s

mfocko added a commit to mfocko/packit-service that referenced this pull request Sep 20, 2023
Follow-up to packit/packit#2085

Related to CentOS Stream support

Signed-off-by: Matej Focko <mfocko@redhat.com>
packit/api.py Outdated Show resolved Hide resolved
self.dist_git_base_url: str = (
dist_git_base_url
or (pkg_tool and DISTGIT_INSTANCES[pkg_tool].url)
or getenv("DISTGIT_URL", DISTGIT_INSTANCES["fedpkg"].url)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd probably have a look at hardly, if it makes sense to default to the packaging tool from the user config, rather than the environment variable.

Allow overriding pkg_tool at the package level rather than being defined
globally in user config. In case it is not defined, fall back to the
global config.

Related to the CentOS Stream support.

Signed-off-by: Matej Focko <mfocko@redhat.com>
Factor out the dist-git information from separate variables

Related to CentOS Stream support

Signed-off-by: Matej Focko <mfocko@redhat.com>
Signed-off-by: Matej Focko <mfocko@redhat.com>
Factor out the lookaside cache code (just the one that uses directly the
cache's API, or pyrpkg API) to a separate class that supports:

* constructing the sources
* checking for existing uploads

Related to CentOS Stream support

Signed-off-by: Matej Focko <mfocko@redhat.com>
mfocko and others added 2 commits September 20, 2023 17:08
Signed-off-by: Matej Focko <mfocko@redhat.com>
Co-authored-by: František Lachman <flachman@redhat.com>
Signed-off-by: Matej Focko <mfocko@redhat.com>
@mfocko mfocko added the mergeit When set, zuul wil gate and merge the PR. label Sep 20, 2023
@mfocko
Copy link
Member Author

mfocko commented Sep 20, 2023

oh… rev-dep tests, needs to be merged manually

@softwarefactory-project-zuul
Copy link
Contributor

Build failed.
https://softwarefactory-project.io/zuul/t/packit-service/buildset/2b4c7ded515e4e00bbe609479db0b1d7

✔️ pre-commit SUCCESS in 1m 31s
✔️ packit-tests-rpm SUCCESS in 13m 12s
✔️ packit-tests-pip-deps SUCCESS in 13m 31s
✔️ packit-tests-git-main SUCCESS in 13m 46s
✔️ packit-tests-pip-deps-sess-rec SUCCESS in 2m 45s
✔️ packit-tests-git-main-sess-rec SUCCESS in 3m 05s
reverse-dep-packit-service-tests FAILURE in 3m 29s

@mfocko mfocko merged commit ffb7d76 into packit:main Sep 20, 2023
17 of 22 checks passed
@mfocko mfocko deleted the c9s branch September 20, 2023 15:33
softwarefactory-project-zuul bot added a commit to packit/packit-service that referenced this pull request Sep 20, 2023
fix(c9s): update the constants from Packit

TODO:

 Write new tests or update the old ones to cover new functionality.
 Update doc-strings where appropriate.
 Update or write new documentation in packit/packit.dev.



Related to packit/packit#2085
Related to CentOS Stream support
Merge after packit/packit#2085

Reviewed-by: František Lachman <flachman@redhat.com>
@cathay4t
Copy link
Contributor

The MR you created to nispor centos stream looks good and merged.

I tried for nmstate new release, still got error: https://gist.github.com/cathay4t/a6eea0f3dc73349e3817207614cb85b0

Somehow your packit enumerate all spec files in my filesystem Local spec files found: [ which is scary.

With your PR, the packit.yaml is ugly than my original patch and not working any more.

@cathay4t
Copy link
Contributor

As the log shows, you are still using fedpkg command for c9s only config.

@mfocko
Copy link
Member Author

mfocko commented Sep 21, 2023

Hi, follow-up from the arch discussion:

  • latest Packit can be installed from packit/packit-dev Copr repo (it contains the builds from main)

  • I've also forgotten to mention that with the packages, I used

    packit -d --config ../nispor.yml propose-downstream -p nispor-centos
    

    to do the release only for CentOS, so you can use this as a shortcut

  • you can define specfile_path in the config to avoid the “lookup” that is done locally

If you hit any issue, feel free to ping me on Slack or create an issue :)

@lsm5
Copy link

lsm5 commented Sep 21, 2023

@mfocko @cathay4t thanks a lot for this. Are there plans to publish a blog post / doc for current packit users? We plan to release podman v4.7 soon so I could try this out if I have a HOWTO.

@lachmanfrantisek
Copy link
Member

It will be (in the ideal case) in production next week, but we still need to test this properly on stage in the meantime.

But @mfocko wants to do a blog post with howto and a few warnings that it's really really fresh and not-battle tested..;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mergeit When set, zuul wil gate and merge the PR.
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

4 participants