Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create a pool of builders that have more storage #3268

Closed
xsuchy opened this issue May 20, 2024 · 2 comments · Fixed by #3408
Closed

Create a pool of builders that have more storage #3268

xsuchy opened this issue May 20, 2024 · 2 comments · Fixed by #3408
Assignees
@nikromen

Comments

@xsuchy
Copy link
Member

xsuchy commented May 20, 2024

Follow up of #3267

We have powerful builders, but this adds "only" CPU and RAM. But not storage.
We have a hardcoded option to mock config to use 140GB for storage using tmpfs
https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18
We must pass an option allowing powerful builders to use more storage.
@xsuchy xsuchy changed the title Create a pool of builders that requires more storage Create a pool of builders that have more storage May 20, 2024
@praiskup
Copy link
Member

There's the on_demand_powerful tag configured in Ansible, which is coming from Frontend's configuration. Then, we need to have a tool that would be able to modify the Mock configuration when on_demand_powerul tag is used.

@praiskup
Copy link
Member

Alternatively, we could bump the storage +10G everywhere.

@xsuchy xsuchy mentioned this issue May 22, 2024
Open
nikromen added a commit to nikromen/copr that referenced this issue Sep 9, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
a9e931f 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
@nikromen nikromen mentioned this issue Sep 9, 2024
Merged
2 tasks
nikromen added a commit to nikromen/copr that referenced this issue Sep 9, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
be206aa 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
@nikromen nikromen mentioned this issue Sep 9, 2024
Merged
nikromen added a commit to nikromen/copr that referenced this issue Sep 9, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
7b54bab 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 16, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
befd48a 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
133840f 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
b45b43a 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
34455a8 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
317b113 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuild: specify mock snippets to mock config via copr-rpmbuild conf…
bb25935 
…ig file

This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
8578e57 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
2d5ca8a 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
e07b957 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
2b5d21c 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
3d2e928 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
48fd69e 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 18, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
ffa70c7 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 20, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
d555403 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 23, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
c52dc98 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 24, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
e869d2a 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 24, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
fb5ff90 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit that referenced this issue Sep 25, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
5d77d36 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See #3268
jaitjacob added a commit to jaitjacob/copr that referenced this issue Sep 25, 2024
@jaitjacob
Squashed commit of the following:
a6d1127 
commit 4b1576f
Merge: 57a303e 8b0977a
Author: jait <jai8.jacob@gmail.com>
Date:   Wed Sep 25 22:59:20 2024 +0530

    Merge branch 'fedora-copr:main' into Webhook-History-UI

commit 8b0977a
Author: Pavel Raiskup <praiskup@redhat.com>
Date:   Tue Sep 24 21:09:24 2024 +0200

    rpmbuild: unblock testsuite

commit 5d77d36
Author: Jiri Kyjovsky <j1.kyjovsky@gmail.com>
Date:   Tue Sep 17 14:35:28 2024 +0200

    rpmbuild: specify snippets to mock config via copr-rpmbuild config file

    This allows us to specify tpm fs size to rpmbuild in order to be able to
    automatically generate its size for performance builders.

    See fedora-copr#3268

commit 57a303e
Author: Jait Jacob <jai8.jacob@gmail.com>
Date:   Wed Sep 25 13:26:34 2024 +0530

    remove unused import & revert localized_time filter method

commit 658230d
Author: Pavel Raiskup <praiskup@redhat.com>
Date:   Mon Sep 23 15:20:09 2024 +0200

    backend: unknown resalloc tickets helper cleanup

    If no tickets are taken (which often happens in the staging
    environment), this script encountered corner case issues.

commit d6a3472
Author: Miroslav Suchý <msuchy@redhat.com>
Date:   Mon Sep 23 22:12:43 2024 +0200

    rpmbuild: do not require rpkg,pyp2rpm,pyp2spec,gem2rpm and fedora-review on rhel

    Resolves: RHBZ#2313878

commit 6ae7c6c
Author: Miroslav Suchý <msuchy@redhat.com>
Date:   Mon Sep 23 22:04:11 2024 +0200

    rpmbuild: do not require qemu-user-static on rhel

    Resolves: RHBZ#2313879

commit 1559b85
Author: Aurélien Bompard <aurelien@bompard.org>
Date:   Mon Jul 29 10:54:02 2024 +0200

    Use `super()` without argument to make pylint happy

    Signed-off-by: Aurélien Bompard <aurelien@bompard.org>

commit f3b1643
Author: Aurélien Bompard <aurelien@bompard.org>
Date:   Mon Jul 29 10:40:13 2024 +0200

    Message schemas: set chroot message severity to DEBUG

    Signed-off-by: Aurélien Bompard <aurelien@bompard.org>

commit c192726
Author: Aurélien Bompard <aurelien@bompard.org>
Date:   Mon Jul 29 10:38:34 2024 +0200

    Message schemas: one-line descriptions should be the summary

    Signed-off-by: Aurélien Bompard <aurelien@bompard.org>

commit 609d369
Author: Pavel Raiskup <praiskup@redhat.com>
Date:   Fri Sep 20 14:14:31 2024 +0200

    frontend: fix the 500 for racy creation attempts

    This is TOCTOU issue.  The other checks for duplications (on so many
    places) seem kinda redundant because nothing but try/except for commit()
    may catch these concurrency problems.

    Fixes: fedora-copr#3372

commit 446dcb3
Author: Jiri Kyjovsky <j1.kyjovsky@gmail.com>
Date:   Wed Sep 18 16:49:52 2024 +0200

    beaker: use podman for testing inside container if installed

commit 030740a
Author: Jait Jacob <jai8.jacob@gmail.com>
Date:   Sun Sep 22 23:17:17 2024 +0530

    remove unaccessed import

commit cf1cb81
Author: Jait Jacob <jai8.jacob@gmail.com>
Date:   Fri Sep 20 02:36:09 2024 +0530

    webhook_history stores UNIX timestamps instead of DateTime

commit 6aeb686
Author: Jiri Kyjovsky <j1.kyjovsky@gmail.com>
Date:   Mon Sep 16 10:15:37 2024 +0200

    docker: set hard ulimits for docker container

    Because of bug in python3-daemon [1] we need to set ulimits inside
    docker container, otherwise backend and dist-git ooms.

    [1] - https://bugzilla.redhat.com/show_bug.cgi?id=2307635

commit 6aa1d5e
Author: Jait Jacob <jai8.jacob@gmail.com>
Date:   Sun Sep 15 19:55:11 2024 +0530

    resolve pylint warnings

commit a2adf81
Author: Jait Jacob <jai8.jacob@gmail.com>
Date:   Sun Sep 15 19:44:07 2024 +0530

    remove unused code

commit 14faa50
Author: Jakub Kadlcik <frostyx@email.cz>
Date:   Thu Sep 5 11:10:15 2024 +0200

    frontend, python, cli: allow admins to set storage for new projects

    See fedora-copr#2533

    This will be useful for beaker tests where we can now add basic tests
    for every supported storage.

commit ad36b8b
Author: Jakub Kadlcik <frostyx@email.cz>
Date:   Thu Sep 5 10:36:46 2024 +0200

    frontend: make the default storage for new projects configurable

    See fedora-copr#2533

commit 286a913
Author: Jakub Kadlcik <frostyx@email.cz>
Date:   Tue Sep 10 08:36:23 2024 +0200

    backend: add a timeout for waiting until a Pulp task finishes

commit 23a2fa3
Author: Jakub Kadlcik <frostyx@email.cz>
Date:   Mon Jul 22 11:23:53 2024 +0200

    backend: actions don't call uses_devel_repo function anymore

commit c5166a1
Author: Jakub Kadlcik <frostyx@email.cz>
Date:   Thu Sep 5 09:43:09 2024 +0200

    backend, frontend: implement project and build deletion in Pulp

    Fix fedora-copr#3318
    Fix fedora-copr#3319

commit 19eff0c
Author: Jakub Kadlcik <frostyx@email.cz>
Date:   Sun Sep 8 12:03:15 2024 +0200

    copr: wait until Pulp publication is finished

commit ff5288d
Author: Miroslav Suchý <msuchy@redhat.com>
Date:   Wed Sep 11 08:19:41 2024 +0200

    common: cleanup - remove six dependency

commit 6c71993
Author: Jait Jacob <jai8.jacob@gmail.com>
Date:   Wed Sep 11 16:21:34 2024 +0530

    fix pylint complaint

commit 649fe51
Author: Jait Jacob <jai8.jacob@gmail.com>
Date:   Wed Sep 11 16:15:14 2024 +0530

    optimize db calls, remove client side js, resolve review comments

commit 31880b4
Author: Jait Jacob <jai8.jacob@gmail.com>
Date:   Sun Sep 8 22:32:14 2024 +0530

    cleanup

commit 593a13f
Author: Jait Jacob <jai8.jacob@gmail.com>
Date:   Sun Sep 8 22:17:58 2024 +0530

    add pagination

commit 99e142e
Merge: 11d7fb1 2bdec45
Author: jait <jai8.jacob@gmail.com>
Date:   Sun Sep 8 15:03:36 2024 +0530

    Merge branch 'fedora-copr:main' into Webhook-History-UI

commit 11d7fb1
Author: Jait Jacob <jai8.jacob@gmail.com>
Date:   Sun Sep 8 15:03:29 2024 +0530

    return webhook history in descending order & without any duplicates

commit b7f2b34
Author: Jait Jacob <jai8.jacob@gmail.com>
Date:   Sat Sep 7 00:22:06 2024 +0530

    frontend: show webhook history table under Setting->Integration
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nikromen nikromen

Labels
None yet
Projects
CPT Kanban
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

rpmbuilds: dynamically specify tmp_fs size nikromen/copr
3 participants
@xsuchy @praiskup @nikromen