Skip to content
This repository has been archived by the owner on Feb 9, 2022. It is now read-only.

DNM: debug packages in composes #41

Merged
merged 3 commits into from Apr 18, 2017

Conversation

ktdreyer
Copy link
Member

@ktdreyer ktdreyer commented Nov 16, 2016

This change needs further work and testing before we merge this.

With this change, rhcephcompose will place all of the debug packages into a separate "-debug" directory.

A new setting include_dbg toggles this behavior. (We'll want to disable include_dbg in Jenkins CI runs in order to save time and disk space).

TODO:

  • generate a deb repository with reprepro in the "-debug" directory, rather than simply dumping all the files into that location.

  • Fix tests so they pass

  • More testing to ensure this does not cause regressions

https://bugzilla.redhat.com/1375078

Copy link
Contributor

@ahills ahills left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added checksum=None to the test_dbg artifact test to make it pass. Overall, looks great. Absolutely love the parallelization!

@@ -48,6 +48,8 @@ def __init__(self, conf):
self.extra_files = conf['extra_files']
# Whether sources composition should be skipped
self.include_sources = conf.get('include_sources', True)
# Whether -dbg composition should be skipped
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticed this comment. It describes the opposite of the behavior; it should probably be:

Whether -dbg composition should happen

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, will rebase this branch and fix

ktdreyer and others added 3 commits April 17, 2017 16:39
Add a new property to BinaryArtifact objects, "dbg_parent". This will
let us identify binaries as "-dbg" or not, and provide us with the
binary's parent name.

The purpose of this change is to expose this data to callers (Compose
class). Eventually we can check this parent name against our comps
manifest. This will allow us to add each package's -dbg bits into a
compose without having to list each "foo-dbg" and "foo-dbg" package pair
explicitly in the comps .xml manifest.
If a BuildArtifact is a -dbg package, and its parent package is listed
in a comps manifest, add that -dbg package to the compose.

The purpose of this change is to ship -dbg packages in the product
deliverable for customers.

Use a separate -dbg directory so we don't duplicate so much data. RHCS 2
is about 8GB when we copy the dbg packages into each variant, and only
2.3GB when we route everything to this single -dbg directory.

Add a "include_dbg" configuration option to control this -dbg dir
(useful for speeding up test composes prior to GA).

A future change could turn this -dbg directory into a proper repository.
@ktdreyer
Copy link
Member Author

I've dropped the commit that runs each variant in its own PID, because they were racing while downloading/using files from the .cache location (sha512 checksum failures). We'll have to optimize the performance with more testing in a future PR.

@ahills ahills merged commit da3a424 into red-hat-storage:master Apr 18, 2017
@ktdreyer ktdreyer deleted the dbg-packages branch April 18, 2017 13:25
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants