Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for supplementary dist-git items #394

Closed
AntStephenson opened this issue Feb 14, 2019 · 14 comments

Comments

Projects
5 participants
@AntStephenson
Copy link

commented Feb 14, 2019

We have a use case for committing another yaml file for CVP (gating.yaml) which would currently be ignored by cekit, is it possible to add capabilities within cekit to define a list of supplementary items that would be synced to the dist-git at runtime? Or at the very least, support this additional file like with content_sets.yaml.

Thank you!

@goldmann

This comment has been minimized.

Copy link
Contributor

commented Feb 14, 2019

As I mentioned earlier, this is an interesting request. I think it would be valuable to make it possible we just need to think what would be the best way to define the files, as well the target locations for them.

Currently the container.yaml and content_sets.yml are special files that are related to the build process in OSBS with native support in Cekit. We could go that route with the gating.yaml file, but it would be nice to get some input from others as well. Implementing support for gating.yaml should be fairly easy. Adding support for arbitrary files could be harder.

@goldmann

This comment has been minimized.

Copy link
Contributor

commented Feb 14, 2019

The simplest implementation would be:

// pseudocode
if (fileFound('gating.yaml')) {
  copyToDistGitRoot('gating.yaml')
}
@goldmann

This comment has been minimized.

Copy link
Contributor

commented Feb 14, 2019

@rnc @jmtd @luck3y @iankko What are your thoughts?

@jmtd

This comment has been minimized.

Copy link
Contributor

commented Feb 14, 2019

@goldmann I wondered about a more generic mechanism, but I haven't sketched out a solution. Some kind of "just copy these files into the image sources" . Then I wondered how different that was from artefact handling, and whether we could use or extend that mechanism for this.

@luck3y

This comment has been minimized.

Copy link

commented Feb 14, 2019

+1 from me, I'd just added gating.yaml to dist-git manually before I'd read this ;)

TBH, I'm not clear on what makes the files 'arbitrary'. If they're in the github repo, don't they end up in dist-git? I'm sure I'm missing something here.

@AntStephenson

This comment has been minimized.

Copy link
Author

commented Feb 14, 2019

@luck3y IIUC if the files are not explicitly handled by cekit (and subsequently referenced in your image yaml) it will not copy them, as an example for copying content_sets.yaml:

packages:
content_sets_file: content_sets.yml

@luck3y

This comment has been minimized.

Copy link

commented Feb 14, 2019

@AntStephenson Yeah, I get that part, I think to better rephrase my question, 'Why don't we just copy whatever is in the src directory into dist-git.' I mean that would cover it, wouldn't it?

This is the part where I'm missing something I think ;)

@goldmann

This comment has been minimized.

Copy link
Contributor

commented Feb 15, 2019

@luck3y This could be possibly dangerous - it could accidentally overwrite some files in dist-git.

What we currently do is that we copy all files in the target/image directory to dist-git. This is a generated directory. One solution to support arbitrary files would be to set up some convention and agree that, for example: content of the dist directory (if existing) would be copied to the target/image directory after generation and later commited to dist-git when the osbs builder would be selected.

This way we would not need to define anything in the schema and have control from where files will be fetched and where these will be placed.

Comments?

@luck3y

This comment has been minimized.

Copy link

commented Feb 15, 2019

@goldmann Ah, sure I get the danger side of it, though you'd see those in the diff I imagine.

I like the compromise of the dist directory and just adding whatever is in there, so +1 from me.

@goldmann goldmann changed the title Add support for supplementary dist-git items or CVP yaml Add support for supplementary dist-git items Feb 18, 2019

@goldmann goldmann added this to the 3.1 milestone Feb 18, 2019

@goldmann

This comment has been minimized.

Copy link
Contributor

commented Feb 18, 2019

Let's schedule it for 3.1 initially since the 3.0 bucket is pretty full ATM.

@jakubschwan

This comment has been minimized.

Copy link

commented Feb 20, 2019

@goldmann Can be increased priority of this issue and also milestone. In QE we need these changes for CVP. We need to be able push gating.yaml file to dist-git repo automatically.

@goldmann

This comment has been minimized.

Copy link
Contributor

commented Feb 20, 2019

An usable workaround would be to push these files manually to dist-git. These do not change often. This is what we used previously and this will work, Cekit will not remove them. Let me know if this works for you.

@jakubschwan

This comment has been minimized.

Copy link

commented Feb 20, 2019

Manual push is something that actually block us, we do not want to do it for every minor release. There is also huge risk that this can be forgotten with every new minor release.

@goldmann goldmann added this to To do in Release 3.1 Mar 13, 2019

@goldmann goldmann added priority/high and removed priority/low labels May 16, 2019

@goldmann goldmann moved this from To do to In progress in Release 3.1 May 17, 2019

goldmann added a commit to goldmann/cekit that referenced this issue May 24, 2019

Allow for supplementary files to be added to dist-git repository
If a directory called `dist` is found next to the image descriptor,
contents of this directory will be copied to the **root** of the
dist-git directory.

Fixes cekit#394.
@goldmann

This comment has been minimized.

Copy link
Contributor

commented May 24, 2019

@AntStephenson @jakubschwan @luck3y @jmtd Could you guys take a look at #519 and let me know what you think about the implementation?

Should we make the directory name (currently dist) configurable? If yes - where, in image descriptor? I do not want to over-complicate this, if possible. I'm happy to change the directory name, if you have better suggestions than the currently used dist.

goldmann added a commit to goldmann/cekit that referenced this issue May 24, 2019

Allow for supplementary files to be added to dist-git repository
If a directory called `dist` is found next to the image descriptor,
contents of this directory will be copied to the **root** of the
dist-git directory.

Fixes cekit#394.

goldmann added a commit to goldmann/cekit that referenced this issue May 27, 2019

Allow for supplementary files to be added to dist-git repository
If a directory called `osbs_extra` is found next to the image descriptor,
contents of this directory will be copied to the **root** of the
dist-git directory.

Directory name is controlled in the image descriptor, in osbs section,
with the `extra_dir` key.

Fixes cekit#394.

Release 3.1 automation moved this from In progress to Done May 29, 2019

goldmann added a commit that referenced this issue May 29, 2019

Allow for supplementary files to be added to dist-git repository
If a directory called `osbs_extra` is found next to the image descriptor,
contents of this directory will be copied to the **root** of the
dist-git directory.

Directory name is controlled in the image descriptor, in osbs section,
with the `extra_dir` key.

Fixes #394.

@goldmann goldmann self-assigned this Jun 3, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.