New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Allow for Fedora or RHEL Images #275

Closed
wants to merge 2 commits into
base: master
from

Conversation

Projects
None yet
3 participants
@ashcrow
Copy link
Collaborator

ashcrow commented Jan 10, 2019

This PR adds the ability to for coreos-assembler to run on either Fedora or RHEL content. Some notes:

  • A maipo directory is needed to build on RHEL. This content is in another repo due to needing some sensitive information.
  • Reuse of existing code was attempted, though there are few copy/paste with minor modification spots. These can be fixed up via follow on PRs as needed.
  • scl's Python3.6 packages are used to allow for modern style Python
  • @miabbott and I worked together on this 馃檶
  • There is one outstanding issue running the container with scl + dumb-init that I'm working on. Once this is figured out AND @miabbott and I have tested the results we will remove WIP.

To build you will need to checkout the internal repo to maipo/, and then podman build -f Dockerfile.rhel.

/cc @imcleod

Makefile: Explicit directory creation
On some distributions with older versions of the install command
explicit creation of full paths seem required.

Signed-off-by: Steve Milner <smilner@redhat.com>

install_rpms_rhel() {

This comment has been minimized.

@miabbott

miabbott Jan 10, 2019

Contributor

I should have seen this earlier, but I think we can just collapse both versions of install_rpms() into a single function that just uses yum. /usr/bin/yum is just a symlink to dnf-3 on F29.

This comment has been minimized.

@miabbott

miabbott Jan 10, 2019

Contributor

Oh, but the deps for RHEL are slightly different...hmm

This comment has been minimized.

@ashcrow

ashcrow Jan 10, 2019

Collaborator

They do point to different deps files though. However, a new function that takes in dep files and then executes the same install commands on them could be done. WDYT?

This comment has been minimized.

@miabbott

miabbott Jan 10, 2019

Contributor

That sounds like a cleaner approach than duplicating the code.

Might require some additional organization/renaming of existing deps.txt, build-deps.txt, and Dockerfile to indicate they are for Fedora builds

Maybe these changes could be a follow-on to this PR.

@ashcrow ashcrow force-pushed the ashcrow:rhel-and-py36 branch from 1fd705c to 0ad6347 Jan 10, 2019

@dustymabe dustymabe added the WIP label Jan 10, 2019

@ashcrow ashcrow changed the title WIP: RHEL WIP: Allow for Fedora or RHEL Images Jan 10, 2019

@@ -0,0 +1,2 @@
# Used by mantle
golang golang-bin golang-src

This comment has been minimized.

@dustymabe

dustymabe Jan 10, 2019

Collaborator

just add these to build-deps.txt with a comment: golang-bin and golang-src don't get pulled in automatically on rhel

they already get installed on fedora so it won't matter:

[coreos-assembler]$ rpm -qa | grep golang
golang-1.10.5-1.fc28.x86_64
golang-bin-1.10.5-1.fc28.x86_64
golang-src-1.10.5-1.fc28.noarch
@dustymabe

This comment has been minimized.

Copy link
Collaborator

dustymabe commented Jan 10, 2019

talked with @ashcrow - AFAIK nothing is blocked on this PR going in today so i'm going to try to dedup some of it to make it easier to maintain. Will try to push something later today

@ashcrow

This comment has been minimized.

Copy link
Collaborator

ashcrow commented Jan 10, 2019

Thanks @dustymabe

@ashcrow

This comment has been minimized.

Copy link
Collaborator

ashcrow commented Jan 10, 2019

I'll submit up one more force push. I have a build right now which should allow dumb-init and scl to work happily together.

rhel: RHEL version of c-a building
Signed-off-by: Steve Milner <smilner@redhat.com>

@ashcrow ashcrow force-pushed the ashcrow:rhel-and-py36 branch from 0ad6347 to 20d1c56 Jan 10, 2019

@ashcrow

This comment has been minimized.

Copy link
Collaborator

ashcrow commented Jan 10, 2019

Updated so that the container now runs coreos-assembler with scl

@ashcrow

This comment has been minimized.

Copy link
Collaborator

ashcrow commented Jan 10, 2019

This should be testable now.

@miabbott

This comment has been minimized.

Copy link
Contributor

miabbott commented Jan 11, 2019

TravisCI failed like this:

^-- SC1008: This shebang was unrecognized. Note that ShellCheck only handles sh/bash/dash/ksh.

..because we are finding files to check with grep -l '^#!.*bash'.

If we standardized the bash shebangs to #!/usr/bin/env bash (or something else) we could adjust the TravisCI job accordingly.

I can open another PR for this....WDYT?

@ashcrow

This comment has been minimized.

Copy link
Collaborator

ashcrow commented Jan 11, 2019

@miabbott SGTM.

@miabbott

This comment has been minimized.

Copy link
Contributor

miabbott commented Jan 11, 2019

I built this PR into a container, but got an error about failed dependencies when using the container:

$ git rev-parse HEAD
20d1c56a5e189dabd96fc6ea0f58b85b573d454d

$ sudo buildah bud -t miabbott/coreos-assembler:rhel275 -f Dockerfile.rhel .

...lots of packages installed...

STEP 15: ENTRYPOINT ["/usr/bin/scl-coreos-assembler"]
ERRO[1518] HOSTNAME is not supported for OCI image format, hostname 1d561c58fd2b will be ignored. Must use `docker` format 
STEP 16: COMMIT containers-storage:[overlay@/var/lib/containers/storage+/var/run/containers/storage:overlay.mountopt=nodev,overlay.override_kernel_check=true]localhost/miabbott/coreos-assembler:rhel275
Getting image source signatures
Skipping fetch of repeat blob sha256:56a763045c4544c21c458f3cd948a46384e4b12f9deacd1aede445af598f6d84
Skipping fetch of repeat blob sha256:ab9227d97750aff30bf47631468e9b69dddcdd4af6a853233d6288d05770fcf8
Copying blob sha256:1f18aabf8206a375991f59b1c382cfce5d01b3e9faa7b4cf74edfacc667bdcbe
 541.51 MiB / 541.51 MiB [=================================================] 26s
Copying config sha256:6a2c7236c58b38982ed760c43f9ff7392eae89950b85619fd71d36f7e8e87715
 3.13 KiB / 3.13 KiB [======================================================] 0s
Writing manifest to image destination
Storing signatures
--> 6a2c7236c58b38982ed760c43f9ff7392eae89950b85619fd71d36f7e8e87715

$ sudo podman images | grep rhel275
localhost/miabbott/coreos-assembler                                             rhel275   6a2c7236c58b   26 minutes ago      1.92 GB

$ alias cosa-rhel='sudo podman run --rm --net=host -ti --privileged --userns=host -v $PWD:/srv localhost/miabbott/coreos-assembler:rhel275'

$ cosa-rhel init --force https://gitlab.cee.redhat.com/coreos/redhat-coreos.git maipo
fatal: Failed to find expected dependencies:  dnf-utils libguestfs-tools python2-gobject-base python3-gobject-base
@ashcrow

This comment has been minimized.

Copy link
Collaborator

ashcrow commented Jan 11, 2019

Dependencies never end ..

  • dnf-utils: May take code change
  • libguestfs-tools: Maybe we need to add another repo again
  • python2-gobject-base: Likely needs a repo
  • python3-gobject-base: I think we are pulling the 3.4 version by accident
@miabbott

This comment has been minimized.

Copy link
Contributor

miabbott commented Jan 11, 2019

Saw this in the build logs:

No package dnf-utils available.                                                                                                                                                                                                                                                          
No package python2-gobject-base available.   
Running transaction
  Erasing    : 1:libguestfs-tools-1.38.2-12.el7_6.1.noarch                                                                                                                                                                                                                            1/6
  Erasing    : 1:libguestfs-tools-c-1.38.2-12.el7_6.1.x86_64                                                                                                                                                                                                                          2/6
  Erasing    : 1:perl-Sys-Guestfs-1.38.2-12.el7_6.1.x86_64                                                                                                                                                                                                                            3/6
  Erasing    : 1:libguestfs-1.38.2-12.el7_6.1.x86_64                                                                                                                                                                                                                                  4/6
  Erasing    : kernel-3.10.0-957.1.3.el7.x86_64                                                                                                                                                                                                                                       5/6
  Erasing    : grubby-8.28-25.el7.x86_64                         
  Installing : python34-gobject-base-3.22.0-4.el7.x86_64                                                                                                                                                                                                                          359/367
@ashcrow

This comment has been minimized.

Copy link
Collaborator

ashcrow commented Jan 11, 2019

I'm going to suggest that we close this PR and build on top of the changes @dustymabe has made in a new PR so we don't do work here and then have the code get out of sync.

@ashcrow

This comment has been minimized.

Copy link
Collaborator

ashcrow commented Jan 11, 2019

Closing in favor of #276

@ashcrow ashcrow closed this Jan 11, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment