Added support for Mageia linux images. #179

Merged
merged 4 commits into from Oct 1, 2014

Projects

None yet

3 participants

@juanluisbaptiste
Contributor

First version, I think I got it right ;)

@tianon
Member
tianon commented Sep 8, 2014

Are you Mageia upstream, by any chance?

Do you happen to know who owns the "mageia" organization on the Hub? (https://registry.hub.docker.com/repos/mageia/)

We'd love to see Mageia added officially, but we would also like to make sure it's supported in the long run. πŸ˜„

@juanluisbaptiste
Contributor

Hi Tiannon,

Yes, I'm a upstream Mageia developer. I have been with Mageia since its
beginning in 2010 when it was forked from Mandriva, I was one of the
multiple devs that went with them, and I was also a Mandriva developer
since 2007 (and previously for PCLinuxOS, another Mandriva fork).

So yes, our Mageia images will be maintained in the long run, I maintain
severa golang packages needed for docker and help with docker-io packaging.
Currently we are a constant three ppl team for docker and related packages
maintenance, I'm working on packaging nsinit and its dependencies, and
later I will package the docker registry. I also maintain another docker
image at the hub[1] for bigbluebutton conferencing servers.

Also. the organization in the hub is mine.

[1]https://registry.hub.docker.com/u/juanluisbaptiste/bigbluebutton/

Cheers,

On Mon, Sep 8, 2014 at 2:52 PM, Tianon Gravi notifications@github.com
wrote:

Are you Mageia upstream, by any chance?

Do you happen to know who owns the "mageia" organization on the Hub? (
https://registry.hub.docker.com/repos/mageia/)

We'd love to see Mageia added officially, but we would also like to make
sure it's supported in the long run. [image: πŸ˜„]

Reply to this email directly or view it on GitHub
#179 (comment)
.

JLB

@yosifkit
Member

@juanluisbaptiste, Thanks, just one code problem: ADD ../rootfs.tar.xz / on your mageia 3 and 4 Dockerfiles. Can you fix the ADDs and then force push an update here on your official-images (formerly: stackbrew) branch?

We will need a short (200 char, plain text) and long (markdown) description for the docker hub. We are finishing up on the guide doc, but examples can be found in docker-library/docs. The finished examples are the language stacks like python and ruby, so we don't have a good example for a base image like debian or ubuntu.

Anything else @tianon?

@tianon
Member
tianon commented Sep 10, 2014

Thanks for clarifying @juanluisbaptiste πŸ‘ (and I think you've covered it well, @yosifkit)

I'd also like to cc @thatsamguy here since he contributed a script in Docker's repo to generate Mageia base images, but it sounds like you're probably already working together with him? πŸ˜„

@juanluisbaptiste
Contributor

On Wed, Sep 10, 2014 at 4:22 PM, yosifkit notifications@github.com wrote:

@juanluisbaptiste https://github.com/juanluisbaptiste, Thanks, just one
code problem: ADD ../rootfs.tar.xz / on your mageia 3 and 4 Dockerfiles.
Can you fix the ADDs and then force push an update here on your
official-images (formerly: stackbrew) branch?

Oops, fixed, I also updated the commit references as I also updated our
images.

We will need a short (200 char, plain text) and long (markdown)
description for the docker hub. We are finishing up on the guide doc, but
examples can be found in docker-library/docs
https://github.com/docker-library/docs. The finished examples are the
language stacks like python and ruby, so we don't have a good example for a
base image like debian or ubuntu

Ok, I'll take a look at this and update the readme file accordingly.

@juanluisbaptiste
Contributor

On Wed, Sep 10, 2014 at 4:29 PM, Tianon Gravi notifications@github.com
wrote:

Thanks for clarifying @juanluisbaptiste
https://github.com/juanluisbaptiste [image: πŸ‘](and I think you've
covered it well, @yosifkit <https://github.com/yosifkit)

I'd also like to cc @thatsamguy https://github.com/thatsamguy here
since he contributed a script in Docker's repo to generate Mageia base
images, but it sounds like you're probably already working together with
him? [image: πŸ˜„]

Yes, my script is based on his, plus some parts from mkimage.sh :)

Juancho

@juanluisbaptiste
Contributor

Here's the pull request for the documentation:

docker-library/docs#9

@yosifkit
Member

LGTM

cc @tianon for test build run

@tianon
Member
tianon commented Sep 19, 2014

Hey @juanluisbaptiste, I agree with @yosifkit; this looks really good. I've got just one last request.

Since this is a big tarball, and Git really doesn't handle big tarballs well, any commit which updates a tarball essentially doubles the size of the repo for no good reason (and thus it takes twice as long to clone on the build server and in our testing, not to mention in your own maintenance, which is not fun for any of us).

There are three main ways I combat this for my own repos (debian and ubuntu):

  1. git checkout --orphan <somebranch> - this is attractive because it creates a brand new "initial commit" if you will, in which you can then make your brand new single commit (which then has no history and clones fast) - the main downside I found with this approach is that you lose the history of any scripts in your repo, which is actually useful history, but if that doesn't matter to you, this is a solid simple option
  2. git rebase -i --root - this lets you interactively rebase all your commits, including squashing them together, reordering them, deleting them outright, etc, so is a pretty advanced option, but gives you a great deal of flexibility in just squashing together your "tarball containing" commits, and even if you don't use it for maintaining the image long-term, it'd be a pretty solid way to rewrite your existing history to just clean up your existing tarball-containing commits
  3. git checkout -b dist master - this is actually how I prefer to maintain my images now (after trying a variety of methods); essentially, you have a "master" branch where you never ever commit tarballs, and it holds the full history of all your nice scripts and scaffolding (directories and the like), and once you're ready to commit and push a new tarball, you create a new "dist" branch based on the "master" branch, and make your single tarball commit there, then force push that to GitHub - once it's time for the next release, you git branch -D dist (to delete your local "dist" branch), and then git checkout -b dist master again to make a brand new one that doesn't contain that older tarball commit

Sorry, that's a lot to digest, but hopefully it helps. Of course, I'm willing to help out in any way you need - I've got a lot of experience dealing with this particular wart of our base image turtles, and I'm happy to help however I can (including collaborating on the repo, if you get really stuck).

@juanluisbaptiste
Contributor

@tianon ok I understand the instructions, but what it isn't clear is what I have to do right now, the instructions seem to be more for future tarball updates ?

@yosifkit
Member

Yes, there is a second set of tarballs in the git history, you should rebase and squash them.

@juanluisbaptiste
Contributor

@yosifkit I'm a little lost with the rebase step, I'm not that expert with git, could you please expand a little that part ?

@yosifkit
Member

No problem, @tianon is the real expert, but he'll pop in if I go wrong.

I found a helpful tutorial on gitready.com. You can ignore the part about only changing stuff that was not pushed remotely.

I would pull git log --stat in one terminal to see which commits you want squash and open git rebase -i --root in the other terminal. Then change the pick to squash on the commits where the old binaries were added.

If there are other changes in the particular commit when the binaries were added, then you will need to put edit instead of pick or squash. It will drop you in at that point of the git history; just git rm the old binary files, git commit --amend to commit the non-binary file changes and then git rebase --continue.

Once you have a cleaned history (you can look at it once the rebase is done with a new git log --stat), then you just git push -f to forcibly update the remote on github.

Let me know if you have any other questions. I am available in the #docker-library channel on freenode for the next hour or so if you need to chat.

@juanluisbaptiste
Contributor

@yosifkit Ok, I think I got it, please review.

@yosifkit
Member

Still looks like there are multiple commits adding different tar files. I did a quick run through and here are my steps. I am assuming you did a git clone https://github.com/juanluisbaptiste/docker-brew-mageia.git, so it is based off master branch using origin for github.

$ git checkout master
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
$ git rebase -i --root

At this point the default text editor opens, in my case vim. In the text editor I changed the second "Initial Commit" with ref 5f8dc030eadcff74953e90dd7a4269faae21fef3 from pick to edit, then write and quit.

$ git rm 3/rootfs-3.tar.xz 4/rootfs-4.tar.xz cauldron/rootfs.tar.xz
$ git commit --amend -v  // (edit commit message if desired, write and quit)
$ git rebase --continue

This will continue rebasing the commits until the last commit where the tarballs were changed with something similar to the following:

error: could not apply 6f7ac29... - Added locales and locales-en to avoid errors setting the locale.

When you have resolved this problem, run "git rebase --continue".
If you prefer to skip this patch, run "git rebase --skip" instead.
To check out the original branch and stop rebasing, run "git rebase --abort".
Could not apply 6f7ac2963f66465cf9128a6dc6af6386d9abc955... - Added locales and locales-en to avoid errors setting the locale. - Updated all images to include locales and locales-en

Then we can git status to see the problem:

$ git status
rebase in progress; onto 04551e5
You are currently rebasing branch 'new-master' on '04551e5'.
  (fix conflicts and then run "git rebase --continue")
  (use "git rebase --skip" to skip this patch)
  (use "git rebase --abort" to check out the original branch)

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    modified:   mkimage-urpmi.sh

Unmerged paths:
  (use "git reset HEAD <file>..." to unstage)
  (use "git add/rm <file>..." as appropriate to mark resolution)

    deleted by us:      3/rootfs-3.tar.xz
    deleted by us:      4/rootfs-4.tar.xz
    deleted by us:      cauldron/rootfs.tar.xz

So we just add our tar files back (the new tars that replaced the old, but now there is nothing to replace since the old ones are no longer in history):

$ git add 3/rootfs-3.tar.xz 4/rootfs-4.tar.xz cauldron/rootfs.tar.xz
$ git rebase --continue

You can see the result in a git log --stat with something similar to the following (which shows us that there is only one set of tars in the history):

commit 32d0bdc60d9560968c4048b38b3d22342dd049fc
Author: Juan Luis Baptiste <juan.baptiste@gmail.com>
Date:   Thu Sep 11 00:03:34 2014 -0500

    - Added locales and locales-en to avoid errors setting the locale.
    - Updated all images to include locales and locales-en

 3/rootfs-3.tar.xz      | Bin 0 -> 38598928 bytes
 4/rootfs-4.tar.xz      | Bin 0 -> 41195440 bytes
 cauldron/rootfs.tar.xz | Bin 0 -> 41349344 bytes
 mkimage-urpmi.sh       |   2 +-
 4 files changed, 1 insertion(+), 1 deletion(-)

The only thing left to do is to force push this to master on github.

$ git push --force origin master

Then feel free to delete your local dist branch git branch -D dist and the one on github git push origin --delete dist.

@juanluisbaptiste
Contributor

Done, please check again.

@yosifkit
Member

Master branch looks great. πŸ‘ Just need to update the git commit refs here. Also, drop the old dist branch to clean out the old tars (it will help our local testing and the build server clone faster).

When you need to update the tar files later, you should be able to git add tar/files with a git commit --amend and a git push --force.

@juanluisbaptiste
Contributor

Just deleted the remote dist branch, but I don't understand your instructions when needing to add a new tar file, I thought that in that case I should use the instructions indicated by @tianon before ?

@tianon
Member
tianon commented Sep 30, 2014

I think it sounds like the easiest method for you in the future is to go down the "amend" route (ie, use git commit --amend each time you need to update the tarballs). It's a lot less confusing, and then you're just constantly updating that one commit that includes the tarballs.

@yosifkit
Member
yosifkit commented Oct 1, 2014

Could you update this branch PR to the new commit refs?

@juanluisbaptiste
Contributor

Done, please review.

@tianon
Member
tianon commented Oct 1, 2014

LGTM

$ ./bashbrew/build.sh 'https://raw.githubusercontent.com/juanluisbaptiste/stackbrew/72cf138fcd5ddf888158c3a62aff7110f12e742c/library/mageia'
Cloning 'git://github.com/juanluisbaptiste/docker-brew-mageia' into '/home/tianon/docker/stackbrew/bashbrew/src/github.com/juanluisbaptiste/docker-brew-mageia' ...
Cloned successfully!
Processing mageia:latest ...
Processing mageia:cauldron ...
Processing mageia:4 ...
Processing mageia:3 ...
@tianon
Member
tianon commented Oct 1, 2014

Also,

$ docker images mageia
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
mageia              3                   672a64b82925        29 seconds ago      147.6 MB
mageia              cauldron            b9662b488d18        37 seconds ago      167.3 MB
mageia              4                   cfcd3e37da6e        45 seconds ago      161.3 MB
mageia              latest              cfcd3e37da6e        45 seconds ago      161.3 MB
@yosifkit
Member
yosifkit commented Oct 1, 2014

LGTM!

@yosifkit yosifkit merged commit 613791f into docker-library:master Oct 1, 2014

1 check passed

continuous-integration/travis-ci The Travis CI build passed
Details
@juanluisbaptiste
Contributor

@tianon I'm not sure if you are just showing that images are ready or if I'm still missing something ?

Also, what means LGTM ? :)

@tianon
Member
tianon commented Oct 1, 2014

"Looks Good To Me"
it's our/the Docker way of saying, "this is ready to merge" - once we have a quorum of the repo maintainers who've stamped the PR with LGTM, we're good to merge it :)

The output there is my build test to show that I've actually successfully build-tested your file before we merge it so we don't have it unexpectedly fail on the build server. πŸ‘

@juanluisbaptiste
Contributor

Excellent !! let me know when it is available at the hub so I can tell the rest of the mageia gang :)

@juanluisbaptiste
Contributor

Thanks to you for this wonderful piece of technology !! I'm like addicted to it and I'm moving all of our sites at work to docker :)

@tianon tianon referenced this pull request Nov 3, 2014
Merged

Updated CRUX to 3.1 #289

@tianon tianon added the new-image label Oct 5, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment