Merge NixOS into Nixpkgs #1041

Closed
edolstra opened this Issue Oct 4, 2013 · 19 comments

Comments

Projects
None yet
@edolstra
Member

edolstra commented Oct 4, 2013

I'd like to merge the NixOS tree into the Nixpkgs repository, as has been discussed a few times in the past. The reasons for merging them are:

  • Changes in Nixpkgs often require a change in NixOS and vice versa. With separate repositories, this means you need to do two commits, which is not atomic and creates the possibility of trees being out of sync with each other.
  • Branches in Nixpkgs often require corresponding branches in NixOS. For instance, a Nixpkgs branch like multiple-outputs requires a branch in NixOS because it changes the structure of some packages.
  • Issue tracking: it's often unclear whether an issue should be reported in the NixOS or Nixpkgs issue tracker, and often the fix is in the other repository.
  • The NixOS version is currently a 2-tuple of Git hashes. Merging NixOS and Nixpkgs allows the NixOS version to be identified using a single Git hash.

There are also downsides to merging:

  • It makes the Nixpkgs tree a bit bigger, which may be slightly annoying for people who only use Nixpkgs and not NixOS.
  • It makes it harder to build a NixOS tree against a different Nixpkgs tree. But given the tight coupling between these trees, that doesn't work very well in practice anyway. OTOH, it becomes much easier to use multiple Nixpkgs trees in a NixOps deployment (since there are no more <nixpkgs> references).

I did a test merge, available here: https://github.com/edolstra/nixpkgs/tree/merge-test. It imports nixos/master as a subdirectory "nixos" in the root directory of Nixpkgs (keeping all history). While I was at it, I also renamed pkgs/lib to lib because it never really made sense to have it in pkgs.

Note that GitHub unfortunately doesn't show history across renames, but "git log --follow" does (and GitHub's "blame" also works fine).

@ghost ghost assigned edolstra Oct 4, 2013

@viric

This comment has been minimized.

Show comment
Hide comment
@viric

viric Oct 4, 2013

Member

It looks to me like a very good idea. I'm for it!

On Fri, Oct 04, 2013 at 03:59:58AM -0700, Eelco Dolstra wrote:

I'd like to merge the NixOS tree into the Nixpkgs repository, as has been discussed a few times in the past. The reasons for merging them are:

  • Changes in Nixpkgs often require a change in NixOS and vice versa. With separate repositories, this means you need to do two commits, which is not atomic and creates the possibility of trees being out of sync with each other.
  • Branches in Nixpkgs often require corresponding branches in NixOS. For instance, a Nixpkgs branch like multiple-outputs requires a branch in NixOS because it changes the structure of some packages.
  • Issue tracking: it's often unclear whether an issue should be reported in the NixOS or Nixpkgs issue tracker, and often the fix is in the other repository.
  • The NixOS version is currently a 2-tuple of Git hashes. Merging NixOS and Nixpkgs allows the NixOS version to be identified using a single Git hash.

There are also downsides to merging:

  • It makes the Nixpkgs tree a bit bigger, which may be slightly annoying for people who only use Nixpkgs and not NixOS.
  • It makes it harder to build a NixOS tree against a different Nixpkgs tree. But given the tight coupling between these trees, that doesn't work very well in practice anyway. OTOH, it becomes much easier to use multiple Nixpkgs trees in a NixOps deployment (since there are no more <nixpkgs> references).

I did a test merge, available here: https://github.com/edolstra/nixpkgs/tree/merge-test. It imports nixos/master as a subdirectory "nixos" in the root directory of Nixpkgs (keeping all history). While I was at it, I also renamed pkgs/lib to lib because it never really made sense to have it in pkgs.

Note that GitHub unfortunately doesn't show history across renames, but "git log --follow" does (and GitHub's "blame" also works fine).


Reply to this email directly or view it on GitHub:
#1041

Member

viric commented Oct 4, 2013

It looks to me like a very good idea. I'm for it!

On Fri, Oct 04, 2013 at 03:59:58AM -0700, Eelco Dolstra wrote:

I'd like to merge the NixOS tree into the Nixpkgs repository, as has been discussed a few times in the past. The reasons for merging them are:

  • Changes in Nixpkgs often require a change in NixOS and vice versa. With separate repositories, this means you need to do two commits, which is not atomic and creates the possibility of trees being out of sync with each other.
  • Branches in Nixpkgs often require corresponding branches in NixOS. For instance, a Nixpkgs branch like multiple-outputs requires a branch in NixOS because it changes the structure of some packages.
  • Issue tracking: it's often unclear whether an issue should be reported in the NixOS or Nixpkgs issue tracker, and often the fix is in the other repository.
  • The NixOS version is currently a 2-tuple of Git hashes. Merging NixOS and Nixpkgs allows the NixOS version to be identified using a single Git hash.

There are also downsides to merging:

  • It makes the Nixpkgs tree a bit bigger, which may be slightly annoying for people who only use Nixpkgs and not NixOS.
  • It makes it harder to build a NixOS tree against a different Nixpkgs tree. But given the tight coupling between these trees, that doesn't work very well in practice anyway. OTOH, it becomes much easier to use multiple Nixpkgs trees in a NixOps deployment (since there are no more <nixpkgs> references).

I did a test merge, available here: https://github.com/edolstra/nixpkgs/tree/merge-test. It imports nixos/master as a subdirectory "nixos" in the root directory of Nixpkgs (keeping all history). While I was at it, I also renamed pkgs/lib to lib because it never really made sense to have it in pkgs.

Note that GitHub unfortunately doesn't show history across renames, but "git log --follow" does (and GitHub's "blame" also works fine).


Reply to this email directly or view it on GitHub:
#1041

@bluescreen303

This comment has been minimized.

Show comment
Hide comment
@bluescreen303

bluescreen303 Oct 4, 2013

Contributor

Sounds good to me.
Will this be combined with the upcoming stable release?

Eelco Dolstra notifications@github.com writes:

I'd like to merge the NixOS tree into the Nixpkgs repository, as has
been discussed a few times in the past. The reasons for merging them
are:

  • Changes in Nixpkgs often require a change in NixOS and vice
    versa. With separate repositories, this means you need to do two
    commits, which is not atomic and creates the possibility of trees
    being out of sync with each other.
  • Branches in Nixpkgs often require corresponding branches in
    NixOS. For instance, a Nixpkgs branch like multiple-outputs requires a
    branch in NixOS because it changes the structure of some packages.
  • Issue tracking: it's often unclear whether an issue should be
    reported in the NixOS or Nixpkgs issue tracker, and often the fix is
    in the other repository.
  • The NixOS version is currently a 2-tuple of Git hashes. Merging
    NixOS and Nixpkgs allows the NixOS version to be identified using a
    single Git hash.

There are also downsides to merging:

  • It makes the Nixpkgs tree a bit bigger, which may be slightly
    annoying for people who only use Nixpkgs and not NixOS.
  • It makes it harder to build a NixOS tree against a different Nixpkgs
    tree. But given the tight coupling between these trees, that doesn't
    work very well in practice anyway. OTOH, it becomes much easier to use
    multiple Nixpkgs trees in a NixOps deployment (since there are no more
    ` references).

I did a test merge, available here:
https://github.com/edolstra/nixpkgs/tree/merge-test. It imports
nixos/master as a subdirectory "nixos" in the root directory of
Nixpkgs (keeping all history). While I was at it, I also renamed
pkgs/lib to lib because it never really made sense to have it in pkgs.

Note that GitHub unfortunately doesn't show history across renames,
but "git log --follow" does (and GitHub's "blame" also works fine).


Reply to this email directly or view it on GitHub:
#1041

Contributor

bluescreen303 commented Oct 4, 2013

Sounds good to me.
Will this be combined with the upcoming stable release?

Eelco Dolstra notifications@github.com writes:

I'd like to merge the NixOS tree into the Nixpkgs repository, as has
been discussed a few times in the past. The reasons for merging them
are:

  • Changes in Nixpkgs often require a change in NixOS and vice
    versa. With separate repositories, this means you need to do two
    commits, which is not atomic and creates the possibility of trees
    being out of sync with each other.
  • Branches in Nixpkgs often require corresponding branches in
    NixOS. For instance, a Nixpkgs branch like multiple-outputs requires a
    branch in NixOS because it changes the structure of some packages.
  • Issue tracking: it's often unclear whether an issue should be
    reported in the NixOS or Nixpkgs issue tracker, and often the fix is
    in the other repository.
  • The NixOS version is currently a 2-tuple of Git hashes. Merging
    NixOS and Nixpkgs allows the NixOS version to be identified using a
    single Git hash.

There are also downsides to merging:

  • It makes the Nixpkgs tree a bit bigger, which may be slightly
    annoying for people who only use Nixpkgs and not NixOS.
  • It makes it harder to build a NixOS tree against a different Nixpkgs
    tree. But given the tight coupling between these trees, that doesn't
    work very well in practice anyway. OTOH, it becomes much easier to use
    multiple Nixpkgs trees in a NixOps deployment (since there are no more
    ` references).

I did a test merge, available here:
https://github.com/edolstra/nixpkgs/tree/merge-test. It imports
nixos/master as a subdirectory "nixos" in the root directory of
Nixpkgs (keeping all history). While I was at it, I also renamed
pkgs/lib to lib because it never really made sense to have it in pkgs.

Note that GitHub unfortunately doesn't show history across renames,
but "git log --follow" does (and GitHub's "blame" also works fine).


Reply to this email directly or view it on GitHub:
#1041

@edolstra

This comment has been minimized.

Show comment
Hide comment
@edolstra

edolstra Oct 4, 2013

Member

@bluescreen303 Yeah, that's basically why I want to do this now. If we do it after the release, we'll have to keep two repositories alive for a while.

Member

edolstra commented Oct 4, 2013

@bluescreen303 Yeah, that's basically why I want to do this now. If we do it after the release, we'll have to keep two repositories alive for a while.

@vcunat

This comment has been minimized.

Show comment
Hide comment
@vcunat

vcunat Oct 4, 2013

Member

For the downsides:

  • The sizes of nixpkgs vs. nixos is on the order of 10:1 and I don't think it would decrease much in future, rather the other way.
  • I think you can get the combination in your work tree in two commands. First git-checkout the whole tree for the "nixpkgs version", and then git-checkout the nixos subtree for the "nixos version".
Member

vcunat commented Oct 4, 2013

For the downsides:

  • The sizes of nixpkgs vs. nixos is on the order of 10:1 and I don't think it would decrease much in future, rather the other way.
  • I think you can get the combination in your work tree in two commands. First git-checkout the whole tree for the "nixpkgs version", and then git-checkout the nixos subtree for the "nixos version".
@vcunat

This comment has been minimized.

Show comment
Hide comment
@vcunat

vcunat Oct 4, 2013

Member

What I'm not clear about: can we somehow "merge" the github issues/PRs? There are quite many for both nixos and nixpkgs.

Member

vcunat commented Oct 4, 2013

What I'm not clear about: can we somehow "merge" the github issues/PRs? There are quite many for both nixos and nixpkgs.

@edolstra

This comment has been minimized.

Show comment
Hide comment
@edolstra

edolstra Oct 4, 2013

Member

There is a script to move issues to another GitHub project, but it doesn't keep comment authorship: https://github.com/collective/collective.developermanual/blob/master/gh-issues-import.py

Member

edolstra commented Oct 4, 2013

There is a script to move issues to another GitHub project, but it doesn't keep comment authorship: https://github.com/collective/collective.developermanual/blob/master/gh-issues-import.py

@the-kenny

This comment has been minimized.

Show comment
Hide comment
@the-kenny

the-kenny Oct 4, 2013

Contributor

I just wrote a mail to Github asking if they know a way to merge projects while keeping all issues with comments.

Contributor

the-kenny commented Oct 4, 2013

I just wrote a mail to Github asking if they know a way to merge projects while keeping all issues with comments.

@edolstra

This comment has been minimized.

Show comment
Hide comment
@edolstra

edolstra Oct 4, 2013

Member

Cool, thanks!

Member

edolstra commented Oct 4, 2013

Cool, thanks!

@bjornfor

This comment has been minimized.

Show comment
Hide comment
@bjornfor

bjornfor Oct 5, 2013

Contributor

I use nixpkgs on Ubuntu on a couple of machines, but still +1 for merging. Looking forward to easily git bisect the whole of NixOS and more easily recreate exact builds (no more manual work to match nixos/nixpkgs versions).

Contributor

bjornfor commented Oct 5, 2013

I use nixpkgs on Ubuntu on a couple of machines, but still +1 for merging. Looking forward to easily git bisect the whole of NixOS and more easily recreate exact builds (no more manual work to match nixos/nixpkgs versions).

@domenkozar

This comment has been minimized.

Show comment
Hide comment
@domenkozar

domenkozar Oct 5, 2013

Member

+1

Member

domenkozar commented Oct 5, 2013

+1

@MarcWeber

This comment has been minimized.

Show comment
Hide comment
@MarcWeber

MarcWeber Oct 5, 2013

Contributor

I'm fine with this change, too.

There are more reasons to merge: Eg should "creating an iso or an usb image of nixos" be a nixpkgs or a nixos project?

The more I think about it the earlier this should be done.

Contributor

MarcWeber commented Oct 5, 2013

I'm fine with this change, too.

There are more reasons to merge: Eg should "creating an iso or an usb image of nixos" be a nixpkgs or a nixos project?

The more I think about it the earlier this should be done.

@garbas

This comment has been minimized.

Show comment
Hide comment
@garbas

garbas Oct 6, 2013

Contributor

awesome! 🍺

Contributor

garbas commented Oct 6, 2013

awesome! 🍺

@offlinehacker

This comment has been minimized.

Show comment
Hide comment
@offlinehacker

offlinehacker Oct 7, 2013

Contributor

+1

On Sun, Oct 6, 2013 at 6:06 PM, Rok Garbas notifications@github.com wrote:

awesome! [image: 🍺]


Reply to this email directly or view it on GitHubhttps://github.com/NixOS/nixpkgs/issues/1041#issuecomment-25770756
.

Contributor

offlinehacker commented Oct 7, 2013

+1

On Sun, Oct 6, 2013 at 6:06 PM, Rok Garbas notifications@github.com wrote:

awesome! [image: 🍺]


Reply to this email directly or view it on GitHubhttps://github.com/NixOS/nixpkgs/issues/1041#issuecomment-25770756
.

@edolstra

This comment has been minimized.

Show comment
Hide comment
@edolstra

edolstra Oct 10, 2013

Member

Okay, I'll do the merge now. Cross fingers...

I'll turn off commits to the NixOS repo first.

Member

edolstra commented Oct 10, 2013

Okay, I'll do the merge now. Cross fingers...

I'll turn off commits to the NixOS repo first.

@offlinehacker

This comment has been minimized.

Show comment
Hide comment
@offlinehacker

offlinehacker Oct 10, 2013

Contributor

Do we have to reopen pull requests now or will they be handled in any other way?

Contributor

offlinehacker commented Oct 10, 2013

Do we have to reopen pull requests now or will they be handled in any other way?

@MarcWeber

This comment has been minimized.

Show comment
Hide comment
@MarcWeber

MarcWeber Oct 10, 2013

Contributor

Excerpts from Jaka Hudoklin's message of Do Okt 10 19:58:59 +0200 2013:

Do we have to reopen pull requests now or will they be handled in any other way?
The thread talked about a request sent to github about whether there is
a way to "merge" issues and the like. Don't know about the reply or
current state.

Closing and reopening is unlikely to be wrong.

Marc Weber

Contributor

MarcWeber commented Oct 10, 2013

Excerpts from Jaka Hudoklin's message of Do Okt 10 19:58:59 +0200 2013:

Do we have to reopen pull requests now or will they be handled in any other way?
The thread talked about a request sent to github about whether there is
a way to "merge" issues and the like. Don't know about the reply or
current state.

Closing and reopening is unlikely to be wrong.

Marc Weber

@domenkozar

This comment has been minimized.

Show comment
Hide comment
@domenkozar

domenkozar Oct 27, 2013

Member

This can be closed?

Member

domenkozar commented Oct 27, 2013

This can be closed?

@edolstra

This comment has been minimized.

Show comment
Hide comment
@edolstra

edolstra Oct 27, 2013

Member

Yes, closing now...

Member

edolstra commented Oct 27, 2013

Yes, closing now...

@edolstra edolstra closed this Oct 27, 2013

@grwlf

This comment has been minimized.

Show comment
Hide comment
@grwlf

grwlf Oct 29, 2013

Contributor

Thanks, +1

Contributor

grwlf commented Oct 29, 2013

Thanks, +1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment