Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC 0032] Phase running changes for better nix-shell use #32

Open
wants to merge 5 commits into
base: master
from

Conversation

@dezgeg
Copy link

@dezgeg dezgeg commented Aug 29, 2018

Making some minor tweaks to how phases are run in the stdenv to improve the UX of nix-shell.

Rendered view: https://github.com/dezgeg/rfcs/blob/phase-changes/rfcs/0032-run-phase-changes-for-better-nix-shell-use.md

@dezgeg dezgeg force-pushed the dezgeg:phase-changes branch from f3bfa1f to 1479cad Aug 29, 2018
@dezgeg dezgeg changed the title [RFCxxxx] Phase running changes for better nix-shell use [RFC0032] Phase running changes for better nix-shell use Aug 29, 2018
@dezgeg dezgeg changed the title [RFC0032] Phase running changes for better nix-shell use [RFC 0032] Phase running changes for better nix-shell use Aug 29, 2018
@samueldr

This comment has been hidden.

@samueldr
Copy link
Member

@samueldr samueldr commented Aug 30, 2018

given that the issue being fixed is a pure UX issue, not fixing it wouldn't prevent any other work from happening.

Good UX is important in everything. Anything that is clumsy will be fumbled with, and possibly be hated by the end-user, that's not a Nix-specific statement! Don't sell the desire for improvement short.


Is there anything preventing your runPhase alternative to be implemented right now? If it were implemented, (I think) it could be also made compatible with the implemented RFC. This would already be a net increase in UX over running eval .... The only drawback I see is that using runPhase would clobber the runPhase name if one would implement a runPhase as a phase.


As for the RFC itself, other than the generic baseline deprecation concerns, the idea seems sound enough. In reality the "new way" isn't that much of a hassle compared to the status quo. This mostly affects the implementation for a default phase; the user overriding a phase won't need to bother, right?.

As a new~ish user and developer running hooks also makes more sense than clobbering the pre parts by setting a custom phase. That is, in addition to making it easier to run phases properly.

@dezgeg
Copy link
Author

@dezgeg dezgeg commented Aug 30, 2018

Is there anything preventing your runPhase alternative to be implemented right now?

No, it could be added straight away. But I am kind of hopeful it won't be needed.

In reality the "new way" isn't that much of a hassle compared to the status quo. This mostly affects the implementation for a default phase; the user overriding a phase won't need to bother, right?.

Correct. You can see the changes needed in this commit: dezgeg/nixpkgs@87959f3

@timokau
Copy link
Member

@timokau timokau commented Aug 30, 2018

Good idea! I was annoyed by this multiple times and the downsides to his seem tiny.

As more future work (out of scope of this RFC, which is good as-is):

  1. make it easier to emulate the build process. For example:
    1. provide a command that will print out a sort of shell script that emulates what nix-build would do. That would eliminate the need to figure out which phases to run and in which order.
    2. provide something like progressUntil installPhase which will execute everything up to but not including installPhase
  2. add a debugging flag to nix-build (--shell-on-failure) that drops the user into a nix-shell in the current build directory if the build fails
  3. figure out a way to make installPhase work in a nix-shell, maybe by mounting some tmpdir to $out

Out of all of those 1.i. should be pretty easy to implement and a great help already.

@dezgeg
Copy link
Author

@dezgeg dezgeg commented Aug 30, 2018

As more future work (out of scope of this RFC, which is good as-is):

I actually experimented with implementing such tool at some point: https://github.com/dezgeg/nix-debug-shell but I am not particularly good with things like documentation or finishing stuff that I've started, so it's in a kind of forgotten state and potentially bitrotted by now. Maybe it could be moved to https://github.com/nix-community/ if there are people willing to work on it.

@Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented Aug 30, 2018

Nice proposal. I definitely want this.

https://github.com/ryantrinkle/nix-build-inplace is another such debug build helper.

@FRidh
Copy link
Member

@FRidh FRidh commented Aug 30, 2018

For Python (mk-python-derivation.nix) we need this

    buildPhase = ''
        buildPhase
        (cd foo; buildPhase)
        (cd bar; buildPhase)
    '';

I'm glad to see this is covered.

This RFC looks good to me, though I think the Summary can be improved. Running the hooks independent of the phases has nothing to do with the UX of nix-shell usage and so that change should already be mentioned there.

@dezgeg
Copy link
Author

@dezgeg dezgeg commented Aug 30, 2018

For Python (mk-python-derivation.nix) we need this

@FRidh do you have a pointer on where is there is such code in Python infra that need changes? I have found (and fixed) some such cases in my branch but none in Python.

[...] I think the Summary can be improved. Running the hooks independent of the phases has nothing to do with the UX of nix-shell usage and so that change should already be mentioned there.

Yeah, the hook changes are an afterthought that came up after I started writing the description... I will reword it a bit.

dezgeg added 2 commits Aug 30, 2018
@Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented Aug 30, 2018

This is probably off topic, but I'd like if we made the findInputs dependency stuff in a phase, which like preHook is run by nix-shell by default for backwards compat. It seems more consistent to do all "real work" in phases.

@FRidh
Copy link
Member

@FRidh FRidh commented Aug 31, 2018

@dezgeg fixupPhase in pkgs/development/interpreters/python/mk-python-derivation.nix


Actually, reading it again it is a bit different.

@dezgeg
Copy link
Author

@dezgeg dezgeg commented Aug 31, 2018

Actually, reading it again it is a bit different.

Yep, it's a different case. The sort of Nix-level code in mk-python-derivation.nix doesn't require any changes. Changes are only needed if inside a fixupPhase = '' ... ''; block (or equivalently, postFixup) there is a call to the Bash function fixupPhase.

@FRidh
Copy link
Member

@FRidh FRidh commented Sep 1, 2018

@dezgeg do you have a branch with this change? I would like to try it out, especially because of NixOS/nixpkgs@9333c1b

@dezgeg
Copy link
Author

@dezgeg dezgeg commented Sep 1, 2018

The branch is mentioned under related-issues: https://github.com/dezgeg/nixpkgs/tree/better-run-phases

@timokau
Copy link
Member

@timokau timokau commented Oct 5, 2018

Since there is nobody to decide and there hasn't been any opposition, should we merge this?

@zimbatm
Copy link
Member

@zimbatm zimbatm commented Oct 5, 2018

It seems like a good target for 19.03. How much pain will it create for out-of-tree nix code?

@veprbl veprbl mentioned this pull request Oct 20, 2018
3 of 9 tasks complete
@lheckemann
Copy link
Member

@lheckemann lheckemann commented Dec 14, 2018

I usually use runHook buildPhase, is there anything wrong with that?

@shlevy
Copy link
Member

@shlevy shlevy commented Jan 17, 2019

@dezgeg Are you ready for us to open nominations for a shepherd team as per the new RFC process?

@shlevy
Copy link
Member

@shlevy shlevy commented Jan 24, 2019

Closing for now. Feel free to reopen when this is ready for general review and shepherding.

@timokau
Copy link
Member

@timokau timokau commented Jan 9, 2020

Unfortunately my todo list is overflowing right now, so I won't be taking on a new responsibility.

@samueldr
Copy link
Member

@samueldr samueldr commented Feb 12, 2020

I must have missed the notifications in the background noise.

I would be interested in shepherding.

@lheckemann lheckemann mentioned this pull request Mar 5, 2020
12 of 12 tasks complete
@Mic92
Copy link
Contributor

@Mic92 Mic92 commented Mar 5, 2020

We decided on @lheckemann as a shepherd leader on this RFC.

@Mic92
Copy link
Contributor

@Mic92 Mic92 commented Apr 2, 2020

@lheckemann Do you think this RFC should have a meeting?

@spacekookie spacekookie mentioned this pull request Apr 16, 2020
10 of 10 tasks complete
@lheckemann
Copy link
Member

@lheckemann lheckemann commented Apr 30, 2020

@dezgeg, @globin has offered to take over primary authorship of the RFC in case you don't currently have the capacity to continue. If you have any objections, let us know :) otherwise we'll proceed with him as author.

@spacekookie spacekookie mentioned this pull request Apr 30, 2020
11 of 11 tasks complete
@lheckemann lheckemann mentioned this pull request May 28, 2020
10 of 10 tasks complete
@edolstra edolstra mentioned this pull request Jun 11, 2020
9 of 9 tasks complete
@Mic92
Copy link
Contributor

@Mic92 Mic92 commented Jun 25, 2020

@lheckemann Do you still plan to make a meeting for this?

@Mic92 Mic92 mentioned this pull request Jun 25, 2020
11 of 12 tasks complete
@lheckemann lheckemann mentioned this pull request Jul 16, 2020
12 of 12 tasks complete
@spacekookie spacekookie mentioned this pull request Jul 30, 2020
12 of 12 tasks complete
@edolstra edolstra mentioned this pull request Aug 13, 2020
12 of 12 tasks complete
@Mic92 Mic92 mentioned this pull request Aug 27, 2020
13 of 13 tasks complete
Ericson2314 referenced this pull request in NixOS/nix Aug 29, 2020
For example, for building the Nix flake, you would do:

  $ nix develop --configure
  $ nix develop --install
  $ nix develop --installcheck
@spacekookie spacekookie mentioned this pull request Sep 10, 2020
2 of 12 tasks complete
@lheckemann lheckemann mentioned this pull request Sep 24, 2020
11 of 11 tasks complete
@nixos-discourse
Copy link

@nixos-discourse nixos-discourse commented Oct 5, 2020

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixpkgs-phases-which-shell-language-how-to-overwrite-phases/9307/3

@nixos-discourse
Copy link

@nixos-discourse nixos-discourse commented Oct 5, 2020

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-create-package-with-multiple-sources/9308/3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

You can’t perform that action at this time.