New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC 0032] Phase running changes for better nix-shell use #32

Open
wants to merge 3 commits into
base: master
from

Conversation

Projects
None yet
5 participants
@dezgeg

dezgeg commented Aug 29, 2018

Making some minor tweaks to how phases are run in the stdenv to improve the UX of nix-shell.

Rendered view: https://github.com/dezgeg/rfcs/blob/phase-changes/rfcs/0032-run-phase-changes-for-better-nix-shell-use.md

@dezgeg dezgeg changed the title from [RFCxxxx] Phase running changes for better nix-shell use to [RFC0032] Phase running changes for better nix-shell use Aug 29, 2018

@dezgeg dezgeg changed the title from [RFC0032] Phase running changes for better nix-shell use to [RFC 0032] Phase running changes for better nix-shell use Aug 29, 2018

@samueldr

This comment was marked as resolved.

Show comment
Hide comment
@samueldr

samueldr Aug 29, 2018

Member

Formatting tip with github-flavoured markdown:

Highlighting

Given a backticks-guarded block of code ``` you can append a language name to the first set of backticks; giving it e.g. diff to highlight a patch.

nix addendum

Parts of attrsets will want their braces.

  foo = "bar";
{
  foo = "bar";
}

I will collapse this comment once acknowledged with either changes or a reaction

Member

samueldr commented Aug 29, 2018

Formatting tip with github-flavoured markdown:

Highlighting

Given a backticks-guarded block of code ``` you can append a language name to the first set of backticks; giving it e.g. diff to highlight a patch.

nix addendum

Parts of attrsets will want their braces.

  foo = "bar";
{
  foo = "bar";
}

I will collapse this comment once acknowledged with either changes or a reaction

@samueldr

This comment has been minimized.

Show comment
Hide comment
@samueldr

samueldr Aug 30, 2018

Member

given that the issue being fixed is a pure UX issue, not fixing it wouldn't prevent any other work from happening.

Good UX is important in everything. Anything that is clumsy will be fumbled with, and possibly be hated by the end-user, that's not a Nix-specific statement! Don't sell the desire for improvement short.


Is there anything preventing your runPhase alternative to be implemented right now? If it were implemented, (I think) it could be also made compatible with the implemented RFC. This would already be a net increase in UX over running eval .... The only drawback I see is that using runPhase would clobber the runPhase name if one would implement a runPhase as a phase.


As for the RFC itself, other than the generic baseline deprecation concerns, the idea seems sound enough. In reality the "new way" isn't that much of a hassle compared to the status quo. This mostly affects the implementation for a default phase; the user overriding a phase won't need to bother, right?.

As a new~ish user and developer running hooks also makes more sense than clobbering the pre parts by setting a custom phase. That is, in addition to making it easier to run phases properly.

Member

samueldr commented Aug 30, 2018

given that the issue being fixed is a pure UX issue, not fixing it wouldn't prevent any other work from happening.

Good UX is important in everything. Anything that is clumsy will be fumbled with, and possibly be hated by the end-user, that's not a Nix-specific statement! Don't sell the desire for improvement short.


Is there anything preventing your runPhase alternative to be implemented right now? If it were implemented, (I think) it could be also made compatible with the implemented RFC. This would already be a net increase in UX over running eval .... The only drawback I see is that using runPhase would clobber the runPhase name if one would implement a runPhase as a phase.


As for the RFC itself, other than the generic baseline deprecation concerns, the idea seems sound enough. In reality the "new way" isn't that much of a hassle compared to the status quo. This mostly affects the implementation for a default phase; the user overriding a phase won't need to bother, right?.

As a new~ish user and developer running hooks also makes more sense than clobbering the pre parts by setting a custom phase. That is, in addition to making it easier to run phases properly.

@dezgeg

This comment has been minimized.

Show comment
Hide comment
@dezgeg

dezgeg Aug 30, 2018

Is there anything preventing your runPhase alternative to be implemented right now?

No, it could be added straight away. But I am kind of hopeful it won't be needed.

In reality the "new way" isn't that much of a hassle compared to the status quo. This mostly affects the implementation for a default phase; the user overriding a phase won't need to bother, right?.

Correct. You can see the changes needed in this commit: dezgeg/nixpkgs@87959f3

dezgeg commented Aug 30, 2018

Is there anything preventing your runPhase alternative to be implemented right now?

No, it could be added straight away. But I am kind of hopeful it won't be needed.

In reality the "new way" isn't that much of a hassle compared to the status quo. This mostly affects the implementation for a default phase; the user overriding a phase won't need to bother, right?.

Correct. You can see the changes needed in this commit: dezgeg/nixpkgs@87959f3

@timokau

This comment has been minimized.

Show comment
Hide comment
@timokau

timokau Aug 30, 2018

Good idea! I was annoyed by this multiple times and the downsides to his seem tiny.

As more future work (out of scope of this RFC, which is good as-is):

  1. make it easier to emulate the build process. For example:
    1. provide a command that will print out a sort of shell script that emulates what nix-build would do. That would eliminate the need to figure out which phases to run and in which order.
    2. provide something like progressUntil installPhase which will execute everything up to but not including installPhase
  2. add a debugging flag to nix-build (--shell-on-failure) that drops the user into a nix-shell in the current build directory if the build fails
  3. figure out a way to make installPhase work in a nix-shell, maybe by mounting some tmpdir to $out

Out of all of those 1.i. should be pretty easy to implement and a great help already.

timokau commented Aug 30, 2018

Good idea! I was annoyed by this multiple times and the downsides to his seem tiny.

As more future work (out of scope of this RFC, which is good as-is):

  1. make it easier to emulate the build process. For example:
    1. provide a command that will print out a sort of shell script that emulates what nix-build would do. That would eliminate the need to figure out which phases to run and in which order.
    2. provide something like progressUntil installPhase which will execute everything up to but not including installPhase
  2. add a debugging flag to nix-build (--shell-on-failure) that drops the user into a nix-shell in the current build directory if the build fails
  3. figure out a way to make installPhase work in a nix-shell, maybe by mounting some tmpdir to $out

Out of all of those 1.i. should be pretty easy to implement and a great help already.

@dezgeg

This comment has been minimized.

Show comment
Hide comment
@dezgeg

dezgeg Aug 30, 2018

As more future work (out of scope of this RFC, which is good as-is):

I actually experimented with implementing such tool at some point: https://github.com/dezgeg/nix-debug-shell but I am not particularly good with things like documentation or finishing stuff that I've started, so it's in a kind of forgotten state and potentially bitrotted by now. Maybe it could be moved to https://github.com/nix-community/ if there are people willing to work on it.

dezgeg commented Aug 30, 2018

As more future work (out of scope of this RFC, which is good as-is):

I actually experimented with implementing such tool at some point: https://github.com/dezgeg/nix-debug-shell but I am not particularly good with things like documentation or finishing stuff that I've started, so it's in a kind of forgotten state and potentially bitrotted by now. Maybe it could be moved to https://github.com/nix-community/ if there are people willing to work on it.

@dezgeg dezgeg referenced this pull request Aug 30, 2018

Merged

Call for Content: 2018/08 #63

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Aug 30, 2018

Member

Nice proposal. I definitely want this.

https://github.com/ryantrinkle/nix-build-inplace is another such debug build helper.

Member

Ericson2314 commented Aug 30, 2018

Nice proposal. I definitely want this.

https://github.com/ryantrinkle/nix-build-inplace is another such debug build helper.

@FRidh

This comment has been minimized.

Show comment
Hide comment
@FRidh

FRidh Aug 30, 2018

Member

For Python (mk-python-derivation.nix) we need this

    buildPhase = ''
        buildPhase
        (cd foo; buildPhase)
        (cd bar; buildPhase)
    '';

I'm glad to see this is covered.

This RFC looks good to me, though I think the Summary can be improved. Running the hooks independent of the phases has nothing to do with the UX of nix-shell usage and so that change should already be mentioned there.

Member

FRidh commented Aug 30, 2018

For Python (mk-python-derivation.nix) we need this

    buildPhase = ''
        buildPhase
        (cd foo; buildPhase)
        (cd bar; buildPhase)
    '';

I'm glad to see this is covered.

This RFC looks good to me, though I think the Summary can be improved. Running the hooks independent of the phases has nothing to do with the UX of nix-shell usage and so that change should already be mentioned there.

@dezgeg

This comment has been minimized.

Show comment
Hide comment
@dezgeg

dezgeg Aug 30, 2018

For Python (mk-python-derivation.nix) we need this

@FRidh do you have a pointer on where is there is such code in Python infra that need changes? I have found (and fixed) some such cases in my branch but none in Python.

[...] I think the Summary can be improved. Running the hooks independent of the phases has nothing to do with the UX of nix-shell usage and so that change should already be mentioned there.

Yeah, the hook changes are an afterthought that came up after I started writing the description... I will reword it a bit.

dezgeg commented Aug 30, 2018

For Python (mk-python-derivation.nix) we need this

@FRidh do you have a pointer on where is there is such code in Python infra that need changes? I have found (and fixed) some such cases in my branch but none in Python.

[...] I think the Summary can be improved. Running the hooks independent of the phases has nothing to do with the UX of nix-shell usage and so that change should already be mentioned there.

Yeah, the hook changes are an afterthought that came up after I started writing the description... I will reword it a bit.

dezgeg added some commits Aug 30, 2018

@Ericson2314

This comment has been minimized.

Show comment
Hide comment
@Ericson2314

Ericson2314 Aug 30, 2018

Member

This is probably off topic, but I'd like if we made the findInputs dependency stuff in a phase, which like preHook is run by nix-shell by default for backwards compat. It seems more consistent to do all "real work" in phases.

Member

Ericson2314 commented Aug 30, 2018

This is probably off topic, but I'd like if we made the findInputs dependency stuff in a phase, which like preHook is run by nix-shell by default for backwards compat. It seems more consistent to do all "real work" in phases.

@FRidh

This comment has been minimized.

Show comment
Hide comment
@FRidh

FRidh Aug 31, 2018

Member

@dezgeg fixupPhase in pkgs/development/interpreters/python/mk-python-derivation.nix


Actually, reading it again it is a bit different.

Member

FRidh commented Aug 31, 2018

@dezgeg fixupPhase in pkgs/development/interpreters/python/mk-python-derivation.nix


Actually, reading it again it is a bit different.

@dezgeg

This comment has been minimized.

Show comment
Hide comment
@dezgeg

dezgeg Aug 31, 2018

Actually, reading it again it is a bit different.

Yep, it's a different case. The sort of Nix-level code in mk-python-derivation.nix doesn't require any changes. Changes are only needed if inside a fixupPhase = '' ... ''; block (or equivalently, postFixup) there is a call to the Bash function fixupPhase.

dezgeg commented Aug 31, 2018

Actually, reading it again it is a bit different.

Yep, it's a different case. The sort of Nix-level code in mk-python-derivation.nix doesn't require any changes. Changes are only needed if inside a fixupPhase = '' ... ''; block (or equivalently, postFixup) there is a call to the Bash function fixupPhase.

@FRidh

This comment has been minimized.

Show comment
Hide comment
@FRidh

FRidh Sep 1, 2018

Member

@dezgeg do you have a branch with this change? I would like to try it out, especially because of NixOS/nixpkgs@9333c1b

Member

FRidh commented Sep 1, 2018

@dezgeg do you have a branch with this change? I would like to try it out, especially because of NixOS/nixpkgs@9333c1b

@FRidh FRidh referenced this pull request Sep 1, 2018

Merged

pythonPackages.pytest: fix .pytest_cache, again #45898

0 of 9 tasks complete
@dezgeg

This comment has been minimized.

Show comment
Hide comment
@dezgeg

dezgeg Sep 1, 2018

The branch is mentioned under related-issues: https://github.com/dezgeg/nixpkgs/tree/better-run-phases

dezgeg commented Sep 1, 2018

The branch is mentioned under related-issues: https://github.com/dezgeg/nixpkgs/tree/better-run-phases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment