Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for nix-shell in JetBrains products #1

Open
jgeerds opened this issue Sep 14, 2016 · 27 comments
Open

Add support for nix-shell in JetBrains products #1

jgeerds opened this issue Sep 14, 2016 · 27 comments

Comments

@jgeerds
Copy link
Member

jgeerds commented Sep 14, 2016

This might be a lot of work but it would be awesome if you could use nix-shell inside JetBrains products (e.g. activating shell.nix of your current project).

I have a lot of friends who use virtualenv instead of nix just because PyCharm (the python IDE by JetBrains) have builtin support for it.

What do you think about it?

@edwtjo
Copy link
Member

edwtjo commented Sep 18, 2016

Yes of course, using profiles or shells for development is one of the main goals of this plugin.

@afranche
Copy link

Hi,

Any update on this project?

@Mic92
Copy link
Member

Mic92 commented Feb 25, 2020

Maybe nix-shell --run 'env' > .env could be used with https://plugins.jetbrains.com/plugin/7861-envfile for the time being.

@evanjs
Copy link
Member

evanjs commented Feb 25, 2020

Just note that EnvFile is not currently supported across all JetBrains applications, unfortunately.

Most notably for me is CLion (for Rust envs, etc.)

Related: ashald/EnvFile#71

@jonringer
Copy link

I generally start my pycharm while inside a nix-shell with my virtualenv. It works really well because pycharm is already able to reason about virtualenvs

@ergose
Copy link

ergose commented Nov 27, 2020

@jonringer - for the work-around win. :) Big thanks. I've been wracking my brain all afternoon trying to get it working. I'm now running NixOS on several machines and wanted to start using nix-shell instead of venv. Another vote for native support.

Some additional details to anyone still trying to figure out:

  1. Create you a default.nix/shell.nix in your Python project directory (if default.nix for packaging, you'll also probably want a setup.py in there also).
  2. Open terminal, nav to project dir, and type nix-shell to load up environment as normal
  3. Open PyCharm from within the nix-shell (ex: nohup pycharm-community >/dev/null 2>&1 & - so can close terminal after pycharm opens.)
  4. Add a Python Interpreter -> System Interpreter -> /run/current-system/sw/bin/python3.8 (assuming 3.8 on NixOS 20.09)
  5. Create you a Run Configuration pointing to the interpreter you just setup.
  6. Press the Run button and cross fingers. For me, main script ran as expected, debug seems to work, and PyCharm sees the packages pulled in from the shell.nix. That said though, I haven't tried to add more packages yet. We may have to update the shell.nix with any new packages and re-load PyCharm to add packages, that or maybe install same version in repo via PyCharm while programming and just remember to add it to shell.nix for next time.

@fehnomenal
Copy link

I created a plugin to load direnvs .envrc files at https://github.com/fehnomenal/intellij-direnv and use it with flakes and lorri in old projects.

It works, but is nothing but a big hack (because the JVM does not want you to modify the environment...). To natively support changing the environment on a per project basis, the IDE could clone the environment from the OS (like the JVM does it) and pass the clone to all processes it launches while allowing modifications. This would allow implementing direnv support without hacks.

@l0b0
Copy link

l0b0 commented Sep 19, 2021

@ergose Thank you for the instructions! That wasn't quite enough to get IDEA to pick up the Python dependencies, I also had to add pip to the mkShell buildInputs, otherwise IDEA wouldn`t find the packages in the project SDK settings (and trying to install the packaging software within IDEA didn't work).

Unfortunately the /run/current-system/sw/bin/python… path does not exist when using Nix on Ubuntu.

Neither the envfile nor intellij-direnv seem to work for wiring up the SDK when using Nix outside of NixOS. The workaround is to change the path to the SDK every time the Nix shell is updated.

@SDAChess
Copy link

Hello,

I came across this plugin that tries to add this feature.
https://plugins.jetbrains.com/plugin/16864-enter-nix-shell
You can find the source here: https://github.com/fctorial/enter_nix_shell

Maybe there are some interesting ideas that could be exploited.
Also, the new flakes and nix develop environments could be a good option to add too.

@JojOatXGME
Copy link
Contributor

JojOatXGME commented Feb 6, 2022

Thanks for your suggestions. I am reluctant to integrate the solution of Enter Nix Shell or intellij-direnv into this plugin. 😔 They both work by changing the environment of the IDEA process. This means that you cannot have different environments per project. While changing the IDEA environment with these plugins might be easier than restarting the whole IDE, it might also work less reliable. For example, see this page from Enter Nix Shell (archive.org). Besides that, you could also end up with a mix of multiple environments, which could have strange effects in some edge cases. While you might very well use one of these plugins, I feel like the implementations are too unstable for an official solution. At least that is my personal opinion.

@JojOatXGME
Copy link
Contributor

I used this weekend to explore the IntelliJ Platform API for alternatives. Unfortunately, I haven't found a good solution.

Run targets

I found Run targets (extension point com.intellij.executionTargetType). They allow users to execute run configurations within Docker containers or within WSL. It might also be usable to move the execution into a nix-shell. However, run targets are an ultimate feature and therefore not available for free. Also note that they do not affect the console panel or SDK detection. So it would only be a partial solution.

Run configuration extensions

EnvFile and ProjectEnv use com.intellij.​run​Configuration​Extension among other extension points. However, this extension point is specifically for IntelliJ (i.e. Java). Other IDEs have their own extension points. So we would have to implement it for every IDE separately. It is also not sure if all these extension points combined cover every IDE and type of run configuration.

Program runner

I also found the extension point com.intellij.programRunner. Program runners are the components which implement the execution of specific run configurations. I tried adding a custom program runner which claims it can execute all run configurations, which then modifies the environment and forwards it to the actual runner. Unfortunately, it looks like I cannot change environment variables at this level. I also couldn't make the runner work reliable in my tests. In many cases, IntelliJ continued to call the actual runner directly. Besides, this solution is also a bit hacky. I could imagine that it breaks some configurations.

Source code

plugin.xml

<extensions defaultExtensionNs="com.intellij">
  <programRunner implementation="org.nixos.idea.shell.NixProgramRunner" order="first"/>
</extensions>

NixProgramRunner.java

public final class NixProgramRunner implements ProgramRunner<RunnerSettings> {
    @Override
    public @NotNull @NonNls String getRunnerId() {
        return "nix-shell-wrapper";
    }

    @Override
    public boolean canRun(@NotNull String executorId, @NotNull RunProfile profile) {
        return true;
    }

    @Override
    public void execute(@NotNull ExecutionEnvironment environment) throws ExecutionException {
        ProgramRunner<?> runner = ProgramRunner.PROGRAM_RUNNER_EP.findFirstSafe(r -> matches(environment, r));
        // Somehow modify the environment?
        runner.execute(environment);
    }

    private boolean matches(@NotNull ExecutionEnvironment environment, @NotNull ProgramRunner<?> runner) {
        String executorId = environment.getExecutor().getId();
        return runner != this && runner.canRun(executorId, environment.getRunProfile());
    }
}

Execution target provider

The extension point com.intellij.executionTargetProvider also got my attention. However, on their own, execution targets only affect the UI. They seem to be used by Android Studio to choose the device where the app gets executed. But the implementation which looks at the configuration actually sits within the specific program runner (see last section) for Android apps. Adding an execution target causes them to be visible in the UI, but it does not impact the behavior at all.

grafik

Conclusion

As written in the first paragraph of this comment, it looks like there is no straightforward solution. From the options I have covered, the approach of EnvFile and ProjectEnv seems to be the most promising, but it might take a lot of effort. So I am still open for suggestions.

@JojOatXGME
Copy link
Contributor

I created a post on the community board of JetBrains. Let's see if someone responds.
https://intellij-support.jetbrains.com/hc/en-us/community/posts/4418065675538-Project-specific-environment-variables
Besides that, if someone knows the process to contribute new extension points to the IntelliJ Platform, I would be interested.

@fehnomenal
Copy link

fehnomenal commented Feb 22, 2022

Yeah, you did the same research as me. The main problem is that the JVMs have no way of modifying the environment (at least not with a public API).

Over a year ago I created a very similar suggestion to yours at https://youtrack.jetbrains.com/issue/IDEA-260725 but nothing happened, so I would not have high hopes... At least someone got assigned I guess 🤷

For reference: https://youtrack.jetbrains.com/issue/IDEA-288756

@l0b0
Copy link

l0b0 commented Jun 1, 2022

FYI Enter Nix Shell is not just abandoned, none of the links (source, issues, documentation) work anymore. Does anyone have a copy of the source code, so we could revive it?

@JojOatXGME
Copy link
Contributor

FYI Enter Nix Shell is not just abandoned, none of the links (source, issues, documentation) work anymore. Does anyone have a copy of the source code, so we could revive it?

Interesting. It was still available in February. Unfortunately, I don't have a local copy of the repository. You can still download the binaries from JetBrains Marketplace (https://plugins.jetbrains.com/plugin/16864-enter-nix-shell) and use a decompiler. According to the page on the Marketplace, the plugin was available under MIT license.

I guess I should invalidate the links in my messages since they seem to point to some NSFW-site now.

@infiniteregrets
Copy link

EnvFile and ProjectEnv use com.intellij.​run​Configuration​Extension among other extension points. However, this extension point is specifically for IntelliJ (i.e. Java). Other IDEs have their own extension points. So we would have to implement it for every IDE separately. It is also not sure if all these extension points combined cover every IDE and type of run configuration.

Hmm it seems like runConfigurationExtension is not available in IDEA 2022.2? Can't really find a way to extend new env in the new API

@JojOatXGME
Copy link
Contributor

Hmm it seems like runConfigurationExtension is not available in IDEA 2022.2? Can't really find a way to extend new env in the new API

Note that you have to add com.intellij.java as dependency because com.intellij.​run​Configuration​Extension is specific to IntelliJ and does not exist in other IDEs of JetBrains.

@qlonik
Copy link

qlonik commented Oct 7, 2022

FYI Enter Nix Shell is not just abandoned, none of the links (source, issues, documentation) work anymore. Does anyone have a copy of the source code, so we could revive it?

I've had a bit of an internet historian moment, and found the website of the developer of this plugin saved on web archive. It might have some interesting information and it also has some videos showing the plugin.

https://web.archive.org/web/20220131211828/https://fctorial.com/posts/nix_intellij.html

@l0b0
Copy link

l0b0 commented Jan 10, 2023

@jonringer - for the work-around win. :) Big thanks. I've been wracking my brain all afternoon trying to get it working. I'm now running NixOS on several machines and wanted to start using nix-shell instead of venv. Another vote for native support.

Thank you for this! It does seem to break when changing the packages in the Nix shell, though, in a weird way:

Screenshot from 2023-01-10 13-49-51

Notice how, in the above, aws_cdk is marked as unknown even though aws-cdk-lib (which provides aws_cdk) is shown in the list of SDK packages. This inconsistency persists across IDEA restarts and even a full "Repair IDE" cycle. It's like IDEA no longer knows about the associated SDK. Even creating a new SDK pointing to /run/current-system/sw/bin/python doesn't seem to work.

Reported

@starquake
Copy link

starquake commented Apr 7, 2023

I use Better Direnv and it seems to work:

@l0b0
Copy link

l0b0 commented Apr 11, 2023

@starquake One problem with that approach is that IDEA gets very confused about which packages are installed. This also happens when using nix-direnv + Better Direnv. We need a solution which works with more than one project.

@GenericNerdyUsername
Copy link

Would it be possible to do something like

nix-shell --run ./ide_launcher.sh & disown
exit

to put the whole ide in the new environment?

Problems I see with this:

  • args to the launch script aren't preserved which could break things (eg remote dev)
    • possible fix; hook com.intellij.idea.Main or read args the normal java way (a quick skim through the launch script shows that no args go to the script itself)
  • If you want to switch between environments quickly, you'll have to wait for the IDE to restart every time
  • There might not be a way to easily make the whole IDE exit

@nhooey
Copy link

nhooey commented Aug 20, 2023

Here's a summary of the most significant things mentioned:

@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/jetbrains-ides-support-for-nix-shell-w-nix-idea-plugin-per-project-environment/32321/1

@infiniteregrets
Copy link

hey we patch the env of all the intellij IDEs, maybe this approach could help?

https://github.com/metalbear-co/mirrord-intellij

@l0b0
Copy link

l0b0 commented Dec 19, 2023

Here's a workaround, after trying many things. tl;dr: Symlink the python executable somewhere static, then point the SDK to that path.

Tested only with IDEA and poetry2nix repos.

Caveats:

  • IDEA seems to require that you include pip in the Python derivation, or else it doesn't find the installed packages.
  • If you're using direnv you have to symlink from both .envrc and shell.nix.
  • IDEA does not seem to pick up on environment changes immediately, and requires a restart for new packages to show up.

Example repos: 1, 2.

@nhooey
Copy link

nhooey commented May 21, 2024

@infiniteregrets: hey we patch the env of all the intellij IDEs, maybe this approach could help?

https://github.com/metalbear-co/mirrord-intellij

@infiniteregrets: Could you elaborate on your approach, or even highlight where in the mirrord JetBrains extension code the environment patching is implemented?

JojOatXGME added a commit that referenced this issue Jul 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests