-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Virtualenv on macOS not working (fish) #1307
Comments
Hi @k-lyda ! I will try to reproduce this later today. Thank you for the feedback! |
Hi @k-lyda , was not able to reproduce it on my MacOS machine. Would you mind to share We'll try anaconda a little bit later today. |
Hi @shcheklein, indeed it looks like an issue is with adding anaconda inside shell profile script. I will do some more testing to check exactly what is causing the issue |
Thanks, @k-lyda. I'll reopen it because I think it's important design question - the way we execute commands with DVC. It seems that ideally behavior should be the same in both cases, does not matter if you run it via DVC or directly from the command line. May be we should use some shell options to prevent it from re-reading profiles/config. What do you think, @efiop ? |
@shcheklein You are right about it being an env vars overwriting issue by bashrc/zshrc/etc when launching a new instance for command execution inside dvc. I will look into it shortly. |
@shcheklein, @efiop, as you mentioned, in my case the issue occurs only when I have in my fish config (I use fish shell), line Best Regards, |
@k-lyda Oh, no wonder I can't reproduce, I should've asked right away about the shell you were using, but I didn't think it could be anything special 🙂 . Unlike other shells like bash/zsh, which don't parse configs when you invoke them like
To be more exact, we would need to modify run() method https://github.com/iterative/dvc/blob/master/dvc/stage.py#L419. Would you like to contribute a patch? 🙂 Thanks, |
Ok, tried to create a patch for this, but ran into problems when sourcing saved env in fish, since it is really angry about overwriting some read-only vars and stuff like that, overcoming which makes the solution even more hackish, at which point I think it is no longer worth it to mess with, since more elegant workarounds are available(such as removing lines from fishrc that overwrite critical env vars; defining SHELL to something like /bin/bash;). Closing this for now. Please feel free to reopen. |
Fixes iterative#1307 Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>
Added a warning just in case: #1313 . |
This might help others that use fish and are confused by the warning message. The recommended way of setting up conda is not to change the My Ideally, based on this, I'd prefer to remove the warning message since it not needed if you set up conda as recommended. |
@svenski the way fish handles env vars is still quite error-prone, even without using conda. I'm afraid that if we remove that warning, users will not know where to start looking for a solution and we'll simply get a lot of bug reports about this same thing. If that warning message is not too annoying for you, I would prefer to leave it as is for now. |
@shcheklein - I don't have enough understanding or context of However, from a (fish-shell) user point of view it would be nice, as I get a warning for each stage that I run now. But I also understand if you don't want to complicate things too much. Ideally, it would be fixed for fish, even when changing the I'm using |
@svenski @k-lyda Guys, we were thinking about maybe making dvc fallback to another shell like |
@efiop , I don't think it would impact my existing pipelines. I don't use fish specific scripts, and if I did, I'd make sure there was a However, out of curiosity, can I ask why you want to do this? Are there any other issues with fish? The original issue in raised in this ticket can be avoided by setting up conda correctly (recommended) in fish (https://github.com/conda/conda/blob/master/CHANGELOG.md#recommended-change-to-enable-conda-in-your-shell). So IMHO I don't think this is issue with dvc, it's with setting up conda in fish. TBH, I would opt for removing the warning unless of course there are other issues with using fish shell that are not mentioned in here. |
Nice to know that. I guess that is that way for most of fish users.
Just something that came to my mind, so decided to ask for an opinion from fish users :) Glad to hear that properly setting up conda works for you!
It is not only about conda though, in the linked PR we've been running into other env-related issues and Thank you for the feedback! |
Hi, I have a similar Issue. The problem was that
Now fish alway starts by activating this specific project environment, but i get to keep everything else. |
Hi @kaiogu ! Indeed, that is a fish limitation. We've even put a special warning for |
I don't think we should close this issue, as a warning message is not a solution. |
I found the following workaround: |
@hanan-vian Do any of the workarounds described above work for you? 🙂
Not at all, you just need to configure your setup to make it work.
That's because your
Sounds too hackish. The proper solution is probably to make fish support an option (e.g. --norc) that will make it usable, but fish folks seem to be against it fish-shell/fish-shell#5394 . Please feel free to comment on that issue, maybe the pressure from actual users will make them reconsider it. As I've mentioned previously, we could consider launching |
See my conda run hack. |
I use fish + conda (miniforge). I tried various approaches listed here, and I found only one approach to solve it : To have a smoother process I leverage fish abbrs:
|
Fish gained a |
@clamydo Thanks for the heads up! 🙏 We would appreciate a PR if you or anyone else is interested in contributing. |
@efiop Today, I started to hack something together. Just adding the option seems easy, however, checking if a fish version above 3.3 is installed and only then apply the option is a tad more fishy. I currently run But I haven't had time to figure out, how to cleanly adapt the I think it is necessary to check for the version to remain backwards-compatible, because fish 3.3.x and above isn't necessarily wide-spread yet. |
I haven't pushed anything publicly yet because I ran out of time figuring out how to properly handle all cases of different shells and versions. It's still on my backlog though. Note (to myself): Fish maintains a |
I have started an early attempt to fix this isse. See #8101 Perhaps somebody can chime in? |
Prevent fish to reload config files in non-interactive mode. Disable warning when using a recent fish version. Fixes iterative#1307
Please provide information about your setup
It looks like the commands run by dvc are ignoring currently used virtualenv
![screenshot 2018-11-08 at 13 48 32](https://user-images.githubusercontent.com/6491400/48199798-82c1c200-e35d-11e8-8f5c-a0522805f596.png)
Steps to reproduce the issue:
The output of two last commands is different, which means, that python... comannd run by DVC is not taking into acount activated virtualenv. I've tested with virtualenv as well as with python venv.
I've done the same steps on linux - it works fine, so it's seems that the issue apears only on macOS
The text was updated successfully, but these errors were encountered: