Don't run PYTHONSTARTUP file if a file or code is passed #5226

bouk · 2014-02-26T15:36:06Z

This modifies Python so the behavior is how Python behaves, this is the current situation:

The same happens when you pass code to the Python binary vs. the IPython binary.

takluyver · 2014-02-26T18:37:13Z

Actually, I think we should differ from plain Python in this situation. IPython is designed for interactive use, and a lot of the time when you run a file, you're using something like the -i flag to go into the interactive shell afterwards. Even when you're not, I think it's more relevant for us to provide a similar environment for running scripts as for interactive use. We treat PYTHONSTARTUP the same way as IPython startup files for this reason.

airhorns · 2014-02-27T00:59:20Z

Differing is dangerous because alias python=ipython works less and less. I use a project which makes use of PYTHONSTARTUP to launch its own interactive shell, but only in the event that a file hasn't been passed for execution on the command line. I'd like to be able to just use the same ipython binary to launch either the shell or a file from the command line and not have to make a wrapper script which makes assumptions about what command line arguments mean.

Also, this difference from normal Python is just surprising for me, I assumed it would work the same way the normal Python binary's worked based on the fact that ipython some_file.py works the same way, and the usage takes care to explain that.

That said, IPython definitely is meant for interactive use... hence the i.

minrk · 2014-03-06T18:21:57Z

We are now inclined to actually merge this. One comment in-line.

minrk · 2014-03-06T18:22:07Z

IPython/core/shellapp.py

@@ -358,7 +358,7 @@ def _run_startup_files(self):
        """Run files from profile startup directory"""
        startup_dir = self.profile_dir.startup_dir
        startup_files = []
-        if self.exec_PYTHONSTARTUP:
+        if self.exec_PYTHONSTARTUP and not self.file_to_run and not self.code_to_run:


add self.module_to_run

This is in line with how Python behaves

bouk · 2014-03-06T18:37:04Z

I addressed the comment

takluyver · 2014-03-06T19:15:28Z

IPython/core/shellapp.py

@@ -358,7 +358,7 @@ def _run_startup_files(self):
        """Run files from profile startup directory"""
        startup_dir = self.profile_dir.startup_dir
        startup_files = []
-        if self.exec_PYTHONSTARTUP:
+        if self.exec_PYTHONSTARTUP and not self.file_to_run and not self.code_to_run and not self.module_to_run:


Nitpick: I think this would be slightly clearer as not (self.file_to_run or self.code_to_run or self.module_to_run), which is logically equivalent.

Agreed, I was wanting to do something like that

Bouke

On Thu, Mar 6, 2014 at 2:15 PM, Thomas Kluyver notifications@github.comwrote:

In IPython/core/shellapp.py:

@@ -358,7 +358,7 @@ def _run_startup_files(self):
"""Run files from profile startup directory"""
startup_dir = self.profile_dir.startup_dir
startup_files = []

if self.exec_PYTHONSTARTUP:

if self.exec_PYTHONSTARTUP and not self.file_to_run and not self.code_to_run and not self.module_to_run:

Nitpick: I think this would be slightly clearer as not (self.file_to_run
or self.code_to_run or self.module_to_run), which is logically equivalent.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/5226/files#r10355918
.

takluyver · 2014-03-06T19:49:13Z

Great, thanks. Merging.

Don't run PYTHONSTARTUP file if a file or code is passed

@dianacarroll

This is based on @dianacarroll's previous pull request #227, and @JoshRosen's comments on #38. Since we do want to allow passing arguments to IPython, this does the following: * It documents that IPython can't be used with standalone jobs for now. (Later versions of IPython will deal with PYTHONSTARTUP properly and enable this, see ipython/ipython#5226, but no released version has that fix.) * If you run `pyspark` with `IPYTHON=1`, it passes your command-line arguments to it. This way you can do stuff like `IPYTHON=1 bin/pyspark notebook`. * The old `IPYTHON_OPTS` remains, but I've removed it from the documentation. This is in case people read an old tutorial that uses it. This is not a perfect solution and I'd also be okay with keeping things as they are today (ignoring `$@` for IPython and using IPYTHON_OPTS), and only doing the doc change. With this change though, when IPython fixes ipython/ipython#5226, people will immediately be able to do `IPYTHON=1 bin/pyspark myscript.py` to run a standalone script and get all the benefits of running scripts in IPython (presumably better debugging and such). Without it, there will be no way to run scripts in IPython. @JoshRosen you should probably take the final call on this. Author: Diana Carroll <dcarroll@cloudera.com> Closes #294 from mateiz/spark-1134 and squashes the following commits: 747bb13 [Diana Carroll] SPARK-1134 bug with ipython prevents non-interactive use with spark; only call ipython if no command line arguments were supplied (cherry picked from commit a599e43) Signed-off-by: Matei Zaharia <matei@databricks.com>

@dianacarroll

This is based on @dianacarroll's previous pull request #227, and @JoshRosen's comments on #38. Since we do want to allow passing arguments to IPython, this does the following: * It documents that IPython can't be used with standalone jobs for now. (Later versions of IPython will deal with PYTHONSTARTUP properly and enable this, see ipython/ipython#5226, but no released version has that fix.) * If you run `pyspark` with `IPYTHON=1`, it passes your command-line arguments to it. This way you can do stuff like `IPYTHON=1 bin/pyspark notebook`. * The old `IPYTHON_OPTS` remains, but I've removed it from the documentation. This is in case people read an old tutorial that uses it. This is not a perfect solution and I'd also be okay with keeping things as they are today (ignoring `$@` for IPython and using IPYTHON_OPTS), and only doing the doc change. With this change though, when IPython fixes ipython/ipython#5226, people will immediately be able to do `IPYTHON=1 bin/pyspark myscript.py` to run a standalone script and get all the benefits of running scripts in IPython (presumably better debugging and such). Without it, there will be no way to run scripts in IPython. @JoshRosen you should probably take the final call on this. Author: Diana Carroll <dcarroll@cloudera.com> Closes #294 from mateiz/spark-1134 and squashes the following commits: 747bb13 [Diana Carroll] SPARK-1134 bug with ipython prevents non-interactive use with spark; only call ipython if no command line arguments were supplied

@dianacarroll

This is based on @dianacarroll's previous pull request apache#227, and @JoshRosen's comments on apache#38. Since we do want to allow passing arguments to IPython, this does the following: * It documents that IPython can't be used with standalone jobs for now. (Later versions of IPython will deal with PYTHONSTARTUP properly and enable this, see ipython/ipython#5226, but no released version has that fix.) * If you run `pyspark` with `IPYTHON=1`, it passes your command-line arguments to it. This way you can do stuff like `IPYTHON=1 bin/pyspark notebook`. * The old `IPYTHON_OPTS` remains, but I've removed it from the documentation. This is in case people read an old tutorial that uses it. This is not a perfect solution and I'd also be okay with keeping things as they are today (ignoring `$@` for IPython and using IPYTHON_OPTS), and only doing the doc change. With this change though, when IPython fixes ipython/ipython#5226, people will immediately be able to do `IPYTHON=1 bin/pyspark myscript.py` to run a standalone script and get all the benefits of running scripts in IPython (presumably better debugging and such). Without it, there will be no way to run scripts in IPython. @JoshRosen you should probably take the final call on this. Author: Diana Carroll <dcarroll@cloudera.com> Closes apache#294 from mateiz/spark-1134 and squashes the following commits: 747bb13 [Diana Carroll] SPARK-1134 bug with ipython prevents non-interactive use with spark; only call ipython if no command line arguments were supplied

Don't run PYTHONSTARTUP file if a file or code is passed

JoshRosen mentioned this pull request Feb 28, 2014

SPARK-1134 pyspark only uses ipython if started interactively apache/spark#38

Closed

minrk reviewed Mar 6, 2014
View reviewed changes

Don't run PYTHONSTARTUP file if a file, code or module is passed

e105ff8

This is in line with how Python behaves

takluyver reviewed Mar 6, 2014
View reviewed changes

Simplify logic

60eeb04

takluyver added a commit that referenced this pull request Mar 6, 2014

Merge pull request #5226 from bouk/fix-pythonstartup

ce824d6

Don't run PYTHONSTARTUP file if a file or code is passed

takluyver merged commit ce824d6 into ipython:master Mar 6, 2014

bouk deleted the fix-pythonstartup branch March 6, 2014 19:54

ivanov mentioned this pull request Mar 18, 2014

PYTHONPATH doesn't work for scripts #101

Open

mateiz mentioned this pull request Apr 2, 2014

[SPARK-1134] Fix and document passing of arguments to IPython apache/spark#294

Closed

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014

Merge pull request ipython#5226 from bouk/fix-pythonstartup

3ebcfbf

Don't run PYTHONSTARTUP file if a file or code is passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Don't run PYTHONSTARTUP file if a file or code is passed #5226

Don't run PYTHONSTARTUP file if a file or code is passed #5226

bouk commented Feb 26, 2014

takluyver commented Feb 26, 2014

airhorns commented Feb 27, 2014

minrk commented Mar 6, 2014

minrk Mar 6, 2014

bouk commented Mar 6, 2014

takluyver Mar 6, 2014

bouk Mar 6, 2014

bouk Mar 6, 2014

takluyver commented Mar 6, 2014

Don't run PYTHONSTARTUP file if a file or code is passed #5226

Don't run PYTHONSTARTUP file if a file or code is passed #5226

Conversation

bouk commented Feb 26, 2014

takluyver commented Feb 26, 2014

airhorns commented Feb 27, 2014

minrk commented Mar 6, 2014

minrk Mar 6, 2014

Choose a reason for hiding this comment

bouk commented Mar 6, 2014

takluyver Mar 6, 2014

Choose a reason for hiding this comment

bouk Mar 6, 2014

Choose a reason for hiding this comment

bouk Mar 6, 2014

Choose a reason for hiding this comment

takluyver commented Mar 6, 2014