Skip to content
This repository

Namespaces for embedding #1140

Merged
merged 7 commits into from over 2 years ago

3 participants

Thomas Kluyver Gabriel de Perthuis Min RK
Thomas Kluyver
Collaborator

This should fix the embedding problem in #1136.

One additional complexity: if you specify only one of user_ns or user_module when instantiating the embedded shell, they are hooked up so that you have a single namespace (i.e. ip.user_ns is ip.user_global_ns). So, the logic at activation time to auto-detect the module and local namespace will only kick in if neither were specified at instantiation.

You should still be able to pass local_ns or module at activation to override them separately.

Gabriel de Perthuis

Thank you. Both my simple example and the scrapy shell are working well.

Min RK minrk commented on the diff December 11, 2011
IPython/frontend/terminal/embed.py
@@ -170,9 +171,17 @@ def mainloop(self, local_ns=None, module=None, stack_depth=0,
170 171
         IPython itself (via %run), but some funny things will happen (a few
171 172
         globals get overwritten). In the future this will be cleaned up, as
172 173
         there is no fundamental reason why it can't work perfectly."""
  174
+        
  175
+        if (global_ns is not None) and (module is None):
  176
+            class DummyMod(object):
  177
+                """A dummy module object for embedded IPython."""
  178
+                pass
  179
+            warnings.warn("global_ns is deprecated, use module instead.", DeprecationWarning)
  180
+            module = DummyMod()
  181
+            module.__dict__ = global_ns
1
Min RK Owner
minrk added a note December 11, 2011

Thanks, I think this is the main thing we need to preserve.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Min RK
Owner

Terrific, thanks for taking care of it. Do we / can we have simple tests for some of the basic cases?

Thomas Kluyver
Collaborator

I'm not quite sure how to automate tests for embedding, but I can tidy up a couple of test scripts and add them in docs/examples to run manually if that's sufficient.

Thomas Kluyver
Collaborator

Added some manual test scripts.

Min RK
Owner

Nice. Manual tests look good, and I checked embedded shells in currently released sympy and django, both of which appear to work happily (at least they don't raise) with your changes. If you think this is good to go, go ahead and merge.

Thomas Kluyver takluyver merged commit feb3b24 into from December 12, 2011
Thomas Kluyver takluyver closed this December 12, 2011
Thomas Kluyver
Collaborator

Thanks, merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
3  IPython/core/interactiveshell.py
@@ -881,6 +881,7 @@ def debugger(self,force=False):
881 881
     #-------------------------------------------------------------------------
882 882
     # Things related to IPython's various namespaces
883 883
     #-------------------------------------------------------------------------
  884
+    default_user_namespaces = True
884 885
 
885 886
     def init_create_namespaces(self, user_module=None, user_ns=None):
886 887
         # Create the namespace where the user will operate.  user_ns is
@@ -919,6 +920,8 @@ def init_create_namespaces(self, user_module=None, user_ns=None):
919 920
         # These routines return a properly built module and dict as needed by
920 921
         # the rest of the code, and can also be used by extension writers to
921 922
         # generate properly initialized namespaces.
  923
+        if (user_ns is not None) or (user_module is not None):
  924
+            self.default_user_namespaces = False
922 925
         self.user_module, self.user_ns = self.prepare_user_module(user_module, user_ns)
923 926
 
924 927
         # A record of hidden variables we have added to the user namespace, so
64  IPython/frontend/terminal/embed.py
@@ -30,6 +30,7 @@
30 30
     from contextlib import nested
31 31
 except:
32 32
     from IPython.utils.nested_context import nested
  33
+import warnings
33 34
 
34 35
 from IPython.core import ultratb
35 36
 from IPython.frontend.terminal.interactiveshell import TerminalInteractiveShell
@@ -74,7 +75,11 @@ class InteractiveShellEmbed(TerminalInteractiveShell):
74 75
     def __init__(self, config=None, ipython_dir=None, user_ns=None,
75 76
                  user_module=None, custom_exceptions=((),None),
76 77
                  usage=None, banner1=None, banner2=None,
77  
-                 display_banner=None, exit_msg=u''):
  78
+                 display_banner=None, exit_msg=u'', user_global_ns=None):
  79
+    
  80
+        if user_global_ns is not None:
  81
+            warnings.warn("user_global_ns has been replaced by user_module. The\
  82
+                           parameter will be ignored.", DeprecationWarning)
78 83
 
79 84
         super(InteractiveShellEmbed,self).__init__(
80 85
             config=config, ipython_dir=ipython_dir, user_ns=user_ns,
@@ -96,24 +101,21 @@ def init_sys_modules(self):
96 101
         pass
97 102
 
98 103
     def __call__(self, header='', local_ns=None, module=None, dummy=None,
99  
-                 stack_depth=1):
  104
+                 stack_depth=1, global_ns=None):
100 105
         """Activate the interactive interpreter.
101 106
 
102  
-        __call__(self,header='',local_ns=None,global_ns,dummy=None) -> Start
  107
+        __call__(self,header='',local_ns=None,module=None,dummy=None) -> Start
103 108
         the interpreter shell with the given local and global namespaces, and
104 109
         optionally print a header string at startup.
105 110
 
106 111
         The shell can be globally activated/deactivated using the
107  
-        set/get_dummy_mode methods. This allows you to turn off a shell used
  112
+        dummy_mode attribute. This allows you to turn off a shell used
108 113
         for debugging globally.
109 114
 
110 115
         However, *each* time you call the shell you can override the current
111 116
         state of dummy_mode with the optional keyword parameter 'dummy'. For
112  
-        example, if you set dummy mode on with IPShell.set_dummy_mode(1), you
113  
-        can still have a specific call work by making it as IPShell(dummy=0).
114  
-
115  
-        The optional keyword parameter dummy controls whether the call
116  
-        actually does anything.
  117
+        example, if you set dummy mode on with IPShell.dummy_mode = True, you
  118
+        can still have a specific call work by making it as IPShell(dummy=False).
117 119
         """
118 120
 
119 121
         # If the user has turned it off, go away
@@ -140,7 +142,7 @@ def __call__(self, header='', local_ns=None, module=None, dummy=None,
140 142
 
141 143
         # Call the embedding code with a stack depth of 1 so it can skip over
142 144
         # our call and get the original caller's namespaces.
143  
-        self.mainloop(local_ns, module, stack_depth=stack_depth)
  145
+        self.mainloop(local_ns, module, stack_depth=stack_depth, global_ns=global_ns)
144 146
 
145 147
         self.banner2 = self.old_banner2
146 148
 
@@ -148,20 +150,20 @@ def __call__(self, header='', local_ns=None, module=None, dummy=None,
148 150
             print self.exit_msg
149 151
 
150 152
     def mainloop(self, local_ns=None, module=None, stack_depth=0,
151  
-                 display_banner=None):
  153
+                 display_banner=None, global_ns=None):
152 154
         """Embeds IPython into a running python program.
153 155
 
154 156
         Input:
155 157
 
156 158
           - header: An optional header message can be specified.
157 159
 
158  
-          - local_ns, global_ns: working namespaces. If given as None, the
159  
-          IPython-initialized one is updated with __main__.__dict__, so that
160  
-          program variables become visible but user-specific configuration
161  
-          remains possible.
  160
+          - local_ns, module: working local namespace (a dict) and module (a
  161
+          module or similar object). If given as None, they are automatically
  162
+          taken from the scope where the shell was called, so that
  163
+          program variables become visible.
162 164
 
163 165
           - stack_depth: specifies how many levels in the stack to go to
164  
-          looking for namespaces (when local_ns and global_ns are None).  This
  166
+          looking for namespaces (when local_ns or module is None).  This
165 167
           allows an intermediate caller to make sure that this function gets
166 168
           the namespace from the intended level in the stack.  By default (0)
167 169
           it will get its locals and globals from the immediate caller.
@@ -170,9 +172,17 @@ def mainloop(self, local_ns=None, module=None, stack_depth=0,
170 172
         IPython itself (via %run), but some funny things will happen (a few
171 173
         globals get overwritten). In the future this will be cleaned up, as
172 174
         there is no fundamental reason why it can't work perfectly."""
  175
+        
  176
+        if (global_ns is not None) and (module is None):
  177
+            class DummyMod(object):
  178
+                """A dummy module object for embedded IPython."""
  179
+                pass
  180
+            warnings.warn("global_ns is deprecated, use module instead.", DeprecationWarning)
  181
+            module = DummyMod()
  182
+            module.__dict__ = global_ns
173 183
 
174 184
         # Get locals and globals from caller
175  
-        if local_ns is None or module is None:
  185
+        if (local_ns is None or module is None) and self.default_user_namespaces:
176 186
             call_frame = sys._getframe(stack_depth).f_back
177 187
 
178 188
             if local_ns is None:
@@ -189,21 +199,26 @@ def mainloop(self, local_ns=None, module=None, stack_depth=0,
189 199
         # Update namespaces and fire up interpreter
190 200
         
191 201
         # The global one is easy, we can just throw it in
192  
-        self.user_module = module
  202
+        if module is not None:
  203
+            self.user_module = module
193 204
 
194 205
         # But the user/local one is tricky: ipython needs it to store internal
195 206
         # data, but we also need the locals. We'll throw our hidden variables
196 207
         # like _ih and get_ipython() into the local namespace, but delete them
197 208
         # later.
198  
-        self.user_ns = local_ns
199  
-        self.init_user_ns()
  209
+        if local_ns is not None:
  210
+            self.user_ns = local_ns
  211
+            self.init_user_ns()
200 212
 
201 213
         # Patch for global embedding to make sure that things don't overwrite
202 214
         # user globals accidentally. Thanks to Richard <rxe@renre-europe.com>
203 215
         # FIXME. Test this a bit more carefully (the if.. is new)
204 216
         # N.B. This can't now ever be called. Not sure what it was for.
205  
-        if local_ns is None and module is None:
206  
-            self.user_global_ns.update(__main__.__dict__)
  217
+        # And now, since it wasn't called in the previous version, I'm
  218
+        # commenting out these lines so they can't be called with my new changes
  219
+        # --TK, 2011-12-10
  220
+        #if local_ns is None and module is None:
  221
+        #    self.user_global_ns.update(__main__.__dict__)
207 222
 
208 223
         # make sure the tab-completer has the correct frame information, so it
209 224
         # actually completes using the frame's locals/globals
@@ -213,8 +228,9 @@ def mainloop(self, local_ns=None, module=None, stack_depth=0,
213 228
             self.interact(display_banner=display_banner)
214 229
         
215 230
         # now, purge out the local namespace of IPython's hidden variables.
216  
-        for name in self.user_ns_hidden:
217  
-            local_ns.pop(name, None)
  231
+        if local_ns is not None:
  232
+            for name in self.user_ns_hidden:
  233
+                local_ns.pop(name, None)
218 234
         
219 235
         # Restore original namespace so shell can shut down when we exit.
220 236
         self.user_module = orig_user_module
10  docs/examples/test_embed/embed1.py
... ...
@@ -0,0 +1,10 @@
  1
+"""This tests standard embedding, automatically detecting the module and
  2
+local namespaces."""
  3
+
  4
+f = set([1,2,3,4,5])
  5
+
  6
+def bar(foo):
  7
+    import IPython
  8
+    IPython.embed(banner1='check f in globals, foo in locals')
  9
+
  10
+bar(f)
5  docs/examples/test_embed/embed2.py
... ...
@@ -0,0 +1,5 @@
  1
+"""This tests passing a dict for the user_ns at shell instantiation."""
  2
+from IPython import embed
  3
+
  4
+user_ns = dict(cookie='monster')
  5
+embed(user_ns=user_ns, banner1="check 'cookie' present, locals and globals equivalent")
7  docs/examples/test_embed/embed3.py
... ...
@@ -0,0 +1,7 @@
  1
+"""This tests passing local_ns and global_ns (for backwards compatibility only)
  2
+at activation of an embedded shell."""
  3
+from IPython.frontend.terminal.embed import InteractiveShellEmbed
  4
+
  5
+user_ns = dict(cookie='monster')
  6
+ISE = InteractiveShellEmbed(banner1='check cookie in locals, and globals empty')
  7
+ISE(local_ns=user_ns, global_ns={})
10  docs/source/whatsnew/development.txt
@@ -145,10 +145,12 @@ Backwards incompatible changes
145 145
   The full path will still work, and is necessary for using custom launchers not in
146 146
   IPython's launcher module.
147 147
 
148  
-* For embedding a shell, note that the parameter ``user_global_ns`` has been
149  
-  replaced by ``user_module``, and expects a module-like object, rather than
150  
-  a namespace dict. The ``user_ns`` parameter works the same way as before, and
  148
+* For embedding a shell, note that the parameters ``user_global_ns`` and ``global_ns``
  149
+  have been deprectated in favour of ``user_module`` and ``module`` respsectively.
  150
+  The new parameters expect a module-like object, rather than a namespace dict.
  151
+  The old parameters remain for backwards compatibility, although ``user_global_ns``
  152
+  is now ignored. The ``user_ns`` parameter works the same way as before, and
151 153
   calling :func:`~IPython.frontend.terminal.embed.embed` with no arguments still
152  
-  works the same way.
  154
+  works as before.
153 155
 
154 156
 .. * use bullet list
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.