Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 333 lines (254 sloc) 11.924 kb
cb48c22 @fabioz Fixing line endings.
authored
1 """
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
2 Based on the python xreload.
3
4 Changes:
5
6 1. we don't recreate the old namespace from new classes. Rather, we keep the existing namespace, load a new version of
7 it and update only some of the things we can inplace. That way, we don't break things such as singletons or end up with
8 a second representation of the same class in memory.
9
10 2. If we find it to be a __metaclass__, we try to update it as a regular class.
11
12 3. We don't remove old attributes (and leave them lying around even if they're no longer used).
13
14 These changes make it more stable, especially in the common case (where in a debug session only the contents of a
15 function are changed).
16
17
18
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i.e....
authored
19
20 Original: http://svn.python.org/projects/sandbox/trunk/xreload/xreload.py
21 Note: it seems https://github.com/plone/plone.reload/blob/master/plone/reload/xreload.py enhances it (to check later)
22
23 Interesting alternative: https://code.google.com/p/reimport/
24
cb48c22 @fabioz Fixing line endings.
authored
25 Alternative to reload().
26
27 This works by executing the module in a scratch namespace, and then
28 patching classes, methods and functions in place. This avoids the
29 need to patch instances. New objects are copied into the target
30 namespace.
31
32 Some of the many limitations include:
33
34 - Global mutable objects other than classes are simply replaced, not patched
35
36 - Code using metaclasses is not handled correctly
37
38 - Code creating global singletons is not handled correctly
39
40 - Functions and methods using decorators (other than classmethod and
41 staticmethod) is not handled correctly
42
43 - Renamings are not handled correctly
44
45 - Dependent modules are not reloaded
46
47 - When a dependent module contains 'from foo import bar', and
48 reloading foo deletes foo.bar, the dependent module continues to use
49 the old foo.bar object rather than failing
50
51 - Frozen modules and modules loaded from zip files aren't handled
52 correctly
53
54 - Classes involving __slots__ are not handled correctly
55 """
56
57 import imp
58 import sys
59 import types
cbefd1a @fabioz Live coding on debugger.
authored
60 from pydev_imports import Exec
54138d5 @fabioz Added a way to skip functions with #@DontTrace. For now must be enabled ...
authored
61 import pydevd_dont_trace
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i.e....
authored
62 import traceback
63
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
64 NO_DEBUG = 0
65 LEVEL1 = 1
66 LEVEL2 = 2
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i.e....
authored
67
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
68 #===================================================================================================================
69 # Helper
70 #=======================================================================================================================
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i.e....
authored
71 class Helper:
72
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
73 DEBUG = NO_DEBUG
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
74
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
75 def write(*args):
76 new_lst = []
77 for a in args:
78 new_lst.append(str(a))
79
80 msg = ' '.join(new_lst)
81 sys.stdout.write('%s\n' % (msg,))
39543ca @fabioz Minors
authored
82 write = staticmethod(write)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
83
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i.e....
authored
84 def info(*args):
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
85 if Helper.DEBUG >= LEVEL1:
86 Helper.write(*args)
39543ca @fabioz Minors
authored
87 info = staticmethod(info)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
88
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
89 def info2(*args):
90 if Helper.DEBUG >= LEVEL2:
91 Helper.write(*args)
39543ca @fabioz Minors
authored
92 info2 = staticmethod(info2)
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i.e....
authored
93
94
95
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
96 #=======================================================================================================================
97 # code_objects_equal
98 #=======================================================================================================================
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i.e....
authored
99 def code_objects_equal(code0, code1):
100 for d in dir(code0):
101 if d.startswith('_') or 'lineno' in d:
102 continue
103 if getattr(code0, d) != getattr(code1, d):
104 return False
105 return True
cb48c22 @fabioz Fixing line endings.
authored
106
107
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
108 #=======================================================================================================================
109 # xreload
110 #=======================================================================================================================
cb48c22 @fabioz Fixing line endings.
authored
111 def xreload(mod):
112 """Reload a module in place, updating classes, methods and functions.
113
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
114 mod: a module object
cb48c22 @fabioz Fixing line endings.
authored
115 """
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
116 r = Reload(mod)
117 r.apply()
118 r = None
54138d5 @fabioz Added a way to skip functions with #@DontTrace. For now must be enabled ...
authored
119 pydevd_dont_trace.clear_trace_filter_cache()
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
120
121
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
122 #=======================================================================================================================
123 # Reload
124 #=======================================================================================================================
125 class Reload:
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
126
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
127 def __init__(self, mod):
128 self.mod = mod
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
129
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
130 def apply(self):
131 mod = self.mod
132 self._on_finish_callbacks = []
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i.e....
authored
133 try:
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
134 # Get the module name, e.g. 'foo.bar.whatever'
135 modname = mod.__name__
136 # Get the module namespace (dict) early; this is part of the type check
137 modns = mod.__dict__
138 # Parse it into package name and module name, e.g. 'foo.bar' and 'whatever'
139 i = modname.rfind(".")
140 if i >= 0:
141 pkgname, modname = modname[:i], modname[i + 1:]
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i.e....
authored
142 else:
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
143 pkgname = None
144 # Compute the search path
145 if pkgname:
146 # We're not reloading the package, only the module in it
147 pkg = sys.modules[pkgname]
148 path = pkg.__path__ # Search inside the package
149 else:
150 # Search the top-level module path
151 pkg = None
152 path = None # Make find_module() uses the default search path
153 # Find the module; may raise ImportError
154 (stream, filename, (suffix, mode, kind)) = imp.find_module(modname, path)
155 # Turn it into a code object
156 try:
157 # Is it Python source code or byte code read from a file?
158 if kind not in (imp.PY_COMPILED, imp.PY_SOURCE):
159 # Fall back to built-in reload()
160 Helper.info('Not patching in place (could not find source)')
161 return reload(mod)
162 if kind == imp.PY_SOURCE:
163 source = stream.read()
164 code = compile(source, filename, "exec")
165 else:
166 import marshal
167 code = marshal.load(stream)
168 finally:
169 if stream:
170 stream.close()
171 # Execute the code. We copy the module dict to a temporary; then
172 # clear the module dict; then execute the new code in the module
173 # dict; then swap things back and around. This trick (due to
174 # Glyph Lefkowitz) ensures that the (readonly) __globals__
175 # attribute of methods and functions is set to the correct dict
176 # object.
177 new_namespace = modns.copy()
178 new_namespace.clear()
179 new_namespace["__name__"] = modns["__name__"]
180 Exec(code, new_namespace)
181 # Now we get to the hard part
182 oldnames = set(modns)
183 newnames = set(new_namespace)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
184
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
185 # Update in-place what we can
186 for name in oldnames & newnames:
187 self._update(modns[name], new_namespace[name])
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
188
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
189 # Create new tokens (note: not deleting existing)
190 for name in newnames - oldnames:
191 Helper.info('Created:', new_namespace[name])
192 modns[name] = new_namespace[name]
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
193
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
194 for c in self._on_finish_callbacks:
195 c()
196 del self._on_finish_callbacks[:]
197 except:
198 traceback.print_exc()
199
200
201 def _update(self, oldobj, newobj):
202 """Update oldobj, if possible in place, with newobj.
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
203
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
204 If oldobj is immutable, this simply returns newobj.
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
205
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
206 Args:
207 oldobj: the object to be updated
208 newobj: the object used as the source for the update
209 """
210 Helper.info2('Updating: ', oldobj)
211 if oldobj is newobj:
212 # Probably something imported
213 return newobj
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
214
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
215 if type(oldobj) is not type(newobj):
216 # Cop-out: if the type changed, give up
217 return newobj
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
218
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
219 if hasattr(newobj, "__reload_update__"):
220 # Provide a hook for updating
221 return newobj.__reload_update__(oldobj)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
222
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
223 if hasattr(types, 'ClassType'):
224 classtype = types.ClassType
225 else:
226 classtype = type
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
227
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
228 if isinstance(newobj, classtype):
229 return self._update_class(oldobj, newobj)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
230
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
231 if isinstance(newobj, types.FunctionType):
232 return self._update_function(oldobj, newobj)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
233
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
234 if isinstance(newobj, types.MethodType):
235 return self._update_method(oldobj, newobj)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
236
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
237 if isinstance(newobj, classmethod):
238 return self._update_classmethod(oldobj, newobj)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
239
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
240 if isinstance(newobj, staticmethod):
241 return self._update_staticmethod(oldobj, newobj)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
242
243 # New: dealing with metaclasses.
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
244 if hasattr(newobj, '__metaclass__') and hasattr(newobj, '__class__') and newobj.__metaclass__ == newobj.__class__:
245 return self._update_class(oldobj, newobj)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
246
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
247 # Not something we recognize, just give up
cb48c22 @fabioz Fixing line endings.
authored
248 return newobj
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
249
250
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
251 # All of the following functions have the same signature as _update()
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
252
253
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
254 def _update_function(self, oldfunc, newfunc):
255 """Update a function object."""
256 oldfunc.__doc__ = newfunc.__doc__
257 oldfunc.__dict__.update(newfunc.__dict__)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
258
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
259 try:
260 newfunc.__code__
261 attr_name = '__code__'
262 except AttributeError:
263 newfunc.func_code
264 attr_name = 'func_code'
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
265
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
266 old_code = getattr(oldfunc, attr_name)
267 new_code = getattr(newfunc, attr_name)
268 if not code_objects_equal(old_code, new_code):
269 Helper.info('Update function:', oldfunc)
270 setattr(oldfunc, attr_name, new_code)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
271
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
272 try:
273 oldfunc.__defaults__ = newfunc.__defaults__
274 except AttributeError:
275 oldfunc.func_defaults = newfunc.func_defaults
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
276
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
277 return oldfunc
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
278
279
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
280 def _update_method(self, oldmeth, newmeth):
281 """Update a method object."""
282 # XXX What if im_func is not a function?
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
283 if hasattr(oldmeth, 'im_func') and hasattr(newmeth, 'im_func'):
284 self._update(oldmeth.im_func, newmeth.im_func)
285 elif hasattr(oldmeth, '__func__') and hasattr(newmeth, '__func__'):
286 self._update(oldmeth.__func__, newmeth.__func__)
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
287 return oldmeth
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
288
289
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
290 def _update_class(self, oldclass, newclass):
291 """Update a class object."""
292 olddict = oldclass.__dict__
293 newdict = newclass.__dict__
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
294
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
295 oldnames = set(olddict)
296 newnames = set(newdict)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
297
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
298 for name in newnames - oldnames:
299 Helper.info('Created:', newdict[name], 'in', oldclass)
300 setattr(oldclass, name, newdict[name])
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
301
302 # Note: not removing old things...
303 # for name in oldnames - newnames:
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
304 # Helper.info('Removed:', name, 'from', oldclass)
305 # delattr(oldclass, name)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
306
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
307 for name in oldnames & newnames - set(['__dict__', '__doc__']):
308 self._update(olddict[name], newdict[name])
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
309
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
310 if hasattr(oldclass, "__after_reload_update__"):
311 # If a client wants to know about it, give him a chance.
312 self._on_finish_callbacks.append(oldclass.__after_reload_update__)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
313
314
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
315 def _update_classmethod(self, oldcm, newcm):
316 """Update a classmethod update."""
317 # While we can't modify the classmethod object itself (it has no
318 # mutable attributes), we *can* extract the underlying function
319 # (by calling __get__(), which returns a method object) and update
320 # it in-place. We don't have the class available to pass to
321 # __get__() but any object except None will do.
322 self._update(oldcm.__get__(0), newcm.__get__(0))
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
323
324
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update what ...
authored
325 def _update_staticmethod(self, oldsm, newsm):
326 """Update a staticmethod update."""
327 # While we can't modify the staticmethod object itself (it has no
328 # mutable attributes), we *can* extract the underlying function
329 # (by calling __get__(), which returns it) and update it in-place.
330 # We don't have the class available to pass to __get__() but any
331 # object except None will do.
332 self._update(oldsm.__get__(0), newsm.__get__(0))
Something went wrong with that request. Please try again.