Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 336 lines (257 sloc) 12.227 kb
cb48c22 @fabioz Fixing line endings.
authored
1 """
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
2 Based on the python xreload.
3
4 Changes:
5
6 1. we don't recreate the old namespace from new classes. Rather, we keep the existing namespace, load a new version of
7 it and update only some of the things we can inplace. That way, we don't break things such as singletons or end up with
8 a second representation of the same class in memory.
9
10 2. If we find it to be a __metaclass__, we try to update it as a regular class.
11
12 3. We don't remove old attributes (and leave them lying around even if they're no longer used).
13
14 These changes make it more stable, especially in the common case (where in a debug session only the contents of a
15 function are changed).
16
17
18
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i…
authored
19
20 Original: http://svn.python.org/projects/sandbox/trunk/xreload/xreload.py
21 Note: it seems https://github.com/plone/plone.reload/blob/master/plone/reload/xreload.py enhances it (to check later)
22
23 Interesting alternative: https://code.google.com/p/reimport/
24
cb48c22 @fabioz Fixing line endings.
authored
25 Alternative to reload().
26
27 This works by executing the module in a scratch namespace, and then
28 patching classes, methods and functions in place. This avoids the
29 need to patch instances. New objects are copied into the target
30 namespace.
31
32 Some of the many limitations include:
33
34 - Global mutable objects other than classes are simply replaced, not patched
35
36 - Code using metaclasses is not handled correctly
37
38 - Code creating global singletons is not handled correctly
39
40 - Functions and methods using decorators (other than classmethod and
41 staticmethod) is not handled correctly
42
43 - Renamings are not handled correctly
44
45 - Dependent modules are not reloaded
46
47 - When a dependent module contains 'from foo import bar', and
48 reloading foo deletes foo.bar, the dependent module continues to use
49 the old foo.bar object rather than failing
50
51 - Frozen modules and modules loaded from zip files aren't handled
52 correctly
53
54 - Classes involving __slots__ are not handled correctly
55 """
56
57 import imp
4e1679f @fabioz Improving pydevd_reload.
authored
58 import traceback
cb48c22 @fabioz Fixing line endings.
authored
59 import sys
60 import types
cbefd1a @fabioz Live coding on debugger.
authored
61 from pydev_imports import Exec
54138d5 @fabioz Added a way to skip functions with #@DontTrace. For now must be enabl…
authored
62 import pydevd_dont_trace
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i…
authored
63
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
64 NO_DEBUG = 0
65 LEVEL1 = 1
66 LEVEL2 = 2
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i…
authored
67
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
68 #===================================================================================================================
69 # Helper
70 #=======================================================================================================================
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i…
authored
71 class Helper:
72
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
73 DEBUG = NO_DEBUG
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
74
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
75 def write(*args):
76 new_lst = []
77 for a in args:
78 new_lst.append(str(a))
79
80 msg = ' '.join(new_lst)
81 sys.stdout.write('%s\n' % (msg,))
39543ca @fabioz Minors
authored
82 write = staticmethod(write)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
83
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i…
authored
84 def info(*args):
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
85 if Helper.DEBUG >= LEVEL1:
86 Helper.write(*args)
39543ca @fabioz Minors
authored
87 info = staticmethod(info)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
88
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
89 def info2(*args):
90 if Helper.DEBUG >= LEVEL2:
91 Helper.write(*args)
39543ca @fabioz Minors
authored
92 info2 = staticmethod(info2)
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i…
authored
93
94
95
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
96 #=======================================================================================================================
97 # code_objects_equal
98 #=======================================================================================================================
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i…
authored
99 def code_objects_equal(code0, code1):
100 for d in dir(code0):
101 if d.startswith('_') or 'lineno' in d:
102 continue
103 if getattr(code0, d) != getattr(code1, d):
104 return False
105 return True
cb48c22 @fabioz Fixing line endings.
authored
106
107
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
108 #=======================================================================================================================
109 # xreload
110 #=======================================================================================================================
cb48c22 @fabioz Fixing line endings.
authored
111 def xreload(mod):
112 """Reload a module in place, updating classes, methods and functions.
113
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
114 mod: a module object
cb48c22 @fabioz Fixing line endings.
authored
115 """
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
116 r = Reload(mod)
117 r.apply()
118 r = None
54138d5 @fabioz Added a way to skip functions with #@DontTrace. For now must be enabl…
authored
119 pydevd_dont_trace.clear_trace_filter_cache()
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
120
121
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
122 #=======================================================================================================================
123 # Reload
124 #=======================================================================================================================
125 class Reload:
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
126
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
127 def __init__(self, mod):
128 self.mod = mod
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
129
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
130 def apply(self):
131 mod = self.mod
132 self._on_finish_callbacks = []
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i…
authored
133 try:
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
134 # Get the module name, e.g. 'foo.bar.whatever'
135 modname = mod.__name__
136 # Get the module namespace (dict) early; this is part of the type check
137 modns = mod.__dict__
138 # Parse it into package name and module name, e.g. 'foo.bar' and 'whatever'
139 i = modname.rfind(".")
140 if i >= 0:
141 pkgname, modname = modname[:i], modname[i + 1:]
7eb4070 @fabioz Improving reload code a bit to deal with Kivy Widget based classes (i…
authored
142 else:
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
143 pkgname = None
144 # Compute the search path
145 if pkgname:
146 # We're not reloading the package, only the module in it
147 pkg = sys.modules[pkgname]
148 path = pkg.__path__ # Search inside the package
149 else:
150 # Search the top-level module path
151 pkg = None
152 path = None # Make find_module() uses the default search path
153 # Find the module; may raise ImportError
154 (stream, filename, (suffix, mode, kind)) = imp.find_module(modname, path)
155 # Turn it into a code object
156 try:
157 # Is it Python source code or byte code read from a file?
158 if kind not in (imp.PY_COMPILED, imp.PY_SOURCE):
159 # Fall back to built-in reload()
160 Helper.info('Not patching in place (could not find source)')
161 return reload(mod)
162 if kind == imp.PY_SOURCE:
163 source = stream.read()
164 code = compile(source, filename, "exec")
165 else:
166 import marshal
167 code = marshal.load(stream)
168 finally:
169 if stream:
170 stream.close()
171 # Execute the code. We copy the module dict to a temporary; then
172 # clear the module dict; then execute the new code in the module
173 # dict; then swap things back and around. This trick (due to
174 # Glyph Lefkowitz) ensures that the (readonly) __globals__
175 # attribute of methods and functions is set to the correct dict
176 # object.
177 new_namespace = modns.copy()
178 new_namespace.clear()
179 new_namespace["__name__"] = modns["__name__"]
180 Exec(code, new_namespace)
181 # Now we get to the hard part
182 oldnames = set(modns)
183 newnames = set(new_namespace)
4e1679f @fabioz Improving pydevd_reload.
authored
184
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
185 # Update in-place what we can
186 for name in oldnames & newnames:
187 self._update(modns[name], new_namespace[name])
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
188
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
189 # Create new tokens (note: not deleting existing)
190 for name in newnames - oldnames:
191 Helper.info('Created:', new_namespace[name])
192 modns[name] = new_namespace[name]
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
193
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
194 for c in self._on_finish_callbacks:
195 c()
196 del self._on_finish_callbacks[:]
197 except:
198 traceback.print_exc()
199
200
201 def _update(self, oldobj, newobj):
202 """Update oldobj, if possible in place, with newobj.
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
203
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
204 If oldobj is immutable, this simply returns newobj.
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
205
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
206 Args:
207 oldobj: the object to be updated
208 newobj: the object used as the source for the update
209 """
4e1679f @fabioz Improving pydevd_reload.
authored
210 try:
211 Helper.info2('Updating: ', oldobj)
212 if oldobj is newobj:
213 # Probably something imported
214 return newobj
215
216 if type(oldobj) is not type(newobj):
217 # Cop-out: if the type changed, give up
218 return newobj
219
220 if hasattr(newobj, "__reload_update__"):
221 # Provide a hook for updating
222 return newobj.__reload_update__(oldobj)
223
224 if isinstance(newobj, types.FunctionType):
225 return self._update_function(oldobj, newobj)
226
227 if isinstance(newobj, types.MethodType):
228 return self._update_method(oldobj, newobj)
229
230 if isinstance(newobj, classmethod):
231 return self._update_classmethod(oldobj, newobj)
232
233 if isinstance(newobj, staticmethod):
234 return self._update_staticmethod(oldobj, newobj)
235
236 if hasattr(types, 'ClassType'):
237 classtype = (types.ClassType, type) #object is not instance of types.ClassType.
238 else:
239 classtype = type
240
241 if isinstance(newobj, classtype):
242 return self._update_class(oldobj, newobj)
243
244 # New: dealing with metaclasses.
245 if hasattr(newobj, '__metaclass__') and hasattr(newobj, '__class__') and newobj.__metaclass__ == newobj.__class__:
246 return self._update_class(oldobj, newobj)
247
248 # Not something we recognize, just give up
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
249 return newobj
4e1679f @fabioz Improving pydevd_reload.
authored
250 except:
251 traceback.print_exc()
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
252
253
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
254 # All of the following functions have the same signature as _update()
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
255
256
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
257 def _update_function(self, oldfunc, newfunc):
258 """Update a function object."""
259 oldfunc.__doc__ = newfunc.__doc__
260 oldfunc.__dict__.update(newfunc.__dict__)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
261
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
262 try:
263 newfunc.__code__
264 attr_name = '__code__'
265 except AttributeError:
266 newfunc.func_code
267 attr_name = 'func_code'
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
268
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
269 old_code = getattr(oldfunc, attr_name)
270 new_code = getattr(newfunc, attr_name)
271 if not code_objects_equal(old_code, new_code):
272 Helper.info('Update function:', oldfunc)
273 setattr(oldfunc, attr_name, new_code)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
274
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
275 try:
276 oldfunc.__defaults__ = newfunc.__defaults__
277 except AttributeError:
278 oldfunc.func_defaults = newfunc.func_defaults
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
279
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
280 return oldfunc
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
281
282
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
283 def _update_method(self, oldmeth, newmeth):
284 """Update a method object."""
285 # XXX What if im_func is not a function?
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
286 if hasattr(oldmeth, 'im_func') and hasattr(newmeth, 'im_func'):
287 self._update(oldmeth.im_func, newmeth.im_func)
288 elif hasattr(oldmeth, '__func__') and hasattr(newmeth, '__func__'):
289 self._update(oldmeth.__func__, newmeth.__func__)
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
290 return oldmeth
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
291
292
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
293 def _update_class(self, oldclass, newclass):
294 """Update a class object."""
295 olddict = oldclass.__dict__
296 newdict = newclass.__dict__
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
297
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
298 oldnames = set(olddict)
299 newnames = set(newdict)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
300
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
301 for name in newnames - oldnames:
302 Helper.info('Created:', newdict[name], 'in', oldclass)
303 setattr(oldclass, name, newdict[name])
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
304
305 # Note: not removing old things...
306 # for name in oldnames - newnames:
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
307 # Helper.info('Removed:', name, 'from', oldclass)
308 # delattr(oldclass, name)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
309
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
310 for name in oldnames & newnames - set(['__dict__', '__doc__']):
311 self._update(olddict[name], newdict[name])
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
312
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
313 if hasattr(oldclass, "__after_reload_update__"):
314 # If a client wants to know about it, give him a chance.
315 self._on_finish_callbacks.append(oldclass.__after_reload_update__)
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
316
317
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
318 def _update_classmethod(self, oldcm, newcm):
319 """Update a classmethod update."""
320 # While we can't modify the classmethod object itself (it has no
321 # mutable attributes), we *can* extract the underlying function
322 # (by calling __get__(), which returns a method object) and update
323 # it in-place. We don't have the class available to pass to
324 # __get__() but any object except None will do.
325 self._update(oldcm.__get__(0), newcm.__get__(0))
c03e6ac @fabioz Fixed reload code to be Python 3 compatible.
authored
326
327
7b2801b @fabioz Making the reload handle less to be more stable (i.e.: only update wh…
authored
328 def _update_staticmethod(self, oldsm, newsm):
329 """Update a staticmethod update."""
330 # While we can't modify the staticmethod object itself (it has no
331 # mutable attributes), we *can* extract the underlying function
332 # (by calling __get__(), which returns it) and update it in-place.
333 # We don't have the class available to pass to __get__() but any
334 # object except None will do.
335 self._update(oldsm.__get__(0), newsm.__get__(0))
Something went wrong with that request. Please try again.