I get the following crash when executing a %macro:
|1> print 10
|2> %macro mm 1
Macro `mm` created. To execute, type its name (without quotes).
IntegrityError Traceback (most recent call last)
/Users/rkern/<ipython-input-3-a39e8f58efe3> in <module>()
----> 1 mm()
/Users/rkern/git/ipython/IPython/core/macro.pyc in __call__(self, *args)
34 self._ip.user_ns['_margv'] = args
---> 35 self._ip.run_cell(self.value)
37 def __getstate__(self):
/Users/rkern/git/ipython/IPython/core/interactiveshell.pyc in run_cell(self, cell)
2101 # Store raw and processed history
-> 2102 self.history_manager.store_inputs(self.execution_count, ipy_cell, cell)
2104 self.logger.log(ipy_cell, cell)
/Users/rkern/git/ipython/IPython/core/history.pyc in store_inputs(self, line_num, source, source_raw)
274 # Trigger to flush cache and write to DB.
275 if len(self.db_input_cache) >= self.db_cache_size:
--> 276 self.writeout_cache()
278 # update the auto _i variables
/Users/rkern/git/ipython/IPython/core/history.pyc in writeout_cache(self)
304 with self.db:
305 self.db.executemany("INSERT INTO history VALUES (?, ?, ?, ?)",
--> 306 self.db_input_cache)
307 self.db.executemany("INSERT INTO output_history VALUES (?, ?, ?)",
IntegrityError: columns session, line are not unique
When I try to view the history using %hist immediately after this, I get a full IPython crash with an IntegrityError in the same place.
It seems that, when you run a macro, both the command calling the macro and the macro itself are stored in history. The database expects a unique line number for each entry, but these are passed with the same line number (shell.execution_count). I think we should only store the macro call, not the content of the macro. I'm not familiar with the various shell.run_* methods, but would it make sense to switch %macro to something like run_source, rather than run_cell?
Nearer to release, it probably makes sense for store_inputs to catch IntegrityError so that it can fail more gracefully. I've left it uncaught for now so that we can pick up on errors like this one.
It might be best for the prefilter mechanism to expand Macro objects explicitly instead of using the IPyAutocall mechanism. Then the history will contain both the raw macro execution and the expanded macro implementation automatically.
That's not a bad idea. I'll look into it tomorrow.
It's working a bit. The trouble is that IPython determines that a cell contains a single block before it does the prefiltering. So the macro command gets expanded, but then compiled in single line mode, and only the first line of the macro is executed. Is there an easy way round this, or would it involve substantial changes to IPython's execution machinery?
It looks like a lot of the logic for prefiltering and transformation has been split up (or simply duplicated) between the PrefilterManager and the InputSplitter. I wouldn't be surprised if there's a bunch of dead code in the prefilter module.
I'll think on it.
The original bug is fixed, and the contents of a macro are now displayed in the translated version of the history. There may well still be duplicate code in prefilter.py and inputsplitter.py - I didn't try to audit them.