Fixes for %paste with special transformations

[fperez]

NOTE: ready for review/merge

I'm just creating the PR for now with one failing test, need to add a second failing test for the ? case, and then actually fix things...

Fix the fact that we use prefilter too aggressively in %paste, which results in applying single-line transformations inside of strings and function definitions.

Closes #1258.

[fperez]

I need to add a test for this as well:

def func():
   if True: #am i true?
       a = 3

if copied and pasted into the IPython interpreter via paste looks like this

[39]: paste
def func():
   if True: #am i true?
       a = 3
## -- End pasted text --
Object `if` not found.

[fperez]

Test results for commit 0c2eef0 merged into master
Platform: linux2

• python2.7: OK
• python3.2: OK (libraries not available: cython matplotlib oct2py pygments pymongo rpy2 wx wx.aui)

Not available for testing: python2.6, python3.1

[minrk]

I'm not well familiar with the prefilter part of the code, so I would ping @takluyver for a more authoritative review, but I tried to read carefully, and it seems solid (and thoroughly commented!).

One functional change this seems to make is that this now dedents pasted text, which the previous implementation did not. Did you add a test for pasting an indented block?

[fperez]

Thanks, @minrk; I'll wait for @takluyver's careful eyes.

BTW, the old code did dedent as well, so nothing has changed on that front (well, the dedent is now unconditional and before it was omitted if a storage name was given; minor change in the name of simplifying an otherwise very spaghetti logic). So that means we do have already good tests in place for indented input (most of them, in fact, start indented).

[minrk]

Test results for commit 0c2eef0 (can't merge cleanly)
Platform: darwin

• python2.6: OK (libraries not available: cython matplotlib oct2py pygments pymongo qt rpy2 tornado wx wx.aui)
• python2.7: OK (libraries not available: oct2py wx wx.aui)
• python3.2: OK (libraries not available: cython matplotlib oct2py pymongo qt rpy2 wx wx.aui)

Not available for testing: python3.1

[minrk]

okay, great. I saw dedent code added, and missed its removal, and there was a comment that said "without dedenting" that you removed, so I thought it was changed behavior.

[fperez]

Yes, that was just indicating that the dedenting used to be optional, I made it global to simplify the unnecessarily branchy logic it led to.

[takluyver]

Just a heads up: I'm going to a conference on Thursday, and there's stuff to prepare for that. I'll try to have a look at this one tomorrow, but if I don't get round to it, I'll be busy for a couple of weeks. Sorry about this, I know I'm disappearing just when we want all hands on deck for the release.

[fperez]

No worries. If it comes to it we'll merge it, as it's well tested and does fix a known bug (with specific tests for it). But I'll leave it open til tomorrow in case you do have a chance to give it a look.

Have fun at your conference!

[takluyver]

Some of this code is using _ip, while most of it seems to be using ip. They're equivalent, but it would be cleaner to use one consistently.

[fperez]

Fixed in 85e9d88, thanks.

[takluyver]

I haven't had time to delve into it much, but from what I can see it looks OK. I'd like proper tests for the new utils functions, but that shouldn't hold this up now.

[fperez]

@takluyver, there's only two new utilities and there are tests for both that I think cover reasonably what they do. What else would make them 'proper'?

[fperez]

Since this fixes a real bug and it looks like we're otherwise good, I'm merging it now. The only remaining point is the question of 'proper' tests, but I know @takluyver has a trip coming up, so that may have to wait. If we clarify that point later on as really needing extra tests (I'm not sure I see the need), we can always add those post-release.

[takluyver]

Oh, I somehow missed the unit tests, and I thought they only had doctests. I see now that they had proper tests all along.

[fperez]

OK, we're good here then.