IndentationErrors caused by mixed tab/space indents not caught

[nedbat]

Originally reported by Danek Duvall (Bitbucket: dhduvall, GitHub: dhduvall)

---

When you have two lines of identical indentation, except that one uses spaces and the other uses tabs (eight spaces to a tabstop), then the tokenize module raises an IndentationError which coverage doesn't catch.

This can be considered a bug in the indentation, but it's only a style bug, as the code works just fine.

This could be considered a bug in tokenize, but I assume it's unlikely to get fixed.

It's a bit of a pain to fix in coverage, I think, because you're seeing it in the looping expression of a for loop. The loop would have to be restructured to pull the tokens from the generator inside the loop, inside a try/except, and either error out usefully (with filename and line number!) or try to go on. Still, it might not be too bad.

The attached file demonstrates the problem.

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/31
• This issue had attachments: testindent.py. See the original issue for details.

[nedbat]

Actually, I explicitly expandtabs(4) just before tokenizing, so the bug is mine. But I wonder how best to know what size tabs you use? I guess defaulting to 8 is a better choice....

[nedbat]

Original comment by Danek Duvall (Bitbucket: dhduvall, GitHub: dhduvall)

---

We use 8, which seems like the sanest default to me. Most code I see which uses 4-space indent either goes all spaces or mixes spaces and tabs. If there are folks relying on the expandtabs(4) behavior, I suppose you could always add a flag to specify tab width; I dunno.

[nedbat]

I'm not sure why I used expandtabs(4), Python is documented as using 8-space tabs: http://docs.python.org/reference/lexical_analysis.html#indentation

Changed to 8-space tabs, and fixed in <<changeset 7a5e06254462 (bb)>>.