Support extraction by filename as well as directory

[ENuge]

Reason for the change: #253 . This obviates the need for parallelizing to make things faster (on the consumer's side, we only scan files that are different in a given branch compared to origin's master, which brings scan time down from ~4min 30s to ~5-20s, depending on how old one's branch is).

Code changes: moved half of the logic from extract_from_dir into its own function. Then added a few "is this a path or directory?" type checks to frontend.py to use the right thing. Those checks are kind of ugly right now but I couldn't think of a more elegant way of doing it. Added/rejiggered a couple of tests.

[sils]

Hi, can you fix the issues pointed out by the gitmate bot and refactor your stuff into atomic commits as described in http://coala.readthedocs.org/en/latest/Getting_Involved/Writing_Good_Commits/ ?

[sils]

Thanks for submitting :)

[codecov-io]

Current coverage is 88.38%

Merging #324 into master will increase coverage by +0.04% as of 59793db

@@            master    #324   diff @@
======================================
  Files           23      23       
  Stmts         3612    3626    +14
  Branches         0       0       
  Methods          0       0       
======================================
+ Hit           3191    3205    +14
  Partial          0       0       
  Missed         421     421       

Review entire Coverage Diff as of 59793db

Powered by Codecov. Updated on successful CI builds.

[ENuge]

Appeased the LineLengthBot with Yelp's preferred syntax (which is vertically long, but eh this line is super-awkward regardless).

I'll refactor my commits later today or tomorrow. (I started this change on Yelp's old fork of babel, so this version was more of a copy/paste job. But I'll make the history make some sense.)

[sils]

@ENuge cool, ping us when it's done.

[akx]

Could you use the same indent style as in the extract_from_dir call above? :)

[ENuge]

Yup! Done.

[akx]

Add a comment like # already normalized

[akx]

Maybe don't extract the tuple here?

for message_tuple in extract...:
   yield (filename,) + message_tuple

[akx]

Would be better to invert this if (avoids over-indentation):

if not pathmatch(pattern, filename):
  continue
options = ....

[akx]

This should not be done in this function; it's fine to just do it in the caller.

EDIT: To expand on what I mean, this function is too low-level for it to be reading facts from the execution environment (the current working directory in this case); it's better to have the callers explicitly pass this in.

[ENuge]

fwiw extract_from_dir(..) above it does the same check. Though I'm fine with having the caller explicitly pass it in.

[akx]

Can you expand a little on this -- namely that this is used to calculate the relative names of files?

[akx]

Is this even possible? :D

[ENuge]

Heh, this is meant to be checking if the path exists. I didn't really think about the much more direct way of doing that...namely, using os.path.exists(..). Will update.

[akx]

Don't expand the tuple here either.

for message_tuple in check...:
    yield message_tuple

is fine and dandy!

[ENuge]

Cool, I'll add a comment/docstring line with the return type or something to make up for the loss in explicitness.

EDIT: The docstring already does that! \o/

[akx]

And aside from the handful of comments I had, great work @ENuge ! :)

I'd appreciate a test that passes in a direct filename to the command line command though. :)

EDIT: Wagh, I'm a dummy who can't scroll down.

[akx]

@ENuge: Looking good!

Please squash all of the commits into one though :)

[sils]

Interesting, travis errored out on mac only for pypy 2.6, maybe a nondeterministic bug?

[akx]

@sils1297 Heh, yes... race condition against the clock, it seems. If the wallclock ticks forward a second while those tests are being run, they may fail.

We should probably freeze time for the duration of those tests or maybe compare the outputs ignoring datetime differences.

[akx]

Super nice work @ENuge!

I'm gonna merge this, but come to think of it, we should add an alias for input-paths called input-dirs for backward compatibility. I'll make an issue out of that. :)