Skip to content

Feature request: event for running all tests #50

Closed
joonty opened this Issue Apr 8, 2013 · 19 comments

2 participants

@joonty
joonty commented Apr 8, 2013

Hi,

Thanks again for tork. It's now firmly a part of my daily development tools - I don't leave the house without it.

Now for the request. I've been hooking into tork's events with tork-remote tork-engine, which is working nicely. I've been using it to pass failed test logs to vim and fire system notification bubbles.

The only slight issue comes when running all tests. If there are lots of failures, I want to be able to delay the handling of the results until the end of the run. However, at the moment I can't differentiate between a single test run and all tests being run - they look the same from the input stream.

How about having start/end marker events for running all tests? Something like all_tests_start and all_tests_end?

Incidentally, I'm working on a project that would take advantage of this: torkify. It's a ruby wrapper around the event notification, allowing users to write callbacks to tork events in ruby.

Thanks for all your work so far, tork is looking great!

@sunaku
Owner
sunaku commented Apr 9, 2013

Sure, we could add such markers. The ending marker is a little tricky though: do we emit it as soon as all tests have been queued for running or after all queued tests have finished running? And what happens when another run_all_test_files command is received while an existing one is being processed? 🐎

@joonty
joonty commented Apr 10, 2013

Yeh, that crossed my mind too.

I would say that the most useful place for an end marker would be after the final test in the queue is completed, as personally I would use it to create a summary report of all the tests in that queue.

And if a run_all_test_files command is received during the run, I think I would just include it in the summary: it wouldn't be the end of the world. It would be a fair bit of work to get around that, e.g. tagging each test/fail/pass event with a queue number that relates to a specific run_all_test_files command.

I might have a play with that last idea, and see where I get. But adding basic marker events would be a useful stopgap if you'd be happy with that feature.

Thanks

@sunaku sunaku pushed a commit that referenced this issue Apr 10, 2013
Suraj N. Kurapati GH-50: add ["ran_all_test_files"] event in driver 7300447
@sunaku sunaku pushed a commit that referenced this issue Apr 10, 2013
Suraj N. Kurapati GH-50: add ["idle"] event to engine daa75c8
@sunaku
Owner
sunaku commented Apr 10, 2013

Here you go @joonty. You may choose from 2 branches: ran_all_test_files and idle. 🍰

I think I prefer the idle event and may end up dropping the absorb event in its favor. :neckbeard:

@sunaku sunaku pushed a commit that referenced this issue Apr 10, 2013
Suraj N. Kurapati GH-50: add ["idle"] event to engine 5a008b3
@joonty
joonty commented Apr 12, 2013

Wow fantastic! Thanks very much.

I also like the idle event, as that's more generic and reusable. Have you thought about implementing a run_all_test_files event yet? I would be more than happy to do a pull request normally, but I'm about to go on holiday for two weeks...

Thanks for the quick turnaround on this, it's much appreciated.

@sunaku
Owner
@joonty
@sunaku
Owner
@joonty
@sunaku sunaku pushed a commit that referenced this issue Apr 13, 2013
Suraj N. Kurapati GH-50: add ["idle"] event to engine ef46db8
@joonty
joonty commented May 2, 2013

Hi, I'm back 😄

The idle event works well, so I think that's a winner.

Have you given a thought to what it would look like to echo tork commands to clients? Perhaps a "command" event, like:

["command", "run_all_test_files"]

In the spirit of open source, I'm happy to take a look into this and open a pull request, if that would be helpful.

Thanks again

@sunaku
Owner
sunaku commented May 3, 2013
@joonty
joonty commented May 3, 2013

Ah ok. Did you throw away your code or is it in some branch somewhere?

@sunaku sunaku pushed a commit that referenced this issue May 5, 2013
Suraj N. Kurapati GH-50: add ["idle"] event to engine 66fffd3
@sunaku sunaku pushed a commit that referenced this issue May 5, 2013
Suraj N. Kurapati GH-50: add ["idle"] event to engine f200b58
@sunaku
Owner
sunaku commented May 5, 2013

@joonty try commit 8ca66ca in the echo branch.

@sunaku sunaku pushed a commit that referenced this issue May 6, 2013
Suraj N. Kurapati GH-50: add ["idle"] event to engine 6b1e78a
@joonty
joonty commented May 16, 2013

@sunaku the echo branch is working fine for me, and I can use it as a solution for this issue subject. Over to you now, as to how you want to proceed in terms of releasing it.

Thanks

@sunaku
Owner
sunaku commented May 17, 2013

@joonty I'll release it this weekend.

@joonty
joonty commented May 21, 2013

@sunaku having played around with this a bit more I've noticed a couple of irregularities. I've pasted a section of output from tork-remote tork-engine on a gist, here.

This was the output of running all test files (as in, sending the run_all_test_files command). It starts by sending the run_test_files event, with the list of test files, but it repeatedly sends the same command at later stages, but with no arguments (take a look at lines 19 and 20).

Also, I would have expected the run_all_test_files to come through in an echo event, but it doesn't seem to.

Are these things expected or not?

Thanks

@sunaku
Owner
sunaku commented Jun 7, 2013

Sorry for the late response. I've no free time nowadays. 😓

No, that isn't expected. I don't see why run_test_files is being echoed multiple times (and in the middle of that batch of test runs) like that. Please try the updated echo branch at b76ab9b which includes the sender's identity in echo messages. Perhaps that may unravel the mystery. 🔮

@joonty
joonty commented Jun 10, 2013

Thanks for that. Don't worry about the no time thing - I have exactly the same problem!

Well, for the life of me I can't reproduce this issue with the duplicated run_test_files. Typical isn't it?

I'll keep a lookout, but it's all looking good now. Thanks for your help!

@sunaku
Owner
sunaku commented Nov 24, 2013

Closing this issue because it seems to have been resolved. Please re-open if needed. 💇

@sunaku sunaku closed this Nov 24, 2013
@joonty
joonty commented Nov 25, 2013

Thanks, still good so far! :shipit:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.