Add :skip to setup in ExUnit

[yrashk]

[josevalim]

I can see two ways to not allow a test to run. I am still undecided which ones will make to ExUnit, so I would like to see some use cases before we implement any solution.

Filtering

The first one I have in mind is command line, test wide filtering. For example, I want to add support to -t and -l options:

elixir test/some/test.rb -l 80

The example above will run only the test on line 80. Similarly:

mix test -t /sample/

Run all tests matching the regex /sample/. We may eventually extend it to allow custom filtering based on user metadata. For example, we could do:

# All these tests access `amazon_web_services`,
# so let's tag them as so.
setup_all do
  { :ok, [amazon_web_services: true] }
end

Since those tests may be slow or flaky, you may want to filter them from your run and run them exclusively in other occasions:

mix test --only amazon_web_services:true
mix test --skip amazon_web_services:true

Skip on callbacks

Another option is the approach proposed here. A callback may return :skip, meaning the next test is not going to run. This is a bit tricky in my opinion because usually callbacks are not tied to specific tests, so this mechanism may only be useful when you want to skip the whole test suite.

Also, I am not sure if it should be a return value OR a function named skip!(). The advantage of having a function (that would throw something instead) is that we could call it from anywhere, including from inside a test.

The filtering and skip on callbacks approaches seems to be ortogonal. There is nothing stopping us from having both. However, it would be nice to have some cases to avoid unnecessarily bloating ExUnit. :)