Split the current "testing" chapter

[japaric]

The current chapter needs to be split:

• intro: The #[test] attribute and rustc --test
• #[should_fail]
• conditional compilation: #[test] is kind of sugar for #[cfg(test)]
• FUT (function under test) have their stdout silenced by default, explain how to override this
• Disable tests using the ignore attribute
• Execute a subset of tests using regexes

---

@alxgnon is working on this

[alxgnon]

I would gladly take care of this.

Additionally, there is some misinformation to fix. #[test] is not equivalent to #[cfg(test)]. #[test] marks a function as a test to be run by the harness - whereas #[cfg(test)] is used to conditionally compile an item. In other words, a function that solely has the attribute #[cfg(test)] will never be run as a test.

[japaric]

I would gladly take care of this.

\o/ Thanks!

Additionally, there is some misinformation to fix. #[test] is not equivalent to #[cfg(test)]. #[test] marks a function as a test to be run by the harness - whereas #[cfg(test)] is used to conditionally compile an item. In other words, a function that solely has the attribute #[cfg(test)] will never be run as a test.

Yeah, that could be better worded.

What I mean is that #[test] === #[cfg(test)] + test-harness, for example see this snippet. Also passing --cfg test to the compiler is equivalent to passing --test, both ways build the crate in test-mode.

[alxgnon]

There could be sections about:

• The ignore attribute
• Running a subset of tests with a regex match

Do you believe this would be useful or will it just bloat the section?

[japaric]

The ignore attribute

I suppose this is to temporarily disable some unit tests? As an alternative to commenting out the test, the ignore attribute would be cleaner.

Running a subset of tests with a regex match

I didn't know this was possible! Is this baked in the test executable as a flag? Can I use this with benchmarks too?

---

Sound like useful topics to me! I'll add them to the TODO list.

[alxgnon]

The ignore attribute also permits ignoring tests by configuration, so platform-specific tests are possible, for example.

Regex matching works for benchmarks and tests. You just pass the regular expression as a free argument to the test runner, and it will filter the tests by match.

I will write those sections up once #159 is ok.

[steveklabnik]

I don't think @alxgnon is still working on this after a few years 😄 If someone else would like to pick up the torch, I'd be happy to mentor!

[serejkus]

I'd like to take on that.

I think that it would be nice to have a chapter on testing (say, chapter 20 and moving meta and the rest after it to +1).

• The intro is a table of chapter's contents + links to doc book about testing and to nursery about testing
• The first subchapter is about unit tests: #[cfg(test)] mod tests { use super::*; /* some unit tests here */ }. Things to note: the purpose of unit tests, assert*! macros, #[ignore] and #[should_panic], ability to test private functions.
• The second subchapter is about doctests. Things no note: "hiding" some code lines, ? + try_main instead of unwrap() in tests
• Third subchapter is about integration tests. Things to note: not forgetting to extern crate. I'm not sure that I understand how interactive examples should work in this case.
• Fourth subchapter is about dev-dependencies. I'm not sure whether it is a proper place, but it is definiteley tests-related

Looking for suggestions about improving this plan

[steveklabnik]

This sounds great!

For the try_main stuff, linking to https://rust-lang-nursery.github.io/api-guidelines/documentation.html#examples-use--not-try-not-unwrap-c-question-mark is probably good :)

[marioidival]

I'm closed this issue because #953 was merged