Add assert helpers

[smackesey]

While correcting an issue over at generator-ember, the need came up for a test to make sure that compass was not installed when the user declines to install compass at the prompt. There was no convenient helper method available to assert the absence of a regex match (needed here for _package.json and Gruntfile.js), so I added it under the name assertNoMatch. It also seemed as if the regex form of assertFile should be aliased to assertMatch for symmetry.

[SBoudrias]

I don't really like the naming we have here. Could you think of something more description? assertFileContent or something similar.

[fayimora]

👍 assertFileContent definitely reads better.

[SBoudrias]

Would you mind adding unit tests for these methods and squashing the commits togheter. Thanks you, it looks good like this.

[smackesey]

In further reading through the generator-ember tests I noticed that every test file defines the function filesDoNotExist, which just takes a list of files and verifies that they don't exist. This is another good candidate for inclusion in the generator helpers, so I've added the methods assertNoFile (checks that file does not exist) and assertNoFiles (checks that no files in the passed-in array exist).

I'm working on writing the unit tests now for all these methods. No tests for helpers other than createGenerator have been written-- it seems like this is because the generator repo does not have a sample generator embedded in it (that generates files, anyway-- I can see there is an example custom-generator and mocha-generator in the fixtures directory) for tests, so there's nothing to test the file assertion methods against. I'm pretty new to testing-- correct me if I'm wrong, but shouldn't there be a basic generator available for testing purposes? If so, I can write one, or should I extend custom-generator-simple to create some files?

[SBoudrias]

You do not need a Generator to test these methods. You simply need to test that the assertion passes when tested against file that exist and contains the given content/regex. And that these assertion will throw when content is not within the file and/or the file exist.

Just put your tests in this file, following the same style: https://github.com/yeoman/generator/blob/master/test/helpers.js

[smackesey]

Squashed my commits and added tests for all methods.

[SBoudrias]

semicolon

[SBoudrias]

I think assertNoFile should accept a single string or an array so we have only one method for both use cases (so I'd merge assertNoFiles functionnality here).

About the naming, I think assertFileContent should be assertFileContains and assertFileNotContains (I'm unsure about the correct way to write contains - with or without s?)

Also, on a side note, I think we should refactor assertion methods for 1.0.0 so we keep them under a namespace (helpers.assert.fileContains).

[SBoudrias]

The assert error messages should describe the expected outcome, not describe a success.

expected <file> to exists

(This is also good for the following methods)

[smackesey]

I think assertNoFile should accept a single string or an array so we have only one method for both use cases (so I'd merge assertNoFiles functionnality here).

The problem with this is that it is inconsistent with assertFile, which may take multiple arguments but uses the second argument as a regex. I see three options:

1. add functionality to assertFile allowing the first argument to be an array, in which case all files are checked for existence and the second argument (the regex) is just ignored. Merge assertNoFile and assertNoFiles.
2. remove the regex-checking functionality from assertFile and leave it to assertFileContent; then neither assertFile nor assertNoFile take a regex argument, so we can pass a variable number of args to both. All passed in args are files whose [un]existence is verified.
3. leave things in the current state, with assertFiles and assertNoFiles separate from their singular counterparts.

I personally like option 2 the best. In order to avoid breaking changes, we could detect whether a regex was passed to assertFile to continue supporting the old functionality, but deprecate this behavior and tell people to use assertFileContent.

This is a comment for all the following tests. Instead of creating an anonymus function, simply bind the arguments to make the assertion single lines.

This is good, I'm relatively new to JS and didn't realize I could do this. Some of the resulting single lines are a little long though, does this repo have a column limit for line length?

The assert error messages should describe the expected outcome, not describe a success.

In writing my assert error messages, I followed what was already in place (assertFile, assertFiles); the error messages there were not as you describe. Instead they seem to say why the test failed (e.g. file + ' did not match \'' + reg + '\'.' was the already-in-place failure message for assertFile. So that's what I aimed at for my methods. The example of mine you quoted is not describing a success, it's describing why the test failed (the test is assertNoFile and the message is file + ' exists.' Personally I think it's more informative to describe why the test failed than just to restate what was expected, which is implicit in the method name. However, I will change the messages (including the ones that were in place) if describing expected outcome is more standard.

[smackesey]

About the naming, I think assertFileContent should be assertFileContains and assertFileNotContains (I'm unsure about the correct way to write contains - with or without s?)

I prefer assertFileContent and assertNoFileContent, mainly because there's not a very graceful way to name the negative version of assertFileContains: assertFileNoContains, assertNoFileContains are both not very good. Using the 's' at the end of contains is correct though.

[coveralls]

Coverage increased (+0.04%) when pulling 4a96b6d on smackesey:add_assert_helpers into 9e96626 on yeoman:master.

[SBoudrias]

I personally like option 2 the best. In order to avoid breaking changes, we could detect whether a regex was passed to assertFile to continue supporting the old functionality, but deprecate this behavior and tell people to use assertFileContent.

I agree with you here. Lets do this.

And you're right about the error messages, I haven't realised I read the assert No File method!

[smackesey]

Should the single method handling multiple args be named assertFile or assertFiles?

[SBoudrias]

I'd go with assertFile

[smackesey]

OK I've updated things and merged the single/multiple functionality into assertFile and assertFileContent. assertFileContent and assertNoFileContent can now take either one filename and one regex or an array of filename/regex pairs (this replaces the functionality of assertFiles. I have left the old matching functionality of assertFile and assertFiles in place but marked them as deprecated in comments. Is there some more formal way I should mark them?

[coveralls]

Coverage increased (+0.04%) when pulling a509df9 on smackesey:add_assert_helpers into 9e96626 on yeoman:master.

[SBoudrias]

Could you rebase on top of the lastest master?

[smackesey]

rebased

[SBoudrias]

Thanks! The build failed after detecting some error to our style guide. Would you mind fixing these?

You can run the build locally with gulp. npm install -g gulp and simply run npm test at the root of the project.

[SBoudrias]

These blocks would be cleaner written like this:

it('reject an array of file/regex pairs when one file\'s content does matches its corresponding regex', function () {
  var arg = [
    ['testFile', /Roses are red/],
    ['testFile2', /Violets are orange/]
  ];
  assert.throws(helpers.assertNoFileContent.bind(helpers, arg));
});

[SBoudrias]

I make a last run on the functions content and leave some comment of how you can simplify some lines.

I think it'd be best if the deprecated assertions methods where to console.log deprecation warning with suggested alternative. Like assertFile deprecated, use X instead.

Also, this is not your doing, but documentation comments aren't valid. We now use JsDoc syntax, so if you have time, it'd be really nice to convert the comments to a valid format. - But I'll merge as is if you don't have the time; so feel no obligation.

[coveralls]

Coverage increased (+1.23%) when pulling c5d38fe on smackesey:add_assert_helpers into 8fcf87d on yeoman:master.

[coveralls]

Coverage remained the same when pulling 365a6e6 on smackesey:add_assert_helpers into 8fcf87d on yeoman:master.

[smackesey]

OK, I've fixed the style errors and added JsDoc comments.

In the course of adding comments to the methods I added, I decided to look at the rest of the helpers. I added JsDoc style comments and made a some internal changes to a few of the helpers:

• changed before to just call testDirectory rather than repeating the same code
• changed name of stubs array to decorated; confusingly, all decorated methods (both stubbed ones and normally decorated ones) were stored in stubs. Since both stubs and normally decorated methods are decorated, but normally decorated methods are not stubs, I changed the name of this array from stubs to decorated.

A few questions/comments:

• I feel like before is just not a good name for a helper. It implies that the only mocha before hook you'd ever want to use is helpers.before. I suggest changing the name to setupTestDirectory
• the comment for helpers.gruntfile says that the method is intended to be used as a mocha handler. I'm pretty new to mocha, can a mocha handler accept an options object (the signature is function gruntfile(options, done))?
• I did not add a JsDoc comment for assertFiles since its deprecated.

[smackesey]

Just let me know if it looks good and I'll squash/rebase

[coveralls]

Coverage remained the same when pulling ab81e31 on smackesey:add_assert_helpers into 8fcf87d on yeoman:master.

[SBoudrias]

This is a very solid PR. Awesome work dude! I'm very happy with the efforts you put in. Thanks a lot! 🌠

I feel like before is just not a good name for a helper. It implies that the only mocha before hook you'd ever want to use is helpers.before. I suggest changing the name to setupTestDirectory

I agree. Could you replace the name, but alias before to setupTestDirectory (with warning?) so we keep backwards compatibility.

the comment for helpers.gruntfile says that the method is intended to be used as a mocha handler. I'm pretty new to mocha, can a mocha handler accept an options object (the signature is function gruntfile(options, done))?

That seems to be wrong. Can you find it used anywhere in our code base? (is there unit test showing how the method is used?)

[coveralls]

Coverage remained the same when pulling e8de9ac on smackesey:add_assert_helpers into 8fcf87d on yeoman:master.

[smackesey]

I agree. Could you replace the name, but alias before to setupTestDirectory (with warning?) so we keep backwards compatibility.

Done.

That seems to be wrong. Can you find it used anywhere in our code base? (is there unit test showing how the method is used?)

The only place it's used in this repo is in before, which calls it after cleaning out the test directory. There are no unit tests for it, so I just deleted that part of the comment.

The build is failing right now and I'm not sure why. On my machine, npm test passes. There seems to be some syntax error with Coveralls at the end of the test on Travis?

[SBoudrias]

Yeah, tests fails because seems like Coveralls returns invalid JSON when it refuse a coverage reports... Anyway, I'll handle this later on.

Thanks a lot!