Templates fail when tests are not run due to API limits

[akrawchyk]

I have a 200 hit/day limit for WebPageTest. I configured a test with more than that many, and webpagetest-mapper did a great job of gracefully handling that:

2015-04-30 17:17:50 ERROR webpagetest-mapper: failed to run test 11 [Viget case study]; The test request will exceed the daily test limit for the given API key
2015-04-30 17:17:50 ERROR webpagetest-mapper: failed to run test 12 [Big Spaceship homepage]; The test request will exceed the daily test limit for the given API key
2015-04-30 17:17:50 ERROR webpagetest-mapper: failed to run test 13 [Big Spaceship article]; The test request will exceed the daily test limit for the given API key
2015-04-30 17:17:50 ERROR webpagetest-mapper: failed to run test 14 [Big Spaceship case study]; The test request will exceed the daily test limit for the given API key

However, after retrieving test results I run into an issue I believe is related to these API limits:

2015-04-30 17:25:26 INFO webpagetest-mapper: mapping result undefined [Viget case study]
2015-04-30 17:25:26 INFO webpagetest-mapper: mapping result undefined [Big Spaceship homepage]
2015-04-30 17:25:26 INFO webpagetest-mapper: mapping result undefined [Big Spaceship article]
2015-04-30 17:25:26 INFO webpagetest-mapper: mapping result undefined [Big Spaceship case study]
TypeError: Cannot call method 'toString' of undefined
at Object.module.exports.compile (/Users/andrew/.nvm/v0.10.38/lib/node_modules/webpagetest-mapper/src/templates.js:29:19)
at Object.eval (eval at createFunctionContext (/Users/andrew/.nvm/v0.10.38/lib/node_modules/webpagetest-mapper/node_modules/handlebars/dist/cjs/handlebars/compiler/javascript-compiler.js:221:23), :25:84)
at prog (/Users/andrew/.nvm/v0.10.38/lib/node_modules/webpagetest-mapper/node_modules/handlebars/dist/cjs/handlebars/runtime.js:193:15)
at execIteration (/Users/andrew/.nvm/v0.10.38/lib/node_modules/webpagetest-mapper/node_modules/handlebars/dist/cjs/handlebars/base.js:155:19)
at Object. (/Users/andrew/.nvm/v0.10.38/lib/node_modules/webpagetest-mapper/node_modules/handlebars/dist/cjs/handlebars/base.js:164:11)
at Object.eval (eval at createFunctionContext (/Users/andrew/.nvm/v0.10.38/lib/node_modules/webpagetest-mapper/node_modules/handlebars/dist/cjs/handlebars/compiler/javascript-compiler.js:221:23), :17:31)
at prog (/Users/andrew/.nvm/v0.10.38/lib/node_modules/webpagetest-mapper/node_modules/handlebars/dist/cjs/handlebars/runtime.js:193:15)
at execIteration (/Users/andrew/.nvm/v0.10.38/lib/node_modules/webpagetest-mapper/node_modules/handlebars/dist/cjs/handlebars/base.js:155:19)
at Object. (/Users/andrew/.nvm/v0.10.38/lib/node_modules/webpagetest-mapper/node_modules/handlebars/dist/cjs/handlebars/base.js:164:11)
at Object.eval (eval at createFunctionContext (/Users/andrew/.nvm/v0.10.38/lib/node_modules/webpagetest-mapper/node_modules/handlebars/dist/cjs/handlebars/compiler/javascript-compiler.js:221:23), :27:31)
Fatal error, exiting.

[philbooth]

Oh yeah! Will publish a fix tomorrow, thanks for reporting this. :)

[philbooth]

Hey @akrawchyk, I've fixed your specific error and published as version 0.0.3.

There may be other places where similar problems exist, I haven't done a thorough test yet. Leaving this issue open until I know that everything is okay across the board.

[akrawchyk]

Thanks for looking into this so quickly @philbooth!

[philbooth]

Just in case anyone is interested, work is now underway to further improve this behaviour in the graceful-failure branch:

https://github.com/nature/webpagetest-mapper/compare/graceful-failure?expand=1

[philbooth]

Version 0.1.2 has been published which fully resolves this issue.

Previously, failed tests were being left in the result array, causing gaps to show up in the charts and labels for the failed tests to be overwritten at the same offset. Now those tests are removed entirely instead.