diff --git a/docs/arrow_cookbook/DetailedArchitectureDiagram.jpg b/docs/arrow_cookbook/DetailedArchitectureDiagram.jpg new file mode 100644 index 00000000..7715aa24 Binary files /dev/null and b/docs/arrow_cookbook/DetailedArchitectureDiagram.jpg differ diff --git a/docs/arrow_cookbook/README.rst b/docs/arrow_cookbook/README.rst new file mode 100644 index 00000000..b64ceeb1 --- /dev/null +++ b/docs/arrow_cookbook/README.rst @@ -0,0 +1,89 @@ +=========== +Arrow Usage +=========== + + +Synopsis +======== +| arrow [OPTION...] [TESTFILE...] + + +Description +=========== +Arrow is a test framework designed to promote test-driven JavaScript development. Arrow provides a consistent test creation and execution environment for both Developers and Quality Engineers. + +Arrow aims to completely remove the line between development’s Unit tests, and Functional and Integration tests by providing a uniform way to create and execute both. + +Arrow itself is a thin, extensible layer that marries JavaScript, NodeJS, PhantomJS and Selenium. Arrow allows you to write tests using YUI-Test and execute those tests using NodeJS, PhantomJS or Selenium. Additionally, Arrow provides a rich mechanism for building, organizing and executing test and test scenarios. + + +Options +======= +--help + display this help page +--version + display installed arrow version +--lib comma separated list of js files needed by the test +--page path to the mock or production html page, for example: http://www.yahoo.com or mock.html +--driver selenium|phantomjs|browser. (default: phantomjs) +--browser firefox|chrome|opera|reuse. Specify browser version with a hypen, ex.: firefox-4.0 or opera-11.0 (default: firefox) +--controller a custom controller javascript file +--reuseSession true/false. Specifies whether to run tests in existing sessions managed by selenium. Visit http://selenuim_host/wd/hub to setup sessions (default: false) +--report true/false. Creates report files in junit and json format, and also prints a consolidated test report summary on console +--testName comma separated list of test names defined in test descriptor. all other tests will be ignored +--group comma separated list of groups defined in test descriptor, all other groups will be ignored +--logLevel DEBUG|INFO|WARN|ERROR|FATAL (default: INFO) +--dimension a custom dimension file for defining ycb contexts +--context name of ycb context + + +Examples +======== +| Below are some examples to help you get started. + +| Unit test: +| arrow --lib=../src/greeter.js test-unit.js + +| Unit test with a mock page: +| arrow --page=testMock.html --lib=./test-lib.js test-unit.js + +| Unit test with selenium: +| arrow --page=testMock.html --lib=./test-lib.js --driver=selenium test-unit.js + +| Integration test: +| arrow --page=http://www.hostname.com/testpage --lib=./test-lib.js test-int.js + +| Integration test: +| arrow --page=http://www.hostname.com/testpage --lib=./test-lib.js --driver=selenium test-int.js + +| Custom controller: +| arrow --controller=custom-controller.js --driver=selenium + + +See Also +======== + +| arrow_server(1) + + +Third Party Libraries +======================= + +The following third-party npm modules are used by Arrow: + +| glob https://github.com/isaacs/node-glob +| mockery https://github.com/nathanmacinnes/Mockery +| nopt https://github.com/isaacs/nopt +| colors https://github.com/Marak/colors.js +| express https://github.com/visionmedia/express +| yui http://github.com/yui/yui3 +| JSV http://github.com/garycourt/JSV +| log4js https://github.com/nomiddlename/log4js-node +| clone https://github.com/pvorb/node-clone +| useragent https://github.com/3rd-Eden/useragent +| ytestrunner https://github.com/gotwarlost/ytestrunner + +Apart from those npm modules, Arrow also uses these two tools + +| selenium https://code.google.com/p/selenium/ +| ghostdriver https://github.com/detro/ghostdriver diff --git a/docs/arrow_cookbook/arrow_CI.rst b/docs/arrow_cookbook/arrow_CI.rst new file mode 100644 index 00000000..4aa6c61a --- /dev/null +++ b/docs/arrow_cookbook/arrow_CI.rst @@ -0,0 +1,31 @@ +================================ +Arrow and Continuous Integration +================================ + +Arrow has always had Continuous Integration in mind. Arrow provides you with all of the necessary tools to run in a CI System. + +Dealing With Different Environments or Hosts +-------------------------------------------- + +In all CI systems, you must deal with varying hosts as your code moves through its validation cycle. For example, as you are developing, the host of the application will likely be *localhost* or perhaps a development machine. Once you commit your code it may go to an *integration* environment where the host will likely be different. Yet as that code moves to your *staging* or even *production* environment, your host will change once more. + +You can manage this using `test descriptors <./arrow_in-depth.html#test-suite-organization>`_ and `dimensions files `_ + +You can also manage this by using the **--baseUrl** parameter in your command: + +:: + + arrow --baseUrl=http://some.base.url.com + +Arrow will override any *baseUrl* value either in the *config* file or it the *test descriptor* and will use it instead. + +Reporting +--------- + +As `described before, --report=true \ No newline at end of file diff --git a/docs/arrow_cookbook/arrow_FAQs.rst b/docs/arrow_cookbook/arrow_FAQs.rst new file mode 100644 index 00000000..f69f8b8e --- /dev/null +++ b/docs/arrow_cookbook/arrow_FAQs.rst @@ -0,0 +1,231 @@ +========== +Arrow FAQs +========== + +If you don't find an answer here, please do not hesitate to email us + +How to run arrow tests from Linux box to Windows machine? +---------------------------------------------------------- + +Make sure you have following setup before running arrow tests + +1. Linux box has arrow installed and it has all the packages listed `here `_ (Only start arrow_server if you are running unit tests) +2. Windows machine has `selenium server `_ up and running (Check http://yourIP:4444/wd/hub is loading fine) +3. Now run the arrow test and do not forget to pass the --seleniumHost=http://yourIP:4444/wd/hub while running your tests + +:: + + arrow test-descriptor.json --seleniumHost=http://110.32.34.65:4444/wd/hub --browser=firefox + +How to debug failures for, "arrow failed to collect report" cases? +------------------------------------------------------------------- + +Usually this happens when there is some JS error inside your test script and to debug it further do following, + +1. Run selenium with default profile + +:: + + java -Dwebdriver.firefox.profile=default -jar selenium-server-standalone-2.25.0.jar + +2. Now from your firefox where you are running selenium, open http://yourIP:4444/wd/hub and create a firefox session +3. Open the firebug console for this newly created session +4. Trigger your tests, this will keep the browser window open after the tests fail and you can see JS errors on the console + + +Error: Cannot find module 'abbrev' +---------------------------------- + +:: + + node.js:201 + throw e; // process.nextTick error, or 'error' event on first tick + ^ + Error: Cannot find module 'abbrev' + at Function._resolveFilename (module.js:334:11) + at Function._load (module.js:279:25) + at Module.require (module.js:357:17) + at require (module.js:368:17) + at Object. (/usr/local/lib/node_modules/arrow/lib/nopt/lib/nopt.js:10:14) + at Module._compile (module.js:432:26) + at Object..js (module.js:450:10) + at Module.load (module.js:351:31) + at Function._load (module.js:310:12) + at Module.require (module.js:357:17) + +Solution +======== + +Install the abbrev module + +:: + + sudo npm install abbrev -g + +Error: SyntaxError?: Unexpected token } +--------------------------------------- + +:: + + SyntaxError?: Unexpected token } + + node.js:201 + throw e; // process.nextTick error, or 'error' event on first tick + ^ + SyntaxError: Unexpected token } + at Object.parse (native) + at DataProvider.getTestData (/usr/local/lib/node_modules/arrow/lib/util/dataprovider.js:21:31) + at SessionFactory.runAllTestSessions (/usr/local/lib/node_modules/arrow/lib/session/sessionfactory.js:55:12) + at Object. (/usr/local/lib/node_modules/arrow/index.js:56:16) + at Module._compile (module.js:432:26) + at Object..js (module.js:450:10) + at Module.load (module.js:351:31) + at Function._load (module.js:310:12) + at Array.0 (module.js:470:10) + at EventEmitter._tickCallback (node.js:192:40) + +Solution +======== + +It's likely you have an extra character in your descriptor JSON file. + +Error: Arrow Server is not Running +---------------------------------- + +:: + + node.js:137 + throw e; // process.nextTick error, or 'error' event on first tick + ^ + arrow_server is not running + +Solution +======== +You need to start Arrow server like this + +:: + + arrow_server + +Because arrow_server needs to KEEP running, start it on a different command prompt than the one you are using for testing + +How do I point Arrow to a Specific Selenium Server +-------------------------------------------------- + +There may be situations where the Selenium Server may not be running on the localhost and/or you may be pointing to a Selenium Grid instance. + +Solution +======== + +You can tell Arrow to point to a specific Selenium Host in two ways + +1. Update the `config file's <./arrow_in-depth.html#configuration>`_ seleniumHost value +2. Use the **--seleniumHost** parameter in your command + +**Note** you need to include the **FULL** path to Selenium Server like this: + +:: + + seleniumHost=http://url.to.server:port/wd/hub + +[ERROR] ArrowServer - + +Solution +======== + +Make sure you have `installed PhantomJS <./arrow_getting_started.html#mac-installation>`_ + +How can I use the Locator Controller to Login? +---------------------------------------------- + +Built-in, Arrow comes with two controllers, default and `locator <./arrow_in-depth.html#the-locator-controller>`_ + +Solution +======== + +You can use the locator controller to *log you into* Yahoo should you need it. In this example we'll do the following: + +1. Open login.yahoo.com with the final URL as the *done* URL +2. Execute the test + +:: + + [ + { + "settings": [ "master" ], + "name": "YahooLogin", + "config": { + "baseUrl": "http://finance.yahoo.com" + }, + "dataprovider" : { + + "Use Locator to Login" : { + "group" : "func", + "params" :{ + "scenario": [ + { + "page": "http://login.yahoo.com/config/login?login=arrowtestuser1@yahoo.com&passwd=123456&.done=$$config.baseUrl$$" + }, + { + "page": "$$config.baseUrl$$" + }, + { + "test": "test-title.js", + "title": "Yahoo! Finance - Business Finance, Stock Market, Quotes, News" + } + ] + } + } + } + }, + { + "settings": [ "environment:development" ] + } + ] + + +How can I install a specific Arrow version? +------------------------------------------- + +Though we don't encourage this, there may be times when you may need to use a specific version of Arrow. + +Solution +======== + +You can install a specific version like this: + +:: + +TODO... needs to be updated + + sudo npm install --registry=http:// arrow@ -g + +To install version 0.0.43 + +:: + +TODO... needs to be updated + + sudo npm install --registry=http:// arrow@0.0.43 -g + + +My Project/Tests are using NodeJS 4.x, the new versions of Arrow expect NodeJS 6+. Will I have problems? +-------------------------------------------------------------------------------------------------------- + +After version XXX of Arrow, Arrow will no longer support NodeJS 4.x. How will this affect my tests, and what impact will this have to me? + +Solution +======== + +Because our dependencies are part of the NPM package, provided you do not upgrade, this should not cause you any problems. In other words, the dependencies for a given version of Arrow are tied to that version. Therefore, you should be able to continue using a previous version of Arrow without any issues. + + + + diff --git a/docs/arrow_cookbook/arrow_extending.rst b/docs/arrow_cookbook/arrow_extending.rst new file mode 100644 index 00000000..21f18e76 --- /dev/null +++ b/docs/arrow_cookbook/arrow_extending.rst @@ -0,0 +1,232 @@ +============================== +Extending and Developing Arrow +============================== + +Extending Controllers +--------------------- + +Controllers are a very important part of Arrow. As described in the `architecture, <./arrow_intro.html#arrow-internals>`_ *controllers* are a way to *control* when and where your test will execute. By default, **Arrow** assumes you want to execute your test against the *page* or *HTTP End-Point* given in the **--page=** parameter. + +Additionally, Arrow provides the concept of `complex scenarios, <./arrow_in-depth.html#complex-test-scenarios>`_ you can create a *scenario* using a combination of the *default* and the `locator controller. <./arrow_in-depth.html#the-locator-controller>`_ + +However, there may still be the case where Arrow's built-in controllers are not sufficient for your needs. In this case, a user can create their own **custom controllers** to satisfy their needs. + +All controllers, including *default* and *locator* extend from the same **controller** interface. The **controller** interface provides you the following methods: + +* setup +* execute +* tearDown + +How To Create A Custom Controller +================================= + +Let's take the example we described in the `complex scenarios <./arrow_in-depth.html#complex-test-scenarios>`_ section. In that case we wanted to go to finance.yahoo.com, enter a ticker value and make sure we got to the correct page. We can convert those steps to into a *custom controller* + +First you need to make sure you can *create* a custom controller by executing this command: + +:: + + sudo ln -s /usr/local/lib /node_modules + +Now you can begin. Here is a sample implementation of what such a *custom controller* could look like: + +:: + + var util = require("util"); + var log4js = require("arrow").log4js; + var Controller = require("arrow").controller; + + function FinanceCustomController(testConfig,args,driver) { + Controller.call(this, testConfig,args,driver); + + this.logger = log4js.getLogger("FinanceCustomController"); + } + + /* + * All controllers MUST implement the Controller interface + */ + util.inherits(FinanceCustomController, Controller); + + + /** + * In the execute method you get full access to webdriver's methods + * Additionally, you can get a handle to the parameters in your descriptor + * file by using this.testParams + * + * Lastly, in this case, the last statement is to execute the test + * You'll note executeTest includes the same parameters as Arrow's CLI + */ + FinanceCustomController.prototype.execute = function(callback) { + var self = this; + + if(this.driver.webdriver){ + + //Get the various parameters needed from the Test Descriptor file + var txtLocator = this.testParams.txtLocator; + var typeText = this.testParams.typeText; + var btnLocator = this.testParams.btnLocator; + var page = this.testParams.page; + + //Get a handle of the WebDriver Object + var webdriver = this.driver.webdriver; + + //Open the page you want to test + webdriver.get(page).then(function() { + self.logger.info(self.config); + + //Navigate the page as necessary + webdriver.findElement(webdriver.By.css(txtLocator)).sendKeys(typeText); + webdriver.findElement(webdriver.By.css(btnLocator)).click(); + self.testParams.page=null; + webdriver.getTitle().then(function(title) { + + //Execute the test + self.driver.executeTest(self.testConfig, self.testParams, function(error, report) { + callback(); + }); + }); + }); + }else{ + this.logger.fatal("Custom Controllers are currently only supported on Selenium Browsers"); + } + } + + module.exports = FinanceCustomController; + +Now let's take a look at what the *Test Descriptor* would look like. + +:: + + [ + { + "settings": [ "master" ], + + "name": "controllers", + + "config": { + "baseUrl": "http://finance.yahoo.com" + }, + + "dataprovider" : { + + "Test YHOO Ticker using Finance Controller" : { + "group" : "func", + "params" :{ + "scenario": [ + { + "controller": "finance-controller.js", + "params": { + "page" : "$$config.baseUrl$$", + "txtLocator": "#txtQuotes", + "typeText": "yhoo", + "btnLocator": "#btnQuotes", + "test": "test-quote.js", + "quote": "Yahoo! Inc. (YHOO)" + } + } + ] + } + } + } + }, + { + "settings": [ "environment:development" ] + } + ] + +The *Test Descriptor* includes all of the information the controller will need; all under the **params** node + +How To Execute +.............. + +Execution is **exactly the same** as in previous examples + +:: + + arrow --driver=selenium + +Developing Arrow +---------------- + +Though the Arrow team members tried their best to think of all possible situations, there may be features which you feel would be good, or perhaps you want to contribute with bug patches. + +How To Get Started +================== + +Obviously, the first step will be to `download the source code `_ + +Once you have become familiar with Arrow, you'll probably want to make small changes to see them reflected locally. + +To force NodeJS to look at your local instance of Arrow do the following: + +1. Navigate to the location where Arrow's source code resides +2. Look for a file called package.json, it will be under: path_to_arrow_source/arrow/node/package.json) +3. From within the *node* folder (*arrow/node*), link your local instance of arrow to node by typing: + +:: + + sudo npm link + +You can confirm if the *link* was successful by checking its version + +:: + + arrow --version + +You should get: + +:: + + 0.0.0 + +How To Submit a Patch +===================== + +Internal Patch Submission Process +................................. + +TODO... needs to be updated + +Code Review Process +................... + +TODO... needs to be updated + +**DO NOT COMMIT your code without following the patch submission process** + +How to Run Unit/Functional Tests? +................................. + +TODO... needs to be updated + +Get the `unit tests _ + +Navigate to the unit tests /arrow_tutorial/unit_test/test + +Run the following command and make sure it passes + +:: + + arrow test-unit.js --lib=../src/greeter.js + + + +**WIP** + +TODO... needs to be updated + + +Get the `functional tests +Navigate to the functional tests /arrow_funtional-tests/node + +Run following command and make sure it passes + +:: + + npm test + + + + + + diff --git a/docs/arrow_cookbook/arrow_getting_started.rst b/docs/arrow_cookbook/arrow_getting_started.rst new file mode 100644 index 00000000..067a0ab4 --- /dev/null +++ b/docs/arrow_cookbook/arrow_getting_started.rst @@ -0,0 +1,316 @@ +========================== +Getting Started With Arrow +========================== + +.. _Installation: + +**Installation** + +Arrow is fully supported on: + + * Mac_ + * Linux_ + +We are working on providing support for Windows. + +Installation consists of three main steps, *PhantomJS* installation, *Arrow* installation, Setting up Arrow and starting *Arrow Server* + +.. _Mac: + +Mac Installation +---------------- + +These instructions assume you do not have NPM and NodeJS installed. If you do, skip step 1 + +1. Install NodeJS and NPM: + + * Go to `Download `_. + * Follow the installation instructions + +2. Due to a bug in PhantomJS v1.5, Arrow's webdriver implementation expects PhantomJS v1.6 or higher. If you are using a version lower than 1.6 please update it. To install/update, please navigate to: http://phantomjs.org/ and follow the instructions. + +3. Copy the phantomjs binary to + +:: + + /usr/local/bin + +4. Make the phantomjs binary executable + +:: + + chmod +x phantomjs + +5. Install Arrow Package **globally** through NPM: + +:: + + sudo npm install yahoo-arrow -g + + +5. To **locally** install Arrow use following + +:: + + sudo npm install yahoo-arrow + +6. Configure Arrow to allow for custom controllers (This step is only needed when you are installing arrow globally) + +:: + + sudo ln -s /usr/local/lib /node_modules + +7. Start Arrow Server on the command prompt (note, this must be left **ON** or the process will die) + +:: + + arrow_server + +7. Local installation of yahoo-arrow will put arrow under ./node_modules/.bin and to run arrow from there use + +:: + + ./node_modules/.bin/arrow --version + ./node_modules/.bin/arrow_server + ./node_modules/.bin/arrow test_descriptor.json + +Now you may proceed to Install Verification_. + +.. _Linux: + +Linux Installation +------------------ + +.. todo we need to rework the installation for Linux + +**Note:** Though the dist package say it only supports RHEL 4.x, we have found ``ynodejs06`` works for +RHEL 5.x (5.0-20070208, 5.2-20080429, 5.4-20090902, 5.6-20110328) as well. + + +1. Install NodeJS06, PhantomJS and NMP: + +:: + +TODO... needs to be updated + + + +2. Set yinst options as follows: + +:: + +TODO... needs to be updated + + + +3. Install Arrow: + +:: + +TODO... needs to be updated + + +4. Configure Arrow to allow for custom controllers + +:: + +TODO... needs to be updated + + +5. Start Arrow Server on the command prompt (note, this must be left **ON** or the process will die) + +:: + + arrow_server + +Now you may proceed to Install Verification_. + + +Selenium Server +--------------- + +Arrow uses `Selenium Server `_ to excute all browser-related tests. Selenium should run on a machine which has access to the browsers you want to use. Normally Selenium Server will run on Mac or Windows Machine. + +Start Selenium +============== + +To start Selenium Server you simply need to do the following: + +1. `Download `_ the Selenium JAR Executable +2. To **Run** the Selenium executable type this: + +:: + + java -jar path/to/selenium-server.jar + +Note Selenium Server is **NOT** required for Arrow to work. However, it is strongly recommended. + +To start Selenium Server using chromedriver and default Firefox profile use + +:: + + java -Dwebdriver.chrome.driver=./chromedriver -Dwebdriver.firefox.profile=default -jar selenium-server-standalone-2.xx.0.jar + + +.. _Verification: + +Verifying the Installation +-------------------------- + +Make sure you are able to execute ALL the commands below: + + +Check Arrow Help +================ + +:: + + arrow --help + +:: + + OPTIONS : + --lib : a comma seperated list of js files needed by the test + --page : (optional) path to the mock or production html page + example: http://www.yahoo.com or mock.html + --driver : (optional) one of selenium|browser. (default: selenium) + --browser : (optional) a comma seperated list of browser names, optionally with a hypenated version number. + Example : 'firefox-12.0,chrome-10.0' or 'firefox,chrome' or 'firefox'. (default: firefox) + --controller : (optional) a custom controller javascript file + --reuseSession : (optional) true/false. Determines whether selenium tests reuse existing sessions. (default: false) + Visit http:///wd/hub to setup sessions. + --parallel : (optional) test thread count. Determines how many tests to run in parallel for current session. (default: 1) + Example : --parallel=3 , will run three tests in parallel + --report : (optional) true/false. creates report files in junit and json format. (default: true) + also prints a consolidated test report summary on console. + --reportFolder : (optional) folderPath. creates report files in that folder. (default: descriptor folder path) + --testName : (optional) comma seprated list of test name(s) defined in test descriptor + all other tests will be ignored. + --group : (optional) comma seprated list of group(s) defined in test descriptor. + all other groups will be ignored. + --logLevel : (optional) one of DEBUG|INFO|WARN|ERROR|FATAL. (default: INFO) + --dimensions : (optional) a custom dimension file for defining ycb contexts + --context : (optional) name of ycb context + --seleniumHost : (optional) override selenium host url (example: --seleniumHost=http://host.com:port/wd/hub) + --capabilities : (optional) the name of a json file containing webdriver capabilities required by your project + + EXAMPLES : + Unit test: + arrow test-unit.js --lib=../src/greeter.js + Unit test with a mock page: + arrow test-unit.js --page=testMock.html --lib=./test-lib.js + Unit test with selenium: + arrow test-unit.js --page=testMock.html --lib=./test-lib.js --driver=selenium + Integration test: + arrow test-int.js --page=http://www.hostname.com/testpage --lib=./test-lib.js + Integration test: + arrow test-int.js --page=http://www.hostname.com/testpage --lib=./test-lib.js --driver=selenium + Custom controller: + arrow --controller=custom-controller.js --driver=selenium + +Check Arrow version +=================== + +:: + + arrow --version + +:: + + [2012-05-17 12:12:06.665] [INFO] console - vX.X.X + +Confirm you can run the Arrow server +==================================== + +:: + + arrow_server + +:: + + l2tp-8-16:test ivan$ arrow_server + [2012-05-17 12:08:31.322] [INFO] console - Server running at: http://Ivans-MacBook-Air.local:4459 + [2012-05-17 12:08:32.105] [INFO] console - GhostDriver Running At : http://Ivans-MacBook-Air.local:4460 + + +Confirm Selenium is Running +=========================== + +Though Selenium Server is NOT required, if you chose to run it, you can confirm it's running successfully like this: + +1. From a Browser, go to: http://host.or.url:port/wd/hub or http://localhost:4444/wd/hub +2. You should be directed to a WebDriver page + + +.. _Creating a test: + +Creating a test +--------------- + +You are now ready to create and execute your first test. For our first test we are going to validate a simple YUI Module. This YUI module has one method called *greet*. *greet* take a first and last name and inverts them as its output + +1. Create a folder called **arrow_test** + +2. Inside arrow_test, create a folder called **src** (this will be our code source folder) + +3. Create a file called **greeter.js** inside src and paste the code below into it + +:: + + YUI.add("arrow-greeter", function (Y) { + Y.namespace("Arrow"); + + var Greeter = Y.Arrow.Greeter = function() {}; + + //This is a simple method which takes two params, first and last name + //It returns it as lastname, firstname + Greeter.prototype.greet = function(firstName, lastName) { + return lastName + ", " + firstName; + } + }, "0.1", {requires:[]}); + +4. Inside arrow_test create a folder called **tests** + +5. Create a file called **test-greeter.js** inside tests and past the code below into it + +:: + + YUI({ useBrowserConsole: true }).use("arrow-greeter", "test", function(Y) { + //Create a basic test suite + //We're calling it "Our First Test" + var suite = new Y.Test.Suite("Our First Test"); + + //Add a test case to the suite; "test greet" + suite.add(new Y.Test.Case({ + "test greet": function() { + var greeter = new Y.Arrow.Greeter(); + + //The method we are testing will inverse the firstname and lastname + //Our test will check for that inversion + Y.Assert.areEqual(greeter.greet("Joe", "Smith"), "Smith, Joe"); + } + })); + + //Note we are not "running" the suite. + //Arrow will take care of that. We simply need to "add" it to the runner + Y.Test.Runner.add(suite); + }); + +Now we are ready to run our test. + +6. Navigate to + +:: + + ~/arrow_test/tests + +7. Type this to execute your test + +:: + + arrow test-greeter.js --lib=../src/greeter.js --driver=nodejs + +Congratulations, you've successfully installed Arrow and created your first test + + + diff --git a/docs/arrow_cookbook/arrow_in-depth.rst b/docs/arrow_cookbook/arrow_in-depth.rst new file mode 100644 index 00000000..bfc47b0b --- /dev/null +++ b/docs/arrow_cookbook/arrow_in-depth.rst @@ -0,0 +1,645 @@ +============== +Arrow In-Depth +============== + +Arrow provides you with a variety of tools to help you organize, execute and configure your tests + +Test Suite Organization +----------------------- + +Test Descriptor files allow you to organize your tests into test suites, while also allowing you to control when and which tests execute at a given phase of your development cycle. + +Consider the following scenario: +You have just finished creating a suite of tests that validate the application we discussed in the `Arrow Tutorial <./arrow_tutorial.html>`_ chapter. + +At this point you have the following test files: + +* Fuctional Tests +* Integration Tests + +For this sample, we'll pretend unit tests are being addressed elsewhere. + +If you recall, to execute the two test files above against our mock and HTTP End-Ponit, we'd type something like this: + +:: + + arrow test-func.js --page=testMock.html --lib=test-lib.js + arrow test-func.js --page=http://www.doctor46.com/tabview.html --lib=test-lib.js + arrow test-int.js --page=http://www.doctor46.com/tabview.html --lib=test-lib.js + +Let's pretend we wanted to easily decide which test files executed and which didn't. Test Descriptors_ allow you to do this easily and in one location + +.. _Descriptors: + +Test Descriptors +================ + +Test descriptors provide a way to describe, organize and factorize your tests. During test development, you'll probably execute each test from the Arrow command line. However, once you have created tests to validate your module, you need a way to organize and factorize the tests. + +Let's look at this test `descriptor: + +TODO... needs to be updated + + +:: + + [ + { + "settings": [ "master" ], + + "name" : "tabview", + + "commonlib" : "./test-lib.js", + + "config" :{ + "baseUrl" : "http://www.doctor46.com" + }, + + "dataprovider" : { + + "dom" : { + "params" : { + "test" : "test-func.js", + "page" : "testMock.html" + }, + "group" : "unit" + }, + + "dom_int" : { + "params" : { + "test" : "test-func.js", + "page" : "$$config.baseUrl$$/tabview.html" + }, + "group" : "smoke" + }, + + "int" : { + "params" : { + "test" : "test-int.js", + "page" : "$$config.baseUrl$$/tabview.html" + }, + "group" : "smoke" + } + + } + + }, + + { + "settings": [ "environment:development" ] + } + + ] + + +**Suite Settings** + +These settings are accessible by all tests in the suite + +:: + + "name" : "tabview", + "commonlib" : "./test-lib.js", + +name and commonlib + +* `name:` Allows you to give your descriptor a name. In Arrow you can run multiple test descriptors in a single execution. Giving it a name allows you to separate the results +* `commonlib:` Behaves like a suite-level `--lib` parameter. Rather than calling each necessary dependency or lib in the tests, you can do that here. `commonlib` is not limited to only one dependency. If you have more than one dependency, you can specify them with commas. + +**Suite Configuration** + +The settings here, allow you to override default config settings and apply to the entire suite + +:: + + "config" :{ + "defaultAppHost" : "http://www.doctor46.com" + }, + +In this example we have a key called `defaultAppHost`. The value assigned to this key can be picked up using the `$$` annotation, for example `$$config.defaultAppHost$$`. + +This is one way we can parametrize our tests and make them easier to execute/share. + +**Individual Test Settings** + +This section uses the `Suite Settings` and the `Suite Configuration` to create instances of your tests. + +:: + + "dom_int" : { + "params" : { + "test" : "test-func.js", + "page" :"testMock.html" + }, + "group" : "unit" + }, + +* The first object is the name of the test. In this case, the test name is `dom_int`. +* The next object, `params`, includes the necessary parameters for the test. +* `test`: Tells Arrow which file to execute +* `page`: Tells Arrow against which page to execute. The `page` value can be a local mock page served by arrow_server, or an HTTP End-Point +* `group`: Allows you to *group* your tests for execution. Each test `file` contains a set of tests or assertions. At the time of creation, tests do not have a context (at least not implied). A `group` gives those test `files` context, enabling you to execute only a given set of tests during a given execution. + +Executing using a Test Descriptor +================================= + +To Execute *All* tests in a given test descriptor file simply type (remember in this example, the name of our file is `test-descriptor.json`): + +:: + + arrow test-descriptor.json + +However, if you wanted to *only* execute tests `grouped` as `func`, you would type: + +:: + + arrow test-descriptor.json --group=func + +Similarly, you can choose to *only* execute a given test, based on its name. You can do that by typing: + +:: + + arrow test-descriptor.json --testName=dom + + +Test Descriptor Best Practices +============================== + +One Test Descriptor Per Module +.............................. + +One test descriptor per module is recommended. You do not need a *parent* test descriptor file to include multiple modules. There are different tools which do this for you. Given a root directory, Arrow traverses the child directories and picks up the required test descriptor files. + +For example, suppose you have the following directory structure, and within each moduleN/test folder you have tests and a test descriptor file. + +:: + + project1 + |____ module1 + | |_____src + | |_____test + | |_____test-descriptor1.json + | + |____ module2 + | |_____src + | |_____test + | |_____test-descriptor2.json + | + |____ module3 + | |_____src + | |_____test + | |_____test-descriptor3.json + | + |____ module4 + |_____src + |_____test + |_____test-descriptor4.json + +To execute *All* test descriptor files *within* each module, simply navigate to the project root (in this case `project1`) and type: + +:: + + arrow "**/*-descriptor.json" + +Arrow will traverse through all sub-folders, pick up the test descriptors which match "**/*-descriptor.json" glob, and execute them sequentially. + +Parametrize Test Descriptors +............................ + +There are tests which require parametrization. Specially in *Integration* tests (int), it is important to have a way to parametrize the host name of your AUT. + +Test descriptors allow you to parametrize like this: + +:: + + "dom_int" : { + "params" : { + "test" : "test-func.js", + "page" :"$$config.defaultAppHost$$/tabview.html" + }, + "group" : "smoke" + }, + + "int" : { + "params" : { + "test" : "test-int.js", + "page" : "$$config.defaultAppHost$$/tabview.html" + }, + "group" : "smoke" + } + +Where `"defaultAppHost" : "http://doctor46.com"` + + +Test Descriptor Parametrization and Test Environments +----------------------------------------------------- + +So far our parametrization examples have only applied to our curent file. If we want to run our tests across different environments (with different hostnames), we'd have to create multiple test-descriptor.json files to do this. However, we can use a `dimension` file to give our paramters additional `dimension` or context. + +At the bottom of our test descriptor file there was this line: + +:: + + { + "settings": [ "environment:development" ] + } + +We can make use of the line above, and a `dimension` file to dynamically change configuration values given a context. + +With this `dimension` file we can set different contexts in our test descriptor: + +:: + + [ + { + "dimensions": [ + { + "environment": + { + "development": { + "test": null + }, + "integration": { + "test": null + }, + "stage": { + "test": null + }, + "production": { + "test": null + } + } + } + ] + } + ] + +Now we can update our test decriptor like this + +:: + + { + "settings": [ "environment:development" ], + + "config" :{ + "defaultAppHost" : "http://development.com" + } + }, + + { + "settings": [ "environment:integration" ], + + "config" :{ + "defaultAppHost" : "http://integration.com" + } + }, + + { + "settings": [ "environment:stage" ], + + "config" :{ + "defaultAppHost" : "http://stage.com" + } + }, + + { + "settings": [ "environment:production" ], + + "config" :{ + "defaultAppHost" : "http://production.com" + } + } + +During execution, we can set the context like this: + +:: + + arrow test-descriptor.json --context=environment:development --dimensions=./dimensions.json + +Or + +:: + + arrow test-descriptor.json --context=environment:stage --dimensions=./dimensions.json + +In each case, Arrow will take the `context` and `dimensions` file and use those to map the correct `config` value for the current execution + + + +Configuration +------------- +There are various ways to configure arrow. Normally, Arrow's configuration file will be installed here + +.. todo need to update the location for NON-Yahoo Linux. + +Configuration Location +====================== + ++-------+--------------------------------------------------------------------------------+ +|MAC | /usr/local/lib/node_modules/arrow/config/config.js | ++-------+--------------------------------------------------------------------------------+ +|Linux | TODO... needs to be updated | ++-------+--------------------------------------------------------------------------------+ +|WIN | `%USERPROFILE%\\AppData\\Roaming\\npm\\node_modules\\arrow\\config\\config.js` | ++-------+--------------------------------------------------------------------------------+ + +The standard arrow config file looks like this + +:: + + var config = {}; + + // User default config + config.seleniumHost = ""; + //example: config.seleniumHost = "http://gridhost:port/wd/hub"; + config.context = ""; + config.defaultAppHost = ""; + config.logLevel = "INFO"; + config.browser = "firefox"; + config.parallel = false; + config.baseUrl = ""; + // Framework config + config.arrowModuleRoot = global.appRoot + "/"; + config.dimensions = config.arrowModuleRoot + "config/dimensions.json"; + config.defaultTestHost = config.arrowModuleRoot + "lib/client/testHost.html"; + config.defaultAppSeed = "http://yui.yahooapis.com/3.6.0/build/yui/yui-min.js"; + config.testSeed = config.arrowModuleRoot + "lib/client/yuitest-seed.js"; + config.testRunner = config.arrowModuleRoot + "lib/client/yuitest-runner.js"; + config.autolib = config.arrowModuleRoot + "lib/common"; + config.descriptorName = "test_descriptor.json"; + + module.exports = config; + +As you can see there are two types of configuration sections: + +* User Config: These are configuration parameters which directly affect how your test or test suite will execute +* Framework Config: These are configuration parameters which indirectly affect how your test or test suite will execute + +Overriding Configuration Values +=============================== + +Obviously, you can update the config file to *override* its settings. However, you can also *override* individual config parameters on a per-execution basis. Every config parameter can be *overridden* during execution like this: + +:: + + arrow --config=value + +Or + +:: + + arrow --seleniumHost=http://some.url.com:1234/wd/hub + +Or + +:: + + arrow --logLevel=debug --baseUrl=http://basesurl.com --browser=chrome + +You can basically override any config parameter in the command line. + +You can also **completely** override all configuration values by placing a config.js file at the root of your execution. Arrow always looks at the current directory for config.js file. If it finds one, it will use **that** file over the default configuration. + + +Complex Test Scenarios +---------------------- + +There are situations where the default arrow controller will not allow you to create the type of test scenario you require. If you recall, the default arrow controller assumes the page you load is the page under test. To solve this you can use a different arrow controller called *locator*. The *locator* controller allows you to navigate to the page under test by allowing you to perform actions such as clicking and typing. + +The controller samples can be found `here. + +TODO... needs to be updated + + +The Locator Controller +====================== + +To use the *locator* controller you need to use a test descriptor with an additional node, **scenario**. + +Suppose you wanted to test finance.yahoo.com's ticker quotes engine. To do that, you would build a scenario like this: + +1. Open http://finance.yahoo.com +2. Use the *locator* controller and look for the *ticker* input textbox and enter *yhoo* +3. Use the *locator* controller and *click* on the submit button +4. Wait for the page to load **and** now test for quotes + +Based on the scenario above, our test descriptor file would look like this: + +:: + + "dataprovider" : { + + "Test YHOO Ticker" : { + "group" : "func", + "params" :{ + "scenario": [ + { + "page": "$$config.baseUrl$$" + }, + { + "controller": "locator", + "params": { + "value": "#txtQuotes", + "text": "yhoo" + } + }, + { + "controller": "locator", + "params": { + "value": "#btnQuotes", + "click": true + } + }, + { + "test": "test-quote.js", + "quote": "Yahoo! Inc. (YHOO)" + } + ] + } + } + } + +Our first step is to open the page (Arrow will use the *default* controller when none is specified). Secondly we look for an input field with a locator value of *#txtQuotes* and we enter *yhoo*. Then we use the *locator* controller to *click* on *#btnQuotes*. Finally we inject our test JS file and using *this.params,* we pass the value in *quote* to the test file. + +Our test continues being a simple YUI test which takes input from the test descriptor in order to do its validation + +:: + + YUI({ useBrowserConsole: true }).use("node", "test", function(Y) { + var suite = new Y.Test.Suite("Quote Page test of the test"); + suite.add(new Y.Test.Case({ + "test quote": function() { + + //In order to paramertize this, instead of having a static quote, we call it from the config + var quote = this.testParams["quote"]; + Y.Assert.areEqual(quote, Y.one(".yfi_rt_quote_summary").one("h2").get('text')); + } + })); + + Y.Test.Runner.add(suite); + }); + +To execute we simply type the following: + +:: + + arrow test-descriptor.json --driver=selenium + +As you can see, the *locator* controller is quite powerful. It can take the following *params* + +* **value**: locator value +* **click**: true or false +* **text**: value ot enter +* **using**: by default, Arrow will assume you want to use *css* locators for *value*. However you can use any **By** strategy supported by WebDriver: className, id, linkText, name, text, xpath, etc. + +For example, you could have the following in your test descriptor + +:: + + { + "controller": "locator", + "params": { + "using": "xpath", + "value": "//*[@id="btnQuotes"]", + "click": true + } + } + + +Re-Using Browser Sessions +------------------------- + +As you develop your tests, you may find it necessary to *test* them against a real browser, such as those supported by Selenium. However, one of the disadvantages of this approach is that normally, for each test file, a new browser sesssion is started and stopped. This is time consuming and counter-productive during development. + +Arrow supports the concept of **Session Reuse**. + +Using Session-Reuse +=================== + +Webdriver has a concept of sessions. Once a Selenium/WebDriver server instance is running, you can tell Selenium to *reuse* a given session. This is a very powerful and helpful idea because: + +* It expedites execution since a new browser window does not need to be instantiated. This greatly cuts down on execution time and puts *real* browser test execution time in-par with PhantomJS +* As a developer, you can tell Selenium to *use* your preferred *profile* for the session. This means that if you have special plugins (such firebug, or developer tools, etc) installed, you can make use of them during test execution. + +However, one should keep in mind that this approach means your test will have a sterile environment as session and cookie information will be **reused** + +To use *Session-Reuse* do the following: + +1. From within the machine running Selenium server go to: http://localhost:4444/wd/hub/static/resource/hub.html +2. Click on *create session* and choose the browser you want +3. A new Browser will start (that is your session) and set itself to a blank page +4. To tell Arrow to **Reuse** that session type: + +:: + + arrow --reuseSession=true + +Arrow will contact the Selenium Server in the config and will ask it if there are any *reusable* sessions. If so, it will direct all tests to them. + +Note Arrow will direct all tests to **ALL OPEN** sessions. If you want to further expedite your test execution time, you can start sessions for different browser and Arrow will execute your tests in parallel against all of them. + +Using Session-Reuse With Specific Profiles +========================================== + +If you want to *reuse* your default profile, or a specific profile you use for developing simply type this when you start Selenium server + +:: + + java -Dwebdriver.firefox.profile=default -jar ./path/to/selenium/sever.jar + +Or + +:: + + java -Dwebdriver.firefox.profile=profile_name -jar ./path/to/selenium/sever.jar + +Once Selenium is started, the same steps for *reusing* sessions apply. + +Parallelism +----------- + +Arrow supports Parallel execution of tests. By default **parallel** is set to *false*. You can update the value to the *maximum number* of threads you want to use. Keep in mind Arrow will try to create one Browser Session **PER** parallel count. It is important that you have enough system resources to support this + +How To Use +========== + +:: + + arrow --parallel=N + +Or + +:: + + arrow --parallel=5 + + +Reporting +--------- + +Arrow supports two reporting formats, the ever-popular JUnit.xml format and Arrow's own JSON format. Reporting is particularly important if you use test descriptors to execute your tests, because each test.js file will have its own set of results. However, using Arrow's reporting feature will merge the individual results into one report. + +How To Use +========== + +To tell Arrow you would like to create reports simply type: + +:: + + arrow --report=true + +After the test executes two files will be created under the location from which you executed Arrow; *report.xml* and *report.json*. + +Running multiple descriptors using 'arrow "**/*-descriptor.json" --report=true' , will create report.xml and report.json under directory structure where each descriptor files reside. + +Hudson supports report globbing, so you can pass **/test-descriptor-report.xml, and it will pick up all your result files. + +report.xml sample +................. + +:: + + + + + + + + + + +report.json sample +.................. + +:: + + [ + { + "passed":1, + "failed":0, + "total":1, + "ignored":0, + "duration":15, + "type":"report", + "name":"Quote Page test of the test", + "testCaseyui_3_2_0_18_133850857473827":{ + "passed":1, + "failed":0, + "total":1, + "ignored":0, + "duration":10, + "type":"testcase", + "name":"testCaseyui_3_2_0_18_133850857473827", + "test quote":{ + "result":"pass", + "message":"Test passed", + "type":"test", + "name":"test quote", + "duration":1 + } + }, + "timestamp":"Thu May 31 16:56:33 2012", + "ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:12.0) Gecko/20100101 Firefox/12.0", + "testName":"Test YHOO Ticker" + } + ] \ No newline at end of file diff --git a/docs/arrow_cookbook/arrow_intro.rst b/docs/arrow_cookbook/arrow_intro.rst new file mode 100644 index 00000000..4dbd4ff7 --- /dev/null +++ b/docs/arrow_cookbook/arrow_intro.rst @@ -0,0 +1,37 @@ +============== +What is Arrow? +============== +Arrow is a test framework designed to promote test-driven JavaScript development. Arrow provides a consistent test creation and execution environment for both, Developers and Quality Engineers. + +Arrow aims to completely remove the line between development's Unit tests, and QE's Functional and Integration tests, by providing a uniform way to create and execute both. + +Arrow itself, is a thin, extensible layer that marriages JavaScript, NodeJS, PhantomJS and Selenium. Arrow allows you to write tests using YUI-Test and Execute those tests using NodeJS, PhantomJS or Selenium. Additionally, Arrow provides a rich mechanism for building, organizing and executing test and test scenarios. + +Why Use Arrow +------------- + +Arrow's design is based on the idea that test purpose should not dictate how tests are created or executed. Using Arrow, users can create **Unit** tests to validate specific units of JavaScript code, create pre-commit **Functional** tests against a mock page, reuse those tests in post-commit scenarios, while also creating cross-page test scenarios that exercise **Integration**. + +What can I do with Arrow? +------------------------- + +The answer is simple, anything you can do with JS and NodeJS or Selenium, you can do with Arrow. In addition to Unit, Functional and Integration tests, you can also test WebServices using NodeJS + +Arrow Internals +--------------- + +Arrow's architecture is very simple. It is made up of 5 main components + +Core components: + +* Test Session: Provides the mechanism for injecting tests onto a page. +* Driver: In Arrow's terminology, this is what determines the driver of your tests. Which mechanism do you want injecting your tests. NodeJS, PhantomJS and Selenium are fully supported. +* Arrow Server: Arrow sever provides two very important functionalities. Firstly, it's a simple file server used to **serve** local mock pages when executing your tests using Selenium. Secondly, it is a reverse-proxy with a Web Driver implementation. This is used as a way to execute tests against browsers and devices not supported by Selenium. With Arrow Server running, **ANY** device can **register** itself with a given instance of Arrow Server, and said device and then be used as a test executor. + +User-facing Components: + +* Controllers: If drivers allow you to dictate *where* to execute your tests, controllers, or scenario atoms, allow you to dictate *when* to execute your tests. Arrow's default controller, or scenario atom, is to open the page and execute the test. This works well for a large number of tests, however, if you require more complex scenario, or more granular control, you can **easily** create your **own** controllers and **control** how and when your tests are executed +* Libs: Libs, or dependencies, are a way to include any necessary dependencies needed by your test. These can be things such as; libraries you've created, the source code against which you are testing, etc. + +.. image:: DetailedArchitectureDiagram.jpg + diff --git a/docs/arrow_cookbook/arrow_tutorial.rst b/docs/arrow_cookbook/arrow_tutorial.rst new file mode 100644 index 00000000..c7f845a6 --- /dev/null +++ b/docs/arrow_cookbook/arrow_tutorial.rst @@ -0,0 +1,193 @@ +============== +Arrow Tutorial +============== +This chapter will walk through the various types of tests for which Arrow is *normally* used; tests such as Unit_, Functional_ and Integration_. + +Prerequisites +================== +You must have: + + +TODO... needs to be updated + +* `Installed `_ the Arrow framework. +* `Downloaded `_ Arrow Tutorial from GitHub + +.. _Unit: + +Unit Tests +=========== + +The Demo includes a *unit_test* folder and includes two files: + +TODO... needs to be updated + + ++---------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+ +| `unit_test/src/greeter.js `_ | Simple YUI module that takes two parameters as input, inverses their order, and returns them as output | ++---------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+ +| `unit_test/test/test-unit.js >`_ | Unit test to validate the function. | ++---------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+ + +In the world of JS, unit tests validate JS functions/classes which do not interact with the DOM. Functionality which interacts with the DOM is usually considered to be functional and should be tested as such. + +Executing the Unit Test +----------------------- + +To execute the unit test example do the following: + +1. Start arrow_server (leave that prompt open): ``arrow_server`` + +2. On a separate command prompt navigate to: ``~/arrow_tutorial/unit_test/test`` + +3. NodeJS is the default driver for JS unit tests and does not require any configuration. To run against NodeJS, type: + +:: + + arrow test-unit.js --lib=../src/greeter.js + +*where*: ``--lib=`` Specifies the location of the source code to test. + + +To run against PhantomJS or a Selenium-supported browser, enter: + +:: + + arrow test-unit.js --lib=../src/greeter.js --driver=selenium --browser=phantomjs + + arrow test-unit.js --lib=../src/greeter.js --driver=selenium --browser=firefox + +TODO... needs to be updated + + +**Note:** `Arrow Server <./arrow_getting_started.html>`_ and `Selenium <./arrow_getting_started.html#start-selenium>`_ need to be running + +When you execute using PhantomJS, a screenshot is captured automatically. The screenshot is stored in the location where you executed the test (in this case, ``unit_test/test/``). + +.. _Functional: + +Functional Tests +================ + +src +--- + +Functional testing is a broad definition for anything that is *not* a unit test. This *may* include tests such as JS UI functional tests, and integration tests. + +As in the *unit_test* demo, under *arrow_tutorial* there is a folder called *func_tests* For the purposes of this demo, we will be working with YUI "multi-tab" module. Furthermore, during this stage of testing, functional, we'll use a *mock* page to mock our ultimate application. + +TODO... needs to be updated + + ++------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| `func_tests/src/tabview.js `_ | Our final application will make use of this file. This small piece of code will allow users to interact with application via tabs | ++------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| `func_tests/test/testMock.html `_ | This is a very simple HTML which acts as our *mock* page container. It has the basic skeleton of the final output and references the JS code the final output will also use | ++------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +Our mock page looks like this: + +.. image:: starting.png + +The final output of the application **will** look like this: + +.. image:: final.png + +test +---- + +TODO... needs to be updated + + +There are are number of test files in our *test* folder + ++---------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| `func_tests/test/test-lib.js `_ | This file acts as our test library. It is a YUI module whose purpose is to execute the various assertion and to facilitate code-sharing across other test files | ++---------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| `func_tests/test/test-func.js `_ | This is the skeleton of a basic functional test. In conjunction with test-lib.js, it makes tests easier to read by turning each statement into an action (validateSelection, validateStructure, etc | ++---------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| `func_tests/test/test-int.js `_ | Similar to test-func.js, test-int.js performs functional tests, however, it makes assertions about the values of each tab. test-func and test-int can be used together to validate the integration of our application | ++---------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| `func_tests/test/test-descriptor.json `_ | A *test descriptor* file is a way in Arrow to organize a test suite. Rather than having a long list of arrow commands, you can group your tests in a *test descriptor* and build test suites out of them | ++---------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +For now, let's look deeper at the test-func.js file. The tests within are pretty simple, they test: + +* The tabs are present. +* You can select on different tabs. + +Executing the Functional Test +----------------------------- + +To execute the func tests do the following: + +1. Start arrow_server (leave that prompt open): ``arrow_server`` + +2. On a separate command prompt navigate to. ``~/arrow_tutorial/func_test/test`` + +3. Because these tests require a web page, Arrow defaults the driver to FireFox (**Note** `Selenium Server <./arrow_getting_started.html#start-selenium>`_ must be running), type: + +:: + + arrow test-func.js --page=testMock.html --lib=test-lib.js + +*where:* ``--page=..`` tells Arrow where the *mock* page resides + +To run against PhantomJS, enter: + + :: + + arrow test-func.js --page=testMock.html --lib=test-lib.js --browser=phantomjs + +**Note** if the *--browser* parameter is used, Arrow will direct traffic to your instance of *Selenium Server* or *arrow_server* + +You do not need to provide the full URL to the mock page; Arrow takes care of that for you. + +**Note** in the commands above, we ``included`` our *test library* file as input in the ``--lib=`` param. We did this in order to satisfy test-func.js's dependency on this file. + +.. _Integration: + +Integration Tests +================= + +In Arrow, the difference between a JavaScript UI functional test and a JavaScript UI integration test is minor. From Arrow's perspective, *integration* tests *can* be functional tests executed against an HTTP End-Point. An HTTP End-Point can be an integration, staging or production environment. + +In other words, if your JavaScript UI *functional_* test is constructed smartly, you could use it for *integration* testing as well. + +TODO... needs to be updated + +Consider the `test-int.js `_ file. It confirms the tabs have specific values. For this simple app, those values *would* come from a WS or some type of integration with another system. + +Executing the Integration Test +------------------------------ + +For this example, we'll suppose our AUT is hosted elsewhere (perhaps in an integration, testing, or staging environment); http://www.doctor46.com/tabview.html + +Execution of the tests follows a familar theme: + +1. Because the AUT is hosted elsewhere, we don't need arrow_server, simply navigate to: ``~/arrow_tutorial/func_test/test`` + +2. To execute type: + +:: + + arrow test-int.js --page=http://www.doctor46.com/tabview.html --lib=test-lib.js + +**Note** the --page parameter is now pointing to an HTTP End-Point rather than our mock page + +**Note:** To run against PhantomJS, enter: + +:: + + arrow test-int.js --page=http://www.doctor46.com/tabview.html --lib=test-lib.js --browser=phantomjs + +Similarly, you can run func tests (test-func.js) against the HTTP End-Point like this: + +:: + + arrow test-func.js --page=http://www.doctor46.com/tabview.html --lib=test-lib.js + +Conclusion +========== + +As you can see, Arrow allows you to execute all types of tests (unit, functional and integration) using the same methodology. Unlike other frameworks, it does not dictate to you how to execute different tests, as far as Arrow is concerned, tests are just tests. \ No newline at end of file diff --git a/docs/arrow_cookbook/arrow_usage.rst b/docs/arrow_cookbook/arrow_usage.rst new file mode 100644 index 00000000..a9960bd8 --- /dev/null +++ b/docs/arrow_cookbook/arrow_usage.rst @@ -0,0 +1,45 @@ +========================== +Arrow Usage +========================== + +.. _Usage: + +**Usage** + +--help display this help page +--version display installed arrow version +--lib comma separated list of js files needed by the test +--page (optional) path to the mock or production html page, for example: http://www.yahoo.com or mock.html +--driver (optional) selenium|phantomjs|browser, default: phantomjs +--browser (optional) firefox|chrome|opera|reuse, default: firefox +--controller (optional) a custom controller javascript file +--reuseSession (optional) true, Used to run tests in existing sessions managed by selenium. Visit http://selenuim_host/wd/hub to setup sessions. +--report (optional) true, creates report files in junit and json format, and also prints a consolidated test report summary on console. +--testName (optional) comma separated list of test name(s) defined in test descriptor, all other tests would be ignored. +--group (optional) comma separated list of group(s) defined in test descriptor, all other groups would be ignored. +--logLevel (optional) DEBUG|INFO|WARN|ERROR|FATAL, default: INFO +--dimension (optional) a custom dimension file for defining ycb contexts +--context (optional) name of ycb context + + +.. _Examples: + +**Examples** + + Unit test: + arrow test-unit.js --lib=../src/greeter.js + + Unit test with a mock page: + arrow test-unit.js --page=testMock.html --lib=./test-lib.js + + Unit test with selenium: + arrow test-unit.js --page=testMock.html --lib=./test-lib.js --driver=selenium + + Integration test: + arrow test-int.js --page=http://www.hostname.com/testpage --lib=./test-lib.js + + Integration test: + arrow test-int.js --page=http://www.hostname.com/testpage --lib=./test-lib.js --driver=selenium + + Custom controller: + arrow --controller=custom-controller.js --driver=selenium diff --git a/docs/arrow_cookbook/arrow_videos.rst b/docs/arrow_cookbook/arrow_videos.rst new file mode 100644 index 00000000..d06345fd --- /dev/null +++ b/docs/arrow_cookbook/arrow_videos.rst @@ -0,0 +1,3 @@ +===================== +Arrow Tutorial Videos +===================== diff --git a/docs/arrow_cookbook/conf.py b/docs/arrow_cookbook/conf.py new file mode 100644 index 00000000..03dc14e7 --- /dev/null +++ b/docs/arrow_cookbook/conf.py @@ -0,0 +1,219 @@ +# +# Arrow documentation build configuration file, created by +# sphinx-quickstart on Sun Apr 22 18:07:15 2012. +# +# This file is execfile()d with the current directory set to its containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +import sys, os + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +#sys.path.insert(0, os.path.abspath('.')) + +# -- General configuration ----------------------------------------------------- + +# If your documentation needs a minimal Sphinx version, state it here. +#needs_sphinx = '1.0' + +# Add any Sphinx extension module names here, as strings. They can be extensions +# coming with Sphinx (named 'sphinx.ext.*') or your custom ones. +extensions = ['sphinx.ext.doctest', 'sphinx.ext.ifconfig'] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix of source filenames. +source_suffix = '.rst' + +# The encoding of source files. +#source_encoding = 'utf-8-sig' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +project = u'Arrow Test Framework' +copyright = u'2012, Yahoo! Inc., 2012' + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. +# +# The short X.Y version. +#version = '0.1' +# The full version, including alpha/beta/rc tags. +#release = '0.1.0.178' + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +#language = None + +# There are two options for replacing |today|: either, you set today to some +# non-false value, then it is used: +#today = '' +# Else, today_fmt is used as the format for a strftime call. +#today_fmt = '%B %d, %Y' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +exclude_patterns = ['.build'] + +# The reST default role (used for this markup: `text`) to use for all documents. +#default_role = None + +# If true, '()' will be appended to :func: etc. cross-reference text. +#add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +#add_module_names = True + +# If true, sectionauthor and moduleauthor directives will be shown in the +# output. They are ignored by default. +#show_authors = False + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +highlight_language = 'javascript' +# A list of ignored prefixes for module index sorting. +#modindex_common_prefix = [] + + +# -- Options for HTML output --------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +#html_theme = 'ydntheme' + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +#html_theme_options = {} + +# Add any paths that contain custom themes here, relative to this directory. +#html_theme_path = [''] + +# The name for this set of Sphinx documents. If None, it defaults to +# " v documentation". +html_title = 'Arrow Test Framework User Guide' + +# A shorter title for the navigation bar. Default is the same as html_title. +#html_short_title = None + +# The name of an image file (relative to this directory) to place at the top +# of the sidebar. +# html_logo = 'images/molotov-cocktail_logo.png' + +# The name of an image file (within the static path) to use as favicon of the +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 +# pixels large. +# html_favicon = 'images/Mojito.ico' + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +#html_static_path = ['.static'] + +# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, +# using the given strftime format. +#html_last_updated_fmt = '%b %d, %Y' + +# If true, SmartyPants will be used to convert quotes and dashes to +# typographically correct entities. +#html_use_smartypants = True + +# Custom sidebar templates, maps document names to template names. +# html_sidebars = { +# '**':["other_links.html"] +# } + +# Additional templates that should be rendered to pages, maps page names to +# template names. +#html_additional_pages = {} + +# If false, no module index is generated. +#html_domain_indices = True + +# If false, no index is generated. +html_use_index = True + +# If true, the index is split into individual pages for each letter. +#html_split_index = False + +# If true, links to the reST sources are added to the pages. +html_show_sourcelink = False + +# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. +html_show_sphinx = False + +# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. +html_show_copyright = True + +# If true, an OpenSearch description file will be output, and all pages will +# contain a tag referring to it. The value of this option must be the +# base URL from which the finished HTML is served. +#html_use_opensearch = '' + +# This is the file name suffix for HTML files (e.g. ".xhtml"). +#html_file_suffix = None + +# Output file base name for HTML help builder. +htmlhelp_basename = 'Arrow' + + +# -- Options for LaTeX output -------------------------------------------------- + +# The paper size ('letter' or 'a4'). +#latex_paper_size = 'letter' + +# The font size ('10pt', '11pt' or '12pt'). +#latex_font_size = '10pt' + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, author, documentclass [howto/manual]). +#latex_documents = [ +# ('index', 'Cocktails.tex', u'Cocktails Documentation', +# u'Joe Catera', 'manual'), +#] + +# The name of an image file (relative to this directory) to place at the top of +# the title page. +#latex_logo = None + +# For "manual" documents, if this is true, then toplevel headings are parts, +# not chapters. +#latex_use_parts = False + +# If true, show page references after internal links. +#latex_show_pagerefs = False + +# If true, show URL addresses after external links. +#latex_show_urls = False + +# Additional stuff for the LaTeX preamble. +#latex_preamble = '' + +# Documents to append as an appendix to all manuals. +#latex_appendices = [] + +# If false, no module index is generated. +#latex_domain_indices = True + + +# -- Options for manual page output -------------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + (u'README', 'arrow', u'run tests with arrow', + [u'Ivan Alonso', u'Pranav Verma'], 1) +] + diff --git a/docs/arrow_cookbook/final.png b/docs/arrow_cookbook/final.png new file mode 100644 index 00000000..11b1f7d4 Binary files /dev/null and b/docs/arrow_cookbook/final.png differ diff --git a/docs/arrow_cookbook/index.rst b/docs/arrow_cookbook/index.rst new file mode 100644 index 00000000..1fbd103a --- /dev/null +++ b/docs/arrow_cookbook/index.rst @@ -0,0 +1,67 @@ +================== +Introducing Arrow +================== +Arrow is a test framework designed to promote test-driven JavaScript development. Arrow provides a consistent test creation and execution environment for both JavaScript developers and quality engineers. + +Arrow erases the line between Development's unit tests and Quality engineering's functional and integration tests. Arrow provides a uniform way to create and execute all type of tests, so the purpose of the test no longer dictates how it is created and/or executed. + +Arrow enables you to: + +* Create unit tests to validate specific units of JavaScript code +* Create pre-commit functional tests against mock pages +* Reuse the tests in post-commit scenarios +* Create cross-page test scenarios + +=========================================== +The Arrow framework documentation includes: +=========================================== + +**Chapter 1: What is Arrow?** + +Describes what Arrow is and how it is different from other testing frameworks. + +**Chapter 2: Getting Started** + +Describes Arrow installation and how to create your first test + +**Chapter 3: Arrow Tutorial** + +Walks through the various types of tests commonly run by Arrow + +**Chapter 4: Arrow In-Depth** + +More in-depth information about Arrow + +**Chapter 5: Continuous Integration** + +Examples on how Arrow works within a CI environment like Hudson. Including different reports provided by Arrow. + +**Chapter 6: Extending & Developing Arrow** + +Since its inception, Arrow was designed to be extensible. Do you want to contribute to arrow? Look here. + +**Chapter 7: Videos** + +Video Tutorials + +**Chapter 8: FAQs** + +Frequently-Asked Questions + +**Chapter 9: Arrow Usage** + +Table of Contents +################# + +.. toctree:: + :maxdepth: 2 + + arrow_intro + arrow_getting_started + arrow_tutorial + arrow_in-depth + arrow_CI + arrow_extending + arrow_videos + arrow_FAQs + README diff --git a/docs/arrow_cookbook/mostpop.jpg b/docs/arrow_cookbook/mostpop.jpg new file mode 100644 index 00000000..3eb19992 Binary files /dev/null and b/docs/arrow_cookbook/mostpop.jpg differ diff --git a/docs/arrow_cookbook/starting.png b/docs/arrow_cookbook/starting.png new file mode 100644 index 00000000..d2d20393 Binary files /dev/null and b/docs/arrow_cookbook/starting.png differ diff --git a/docs/arrow_tutorial/action_test/test/locator-action.js b/docs/arrow_tutorial/action_test/test/locator-action.js new file mode 100644 index 00000000..e2e830a9 --- /dev/null +++ b/docs/arrow_tutorial/action_test/test/locator-action.js @@ -0,0 +1,52 @@ +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + +YUI.add("navigate-action", function (Y) { + var buttonNode = null; + + console.log("action initialized"); + + Y.namespace("arrow").action = { + setUp: function (callback) { + var locators, + locator, + doc = Y.one(document.body), + node, + buttonTarget, + i; + + console.log("action setUp"); + + locators = this.testParams["locators"]; + if (!locators) { + locators = [this.testParams["locator"]]; + } + + // all locators except the last one enter texts + for (i = 0; i < (locators.length - 1); i += 1) { + locator = locators[i]; + node = doc.one(locator.target); + node.set("value", locator.text); + } + + // save the buttonNode to click later + buttonTarget = locators[locators.length - 1].target; + buttonNode = doc.one(buttonTarget); + if (!buttonNode) { + callback("Could not find button node to click: " + buttonTarget); + } else { + callback(); + } + }, + + execute: function() { + console.log("action executed"); + buttonNode.simulate("click"); + } + } + +}, "0.1", {requires: ["node", "node-event-simulate"]}); + diff --git a/docs/arrow_tutorial/action_test/test/simple-action.js b/docs/arrow_tutorial/action_test/test/simple-action.js new file mode 100644 index 00000000..396d10d2 --- /dev/null +++ b/docs/arrow_tutorial/action_test/test/simple-action.js @@ -0,0 +1,24 @@ +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + +YUI.add("simple-action", function (Y) { + + console.log("action initialized"); + + Y.namespace("arrow").action = { + setUp: function (callback) { + console.log("action setUp"); + callback(); + }, + + execute: function() { + console.log("action executed"); + window.location.href = "http://www.yahoo.com"; + } + } + +}, "0.1", {requires: []}); + diff --git a/docs/arrow_tutorial/action_test/test/test-quote.js b/docs/arrow_tutorial/action_test/test/test-quote.js new file mode 100644 index 00000000..63369bae --- /dev/null +++ b/docs/arrow_tutorial/action_test/test/test-quote.js @@ -0,0 +1,18 @@ +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + + +YUI({ useBrowserConsole: true }).use("node", "test", function(Y) { + var suite = new Y.Test.Suite("Quote Page test of the test"); + suite.add(new Y.Test.Case({ + "test quote": function() { + Y.Assert.areEqual("Yahoo! Inc. (YHOO)", Y.one(".yfi_rt_quote_summary").one("h2").get('text')); + } + })); + + Y.Test.Runner.add(suite); +}); + diff --git a/docs/arrow_tutorial/action_test/test/test_descriptor.json b/docs/arrow_tutorial/action_test/test/test_descriptor.json new file mode 100755 index 00000000..735048e0 --- /dev/null +++ b/docs/arrow_tutorial/action_test/test/test_descriptor.json @@ -0,0 +1,42 @@ +[ + { + "settings": [ "master" ], + + "name": "controllers", + + "config": { + }, + + "dataprovider" : { + + "text_enter" : { + "group" : "func", + "params" :{ + "scenario": [ + { + "page": "http://finance.yahoo.com", + "action": "locator-action.js", + "locators": [ + { + "target": "#txtQuotes", + "text": "yhoo" + }, + { + "target": "#btnQuotes" + } + ] + }, + { + "test": "test-quote.js" + } + ] + } + } + } + }, + + { + "settings": [ "environment:development" ] + } + +] diff --git a/docs/arrow_tutorial/controllers/test/finance-controller.js b/docs/arrow_tutorial/controllers/test/finance-controller.js new file mode 100644 index 00000000..cd3c57b6 --- /dev/null +++ b/docs/arrow_tutorial/controllers/test/finance-controller.js @@ -0,0 +1,69 @@ +/*jslint forin:true sub:true anon:true, sloppy:true, stupid:true nomen:true, node:true continue:true*/ + +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + + +var util = require("util"); +var log4js = require("arrow").log4js; +var Controller = require("arrow").controller; + +function FinanceCustomController(testConfig,args,driver) { + Controller.call(this, testConfig,args,driver); + + this.logger = log4js.getLogger("FinanceCustomController"); +} + +/* + * All controllers MUST implement the Controller interface + */ +util.inherits(FinanceCustomController, Controller); + + +/** + * In the execute method you get full access to webdriver's methods + * Additionally, you can get a handle to the parameters in your descriptor + * file by using this.testParams + * + * Lastly, in this case, the last statement is to execute the test + * You'll note executeTest includes the same parameters as Arrow's CLI + */ +FinanceCustomController.prototype.execute = function(callback) { + var self = this; + + if(this.driver.webdriver){ + + //Get the various parameters needed from the Test Descriptor file + var txtLocator = this.testParams.txtLocator; + var typeText = this.testParams.typeText; + var btnLocator = this.testParams.btnLocator; + var page = this.testParams.page; + + //Get a handle of the WebDriver Object + var webdriver = this.driver.webdriver; + + //Open the page you want to test + webdriver.get(page).then(function() { + self.logger.info(self.config); + + //Navigate the page as necessary + webdriver.findElement(webdriver.By.css(txtLocator)).sendKeys(typeText); + webdriver.findElement(webdriver.By.css(btnLocator)).click(); + self.testParams.page=null; + webdriver.getTitle().then(function(title) { + + //Execute the test + self.driver.executeTest(self.testConfig, self.testParams, function(error, report) { + callback(); + }); + }); + }); + }else{ + this.logger.fatal("Custom Controllers are currently only supported on Selenium Browsers"); + } +} + +module.exports = FinanceCustomController; \ No newline at end of file diff --git a/docs/arrow_tutorial/controllers/test/finance-controller_descriptor.json b/docs/arrow_tutorial/controllers/test/finance-controller_descriptor.json new file mode 100644 index 00000000..89285f15 --- /dev/null +++ b/docs/arrow_tutorial/controllers/test/finance-controller_descriptor.json @@ -0,0 +1,36 @@ +[ + { + "settings": [ "master" ], + + "name": "controllers", + + "config": { + "baseUrl": "http://finance.yahoo.com" + }, + + "dataprovider" : { + + "Test YHOO Ticker using Finance Controller" : { + "group" : "func", + "params" :{ + "scenario": [ + { + "controller": "finance-controller.js", + "params": { + "page" : "$$config.baseUrl$$", + "txtLocator": "#txtQuotes", + "typeText": "yhoo", + "btnLocator": "#btnQuotes", + "test": "test-quote.js", + "quote": "Yahoo! Inc. (YHOO)" + } + } + ] + } + } + } + }, + { + "settings": [ "environment:development" ] + } +] \ No newline at end of file diff --git a/docs/arrow_tutorial/controllers/test/test-quote.js b/docs/arrow_tutorial/controllers/test/test-quote.js new file mode 100644 index 00000000..95314275 --- /dev/null +++ b/docs/arrow_tutorial/controllers/test/test-quote.js @@ -0,0 +1,20 @@ +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + +YUI({ useBrowserConsole: true }).use("node", "test", function(Y) { + var suite = new Y.Test.Suite("Quote Page test of the test"); + suite.add(new Y.Test.Case({ + "test quote": function() { + + //In order to paramertize this, instead of having a static quote, we call it from the config + var quote = this.testParams["quote"]; + Y.Assert.areEqual(quote, Y.one(".yfi_rt_quote_summary").one("h2").get('text')); + } + })); + + Y.Test.Runner.add(suite); +}); + diff --git a/docs/arrow_tutorial/controllers/test/test-title.js b/docs/arrow_tutorial/controllers/test/test-title.js new file mode 100644 index 00000000..29621fb2 --- /dev/null +++ b/docs/arrow_tutorial/controllers/test/test-title.js @@ -0,0 +1,32 @@ +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + +/* + * Like other tests, this is a YUI test module + * + */ +YUI({ useBrowserConsole: true }).use("node", "test", function(Y) { + + //We initialize the suite object as a YUI test suite and with a suite title + var suite = new Y.Test.Suite("Title test of the test"); + suite.add(new Y.Test.Case({ + + //We are creating a simple test called "test title" + "test title": function() { + + //In order to paramertize this, instead of having a static title, we call it from the config + var title = this.testParams["title"]; + + //If the title is null, set it to empty + if(!title) title = ""; + Y.Assert.areEqual(title, document.title); + } + })); + + //Never "run" the tests, simply add them to the suite. Arrow takes care of running them + Y.Test.Runner.add(suite); +}); + diff --git a/docs/arrow_tutorial/controllers/test/test_descriptor.json b/docs/arrow_tutorial/controllers/test/test_descriptor.json new file mode 100755 index 00000000..28602e24 --- /dev/null +++ b/docs/arrow_tutorial/controllers/test/test_descriptor.json @@ -0,0 +1,47 @@ +[ + { + "settings": [ "master" ], + + "name": "controllers", + + "config": { + "baseUrl": "http://finance.yahoo.com" + }, + + "dataprovider" : { + + "Test YHOO Ticker" : { + "group" : "func", + "params" :{ + "scenario": [ + { + "page": "$$config.baseUrl$$" + }, + { + "controller": "locator", + "params": { + "value": "#txtQuotes", + "text": "yhoo" + } + }, + { + "controller": "locator", + "params": { + "value": "#btnQuotes", + "click": true + } + }, + { + "test": "test-quote.js", + "quote": "Yahoo! Inc. (YHOO)" + } + ] + } + } + } + }, + { + "settings": [ "environment:development" ] + } + +] \ No newline at end of file diff --git a/docs/arrow_tutorial/controllers/yahooLogin/test-title.js b/docs/arrow_tutorial/controllers/yahooLogin/test-title.js new file mode 100644 index 00000000..a8382f7f --- /dev/null +++ b/docs/arrow_tutorial/controllers/yahooLogin/test-title.js @@ -0,0 +1,33 @@ +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + + +/* + * Like other tests, this is a YUI test module + * + */ +YUI.add("test-title-tests", function (Y) { + + //We initialize the suite object as a YUI test suite and with a suite title + var suite = new Y.Test.Suite("Title test of the test"); + suite.add(new Y.Test.Case({ + + //We are creating a simple test called "test title" + "test title": function() { + + //In order to paramertize this, instead of having a static title, we call it from the config + var title = this.testParams["title"]; + + //If the title is null, set it to empty + if(!title) title = ""; + Y.Assert.areEqual(title, document.title); + } + })); + + //Never "run" the tests, simply add them to the suite. Arrow takes care of running them + Y.Test.Runner.add(suite); +}, "0.1", {requires:["test","node"]}); + diff --git a/docs/arrow_tutorial/controllers/yahooLogin/test_descriptor.json b/docs/arrow_tutorial/controllers/yahooLogin/test_descriptor.json new file mode 100755 index 00000000..71bca722 --- /dev/null +++ b/docs/arrow_tutorial/controllers/yahooLogin/test_descriptor.json @@ -0,0 +1,53 @@ +[ + { + "settings": [ "master" ], + "name": "YahooLogin", + "config": { + "baseUrl": "https://login.yahoo.com" + }, + "dataprovider" : { + + "Use Locator to Login" : { + "group" : "func", + "params" :{ + "scenario": [ + { + "page": "$$config.baseUrl$$" + }, + { + "controller": "locator", + "params": { + "value": "#username", + "text": "arrowtestuser1" + } + }, + { + "controller": "locator", + "params": { + "value": "#passwd", + "text": "123456" + } + }, + { + "controller": "locator", + "params": { + "value": "#submit", + "click": true + } + }, + { + "page": "http://finance.yahoo.com" + }, + { + "test": "test-title.js", + "title": "Yahoo! Finance - Business Finance, Stock Market, Quotes, News" + } + ] + } + } + } + }, + { + "settings": [ "environment:development" ] + } +] \ No newline at end of file diff --git a/docs/arrow_tutorial/func_test/src/tabview.html b/docs/arrow_tutorial/func_test/src/tabview.html new file mode 100644 index 00000000..f05b2bf7 --- /dev/null +++ b/docs/arrow_tutorial/func_test/src/tabview.html @@ -0,0 +1,88 @@ + + + + + Integration test example + + + + + + + + + + + + +
+

Tab view test page

+
+ +
+
+
+
Mail
+
Finance
+
News
+
+
+ +
+
+ + + +
+
+ + Asparagus + +
+
+ + Bird + +
+
+ + Coffee + +
+
+ +
+
+ +
+
+
+ +
+ + +
+ + + + + diff --git a/docs/arrow_tutorial/func_test/src/tabview.js b/docs/arrow_tutorial/func_test/src/tabview.js new file mode 100644 index 00000000..71de298f --- /dev/null +++ b/docs/arrow_tutorial/func_test/src/tabview.js @@ -0,0 +1,17 @@ +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + +YUI.add("arrow-tabview", function (Y) { + Y.namespace("Arrow"); + + Y.Arrow.TabView = { + load : function(tabid) { + var tabview = new Y.TabView({srcNode: tabid}); + tabview.render(); + } + } +}, "0.1", {requires:["tabview"]}); + diff --git a/docs/arrow_tutorial/func_test/test/test-func.js b/docs/arrow_tutorial/func_test/test/test-func.js new file mode 100644 index 00000000..a1ab861f --- /dev/null +++ b/docs/arrow_tutorial/func_test/test/test-func.js @@ -0,0 +1,42 @@ +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + +/* + * This the skeleton of a basic func test for a multi-tab module + * It delegates actual testing to test-lib.js, which performs the actual assertions + */ +YUI.add("test-func-tests", function (Y) { + + // Get the tab view + var tabid = "#things"; + var tabNode = Y.one(document.body).one(tabid); + + //Get access to test-lib.js + var tabTest = Y.Arrow.Test.TabView; + + //Create a test suite and name it "TabView unit test suite" + var suite = new Y.Test.Suite("TabView functional test suite"); + + /* + * Add a new test, with three validations to the suite + * We are going to use the "validateStructure" and "validateSeelection" methods + * in "media-test-tabview" for the actual validation. + * Note, the values we are passing are relevant to our "mock" page (testMoke.html) + */ + suite.add(new Y.Test.Case({ + "test tab structure": function() { + tabTest.validateStructure(tabNode, ["#tab1", "#tab2", "#tab3"], ["#mod1", "#mod2", "#mod3"]); + }, + "test tab selection": function() { + tabTest.validateSelection(tabNode, "#tab2", "#mod2"); + tabTest.validateSelection(tabNode, "#tab1", "#mod1"); + } + })); + + //Never "run" the tests, simply add them to the suite. Arrow takes care of running them + Y.Test.Runner.add(suite); +}, "0.1", {requires:["test","arrow-test-tabview"]}); + diff --git a/docs/arrow_tutorial/func_test/test/test-int.js b/docs/arrow_tutorial/func_test/test/test-int.js new file mode 100644 index 00000000..da6b7c14 --- /dev/null +++ b/docs/arrow_tutorial/func_test/test/test-int.js @@ -0,0 +1,38 @@ +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + +/* + * Like test_func.js, this is just another test as far as arrow is concerned + * The difference here is the expected values of test. + * It delegates actual testing to test-lib.js, which performs the actual assertions + */ +YUI.add("test-int-tests", function (Y) { + + // render tab view + var tabid = "#things"; + var tabNode = Y.one(document.body).one(tabid); + + //Get access to test-lib.js + var testLib = Y.Arrow.Test.TabView; + + //Create a test suite and name it "TabView unit test suite" + var suite = new Y.Test.Suite("TabView Integration test suite"); + + /* + * Add a new test, with one validation to the suite + * We are going to use the "validatePresence" method to check for specific values + * Note, the values we are passing are relevant to our "integration" page (tabview.html) + */ + suite.add(new Y.Test.Case({ + "test tab presence": function() { + testLib.validatePresence(tabNode, ["Asparagus", "Bird", "Coffee"]); + } + })); + + //Never "run" the tests, simply add them to the suite. Arrow takes care of running them + Y.Test.Runner.add(suite); +}, "0.1", {requires:["test","arrow-test-tabview"]}); + diff --git a/docs/arrow_tutorial/func_test/test/test-lib.js b/docs/arrow_tutorial/func_test/test/test-lib.js new file mode 100644 index 00000000..95e6cace --- /dev/null +++ b/docs/arrow_tutorial/func_test/test/test-lib.js @@ -0,0 +1,85 @@ +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + + +/* + * The purpose of this class is to abstract the various tests you'd have on a tab-view + * There are several methods and each of them are abstracted in such a way as to + * not assume how the module was implemented + * The name of this YUI module is "media-test-tabview" + */ +YUI.add("arrow-test-tabview", function(Y) { + + var A = Y.Assert; + var L = Y.Lang; + + var self = Y.namespace("Arrow.Test").TabView = { + + //Confirms the tabs are present + validateStructure: function(node, tabs, mods) { + for(tab in tabs) { + var tabNode = node.one(tabs[tab]); + A.isNotNull(tabNode, "tab is present: " + tabs[tab]); + } + + //Confirms the first module displays correctly, with the associated content below it + for(mod in mods) { + var modNode = node.one(mods[mod]); + A.isNotNull(modNode, "module is present: " + mods[mod]); + if(0 == mod) { + A.areNotEqual(modNode.getStyle("display"), "none", "module should be displayed: " + mods[mod]); + } else { + A.areEqual(modNode.getStyle("display"), "none", "module should not be displayed: " + mods[mod]); + } + } + }, + + //Confirms each tab is represented as a link + validatePresence: function(node, tabNames) { + for(i in tabNames) { + var tabName = tabNames[i]; + var link = node.oneLink(tabName); + A.isNotNull(link, "Tab must exist: " + tabName); + } + }, + + //Clicks through the tabs and makes sure they are displayed correctly + validateSelection: function(node, tabName, modName) { + var tabNode = node.one(tabName); + var modNode = node.one(modName); + + A.areEqual(modNode.getStyle("display"), "none", "module should not be displayed: " + modName); + + A.isNotNull(tabNode, "tab is present: " + tabName); + tabNode.simulate("click"); + + A.areNotEqual(modNode.getStyle("display"), "none", "module should be displayed: " + modName); + } + + }; + + var Node = function() {} + Node.prototype.oneLink = function(linkText, tagName) { + if(!tagName) tagName = "a"; + var useRegex = !L.isString(linkText); + + var links = this.getElementsByTagName(tagName); + for(var i = 0; i < links.size(); i++) { + var link = links.item(i); + if(useRegex) { + if(linkText.test(link.get("text"))) { + return link; + } + } else if(linkText == link.get("text")) { + return link; + } + } + + return null; + } + Y.augment(Y.Node, Node); + +}, "0.1", { requires:["event", "node", "node-event-simulate", "test"]}); diff --git a/docs/arrow_tutorial/func_test/test/testMock.html b/docs/arrow_tutorial/func_test/test/testMock.html new file mode 100644 index 00000000..0c1adb94 --- /dev/null +++ b/docs/arrow_tutorial/func_test/test/testMock.html @@ -0,0 +1,72 @@ + + + + + Unit test example + + + + + + + + + + + + +
+

Unit test mock

+
+ +
+
+
+
+ +
+
+ + + +
+
ONE
+
TWO
+
THREE
+
+ +
+
+ +
+
+
+ +
+ +
+ + + + + diff --git a/docs/arrow_tutorial/func_test/test/test_descriptor.json b/docs/arrow_tutorial/func_test/test/test_descriptor.json new file mode 100755 index 00000000..642bd651 --- /dev/null +++ b/docs/arrow_tutorial/func_test/test/test_descriptor.json @@ -0,0 +1,47 @@ +[ + { + "settings": [ "master" ], + + "name" : "tabview", + + "commonlib" : "./test-lib.js", + + "config" :{ + "baseUrl" : "http://www.doctor46.com" + }, + + "dataprovider" : { + + "dom" : { + "params" : { + "test" : "test-func.js", + "page" : "testMock.html" + }, + "group" : "unit" + }, + + "dom_int" : { + "params" : { + "test" : "test-func.js", + "page" : "$$config.baseUrl$$/tabview.html" + }, + "group" : "smoke" + }, + + "int" : { + "params" : { + "test" : "test-int.js", + "page" : "$$config.baseUrl$$/tabview.html" + }, + "group" : "smoke" + } + + } + + }, + + { + "settings": [ "environment:development" ] + } + +] diff --git a/docs/arrow_tutorial/html_test/src/greeter.js b/docs/arrow_tutorial/html_test/src/greeter.js new file mode 100644 index 00000000..7edb7886 --- /dev/null +++ b/docs/arrow_tutorial/html_test/src/greeter.js @@ -0,0 +1,17 @@ +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + +YUI.add("media-greeter", function (Y) { + Y.namespace("Media"); + + var Greeter = Y.Media.Greeter = function() {}; + + //This is a simple method which takes two params, first and last name + //It returns it as lastname, firstname + Greeter.prototype.greet = function(firstName, lastName) { + return lastName + ", " + firstName; + } +}, "0.1", {requires:[]}); diff --git a/docs/arrow_tutorial/html_test/test/test.html b/docs/arrow_tutorial/html_test/test/test.html new file mode 100644 index 00000000..cfe24ba1 --- /dev/null +++ b/docs/arrow_tutorial/html_test/test/test.html @@ -0,0 +1,37 @@ + + + + + Greeter test + + + + + + +Greeter test + + + + + diff --git a/docs/arrow_tutorial/node_test/src/jsoncopy.js b/docs/arrow_tutorial/node_test/src/jsoncopy.js new file mode 100644 index 00000000..2de720cd --- /dev/null +++ b/docs/arrow_tutorial/node_test/src/jsoncopy.js @@ -0,0 +1,66 @@ +/* + * Make a deep copy of the supplied object. This function reliably copies only + * what is valid for a JSON object, array, or other element. + */ +var deepCopy = function (o) { + var newArr, ix, newObj, prop; + + if (!o || typeof o !== 'object') { + return o; + } + if (Array.isArray(o)) { + newArr = []; + for (ix = 0; ix < o.length; ix += 1) { + newArr.push(deepCopy(o[ix])); + } + return newArr; + } else { + newObj = {}; + for (prop in o) { + if (o.hasOwnProperty(prop)) { + newObj[prop] = deepCopy(o[prop]); + } + } + return newObj; + } +}; + +/* + * Make a shallow (i.e. top level only) copy of the supplied object. This + * function reliably copies only what is valid for a JSON object, array, or + * other element. + */ +var shallowCopy = function (o) { + var newObj, prop; + + if (!o || typeof o !== 'object') { + return o; + } + if (Array.isArray(o)) { + return o.slice(0); + } else { + newObj = {}; + for (prop in o) { + if (o.hasOwnProperty(prop)) { + newObj[prop] = o[prop]; + } + } + return newObj; + } +}; + +/* + * Make a copy of the supplied object, either shallow or deep, according to the + * second argument. This function reliably copies only what is valid for a JSON + * object, array, or other element. + */ +var copy = function (o, shallow) { + var copyfn = shallow ? shallowCopy : deepCopy; + + return copyfn(o); +}; + +exports.deepCopy = deepCopy; +exports.shallowCopy = shallowCopy; +exports.copy = copy; + diff --git a/docs/arrow_tutorial/node_test/test/test-unit.js b/docs/arrow_tutorial/node_test/test/test-unit.js new file mode 100644 index 00000000..808d6e7d --- /dev/null +++ b/docs/arrow_tutorial/node_test/test/test-unit.js @@ -0,0 +1,196 @@ +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + +YUI.add("media-greeter-tests", function (Y) { + + // NOTE: sourceUnderTest could also be passed as a parameter to test against dev or the + // instrumented version of the same code + var sourceUnderTest = "../src/jsoncopy.js"; + var Assert = Y.Assert; + + var testThing1, + testThing2 = null, + testThing3 = "A string", + testThing4 = 42, + testThing5 = [ "one", 2, true ], + testThing6 = { one: 1, two: "two", three: false }, + testThing7 = [ + { one1: 1, one2: "two", one3: false }, + [ "two1", 22, true ] + ], + testThing8 = { + one: { one1: 1, one2: "two", one3: false }, + two: [ "two1", 22, true ] + }; + + + var jsoncopyTestCase = new Y.Test.Case({ + name: "jsoncopy Test Case", + + testRequire: function () { + var jsoncopy = require(sourceUnderTest); + Assert.isNotNull(jsoncopy); + }, + + // Shallow copy tests + + testShallowCopySimpleTypes: function () { + var jsoncopy = require(sourceUnderTest); + + Assert.isUndefined(jsoncopy.shallowCopy(testThing1)); + Assert.isNull(jsoncopy.shallowCopy(testThing2)); + Assert.areSame(testThing3, jsoncopy.shallowCopy(testThing3)); + Assert.areSame(testThing4, jsoncopy.shallowCopy(testThing4)); + }, + + testShallowCopySimpleArray: function () { + var jsoncopy = require(sourceUnderTest), + actual, + i; + + actual = jsoncopy.shallowCopy(testThing5); + Assert.isArray(actual); + Assert.areNotEqual(testThing5, actual); + Assert.areSame(testThing5.length, actual.length); + for (i = 0; i < actual.length; i++) { + Assert.areSame(testThing5[i], actual[i]); + } + }, + + testShallowCopySimpleObject: function () { + var jsoncopy = require(sourceUnderTest), + actual; + + actual = jsoncopy.shallowCopy(testThing6); + Assert.isObject(actual); + Assert.areNotEqual(testThing6, actual); + Assert.areSame(Object.keys(testThing6).length, Object.keys(actual).length); + Object.keys(actual).forEach(function (key) { + Assert.areSame(testThing6[key], actual[key]); + }); + }, + + testShallowCopyComplexArray: function () { + var jsoncopy = require(sourceUnderTest), + actual; + + actual = jsoncopy.shallowCopy(testThing7); + Assert.isArray(actual); + Assert.areNotEqual(testThing7, actual); + Assert.areSame(testThing7.length, actual.length); + Assert.areSame(testThing7[0], actual[0]); + Assert.areSame(testThing7[1], actual[1]); + }, + + testShallowCopyComplexObject: function () { + var jsoncopy = require(sourceUnderTest), + actual; + + actual = jsoncopy.shallowCopy(testThing8); + Assert.isObject(actual); + Assert.areNotEqual(testThing8, actual); + Assert.areSame(Object.keys(testThing8).length, Object.keys(actual).length); + Assert.areSame(testThing8.one, actual.one); + Assert.areSame(testThing8.two, actual.two); + }, + + // Deep copy tests + + testDeepCopySimpleTypes: function () { + var jsoncopy = require(sourceUnderTest); + + Assert.isUndefined(jsoncopy.deepCopy(testThing1)); + Assert.isNull(jsoncopy.deepCopy(testThing2)); + Assert.areSame(testThing3, jsoncopy.deepCopy(testThing3)); + Assert.areSame(testThing4, jsoncopy.deepCopy(testThing4)); + }, + + testDeepCopySimpleArray: function () { + var jsoncopy = require(sourceUnderTest), + actual, + i; + + actual = jsoncopy.deepCopy(testThing5); + Assert.isArray(actual); + Assert.areNotEqual(testThing5, actual); + Assert.areSame(testThing5.length, actual.length); + for (i = 0; i < actual.length; i++) { + Assert.areSame(testThing5[i], actual[i]); + } + }, + + testDeepCopySimpleObject: function () { + var jsoncopy = require(sourceUnderTest), + actual; + + actual = jsoncopy.deepCopy(testThing6); + Assert.isObject(actual); + Assert.areNotEqual(testThing6, actual); + Assert.areSame(Object.keys(testThing6).length, Object.keys(actual).length); + Object.keys(actual).forEach(function (key) { + Assert.areSame(testThing6[key], actual[key]); + }); + }, + + testDeepCopyComplexArray: function () { + var jsoncopy = require(sourceUnderTest), + actual, + i; + + actual = jsoncopy.deepCopy(testThing7); + Assert.isArray(actual); + Assert.areNotEqual(testThing7, actual); + Assert.areSame(testThing7.length, actual.length); + Assert.areNotSame(testThing7[0], actual[0]); + Object.keys(actual[0]).forEach(function (key) { + Assert.areSame(testThing7[0][key], actual[0][key]); + }); + Assert.areNotSame(testThing7[1], actual[1]); + for (i = 0; i < actual[1].length; i++) { + Assert.areSame(testThing7[1][i], actual[1][i]); + } + }, + + testDeepCopyComplexObject: function () { + var jsoncopy = require(sourceUnderTest), + actual, + i; + + actual = jsoncopy.deepCopy(testThing8); + Assert.isObject(actual); + Assert.areNotEqual(testThing8, actual); + Assert.areSame(Object.keys(testThing8).length, Object.keys(actual).length); + Assert.areNotSame(testThing8.one, actual.one); + Object.keys(actual.one).forEach(function (key) { + Assert.areSame(testThing8.one[key], actual.one[key]); + }); + Assert.areNotSame(testThing8.two, actual.two); + for (i = 0; i < actual.two.length; i++) { + Assert.areSame(testThing8.two[i], actual.two[i]); + } + }, + + // Parameterized copy tests + + testCopySimpleTypes: function () { + var jsoncopy = require(sourceUnderTest); + + Assert.isUndefined(jsoncopy.copy(testThing1, false)); + Assert.isNull(jsoncopy.copy(testThing2, false)); + Assert.areSame(testThing3, jsoncopy.copy(testThing3, false)); + Assert.areSame(testThing4, jsoncopy.copy(testThing4, false)); + + Assert.isUndefined(jsoncopy.copy(testThing1, true)); + Assert.isNull(jsoncopy.copy(testThing2, true)); + Assert.areSame(testThing3, jsoncopy.copy(testThing3, true)); + Assert.areSame(testThing4, jsoncopy.copy(testThing4, true)); + } + }); + + Y.Test.Runner.add(jsoncopyTestCase); + + +}, "0.1", {requires:["test"]}); diff --git a/docs/arrow_tutorial/unit_test/src/greeter.js b/docs/arrow_tutorial/unit_test/src/greeter.js new file mode 100644 index 00000000..7edb7886 --- /dev/null +++ b/docs/arrow_tutorial/unit_test/src/greeter.js @@ -0,0 +1,17 @@ +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + +YUI.add("media-greeter", function (Y) { + Y.namespace("Media"); + + var Greeter = Y.Media.Greeter = function() {}; + + //This is a simple method which takes two params, first and last name + //It returns it as lastname, firstname + Greeter.prototype.greet = function(firstName, lastName) { + return lastName + ", " + firstName; + } +}, "0.1", {requires:[]}); diff --git a/docs/arrow_tutorial/unit_test/test/test-unit.js b/docs/arrow_tutorial/unit_test/test/test-unit.js new file mode 100644 index 00000000..89bcaa28 --- /dev/null +++ b/docs/arrow_tutorial/unit_test/test/test-unit.js @@ -0,0 +1,26 @@ +/* + * Copyright (c) 2012, Yahoo! Inc. All rights reserved. + * Copyrights licensed under the New BSD License. + * See the accompanying LICENSE file for terms. + */ + +YUI.add("media-greeter-tests", function (Y) { + //Create a basic test suite + //We're calling it "unit test suite" + var suite = new Y.Test.Suite("unit test suite"); + + //Add a test case to the suite; "test greet" + suite.add(new Y.Test.Case({ + "test greet": function() { + var greeter = new Y.Media.Greeter(); + + //The method we are testing will inverse the firstname and lastname + //Our test will check for that inversion + Y.Assert.areEqual(greeter.greet("Joe", "Smith"), "Smith, Joe"); + } + })); + + //Note we are not "running" the suite. + //Arrow will take care of that. We simply need to "add" it to the runner + Y.Test.Runner.add(suite); +}, "0.1", {requires:["test","media-greeter"]});