New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow running of additional PhantomJS code before pa11y runs #98

Closed
rowanmanning opened this Issue Oct 8, 2015 · 20 comments

Comments

Projects
None yet
5 participants
@rowanmanning
Member

rowanmanning commented Oct 8, 2015

Currently there are some issues around login forms and pages that rely on a lot of JavaScript to render. We have the wait option for the latter, but it's not ideal.

Proposing adding the ability to run additional custom PhantomJS code before pa11y to prepare the page for testing. Not sure how this would look yet, but I'll throw together some ideas when I have time.

GlynnPhillips pushed a commit that referenced this issue Dec 19, 2015

Glynn Phillips
Adding an option for running JS before testing
This is an attempt to add the new functionality suggested in issue #98.

This will add the `--javascript` option to pass in a file path to a
JavaScript file to be run pa11y is run.

That JavaScript file has to `console.log('pa11y: script complete')`
before pa11y will be run.

I am not 100% sure how I feel about this `console.log` approach but it
felt like it kept the supplied JS simple and without any phantomjs
knowledge required compared to the callback approach.

Tests still need to be written.

GlynnPhillips pushed a commit that referenced this issue Dec 20, 2015

Glynn Phillips
Adding an option for running JS before testing
This is an attempt to add the new functionality suggested in issue #98.

This will add the `--injectJs` option to pass in a file path to a
JavaScript file to be run pa11y is run.

That JavaScript file has to `console.log('pa11y: script complete')`
before pa11y will be run.

I am not 100% sure how I feel about this `console.log` approach but it
felt like it kept the supplied JS simple and without any phantomjs
knowledge required compared to the callback approach.

Tests still need to be written.
@GlynnPhillips

This comment has been minimized.

Show comment
Hide comment
@GlynnPhillips

GlynnPhillips Dec 21, 2015

Member

I started looking at this but then realised my solution was not the best for all cases so it might be helpful to figure out what the user cases we are trying to cover. A couple I can think of:

  • Allowing for UI changes e.g. clicking on an element to open a hidden section
  • Adding data and submitting form elements e.g. Log in
  • Allowing for ajax requests to be made to update content

Some of the problems I was hitting when thinking about these was how to pass back a message to pa11y that the script had finished and the testing could continue. Phantom has a couple options which could work like onCallback and onConsoleMessage but I couldn't make my mind up as to which would be right in this case.

It also might be worth defining if these interactions should/would allow for page refreshes or location changes so this might effect the way in which the message is passed back that the script has finished, or might not be possible in cases.

Member

GlynnPhillips commented Dec 21, 2015

I started looking at this but then realised my solution was not the best for all cases so it might be helpful to figure out what the user cases we are trying to cover. A couple I can think of:

  • Allowing for UI changes e.g. clicking on an element to open a hidden section
  • Adding data and submitting form elements e.g. Log in
  • Allowing for ajax requests to be made to update content

Some of the problems I was hitting when thinking about these was how to pass back a message to pa11y that the script had finished and the testing could continue. Phantom has a couple options which could work like onCallback and onConsoleMessage but I couldn't make my mind up as to which would be right in this case.

It also might be worth defining if these interactions should/would allow for page refreshes or location changes so this might effect the way in which the message is passed back that the script has finished, or might not be possible in cases.

@JeffHall

This comment has been minimized.

Show comment
Hide comment
@JeffHall

JeffHall Dec 28, 2015

I've just started experimenting with pa11y, and I'm very impressed by what I've seen so far!

So the obvious caveat is that I'm a newbie, but in thinking about this issue and the user cases that @GlynnPhillips mentions above, these are the ones that jump out at me:

  • Allowing for UI changes e.g. clicking on an element to open a hidden section
  • Allowing for ajax requests to be made to update content

The AngularJS app that I'm working with has several ngClick directives that change the state of the page display (essentially show/hide behaviors), so using PhantomJS to navigate to the expected page "state" would be ideal before the pa11y verification itself is run.

Now my challenge is to learn enough JavaScript and Node so that I can actually contribute to this great project!

JeffHall commented Dec 28, 2015

I've just started experimenting with pa11y, and I'm very impressed by what I've seen so far!

So the obvious caveat is that I'm a newbie, but in thinking about this issue and the user cases that @GlynnPhillips mentions above, these are the ones that jump out at me:

  • Allowing for UI changes e.g. clicking on an element to open a hidden section
  • Allowing for ajax requests to be made to update content

The AngularJS app that I'm working with has several ngClick directives that change the state of the page display (essentially show/hide behaviors), so using PhantomJS to navigate to the expected page "state" would be ideal before the pa11y verification itself is run.

Now my challenge is to learn enough JavaScript and Node so that I can actually contribute to this great project!

@rowanmanning

This comment has been minimized.

Show comment
Hide comment
@rowanmanning

rowanmanning Jan 27, 2016

Member

After a bit of thought I think that the simplest way to do this would be to give the end user a place to hook into PhantomJS directly. We could just stick an extra function in here which just runs the user code if there is some, then they can do whatever the hell they want and we cater for all the above use-cases (and more that we can't think of).

API-wise it'd be nice to do something like:

Command Line

pa11y --extras ./my-function.js nature.com

Where ./my-function.js is a Node.js module that exports a single function (see below for the expected args for that function).

JavaScript

pa11y({
    extras: function(page, done) {
        // do stuff here
    }
});

Where page is the PhantomJS page you're working with, and done is a callback to trigger when you've finished.

Not convinced on extras as a name, but you get the picture. @GlynnPhillips, @JeffHall, thoughts on this?

Member

rowanmanning commented Jan 27, 2016

After a bit of thought I think that the simplest way to do this would be to give the end user a place to hook into PhantomJS directly. We could just stick an extra function in here which just runs the user code if there is some, then they can do whatever the hell they want and we cater for all the above use-cases (and more that we can't think of).

API-wise it'd be nice to do something like:

Command Line

pa11y --extras ./my-function.js nature.com

Where ./my-function.js is a Node.js module that exports a single function (see below for the expected args for that function).

JavaScript

pa11y({
    extras: function(page, done) {
        // do stuff here
    }
});

Where page is the PhantomJS page you're working with, and done is a callback to trigger when you've finished.

Not convinced on extras as a name, but you get the picture. @GlynnPhillips, @JeffHall, thoughts on this?

@GlynnPhillips

This comment has been minimized.

Show comment
Hide comment
@GlynnPhillips

GlynnPhillips Jan 27, 2016

Member

That sounds good to me and solves the issues I was having with how to let pa11y know when the script is finished.

I had previously played with adding the function at pretty much the exact point you referenced inside lib/pally.js. Here is an example of what I had.

function(next) {
    if (options.injectJs) {
        options.log.debug('Injecting supplied JavaScript');
        page.injectJs(options.injectJs, function(error, injected) {
            if (error) {
                return next(error);
            }
            if (!injected) {
                return next(new Error('Pa11y was unable to inject supplied scripts into the page'));
            }

        });

        // Here I was playing with different ways of knowing when the script had finished such as onCallback or onConsoleMessage before calling next
        next();
    } else {
        next();
    }
}
Member

GlynnPhillips commented Jan 27, 2016

That sounds good to me and solves the issues I was having with how to let pa11y know when the script is finished.

I had previously played with adding the function at pretty much the exact point you referenced inside lib/pally.js. Here is an example of what I had.

function(next) {
    if (options.injectJs) {
        options.log.debug('Injecting supplied JavaScript');
        page.injectJs(options.injectJs, function(error, injected) {
            if (error) {
                return next(error);
            }
            if (!injected) {
                return next(new Error('Pa11y was unable to inject supplied scripts into the page'));
            }

        });

        // Here I was playing with different ways of knowing when the script had finished such as onCallback or onConsoleMessage before calling next
        next();
    } else {
        next();
    }
}
@rowanmanning

This comment has been minimized.

Show comment
Hide comment
@rowanmanning

rowanmanning Jan 27, 2016

Member

Oh yeah, cool. Well it's similar but the person writing the code is responsible for everything, and can set their own timeouts etc :)

Member

rowanmanning commented Jan 27, 2016

Oh yeah, cool. Well it's similar but the person writing the code is responsible for everything, and can set their own timeouts etc :)

@GlynnPhillips

This comment has been minimized.

Show comment
Hide comment
@GlynnPhillips

GlynnPhillips Jan 27, 2016

Member

Ohh yeah this was doing that as well I think (Maybe I am wrong) but I just removed the section where I was handling the callback from the user and added that comment because I didn't feel my handling of that callback was very elegant

Member

GlynnPhillips commented Jan 27, 2016

Ohh yeah this was doing that as well I think (Maybe I am wrong) but I just removed the section where I was handling the callback from the user and added that comment because I didn't feel my handling of that callback was very elegant

@qaDream77

This comment has been minimized.

Show comment
Hide comment
@qaDream77

qaDream77 Jan 27, 2016

My biggest concern is related to authentication, although i do use quite a bit of angular. I was wondering how this method/solution would work with CAS or LDAP authentication which has redirects?

Example.

www.mytestsite.com/myapp/login.html
i enter my username, password and any other parameters and post this page. This will redirect me to an intermediary page which generates a token and then to

www.mytestsite.com/myapp/index.html

at this point i want pa11t to scan
www.mytestsite.com/myapp/index.html
www.mytestsite.com/myapp/contact.html
etc
as an authenticated user, should still be using the same session, cookies, etc that were generated on a successful authentication.

My question is, ideally all of this would be handled by phantomjs, which i don't think should be a problem, right? Pa11y would then just execute on the current page?

Cheers.

FYI: @GlynnPhillips if you need any help, let me know.

qaDream77 commented Jan 27, 2016

My biggest concern is related to authentication, although i do use quite a bit of angular. I was wondering how this method/solution would work with CAS or LDAP authentication which has redirects?

Example.

www.mytestsite.com/myapp/login.html
i enter my username, password and any other parameters and post this page. This will redirect me to an intermediary page which generates a token and then to

www.mytestsite.com/myapp/index.html

at this point i want pa11t to scan
www.mytestsite.com/myapp/index.html
www.mytestsite.com/myapp/contact.html
etc
as an authenticated user, should still be using the same session, cookies, etc that were generated on a successful authentication.

My question is, ideally all of this would be handled by phantomjs, which i don't think should be a problem, right? Pa11y would then just execute on the current page?

Cheers.

FYI: @GlynnPhillips if you need any help, let me know.

@GlynnPhillips

This comment has been minimized.

Show comment
Hide comment
@GlynnPhillips

GlynnPhillips Feb 9, 2016

Member

We have now added this functionality as part of 3.5.0 and you can see the changes here

@qaDream77 I believe this should cover your user case and you can see an example we have put together in the commonly asked questions here

Cheers

Member

GlynnPhillips commented Feb 9, 2016

We have now added this functionality as part of 3.5.0 and you can see the changes here

@qaDream77 I believe this should cover your user case and you can see an example we have put together in the commonly asked questions here

Cheers

@qaDream77

This comment has been minimized.

Show comment
Hide comment
@qaDream77

qaDream77 Mar 15, 2016

Thank you very much! I'm going back to this now :) I have modified the example above to suit my website, but how do i run it?

phantomjs login_test.js

This throws a reference error: can't find variable pa11y

qaDream77 commented Mar 15, 2016

Thank you very much! I'm going back to this now :) I have modified the example above to suit my website, but how do i run it?

phantomjs login_test.js

This throws a reference error: can't find variable pa11y

@GlynnPhillips

This comment has been minimized.

Show comment
Hide comment
@GlynnPhillips

GlynnPhillips Mar 16, 2016

Member

Hi @qaDream77 you need to run pa11y with the beforeScript option set within your config.

If you are running pa11y via the command line here is an example of how that can be achieved within a config file.

config.js file example

module.exports = {
    beforeScript: function(page, options, next) {
        var waitUntil = function(condition, retries, waitOver) {
            page.evaluate(condition, function(error, result) {
                if (result || retries < 1) {
                    waitOver();
                } else {
                    retries -= 1;
                    setTimeout(function() {
                        waitUntil(condition, retries, waitOver);
                    }, 200);
                }
            });
        };

        page.evaluate(function() {
            var user = document.querySelector('#username');
            var password = document.querySelector('#password');
            var submit = document.querySelector('#submit');

            user.value = 'exampleUser';
            password.value = 'password1234';

            submit.click();

        }, function() {

            waitUntil(function() {
                return window.location.href === 'http://example.com/myaccount';
            }, 20, next);
        });
    }
}

Command example

pa11y --config ./path/to/config.js example.com/login
Member

GlynnPhillips commented Mar 16, 2016

Hi @qaDream77 you need to run pa11y with the beforeScript option set within your config.

If you are running pa11y via the command line here is an example of how that can be achieved within a config file.

config.js file example

module.exports = {
    beforeScript: function(page, options, next) {
        var waitUntil = function(condition, retries, waitOver) {
            page.evaluate(condition, function(error, result) {
                if (result || retries < 1) {
                    waitOver();
                } else {
                    retries -= 1;
                    setTimeout(function() {
                        waitUntil(condition, retries, waitOver);
                    }, 200);
                }
            });
        };

        page.evaluate(function() {
            var user = document.querySelector('#username');
            var password = document.querySelector('#password');
            var submit = document.querySelector('#submit');

            user.value = 'exampleUser';
            password.value = 'password1234';

            submit.click();

        }, function() {

            waitUntil(function() {
                return window.location.href === 'http://example.com/myaccount';
            }, 20, next);
        });
    }
}

Command example

pa11y --config ./path/to/config.js example.com/login
@rowanmanning

This comment has been minimized.

Show comment
Hide comment
@rowanmanning

rowanmanning Mar 16, 2016

Member

@GlynnPhillips: your example JS file needs a module.exports, otherwise 👌

Member

rowanmanning commented Mar 16, 2016

@GlynnPhillips: your example JS file needs a module.exports, otherwise 👌

@GlynnPhillips

This comment has been minimized.

Show comment
Hide comment
@GlynnPhillips

GlynnPhillips Mar 16, 2016

Member

oops, silly copy and paste error. Thanks and good catch Rowan.

Now updated

Member

GlynnPhillips commented Mar 16, 2016

oops, silly copy and paste error. Thanks and good catch Rowan.

Now updated

@GlynnPhillips

This comment has been minimized.

Show comment
Hide comment
@GlynnPhillips

GlynnPhillips Mar 16, 2016

Member

Sorry @qaDream77 maybe it would be clearer is my example referenced example.com all the time instead of nature.com. I will change the above now to make that clearer

Your example is still trying to do everything with phantomjs when there is no need. pa11y spins up an instance of phantomjs for you so there is no need for some of the above.

Also from your example I guess you are trying to use pa11y from within a node/JavaScript project and not from the command line. Can you confirm for me if this is correct or if you just want to run a command on the command line?

Then I can provide an example with comments on how to do this.

Member

GlynnPhillips commented Mar 16, 2016

Sorry @qaDream77 maybe it would be clearer is my example referenced example.com all the time instead of nature.com. I will change the above now to make that clearer

Your example is still trying to do everything with phantomjs when there is no need. pa11y spins up an instance of phantomjs for you so there is no need for some of the above.

Also from your example I guess you are trying to use pa11y from within a node/JavaScript project and not from the command line. Can you confirm for me if this is correct or if you just want to run a command on the command line?

Then I can provide an example with comments on how to do this.

@qaDream77

This comment has been minimized.

Show comment
Hide comment
@qaDream77

qaDream77 Mar 16, 2016

I accidentally deleted the wrong comment :( Sorry!

I am trying to run form command line :) I modified your above code to the following:

config.js


module.exports = {
    beforeScript: function(page, options, next) {
        var waitUntil = function(condition, retries, waitOver) {
            page.evaluate(condition, function(error, result) {
                if (result || retries < 1) {
                    waitOver();
                } else {
                    retries -= 1;
                    setTimeout(function() {
                        waitUntil(condition, retries, waitOver);
                    }, 200);
                }
            });
        };

        page.evaluate(function() {
            var user = document.querySelector('#username');
            var password = document.querySelector('#password');
            var submit = document.querySelector('#login');

            user.value = 'user1';
            password.value = 'abc';

            submit.submit();
        }, function() {
            waitUntil(function() {
                return window.location.href === 'http://example.com/app/';
            }, 20, next);
        });
    }
}

The commands to test different pages are:

pa11y --config ./path/to/config.js "http://example.com/app/page_1.html"
pa11y --config ./path/to/config.js "http://example.com/app/page_2.html"
pa11y --config ./path/to/config.js "http://example.com/app/page_3.html"
pa11y --config ./path/to/config.js "http://example.com/app/page_4.html"

This works for me! Thank you! One question is, where do i add a sanity check? I would like phantom to take a screenshot of the loaded page!

page.render("page.png");

qaDream77 commented Mar 16, 2016

I accidentally deleted the wrong comment :( Sorry!

I am trying to run form command line :) I modified your above code to the following:

config.js


module.exports = {
    beforeScript: function(page, options, next) {
        var waitUntil = function(condition, retries, waitOver) {
            page.evaluate(condition, function(error, result) {
                if (result || retries < 1) {
                    waitOver();
                } else {
                    retries -= 1;
                    setTimeout(function() {
                        waitUntil(condition, retries, waitOver);
                    }, 200);
                }
            });
        };

        page.evaluate(function() {
            var user = document.querySelector('#username');
            var password = document.querySelector('#password');
            var submit = document.querySelector('#login');

            user.value = 'user1';
            password.value = 'abc';

            submit.submit();
        }, function() {
            waitUntil(function() {
                return window.location.href === 'http://example.com/app/';
            }, 20, next);
        });
    }
}

The commands to test different pages are:

pa11y --config ./path/to/config.js "http://example.com/app/page_1.html"
pa11y --config ./path/to/config.js "http://example.com/app/page_2.html"
pa11y --config ./path/to/config.js "http://example.com/app/page_3.html"
pa11y --config ./path/to/config.js "http://example.com/app/page_4.html"

This works for me! Thank you! One question is, where do i add a sanity check? I would like phantom to take a screenshot of the loaded page!

page.render("page.png");

@GlynnPhillips

This comment has been minimized.

Show comment
Hide comment
@GlynnPhillips

GlynnPhillips Mar 16, 2016

Member

So the problem is with your call to the waitUntil function.

You are returning whenever the url is equal to https://sample.ca/app/client/page_one.html and this is the same as the url that you are passing to pa11y on the command line.

This check of the url is done to validate that the login action has completed. In your example pa11y will continue and run the tests straight away before it has had time to perform your login because the url is already equal to https://sample.ca/app/client/page_one.html.

If on login your site doesn't redirect to a different url then you will need to use something else to validate that the login has been completed, for instance a using an element in the page that might be different once logged in.

This could be checking for the existence of an element that is not on the page until the user is logged in. Without knowing your code it might look something like this.

waitUntil(function() {
    // Element that doesn't exist until logged in
    var exampleElement = document.querySelector('#exampleElement');
    // returns true when element exists
    return exampleElement !== null;
}, 20, next);
Member

GlynnPhillips commented Mar 16, 2016

So the problem is with your call to the waitUntil function.

You are returning whenever the url is equal to https://sample.ca/app/client/page_one.html and this is the same as the url that you are passing to pa11y on the command line.

This check of the url is done to validate that the login action has completed. In your example pa11y will continue and run the tests straight away before it has had time to perform your login because the url is already equal to https://sample.ca/app/client/page_one.html.

If on login your site doesn't redirect to a different url then you will need to use something else to validate that the login has been completed, for instance a using an element in the page that might be different once logged in.

This could be checking for the existence of an element that is not on the page until the user is logged in. Without knowing your code it might look something like this.

waitUntil(function() {
    // Element that doesn't exist until logged in
    var exampleElement = document.querySelector('#exampleElement');
    // returns true when element exists
    return exampleElement !== null;
}, 20, next);
@qaDream77

This comment has been minimized.

Show comment
Hide comment
@qaDream77

qaDream77 Mar 16, 2016

To answer your previous question, i know i can look for a element on the screen but i would prefer to be able to take a screen shot, if this is not doable that is fine. I was just wondering if it was :)

With phantomjs, it would be page.render("page.png");

Thanks for all the great help! I'll be looking into the ajax part of it now, and let you know if i have any issues.

EDIT: I just want to add GREAT work on this, very well done =D

qaDream77 commented Mar 16, 2016

To answer your previous question, i know i can look for a element on the screen but i would prefer to be able to take a screen shot, if this is not doable that is fine. I was just wondering if it was :)

With phantomjs, it would be page.render("page.png");

Thanks for all the great help! I'll be looking into the ajax part of it now, and let you know if i have any issues.

EDIT: I just want to add GREAT work on this, very well done =D

@qaDream77

This comment has been minimized.

Show comment
Hide comment
@qaDream77

qaDream77 Mar 16, 2016

Just one more question about config.json! The values for username and password, as well as the url. Is there any ways to pass those in as command line arguments?

Want to write a shell script to go through a list of websites and perform a scan on them!

qaDream77 commented Mar 16, 2016

Just one more question about config.json! The values for username and password, as well as the url. Is there any ways to pass those in as command line arguments?

Want to write a shell script to go through a list of websites and perform a scan on them!

@GlynnPhillips

This comment has been minimized.

Show comment
Hide comment
@GlynnPhillips

GlynnPhillips Mar 16, 2016

Member

Ok, so I was answering your previous questions which you accidentally deleted where you mentioned that it was taking the screen shot and running pa11y before the login had completed which is why I suggested checking for an element instead of checking the url as it seemed like your url didn't change.

Anyway you have something working now so onto your next question about taking a screen shot. You would want to do this right before waitOver() is called, like the below.

var waitUntil = function(condition, retries, waitOver) {
   page.evaluate(condition, function(error, result) {
       if (result || retries < 1) {
             page.render('page.png');
             waitOver();
        } else {
            retries -= 1;
            setTimeout(function() {
                waitUntil(condition, retries, waitOver);
            }, 200);
        }
    });
};

As far as I am aware there is no way to pass in the credentials as an argument or environment variable. This would be handled by using separate config files or logic in your config file to determine which credentials to use.

Member

GlynnPhillips commented Mar 16, 2016

Ok, so I was answering your previous questions which you accidentally deleted where you mentioned that it was taking the screen shot and running pa11y before the login had completed which is why I suggested checking for an element instead of checking the url as it seemed like your url didn't change.

Anyway you have something working now so onto your next question about taking a screen shot. You would want to do this right before waitOver() is called, like the below.

var waitUntil = function(condition, retries, waitOver) {
   page.evaluate(condition, function(error, result) {
       if (result || retries < 1) {
             page.render('page.png');
             waitOver();
        } else {
            retries -= 1;
            setTimeout(function() {
                waitUntil(condition, retries, waitOver);
            }, 200);
        }
    });
};

As far as I am aware there is no way to pass in the credentials as an argument or environment variable. This would be handled by using separate config files or logic in your config file to determine which credentials to use.

@JordanGS

This comment has been minimized.

Show comment
Hide comment
@JordanGS

JordanGS Mar 16, 2016

@GlynnPhillips

I am so glad that you guys have implemented this! I have been tracking this project for a while now and and super excited that i am able to authenticate into my website! Thank you very much!

Sorry if i am interrupting on this discussion but i wanted to say that I like the idea of using screenshots, it allows for validation when integrating in Jenkins, so you can manually confirm that it authenticated!

I would like to ask one thing though, how would you analyze the following:

Say you have page_one.html and page_two.html but you cannot get to two unless page one was submitted with valid form data. I understand this is a bit more complicated and i understand how to do this in phantomjs, a little unsure though how to do multiple redirects in the config.js script. Also confused what my two URL's would be in this instance because if you try to go to page_two.html before page_one.html is validated, then you get an error_page.html.

Sorry for my English, please let me know if my question is unclear!

@qaDream77 Create a python script or shell script and generate a temporary file called temp.js with command line arguments injected in, keep your urls in an array and just iterate. Make sure to pipe the output to a unique out file each time. That's the easiest thing i can think of.

JordanGS commented Mar 16, 2016

@GlynnPhillips

I am so glad that you guys have implemented this! I have been tracking this project for a while now and and super excited that i am able to authenticate into my website! Thank you very much!

Sorry if i am interrupting on this discussion but i wanted to say that I like the idea of using screenshots, it allows for validation when integrating in Jenkins, so you can manually confirm that it authenticated!

I would like to ask one thing though, how would you analyze the following:

Say you have page_one.html and page_two.html but you cannot get to two unless page one was submitted with valid form data. I understand this is a bit more complicated and i understand how to do this in phantomjs, a little unsure though how to do multiple redirects in the config.js script. Also confused what my two URL's would be in this instance because if you try to go to page_two.html before page_one.html is validated, then you get an error_page.html.

Sorry for my English, please let me know if my question is unclear!

@qaDream77 Create a python script or shell script and generate a temporary file called temp.js with command line arguments injected in, keep your urls in an array and just iterate. Make sure to pipe the output to a unique out file each time. That's the easiest thing i can think of.

@GlynnPhillips

This comment has been minimized.

Show comment
Hide comment
@GlynnPhillips

GlynnPhillips Mar 17, 2016

Member

Hi @JordanGS no worries, if I am understanding your problem correctly it should be pretty straight forward to achieve using the login example in the FAQ's.

When running pa11y from the command line you would pass it the first url which contains the form that needs to be submitted with valid data.

pa11y --config ~/path/to/config.js http://example.com/page_one.html

Then your config file would look something like this

module.exports = {
    beforeScript: function(page, options, next) {
        var waitUntil = function(condition, retries, waitOver) {
            page.evaluate(condition, function(error, result) {
                if (result || retries < 1) {
                    waitOver();
                } else {
                    retries -= 1;
                    setTimeout(function() {
                        waitUntil(condition, retries, waitOver);
                    }, 200);
                }
            });
        };

        page.evaluate(function() {
            // Code to populate and submit your form with valid data
        }, function() {
            waitUntil(function() {
                return window.location.href === 'http://example.com/page_two.html';
            }, 50, next);
        });
    }
}

The waitUntil function here is checking that the window has redirected to the second url. When this is equal to true the beforeScript is complete and will make its callback to pa11y allowing the pa11y tests to be run.

I hope that makes sense.

Member

GlynnPhillips commented Mar 17, 2016

Hi @JordanGS no worries, if I am understanding your problem correctly it should be pretty straight forward to achieve using the login example in the FAQ's.

When running pa11y from the command line you would pass it the first url which contains the form that needs to be submitted with valid data.

pa11y --config ~/path/to/config.js http://example.com/page_one.html

Then your config file would look something like this

module.exports = {
    beforeScript: function(page, options, next) {
        var waitUntil = function(condition, retries, waitOver) {
            page.evaluate(condition, function(error, result) {
                if (result || retries < 1) {
                    waitOver();
                } else {
                    retries -= 1;
                    setTimeout(function() {
                        waitUntil(condition, retries, waitOver);
                    }, 200);
                }
            });
        };

        page.evaluate(function() {
            // Code to populate and submit your form with valid data
        }, function() {
            waitUntil(function() {
                return window.location.href === 'http://example.com/page_two.html';
            }, 50, next);
        });
    }
}

The waitUntil function here is checking that the window has redirected to the second url. When this is equal to true the beforeScript is complete and will make its callback to pa11y allowing the pa11y tests to be run.

I hope that makes sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment