New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

w3c tests out-of-date and failing #314

Open
appsforartists opened this Issue Sep 21, 2016 · 2 comments

Comments

Projects
None yet
1 participant
@appsforartists
Contributor

appsforartists commented Sep 21, 2016

tests/w3c seems to be populated by the SHA referenced in package.json['web-platform-tests'].repo. It's currently pointing at a really old SHA (from 2015). When I try to update it, I receive this error:

Running "pretest" task
Unhandled rejection Error: EISDIR: illegal operation on a directory, open '~/Projects/PEP/tests/w3c/resources'
    at Error (native)

If I kill the process and run it again, it goes further, but most of the tests fail:

NoSuchElement: [POST http://localhost:4444/wd/hub/session/f93260d5-52de-439a-bf86-bca9471abfa0/element / {"using":"id","value":"target0"}] no such element: Unable to locate element: {"method":"id","selector":"target0"}

or

ScriptTimeout: [POST http://localhost:4444/wd/hub/session/f93260d5-52de-439a-bf86-bca9471abfa0/execute_async / {"script":"return (function (poller, args, timeout, pollInterval, done) {\n\t\t\t\t\t\t/* jshint evil:true */\n\t\t\t\t\t\tpoller = new Function(poller);\n\n\t\t\t\t\t\tvar endTime = Number(new Date()) + timeout;\n\n\t\t\t\t\t\t(function poll() {\n\t\t\t\t\t\t\tvar result = poller.apply(this, args);\n\n\t\t\t\t\t\t\t/*jshint evil:true */\n\t\t\t\t\t\t\tif (result != null) {\n\t\t\t\t\t\t\t\tdone(result);\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\telse if (Number(new Date()) < endTime) {\n\t\t\t\t\t\t\t\tsetTimeout(poll, pollInterval);\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\telse {\n\t\t\t\t\t\t\t\tdone(null);\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t})();\n\t\t\t\t\t}).apply(this, arguments);","args":["return (function () {\n\t\t\t\t\treturn window.w3cTests;\n\t\t\t\t}).apply(this, arguments);",[],1000,67]}] asynchronous script timeout: result was not received in 1 seconds

My current strategy is to look at the git history (back 20 pages, back 10 pages, etc.) to figure out which commit broke the PEP tests. It's effectively git bisect, but across repos.

@appsforartists

This comment has been minimized.

Show comment
Hide comment
@appsforartists

This comment has been minimized.

Show comment
Hide comment
@appsforartists

appsforartists Sep 21, 2016

Contributor

npm run test isn't copying the HTML files in resources correctly:

screen shot 2016-09-21 at 4 43 20 pm

However, adding those files in manually doesn't fix the NoSuchElement error.

Contributor

appsforartists commented Sep 21, 2016

npm run test isn't copying the HTML files in resources correctly:

screen shot 2016-09-21 at 4 43 20 pm

However, adding those files in manually doesn't fix the NoSuchElement error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment