Add a mach command to test WebIDL.py

[psdh]

We should run runtests.py.
You could possibly name it mach test-webidl

[jdm]

This would belong in python/testing_commands.py.

[nox]

We probably should update update.sh to make it update the tests from mozilla-central too.

[kishorbhat]

I'd like to do this.

[jdm]

@therealkbhat It's yours! Let us know if you've got any questions!

[kishorbhat]

We probably should update update.sh to make it update the tests from mozilla-central too.

Isn't it already fetching from mozilla-central?

[psdh]

@therealkbhat Note that it just fetches the WebIDL.py file. nox was talking updating the tests dir too

[kishorbhat]

@psdh ah, right.

Also, I'm having trouble putting my finger on how exactly I need to run runtests.py. Here's what I have so far: gist.

[nxnfufunezn]

I guess putting the paren in print will solve the issue..

[kishorbhat]

@nxnfufunezn that worked. :)

Relative imports is throwing me off, though. It says ImportError: No module named WebIDL.
Is this because runtests.py is running in the context of testing_commands.py, so it doesn't see WebIDL.py? How do I get around this?

[nxnfufunezn]

import sys
sys.path.insert(0, "/path/to/your/package|module")

[kishorbhat]

@nxnfufunezn thanks again.

This is a bit more confusing than I thought it'd be... how do I pass run_tests() the tests and options.verbose that it expects? (code as previously mentioned in gist)

Error running mach:

    ['test-webidl']

The error occurred in the implementation of the invoked mach command.

This should never occur and is likely a bug in the implementation of that
command. Consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

TypeError: run_tests() takes exactly 2 arguments (0 given)

  File "/home/kishor/projects/servo/python/servo/testing_commands.py", line 343, in test_webidl
    return run_globals["run_tests"](**kwargs)

[psdh]

@therealkbhat Try importing the runtests module and add a command argument for verbose output :)

[kishorbhat]

Try importing the runtests module and add a command argument for verbose output :)

In runtests.py:

sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))

So I need to import runtests in testing_commands.py instead?

In testing_commands.py:

@CommandArgument('--quiet', '-q', action='store_false',
                     help="Don't print passing tests")

This is what I've added to my previous code. What am I doing wrong? I'm getting

TypeError: run_tests() got an unexpected keyword argument 'quiet'

EDIT: updated gist.

[kishorbhat]

@jdm, feel free to unassign me for now.

I won't be able to work on this for a while, and I found that I need to do a bit more understanding to figure out how everything fits together! In the meantime, someone else might be able to get this fixed faster. :)

[wafflespeanut]

No worries, thanks for trying! :)

For future reference, it would be of great help if you push your commits to your fork and link them here, instead of putting them in gists.

[kishorbhat]

@wafflespeanut duly noted. :)

[shinglyu]

Can I take this bug if no one is working on it?

[wafflespeanut]

Great! Feel free to ask questions here or at the #servo channel in IRC :)

[shinglyu]

So here is my plan, please let me know if its OK.

1. When run ./mach test-webidl
2. Run update.sh once
3. Create a virtualenv
4. Install pip install ply in the virtualenv
5. run `runtest.py

I can run runtest.py successfully, but when I run update.sh the auto-merge got rejected. How should I handle a failed patching? Should I still run the original WebIDL.py or just stop there and show an error?

[shinglyu]

Looking into the python folder, I found that we already have a requirements.txt and _virtualenv created, is there any example of other commands installing dependencies and creating virtualenv ?

[shinglyu]

Hi I wrote a draft version that doesn't update from mozilla-central

master...shinglyu:testwebidl

So we should download the tests/*.py from m-c in update.sh right?

[wafflespeanut]

@shinglyu That looks good. Go ahead and open a PR, so that people can comment on your changes :)

@nox What should we do when the patch fails?

[psdh]

Do we want to run the update script each time I test-webidl?

[nox]

@wafflespeanut Fix it. We store patches to be sure we don't lose stuff when updating the parser.

[wafflespeanut]

@psdh No, we don't!

@shinglyu I misinterpreted nox's comment. We just want to update the update.sh to include the tests folder. We should never invoke that during testing!

[shinglyu]

@wafflespeanut OK, I'll create a PR for the test-webidl part.

@nox For the tests folder, I don't see a clean way to download the folder from either mxr or hg.mozilla.org. I thought about reading the filenames from the local tests folder, but I fear that the mozilla-central folder may have new files or renamed files. What do you think?

[shinglyu]

I got an warning in the PR saying that I don't have tests. Where can I add a test for mach command?

[wafflespeanut]

That's just the bot shouting at any change made to script. We don't have to worry about it :)

[shinglyu]

@wafflespeanut Thanks!

[wafflespeanut]

fixed by #9459 and superseded by #9475