Skip to content

/ servo

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create a ./mach test-perf command #12792

Closed
shinglyu opened this issue Aug 10, 2016 · 8 comments
Closed

Create a ./mach test-perf command #12792

shinglyu opened this issue Aug 10, 2016 · 8 comments
Labels
A-mach A-testing C-assigned E-less easy L-python

Comments

@shinglyu
Copy link
Member

@shinglyu shinglyu commented Aug 10, 2016 •
edited by jdm

Since the page load performance test runner is now in tree, we can create a mach command that runs it on the locally built Servo binary.

Some thing so consider:

  • The test runner whet through a big rewrite during the review, so expect some corner case to be broken.
  • It only makes sense to run performance on release build, so show an error if there is only the debug build
  • Currently the nightly tests are running on my private server, we need to make sure the command can be run in the CI without manual setup.

This sounds like a easy to less-easy task, if someone is interested please give it a try first.
@highfive
Copy link

@highfive highfive commented Aug 10, 2016

cc @wafflespeanut
@shinglyu shinglyu added the L-python label Aug 10, 2016
@highfive
Copy link

@highfive highfive commented Aug 10, 2016

cc @wafflespeanut
@asajeffrey
Copy link
Member

@asajeffrey asajeffrey commented Aug 18, 2016

This would be very nice to have, especially if we could use it to compare performance between branches.
@metajack
Copy link
Contributor

@metajack metajack commented Aug 18, 2016

It would be nice if this worked like arewecompressedyet.com where branches can be tried and various results compared easily. I assume this is possible with perfherder, but it's just so easy to use arewecompressedyet that everyone does it for almost all changes.
@shinglyu
Copy link
Member Author

@shinglyu shinglyu commented Aug 19, 2016

@metajack I think that might be something we can bring up with the Perfherder team, AFAIK they are still seeking ideas to improve the Perfherder UI. It will also benefit the Platform team.
@shinglyu shinglyu mentioned this issue Aug 27, 2016
Closed
0 of 5 tasks complete
@shinglyu
Copy link
Member Author

@shinglyu shinglyu commented Aug 27, 2016

From #12405 :

This is a followup for #11107 . There are a few things that needs to be done:

  • Create a script to download tp5.zip and put it in the correct folder
  • Create a test-perf command in test_commands.py to bootstrap the test env and run.
  • Pretty print the test result for human to read (currently a json) (Some of the work is done in #12946)
  • Figure out what kind of log format we want. Maybe use mozlog?
  • Deprecate the shinglyu/servo-perf repo and redirect people to the in-tree folder in the README.md file.
@shinglyu shinglyu added this to the Taiwan Code Sprint milestone Aug 27, 2016
@fuchiao
Copy link
Contributor

@fuchiao fuchiao commented Aug 27, 2016

I am trying.
@jdm jdm added the C-assigned label Aug 27, 2016
@jdm
Copy link
Member

@jdm jdm commented Aug 27, 2016

Great!
@fuchiao fuchiao mentioned this issue Aug 28, 2016
Merged
2 of 5 tasks complete
bors-servo added a commit that referenced this issue Sep 26, 2016
@bors-servo
Auto merge of #13091 - zack1030:issue-12792, r=shinglyu
Loading status checks…
9943566 
Create ./mach test-perf command

<!-- Please describe your changes on the following line: -->

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [ ] `./mach build -d` does not report any errors
- [ ] `./mach test-tidy` does not report any errors
- [X] These changes fix #12792 (github issue number if applicable).

<!-- Either: -->
- [ ] There are tests for these changes OR
- [X] These changes do not require tests because it is low-risk

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

in **testing_commands.py**, I linked the new command to **test_perf.sh**
in **test_perf.sh**, it prepares tp5n.zip and virtual environment
then runs **test_all.sh**

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/13091)
<!-- Reviewable:end -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-mach A-testing C-assigned E-less easy L-python
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

Create ./mach test-perf command
6 participants
@jdm @metajack @asajeffrey @fuchiao @shinglyu @highfive
You can’t perform that action at this time.