Documentation Improvements

[Ipstenu]

The configuration directions imply that someone can use Travis, but offer no explicit walkthroughs as to how one might do so. This is made more complicated by the fact that installation directions are muddled with configurations, and in neither case are the two actually detailed.

This should be changed to something like the following:

0 . Requirements

Move this UP from hidden down in 'running' :) If you HAVE to have these things, get them installed first.

1. Installation Instructions

What has to be installed? One presumes you always want to gitclone, but this isn't expressly stated. This would have the "How to install the runner scripts" (git clone?) and then a subsection for Travis on the SSH stuff.

If there are different directions for Travis vs local, then break the sections apart clearly.

2. Configuration

Here, we should offer some help for newbies. Travis. Cool! What does that mean? :) We may eventually need an example (use the wiki?) for a walk through of how someone sets up Travis.

This is where we separate Local and Travis/CI again.

In a CI service, you can set these environment variables through the service's web console. Importantly, the WPT_SSH_CONNECT environment variable determines whether the test suite is run locally or against a remote environment.

There's not enough detail there for someone to understand what that means without the explanations of what a CI is for. Even just an example of "For Travis you want..." will get people started.

3. Prepare

This can be as is, though it would help to explicitly state "Run the command with php prepare.php as a normal user (i.e. not ROOT)" - this should be obvious, but it may not be.

4. Running

It's not actually clear that you want to run the test and report commands sequentially, so here we need to step back and explain that test.php actually runs a test. This is the thing you're trying to get to :)

Under that you could have a subsection for reporting. "If you're going to automagically send the report to somewhere, like WordPress.org (link to that handbook), you'll need to use this." Explicitly state what values need to be set and where in the .env file for that if you can.

5. Cleanup

There's no information WHY a person would want to run this. Is this for uninstall? Should I always run it after the report? If it's uninstall, let's explain that. If it's meant to run every time, then put it up with the running section.

6. Automation

FUN TIME! We don't explain this at all :D It should probably be a wiki page instead of in the readme, but we should offer some examples like "If you're using local and want to cron this every X hours..." That would let us have a whole Travis-CI section AND let other people contribute 'how to with this CI thingy' as well.

[g-taniguchi]

Hi, we already setup them, but had some troubles. The reason why our environments are pretty legacy (FreeBSD9,11) so we couldn't install latest node.js. And we were confused how to test on multiple php versions. So this improvements proposal is very important for new comer I think.

Configuring
OK, certainly CI's instruction is just too long and it causes confusion, so it should be seperated section like this.

You can choose 2 way to run the test.

To use Travis CI

(instructions)

To use cron jobs

(instructions)

And we should emphasise that CI link is just one of the way to execute automate test.
I think we don't have to use Wiki. This readme.md is enough for them. Configurering section seems too long but it'll be better to seperate CI and cron section.

Running
0. Requirements
We should make sure these points

• Prep environment and Test one are not have to be on same server.
  (If production environment is too legacy, some apps can't be installed)
• If you want to test on multiple versions of php, you have to sign up another wordpress.org account and API key.

2. Test
   We are having a problem test.php's high CPU time consumption, so it has to make sure.

• Sometimes test.php use much CPU times, so be careful for running on production environment.

[javiercasares]

The documentation we have is up-to-date. We can try to add new documentation in the future if we add more ways to do the testings.