Refactor integration tests to make requests to wordpress.org

[crazytonyli]

Note

This PR is built on top of #415

The integration tests now make HTTP requests to api.wordpress.org instead of using cached content.

There are two kinds of tests:

1. Verify action=query_plugins, which returns a list of plugins (with a subset of properties).
2. Verify action=plugin_information, which returns full plugin details.

There are 44k plugins in wordpress.org. That means the first set of tests makes 250+ HTTP requests with 200 plugins per request, and the second set of tests makes 44k HTTP requests for each plugin.

I tried a few different ways to run those tests. I believe the one in this PR is the most optimal. To try it locally, run:

./wordpress_org_api_integration_tests/replace-test-parameters.sh
make test-rust-integration-wordpress-org-api

On CI, ./wordpress_org_api_integration_tests/replace-test-parameters.sh makes requests to api.wordpress.org and generates test parameters (like 44k+ plugin slugs) for the integration tests. The integration tests are configured to load those parameters and generate individual test cases for them. That means there will be 250+ tests for action=query_plugins and 44k+ tests for action=plugin_information. Cargo runs them in parallel and reports their failures individually. They won't interfere with each other.

When running locally during development, we don't need to run the shell script and can simply run make test-rust-integration-wordpress-org-api (or cargo test --package wordpress_org_api_integration_tests) to run the integration tests. However, this will only run a few tests against api.wordpress.org. I don't see the need to run the full test suite locally, given the full test suite runs on CI regularly. If the CI job failed and we'd like to investigate, we can find the failed parameter (i.e. plugin slug) from CI job output, set it in the wordpress_org_api_integration_tests/tests/plugin_directory_parameters.rs file, and run the tests locally to debug.

[crazytonyli]

I just noticed compiling the tests takes up 14 GB RAM. Need to look into optimizing that.

[oguzkocer]

I can't agree with the general approach in this PR. Particularly I found the fact that we are replacing Rust code from a bash script concerning. In my opinion, we shouldn't make changes to checked in files in general.

If we wanted to do this - and I don't think we do - we should do it by using a Cargo build script and generate the Rust files. At least that way it's part of the general Cargo build logic and properly placed and read from target folder.

---

In my opinion, we should use simple configuration files to drive the execution of the tests. All the implementation details should live in Rust. I am happy to offer a concrete approach - or make quick changes to the PR if you can can write a documentation of your desired differences between local and CI runs.

[crazytonyli]

Closing this PR as we'll test plugins one-by-one instead.