Collector automated testing for storage module

[peng-yongsheng]

Please answer these questions before submitting your issue.

• Why do you submit this issue?
• Feature or performance improvement

---

Requirement or improvement

• Please describe about your requirements or improvement suggestions.

In the feature, there will be several storage implementations, such as H2, ElasticSearch transport client, ElasticSearch Rest client, ShardingJDBC. But to this day, we determine whether a system functions as intended by the manual test. It's too hard to ensure all of them have the same implementations.

Discuss how to do it. @wu-sheng @acurtain @wengangJi

Overview of automated testing projects

• Forked source codes
• Test toolkit
• Docker compose

Forked source code

The source codes where the project resides, contributors fork the "incubator-skywalking" project and program the storage module implementation with different database.

[Source code project](https://github.com/{Account ID}/{forked project name from incubator-skywalking})

Test toolkit

The source codes which used to mock datas, send to the collector, verify the datas, shell script for startup.

Shell script:

1. Variable definition: Forked project git address, Docker compose project git address
2. Maven package the forked project
3. Maven package toolkit project
4. Get the docker compose define and run

Test toolkit

Docker compose( Create new project when it not exist )

This project contains the docker compose define, collector application config file.

Docker compose:

• Define database service uses a public image
• Bind mount for forked project package and toolkit project package

Config file:

• Config the database information because of this file could not PR into "incubator-skywalking" project. The "incubator-skywalking" project default storage must use H2 implementation.

[Docker compose](https://github.com/SkywalkingTest/collector-[database name]-docker)
Database name for example: h2, es-transport, es-rest

[wu-sheng]

[Source code project](https://github.com/{Account ID}/{forked project name from incubator-skywalking})

Forked project name is normally as same as ours. So, we can use the same name in default.

Test toolkit

Test toolkit should be provided as a public image, just download and use.

Docker compose

Because only two files are required, no need to create so many repositories in our org. I prefer use different folders. In pr, contributor can choose to use our provided docker-compose to test their bug-fix or feature, or they can provided one in our docker-compose-fork repo.

After docker-compose review, tested, passed, merged, the pr can be merged finally. In that cases, we have more test vessels for support different scenario.

Finally, we can use some ci env to run all cases(docker-compose) for full check, before release, or test in some check point.

[wu-sheng]

Closing this discussion.