Add more container-based 3rd party application tests #172
Comments
|
Add Elasticsearch under thirdparty container tests |
smlambert
added
the
type:external
|
In additional to a broad set of application tests to prove the quality of the binaries, it would be useful to have a suite of MicroProfile tests to demonstrate that our binaries are a good foundation for MicroProfile. Microprofile doesn't have a reference implementation, rather there are a number of popular implementations that cover different proportions of the specification. The following projects will provide a good test environment for our binaries:
The tests would be taken from the MicroProfile TCKs, e.g. fault tolerance APIs, metrics APIs, etc. Note that not all the implementations implement every part of the MicroProfile spec, so we'd need to define the appropriate set of TCKs to run against each implementation. As the MicroProfile projects are open communities, I expect we can get help in putting together the tests and running them on our test farm. |
|
This is a little bit longer term, but you should also consider the use of the Jakarta EE test suite (CTS) when that becomes available under Eclipse licensing. I know from personal experience that running the CTS can be complicated and time-consuming. So, maybe we can work with the various vendors and developers that have or will be running the Jakarta EE CTS to also run with AdoptOpenJDK binaries. Not as a requirement, but as a "friend of open source". We're still a few months away before these test suites will be available and usable, but something to keep in your back pocket for future testing efforts. |
|
Thanks @tellison and @kwsutter - we are in process of updating our test doc to state our goals around running these external tests and engaging with the open-source communities (Summarized mission: us running their tests, them using adopt SDKs, attempting to make both aspects of testing drop-dead easy). Both your suggestions fit in with the overall mission. 👍 Link to updated mission statement to come shortly. |
|
related: adoptium/adoptium-support#30 |
|
Does mybatis worth adding if we are considering hibernate? |
# Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. --> # External Payara-mp-tck Tests Payara Platform is cloud-native, optimized for production environments. payara-mp-tck test(payara MicroProfile-tck) tests are part of the third-party application tests that helps verify that the Adoptium binaries are good by running a variety of Java applications inside of Docker containers. AdoptOpenJDK/openjdk-tests/Issue adoptium#172 lists the applications that we have initially targeted to best exercise the Adoptium binaries. For each application, we choose to run a selection of their functional tests.payara-mp-tck tests are functional tests pulled from the [payara/MicroProfile-TCK-Runners](https://github.com/payara/MicroProfile-TCK-Runners.git) repository. ## Running External Payara-mp-tck tests locally To run any AQA tests locally, you follow the same pattern: 0. Ensure your test machine is set up with [test prereqs](https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/doc/Prerequisites.md). For external tests, you do need Docker installed. 1. Download/unpack the SDK that you want to test to your test machine. 2. `export TEST_JDK_HOME=</pathToWhereYouInstalledSDK>` 3. `git clone https://github.com/AdoptOpenJDK/openjdk-tests.git` 4. `cd openjdk-tests` 5. `./get.sh` 6. `cd TKG` 7. Export required environment variables, BUILD_LIST and EXTRA_DOCKER_ARGS (example: `export BUILD_LIST=external/payara-mp-tck` and `export EXTRA_DOCKER_ARGS="-v $TEST_JDK_HOME:/opt/java/openjdk"` 8. `make compile` (This fetches test material and compiles it, based on build.xml files in the test directories). 9. `make _payara-mp-tck_test` (When you defined BUILD_LIST to point to a directory in [openjdk-tests/external](https://github.com/AdoptOpenJDK/openjdk-tests/tree/master/external), then this is a testCaseName from the playlist.xml file within the directory you chose). When [running these from the command-line](https://github.com/AdoptOpenJDK/openjdk-tests/blob/master/doc/userGuide.md#local-testing-via-make-targets-on-the-commandline), these tests are grouped under a make target called 'external', so 'make external' would run the entire set of tests found in the openjdk-tests/external directory. This is unadvisable! Limit what you compile and run, BUILD_LIST=external/`<someSubDirectory>`, and TARGET=`<testCaseNameFromSubdirPlaylist>` These tests run regularly and results can be found in [TRSS Third Party Application view](https://trss.adoptopenjdk.net/ThirdPartyAppView). Signed-off-by: Benny-coy <benytm@gmail.com>
|
We may also consider adding jython test suite: https://github.com/jython/jython/tree/master/tests |
|
Maybe consider adding rocketmq |
As an example of this type of testing, we have added scala tests: https://github.com/AdoptOpenJDK/openjdk-tests/tree/master/thirdparty_containers/scala
We have derby and solr_lucene tests running in the same "dockerized" way, so we should also add those test sub-folders.
Once we have those 3 examples of 3rd party application tests added, we should consider the list of other applications we would like to add and prioritize them (eclipse, cassandra, etc). It will also be good to engage with these communities so that they are aware that we will be running their community test suites against the different flavours of JVM at AdoptOpenJDK (openjdk8, openjdk9, openjdk+openj9...).
Here is a list of potential ones to add, not in any priority order:
Also, this Testing Java link and this Quality Outreach link relates to this effort (engaging with Java projects, unblocking them if they have issues)...
Consider new tests via external_custom test target.
Consider startup tests for quick smoke test of each application.
The text was updated successfully, but these errors were encountered: