Skip to content

Commit

Permalink
Merge pull request #145 from apache/standalone-test
Browse files Browse the repository at this point in the history 
Make the automated test suite standalone from the office build
  • Loading branch information
@cbmarcum
cbmarcum committed Apr 15, 2022
2 parents bdf8903 + f9405ed commit e266681
Show file tree
Hide file tree
Showing 8 changed files with 519 additions and 9 deletions.
128 changes: 128 additions & 0 deletions test/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# Apache OpenOffice<sup>®</sup> Test
Apache OpenOffice contains automated test suites that can be ran against the office. The test framework is written in **Java**
and based on **JUnit 4**.

Currently, the test suite in **trunk** and other main branches like **AOO41X** and **AOO42X** can be ran against a normal installation
of the office or a just built "installed" office.

This currently requires the user running the tests to have previously built the office from source as some required dependencies
for compiling the tests are used from the office build artifacts located in `main/solver`.
Primarily these dependencies are things the Java UNO jars like juh, jurt, ridl, and unoil and tools like javamaker, regmerge,
and idlc.

This README is documentation for using the test suites without requiring an office build environment or having built the
office from source in this **standalone-test** branch.

## Prerequisites

Apache Ant 1.10.x and Java JDK 1.8 are known to work. Windows platforms can use the 64-bit JDK even if OpenOffice has to use a 32-bit JRE.

## Caution
When switching branches especially between AOO versions it's recommended to delete the jars in `test/lib` which will make
sure you get the correct versions of **JUnit** and **Hamcrest** libraries for your build since the test build will try to use the jars that are present but download the correct version if not found.

## Installed Type Build Setup
When we refer to an "installed" type build this means that you have built OpenOffice from source using build
config flag `--with-package-format="installed"`.

Compiling the tests now require an SDK to be included in the office directory layout. When compiling tests using an "installed"
type build, the office and the SDK are in built into separate directories and need combined. One method is to copy them into
the test directory layout.

If you have just built an "installed" build of the office you should have environment variables set you can use.
For example on my Linux machine my local OpenOffice source repo is `/home/carl/dev-git/openoffice` and my variables are set like this:
```shell
SOURCE_ROOT_DIR=/home/carl/dev-git/openoffice
SRC_ROOT=/home/carl/dev-git/openoffice/main
INPATH=unxlngx6.pro
```

If you are using a new terminal you can run your environment setup script from `main` like:
```shell
source ./LinuxX86-64Env.Set.sh
```

If you have not previously ran tests you may not have a `testspace` directory under `test`. If not create it:
```shell
mkdir $SOURCE_ROOT_DIR/test/testspace
```

And the directory for the office:
```shell
mkdir $SOURCE_ROOT_DIR/test/testspace/install
```

Now copy the "installed" office from the build location:
```shell
cp -r $SRC_ROOT/instsetoo_native/$INPATH/Apache_OpenOffice/installed/install/en-US/openoffice4 $SOURCE_ROOT_DIR/test/testspace/install/
```

Then copy the SDK into the office:
```shell
cp -r $SRC_ROOT/instsetoo_native/$INPATH/Apache_OpenOffice_SDK/installed/install/en-US/openoffice4/sdk $SOURCE_ROOT_DIR/test/testspace/install/openoffice4/
```
## Compiling the Tests
The tests are built using Ant and passing an `openoffice.home` property with the location of the office needed for dependencies.

"installed" build example:
```shell
ant -Dopenoffice.home="$PWD/testspace/install/openoffice4" compile
```
Or an office in the standard Linux installation directory:
```shell
ant -Dopenoffice.home="/opt/openoffice4" compile
```
Or Windows
```shell
ant -Dopenoffice.home="C:\Program Files (x86)\OpenOffice 4" compile
```

## Running Tests
There are three test suites available. Build Verification Test (BVT), Function Verification Test (FVT), and Performance Verification Test (PVT).

The **Build Verification Test** Suite includes approximately fifty unit tests used for general build verification.

The **Functional Verification Test** Suite includes almost nine hundred unit tests to more thoroughly test the office features and UI.
_Not all tests are ran for each platform._

### Running a Complete Test Suite
BVT Example (Linux):
```shell
./run -Dopenoffice.home="$PWD/testspace/install/openoffice4" -tp bvt
```
BVT Example (Windows):
```shell
.\run.bat -Dopenoffice.home="C:\Program Files (x86)\OpenOffice 4" -tp bvt
```
FVT Example:
```shell
./run -Dopenoffice.home="$PWD/testspace/install/openoffice4" -tp fvt
```
### Running a Single Class
```shell
./run -Dopenoffice.home="$PWD/testspace/install/openoffice4" -tc bvt.gui.BasicFunctionTest
```
### Running a Single Test Method
```shell
./run -Dopenoffice.home="$PWD/testspace/install/openoffice4" -tm bvt:bvt.gui.BasicFunctionTest.testFindFormulasAndValues
```
### Run Help
```shell
./run -help
```

## Test Results
By default, the testing output is stored in `test/testspace/output.***` where *** is the test suite type.

* result.xml: Test result in HTML.
* result.html: Test result in XML.
* screenshot/: Screenshot pictures when test assert is failed.

Open `test/testspace/output.***/result.html` in your browser to see the testing report.

## Cleaning Up
Running clean will delete compiled classes and build directories.
```shell
ant clean
```
The `test/testspace` directory, test results and "installed" office if used need to be manually deleted when no longer needed.
1 change: 1 addition & 0 deletions test/build.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,7 @@
</condition>
<property name="test.executable" value="./run"/>
<property name="test.args" value="-tp bvt"/>
<echo message="Executing: ${test.executable} ${test.arg0} ${test.args}" level="info"/>
<exec executable="${test.executable}">
<arg value="${test.arg0}"/>
<arg line="${test.args}"/>
Expand Down
243 changes: 243 additions & 0 deletions test/smoketestdoc/aoo-ant.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,243 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--***********************************************************
*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*
***********************************************************-->

<!-- copied from main/solenv/ant and modified for standalone tests -->

<project name="aoo-ant" default="main" xmlns:if="ant:if" xmlns:unless="ant:unless">

<!-- ================================================================= -->
<!-- settings -->
<!-- ================================================================= -->

<dirname property="aoo-ant.basedir" file="${ant.file.aoo-ant}"/>

<!-- RSCREVISION: -->
<property file="${aoo-ant.basedir}/../../main/solenv/inc/minor.mk"/>

<property name="main.src.dir" location="src/main/java"/>
<property name="res.src.dir" location="src/main/resources"/>
<property name="idl.src.dir" location="src/main/idl"/>
<property name="test.src.dir" location="src/test/java"/>
<property name="test-res.src.dir" location="src/test/resources"/>
<property name="test-idl.src.dir" location="src/test/idl"/>

<property name="main.build.dir" location="${build.base.dir}/main"/>
<property name="idl.build.dir" location="${build.base.dir}/idl"/>
<property name="idl.build.classes.dir" location="${build.base.dir}/idl/classes"/>
<property name="test.build.dir" location="${build.base.dir}/test"/>
<property name="test-idl.build.dir" location="${build.base.dir}/test-idl"/>
<property name="test-idl.classes.build.dir" location="${build.base.dir}/test-idl/classes"/>
<property name="test.reports.dir" location="${build.base.dir}/test-reports"/>

<property name="java.baseline.version" value="1.7"/>

<property name="main.debug" value="true"/>
<property name="main.deprecation" value="false"/>
<property name="test.debug" value="true"/>
<property name="test.deprecation" value="false"/>

<import file="${aoo-ant.basedir}/idl.xml"/>

<target name="init-project"/>

<target name="prepare" depends="init-project">
<property name="jar.enabled" value="true"/>
<property name="jar.name" value="${ant.project.name}"/>
<property name="jar.classpath" value=""/>
<property name="jar.manifest" value ="${aoo-ant.basedir}/manifest.empty"/>

<local name="has.main.classpath"/>
<condition property="has.main.classpath">
<isreference refid="main.classpath"/>
</condition>
<path id="internal.main.classpath">
<path refid="main.classpath" if:set="has.main.classpath"/>
<pathelement location="${idl.classes.build.dir}"/>
</path>

<local name="has.test.classpath"/>
<condition property="has.test.classpath">
<isreference refid="test.classpath"/>
</condition>
<path id="internal.test.classpath">
<pathelement location="${main.build.dir}"/>
<pathelement location="${idl.classes.build.dir}"/>
<pathelement location="${test.build.dir}"/>
<pathelement location="${test-idl.classes.build.dir}"/>
<path refid="main.classpath"/>
<path refid="test.classpath" if:set="has.test.classpath"/>
<pathelement location="${OOO_JUNIT_JAR}"/>
<pathelement location="${HAMCREST_CORE_JAR}" if:set="HAMCREST_CORE_JAR"/>
</path>
</target>

<target name="dependencies" depends="prepare">
<local name="has.main.classpath"/>
<condition property="has.main.classpath">
<isreference refid="main.classpath"/>
</condition>

<pathconvert refid="main.classpath" setonempty="true" pathsep=" " property="deps" if:set="has.main.classpath"/>
<property name="deps" value=""/>
<echo message="${deps}" file="${dependencies.outfile}"/>
</target>

<target name="res" depends="prepare">
<mkdir dir="${main.build.dir}"/>
<copy todir="${main.build.dir}" failonerror="false" quiet="true">
<fileset dir="${res.src.dir}">
<include name="**/*"/>
</fileset>
</copy>
</target>

<extension-point name="pre-compile" depends="prepare,res"/>

<target name="idl" depends="pre-compile">
<fileset id="idl.files" dir="${idl.src.dir}" includes="**/*.idl" erroronmissingdir="false"/>
<idl-javamaker
idlFiles="idl.files"
idlBuildDir="${idl.build.dir}"/>
</target>

<target name="compile" depends="idl">
<mkdir dir="${main.build.dir}"/>
<javac srcdir="${main.src.dir}"
destdir="${main.build.dir}"
source="${java.baseline.version}"
target="${java.baseline.version}"
debug="${main.debug}"
debuglevel="lines,vars,source"
deprecation="${main.deprecation}"
classpathref="internal.main.classpath"
includeantruntime="false"/>
</target>

<macrodef name="check-test">
<sequential>
<local name="tests.present"/>
<available type="dir" file="${test.src.dir}" property="tests.present"/>
<echo message="No tests" unless:set="tests.present"/>

<local name="only.junit.absent"/>
<condition property="only.junit.absent">
<and>
<isset property="tests.present"/>
<not><isset property="OOO_JUNIT_JAR"/></not>
</and>
</condition>
<echo message="No junit, skipping tests" if:set="only.junit.absent"/>

<condition property="test.skip">
<or>
<not><isset property="tests.present"/></not>
<not><isset property="OOO_JUNIT_JAR"/></not>
</or>
</condition>
</sequential>
</macrodef>

<target name="test-res" depends="pre-compile">
<mkdir dir="${test.build.dir}" unless:set="test.skip"/>
<copy todir="${test.build.dir}" failonerror="false" quiet="true" unless:set="test.skip">
<fileset dir="${test-res.src.dir}">
<include name="**/*"/>
</fileset>
</copy>
</target>

<target name="test-idl" depends="test-res">
<fileset id="test-idl.files" dir="${test-idl.src.dir}" includes="**/*.idl" erroronmissingdir="false"/>
<idl-javamaker
idlFiles="test-idl.files"
idlBuildDir="${test-idl.build.dir}"/>
</target>

<target name="test-compile" depends="compile,test-idl">
<check-test/>
<mkdir dir="${test.build.dir}" unless:set="test.skip"/>
<javac srcdir="${test.src.dir}"
destdir="${test.build.dir}"
source="${java.baseline.version}"
target="${java.baseline.version}"
debug="${test.debug}"
debuglevel="lines,vars,source"
deprecation="${test.deprecation}"
classpathref="internal.test.classpath"
includeantruntime="false"
unless:set="test.skip"/>
</target>

<!-- fork="true" is sadly necessary on Ubuntu due to multiple versions of junit confusing Ant,
see https://github.com/real-logic/simple-binary-encoding/issues/96 -->
<target name="test" depends="test-compile" unless="${test.skip}">
<mkdir dir="${test.reports.dir}"/>
<junit printsummary="yes" haltonfailure="yes" showoutput="true" filtertrace="false" fork="true">
<classpath refid="internal.test.classpath"/>
<formatter type="plain"/>
<batchtest todir="${test.reports.dir}">
<fileset dir="${test.src.dir}">
<include name="**/*_Test.java"/>
</fileset>
</batchtest>
</junit>
</target>

<target name="jar" depends="compile" if="${jar.enabled}">
<jar destfile="${jar.dir}/${jar.name}.jar"
manifest="${jar.manifest}"
duplicate="fail">
<manifest>
<attribute name="Class-Path" value="${jar.classpath}" unless:blank="${jar.classpath}"/>
<attribute name="Solar-Version" value="${RSCREVISION}"/>
</manifest>
<fileset dir="${main.build.dir}"/>
<!-- Breaks trunk/test/smoketestdoc by packaging different classes with the same name: -->
<!-- <fileset dir="${idl.build.classes.dir}" erroronmissingdir="false"/> -->
<include name="**/*.class"/>
<include name="**/*.properties"/>
<include name="**/*.css"/>
<include name="**/*.dtd"/>
<include name="**/*.form"/>
<include name="**/*.gif "/>
<include name="**/*.htm"/>
<include name="**/*.html"/>
<include name="**/*.js"/>
<include name="**/*.mod"/>
<include name="**/*.sql"/>
<include name="**/*.xml"/>
<include name="**/*.xsl"/>
<include name="**/*.map"/>
</jar>
</target>

<extension-point name="pre-clean" depends="prepare"/>

<target name="clean" depends="pre-clean">
<delete dir="${build.base.dir}"/>
<delete file="${jar.dir}/${jar.name}.jar"/>
</target>

<target name="main" depends="test,jar"/>

</project>

0 comments on commit e266681

Please sign in to comment.