Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Java
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
boot-lib
src
README.txt
build.xml
generate-all.sh
generate.sh
verify.sh

README.txt

Neo4j store compatibility tests
===============================

This project aims to create a backward compatibility test suite for stores,
and it is intended for continuous use and maintenance. Its goal is to detect
problems related to store upgrades between versions.

It does this by storing a set of generated test stores created by so-called
agents. The tool will produce a bundle of test stores from a given Neo4j
version. Each agent produces a store for the bundle. An agent is a Java class
implementing StoreAgent, and it is capable of generating a store, and
verifying that a store matches what it expects.


File structure
--------------

src/
 framework/     Testing framework classes
 versions/      Version-specific tests
  default/      Default test suite
   agents/      Java package for agents
   META-INF/
    services/
     org.neo4j.compatibility.StoreAgent   References agents to be used
  1.2.M06/      Version-specific test, merged with default suite for
                compatibility of certain versions.
  1.2.M05/
    ...


Scripts
-------

./generate.sh <version>
  
  Generate new bundle of test stores for the specified Neo4j version.

./generate-all.sh

  Generate new bundles of test stores for all Neo4j versions specified in the
  script.

./verify.sh <version>

  Verify that the Neo4j version specified can verify all previously stored
  test store bundles.


Examples
--------

./generate.sh 1.2.M04

  Generates a bundle of test stores in repo/db-1.2.M04.zip.

./generate-all.sh

  Generates bundles of test stores in repo for each version in script.

./verify.sh 1.3-SNAPSHOT

  Verifies that version 1.3-SNAPSHOT can verify every bundle in repo.


New versions
------------

When a new version of Neo4j is released, the previous version needs to be 
rotated into the set of versions tested against. Follow these steps when
a new version X is released:

1. Update the test suite to support version X.

2. Generate bundles of test stores for version X:

  ./generate.sh X

3. Add version X to generate-all.sh.

4. Verify the new version against all stored version bundles:

  ./verify.sh X

Note: This only covers the overall procedure of rotating versions. Adding of
tests is a separate process, described below.


Adding new tests
----------------

As new functionality is added to Neo4j, we will be wanting to test new things.
This is done by adding new test cases, generating and verify new test stores.
When a new test is added, depending on how it is done, it will also be
included when generating bundles of test stores for older versions. Therefore
it is advantageous to regenerate all test store bundles after adding a new
test. This can be done using generate-all.sh.

The default behavior should be to add new tests to src/versions/default. 
Simply give your class a unique name (that identifies the test store), and
make your class implement StoreAgent. Finally, add your class name to
src/versions/default/META-INF/services/org.neo4j.compatibility.StoreAgent. It
will then be picked up when generating and verifying test stores.

To make development a little bit easier, one can set the dev.version property
in build.xml, and run:

  ant retrieve-dev-dependencies

It will place the necessary libraries in the dev-lib/ directory. These can
then be included by the IDE. For example, if going back to change a
version-specific agent for 1.2.M01, then dev.version can be set to 1.2.M01,
and the change can be made by using src/versions/1.2.M01 as source root.


Managing code incompatibility
-----------------------------

Sometimes changes to new code break old code. In general it is good to preserve
working test code for each version of Neo4j. 

If a new release breaks code in src/versions/default, follow these steps:

1. Update src/version/default to be compatible with the latest changes.

2. Run a cycle of generate-all.sh -> verify.sh <new_release>.

3. If the updated agent code is now incompatible with an older version, then
   create a compatible agent in a version-specific source directory.
   Version-specific agents can opt to not implement any verification step,
   as one seldomly will be verifying against versions older than the latest
   one.


How do all the versions come together?
--------------------------------------

When a version is to be generated or verified, first it is determined whether 
a specific version directory for that version exists or not. If none is found,
then the default is used, and that is that.

If a version directory is found, existence for a file named "no-default" in
it is checked. If such a file exists, the default will not be used as the
template. Then the classes used will simply be the ones in that version
directory.

If there is no "no-default" file, then the defaults are used as a template, and
any files that exist in the version directory replace the ones in the default
set. Note that this includes the META-INF/... file. Typically there will be two
use cases:

1. The tested version's code does not compile with the default, and therefore
   requires special changes. An example is graphDb.index() vs IndexService.

2. A specific test case class (agent) should not be used for this version. This
   is easily achieved by making an empty class with the same name, that 
   inherits from IgnoringStoreAgent. It will not be picked up as a store agent.

So to sum up:

exists( version/no-default ) ? version : default + version


Code compatibility relationships
--------------------------------

Every agent can generate and verify a store. An agent is identified by its
class name. Any store generated by an agent with any version of Neo4j has to
be verifiable with the latest version of Neo4j. This means that the
verification code in the agent should stay backward compatible. If the test
needs to cover something new, then a new test should be created instead.

That being said, it is still useful to keep the versions hierarchy maintained,
so that generate-all.sh generates valid store bundles for each version of
Neo4j.

Something went wrong with that request. Please try again.