Test to ensure that the behavior of the world object does not change #42
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
To prevent the need to make the minecraft jar file and libraries available on travis ci, this test uses previously generated testdata to ensure that the generated world icons are still the same as when the test data where generated. It is true, that this uses the production code to generate the test data, however, this is not an issue, because the test data are stored and not generated every time the test is executed. All in all, the test checks whether the generated world icons have changed since the test data where generated. It does not ensure, that the world icons are correct (meaning like minecraft generates them). The test: * loads the biome data from the test data * loads the world icons * creates a fake minecraft interface that provides the stored biome data * creates a world object using the fake minecraft interface * generates all world icons using the world object * compares the generated world icons to the loaded world icons The test data contains all biome data in the test area as well as all world icons. The test area is 10 fragments in all directions from [0, 0] ([-5120, -5120] to [5120, 5120]). The world icons are generated for the inner 10 - 1 = 9 fragments ([-4608, -4608] to [4608, 4608]). This prevents the world icon producers to request biome data that have not been stored. The test data are stored in a zip-file to prevent them from taking up to much space. One world requires about 6.5 MB (compressed). Thus, we should not add to much test worlds and not change them very often to prevent the repository from getting too big. An alternative would be stored the testdata somewhere else than in the repository. However, they have to be available for travis ci, so this will be the solution for now. * The End Cities are currently not included. * The current seed is "amidst-test-seed" using the default world type and minecraft version 1.8.9. * The test data are generated by running the test in the class DevToolRunner.
... to make it available to the test classes. The tests actually do not need it. However, it is used in the test data generator. In the future, we should move the devtools to the test source folder and removed the extra devtools source folder to prevent stuff like this.
|
@Treer and @BigAlanM maybe one of you knows a seed that is good for testing the world icon generation? As described above, this test uses a limited area around 0, 0 for a given Minecraft version and a given seed. So, if you know a combination of seed and Minecraft version that generates a sufficient amount of world icons of each type in the give region, that would be really helpful. It would be even more helpful if the seed includes world icons that are easily generated wrong, for example because they are on a border between two biomes. For example the current test data do not include a single igloo, so this algorithm might still break without a failing test. |
|
Sounds like a job for a seedfinder search, which BigAlanM knows more about. Here's one with all biomes within 1024, and -9223372036681075344 is similar. FWIW -1364077613 was interesting because it misses an ocean monument if the quarter-resolution map is used for the structure test, but there's no nearby igloos. Often I use 1 (it's easy to type), which by luck happened to illustrate the minecraft bug in how the new 1.9 strongholds are placed, which we need to keep an eye on - awkwardly reported here. But yeah, a seed wth all the biomes makes more sense. |
* improved the code
* switched biome data storage format to protobuf (binary)
* switched back to json (see reasons below)
* switched biome data serialization strategy
* added mechanism to test specific entries for specific worlds
* squashed commits to prevent git from storing the outdated big testdata files forever
And this are the old detailed commit messages:
changed strategy for biome data storage
* it now tracks and stores all biome data requests while the other test data are created
* the same request during the running test will offer the same response as during the test data generation
* this greatly simplifies the code, makes the size of the test data much smaller and the testdata creation and loading much faster
removed the binary test data storage
* switched back to json
* did not serve a significant better result (time and space)
* protobuf documentation discourages the use for large messages
* I want to not use code generation whenever possible because
* it contains warnings
* it is differently formatted than the hand written code
* it makes the build process more complicated
Here are the stats:
json binary binary packed
test time: 13,193 s 9,838 s 9,333 s
complete: 6,4 mb 6,1 mb 6,1 mb
uncompressed: 260,8 mb 212,5 mb 212,5 mb
complete is the compressed size of all zip files. uncompressed is the uncompressed size of the full-resolution biome data. We see, that the compression removes a big portion of the space advantage that would be served by the binary format. The binary packed format seems to not be any better than the default binary format. Just the few seconds are not really worth the disadvantages listed above.
fixed bugs in test data generation
* test world generation now omits unsupported entries
* creation of an empty coordinates collection throws an exception to prevent the usage of empty testdata
switched BiomeData to binary storage format
rename refactorings and adjustments to the protobuf format
added equalityChecker to TestWorldEntryDeclaration
used the method TestWorldDeclaraion.isSupported() ...
... to allow that only specific features are tested for special test worlds
enhanced readability of test world entry declaration
more refactorings for the testworld code
extracted the class TestWorld from the class TestWorldDirectory
renamed and moved classes
moved write and read methods to entry declaration
added protobuf
* library
* license
* testdata declarations
* generated code from testdata declarations
improved testdata storage
* split up the single zip file into multiple small ones to only change as few data as possible, when they need to be changed
* removed timestamp from zip files to make them reproducible
removed many usages of TestWorldDeclaration by exposing the regocnised version from the world object
|
I got the test world size to under 50 kB. Thus, we no longer need to be super economically with the test worlds. |
* if it contains less than 5 entries, the test data generation will throw an exception * this is to prevent meaningless testdata * this rule does not apply to strongholds and the spawn * now all strongholds will be taken into account, not only the strongholds that are placed into to test area
... for an area 2*2 fragments around origin
stefandollase
changed the title
|
@Treer Thanks for the seed suggestions. I added all of them, since one test world now became relatively cheap. After I added all the data from the world object it is still less that 100 kB per world. However, we should nevertheless keep an eye on the number of test worlds to keep the tests and thus the automatic build process quick. |
stefandollase
added a commit
that referenced
this pull request
Jan 31, 2016
Test to ensure that the behavior of the world object does not change
This was referenced Jan 31, 2016
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To prevent the need to make the minecraft jar file and libraries available on travis ci, this test uses previously generated testdata to ensure that the generated world icons are still the same as when the test data where generated. It is true, that this uses the production code to generate the test data, however, this is not an issue, because the test data are stored and not generated every time the test is executed. All in all, the test checks whether the generated world icons have changed since the test data where generated. It does not ensure, that the world icons are correct (meaning like minecraft generates them).
The test:
The test data contains all biome data in the test area as well as all world icons. The test area is 10 fragments in all directions from 0, 0 ([-5120, -5120] to [5120, 5120]). The world icons are generated for the inner 10 - 1 = 9 fragments ([-4608, -4608] to [4608, 4608]). This prevents the world icon producers to request biome data that have not been stored. The test data are stored in a zip-file to prevent them from taking up to much space. One world requires about 6.5 MB (compressed). Thus, we should not add to much test worlds and not change them very often to prevent the repository from getting too big. An alternative would be stored the testdata somewhere else than in the repository. However, they have to be available for travis ci, so this will be the solution for now.