integration Junit for OSS #778
Conversation
| } | ||
|
|
||
| @Override | ||
| public void validateResults() { |
There was a problem hiding this comment.
we should validate that a model has been created - you can add a separate issue for that and we can take it up later
spark/spark-test/src/test/java/zingg/spark/core/executor/TestSparkExecutors.java
Show resolved
Hide resolved
|
|
||
|
|
||
| // returns df1.intersect(df2) | ||
| public abstract ZFrame<D, R, C> intersect(ZFrame<D, R, C> df1, ZFrame<D, R, C> df2); |
There was a problem hiding this comment.
we should define these test methods in Zframe itself so that we dont have to go down to spark/snowflake specific stuff in the tests.
There was a problem hiding this comment.
done in commit 2c7f744
|
|
||
| protected SparkTrainingDataFinder getTrainingDataFinder() throws ZinggClientException { | ||
| SparkTrainingDataFinder stdf = new SparkTrainingDataFinder(ctx); | ||
| stdf.init(args); |
There was a problem hiding this comment.
move init to executortester.
There was a problem hiding this comment.
What would be the benefit in that? Moreover if something special needs to be done for an instance let it be done by concrete implementation
There was a problem hiding this comment.
That specific thing will anyways be done in the derived class. Setting up the test in one places ensures no code repetition and that all derived classes are executing the same tests, unless explicitly overridden
There was a problem hiding this comment.
done in commit 6423793
| } | ||
|
|
||
| @Test | ||
| public void testExecutors() throws ZinggClientException { |
There was a problem hiding this comment.
move this to the base class with S,D,R,C,T. Only getTrainingDataFinder etc should be overwritten to return concrete class here
There was a problem hiding this comment.
Done in commit e18399e
common/common-test/pom.xml
Outdated
| <packaging>jar</packaging> | ||
| <dependencies> | ||
| <dependency> | ||
| <groupId>zingg</groupId> |
There was a problem hiding this comment.
move tests to common test as discussed
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
common/common-test/src/test/java/zingg/common/core/executor/LabellerTester.java
Fixed
Show fixed
Hide fixed
common/common-test/src/test/java/zingg/common/core/executor/MatcherTester.java
Fixed
Show fixed
Hide fixed
common/common-test/src/test/java/zingg/common/core/executor/MatcherTester.java
Fixed
Show fixed
Hide fixed
common/common-test/src/test/java/zingg/common/core/executor/MatcherTester.java
Fixed
Show fixed
Hide fixed
common/common-test/src/test/java/zingg/common/core/executor/MatcherTester.java
Fixed
Show fixed
Hide fixed
common/common-test/src/test/java/zingg/common/core/executor/MatcherTester.java
Fixed
Show fixed
Hide fixed
common/common-test/src/test/java/zingg/common/core/executor/TrainingDataFinderTester.java
Fixed
Show fixed
Hide fixed
common/common-test/src/test/java/zingg/common/core/executor/MatcherTester.java
Fixed
Show fixed
Hide fixed
common/common-test/src/test/java/zingg/common/core/executor/MatcherTester.java
Fixed
Show fixed
Hide fixed
common/common-test/src/test/java/zingg/common/core/executor/MatcherTester.java
Fixed
Show fixed
Hide fixed
common/common-test/src/test/java/zingg/common/core/executor/MatcherTester.java
Fixed
Show fixed
Hide fixed
| assertTrue(matchCount > 1); | ||
| long unmatchCount = dfMarked.filter(notMatchCond).count(); | ||
| assertTrue(unmatchCount > 1); | ||
| LOG.info("matchCount : "+ matchCount + ", unmatchCount : " + unmatchCount); |
Check warning
Code scanning / PMD
Logger calls should be surrounded by log level guards. Warning test
| double score1 = tpCount*1.0d/(tpCount+fpCount); | ||
| double score2 = tpCount*1.0d/(tpCount+fnCount); | ||
|
|
||
| LOG.info("False negative " + fnCount); |
Check warning
Code scanning / PMD
Logger calls should be surrounded by log level guards. Warning test
| double score2 = tpCount*1.0d/(tpCount+fnCount); | ||
|
|
||
| LOG.info("False negative " + fnCount); | ||
| LOG.info("True positive " + tpCount); |
Check warning
Code scanning / PMD
Logger calls should be surrounded by log level guards. Warning test
|
|
||
| LOG.info("False negative " + fnCount); | ||
| LOG.info("True positive " + tpCount); | ||
| LOG.info("False positive " + fpCount); |
Check warning
Code scanning / PMD
Logger calls should be surrounded by log level guards. Warning test
| LOG.info("False negative " + fnCount); | ||
| LOG.info("True positive " + tpCount); | ||
| LOG.info("False positive " + fpCount); | ||
| LOG.info("precision " + score1); |
Check warning
Code scanning / PMD
Logger calls should be surrounded by log level guards. Warning test
| LOG.info("True positive " + tpCount); | ||
| LOG.info("False positive " + fpCount); | ||
| LOG.info("precision " + score1); | ||
| LOG.info("recall " + tpCount + " denom " + (tpCount+fnCount) + " overall " + score2); |
Check warning
Code scanning / PMD
Logger calls should be surrounded by log level guards. Warning test
|
|
||
| long trainingDataCount = df.count(); | ||
| assertTrue(trainingDataCount > 10); | ||
| LOG.info("trainingDataCount : "+ trainingDataCount); |
Check warning
Code scanning / PMD
Logger calls should be surrounded by log level guards. Warning test
integration Junit for OSS