Cross test rig for running Daffodil-flavor TDML on IBM DFDL #1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mbeckerle
commented
Oct 26, 2018
mbeckerle
commented
Oct 26, 2018
mbeckerle
commented
Oct 26, 2018
mbeckerle
commented
Oct 26, 2018
mbeckerle
commented
Oct 26, 2018
mbeckerle
commented
Oct 26, 2018
Dynamically loaded by Daffodil if found on the class path. Using DFDLCatalogResolver from Daffodil to insure include/imports resolve the same as on Daffodil. There are test-rig-tests here, and also the sample that comes with IBM DFDL can be run as TDML tests. You just have to copy files from the IBM DFDL installation to the lib and src/test/resources directories. All the tests pass on Daffodil and IBM DFDL. See the README.txt file for how to setup and run.
org.apache.daffodil.tdml.processor package, to be found on the classpath. Depends only on daffodil's TDML runner, not the rest of daffodil.
Current test results from this are: [error] Failed: Total 112, Failed 15, Errors 0, Passed 79, Skipped 18 [error] Failed tests: [error] org.apache.daffodil.IBMTestsThatPass2 [error] org.apache.daffodil.IBMTestsThatPass [error] org.apache.daffodil.TresysTests
Fixed bug in IBM DFDLToSAXEventAdapter not escaping strings. Now escapes them with Daffodil's escape util, which should allow daffodil-tests that expect daffodil-style XML-escaping (such as NUL to  to work. This illustrates how to add more DFDLSchemas to this cross-test rig. Setup source and resource folders for eclipse, so that you can test daffodil-test-ibm1 tests from here locally under eclipse, as well as run NACHA tests from here locally under eclipse. Also works from sbt.
This is symmetric with changes that map XML-illegals to the Unicode Private Use Area on parsing. Ex: On parse NUL (aka U+0) becomes U+E000. On unparse U+E000 becomes 0, that is NUL again. This should allow daffodil TDML tests that depend on this escaping and substitution convention to be portable to IBM DFDL as well.
This is added whenver trace is on.
Must be published i.e., 'sbt publishLocal' so test modules can load it as a managed dependency. A plugin will auto-edit classpaths to use the IBM DFDL tdml processor. See README.md for instructions on the plugin.
| * | ||
| * US Government Users Restricted Rights - Use, duplication, or | ||
| * disclosure restricted by GSA ADP Schedule Contract with IBM Corp. | ||
| **********************************************************************/ |
There was a problem hiding this comment.
The licensing isn't clear to me for some of these files. Are we allowed to redistribute these files from IBM?
There was a problem hiding this comment.
We are not. I'll have to pull this all down.
There was a problem hiding this comment.
Actually, my plan is to write a replacement of this class in scala.
Format the trees correctly in README.md
|
|
||
| @Override | ||
| public void entry(IDFDLTrace traceItem) { | ||
| // System.err.println("TraceListener : entry() "+traceItem.getMessage() ); |
There was a problem hiding this comment.
Any reason why this and "exit" below have commented out print statements, but the rest don't?
There was a problem hiding this comment.
Yes. They aren't desirable to see in a trace. They're really in case you implement indenting. For a basic simple trace output you don't need to see these events.
| Assert.invariant(schemaSource.isInstanceOf[URISchemaSource]) | ||
| val grammar = | ||
| try { | ||
| // LoggingDefaults.setLoggingLevel(LogLevel.Resolver) |
There was a problem hiding this comment.
Commented out logging and print statement below
jadams-tresys
approved these changes
Nov 30, 2018
mbeckerle
added a commit
that referenced
this pull request
Nov 30, 2018
* Cross test rig for running Daffodil-flavor TDML on IBM DFDL Dynamically loaded by Daffodil if found on the class path. Using DFDLCatalogResolver from Daffodil to insure include/imports resolve the same as on Daffodil. There are test-rig-tests here, and also the sample that comes with IBM DFDL can be run as TDML tests. You just have to copy files from the IBM DFDL installation to the lib and src/test/resources directories. All the tests pass on Daffodil and IBM DFDL. See the README.txt file for how to setup and run. * Must be named DaffodilTDMLDFDLProcessorFactory in the org.apache.daffodil.tdml.processor package, to be found on the classpath. Depends only on daffodil's TDML runner, not the rest of daffodil. * Incorporate review feedback as of 2018-11-19. * Update README.txt to match current behavior. * Works from sbt now, and tests daffodil-test-ibm1 tests. Current test results from this are: [error] Failed: Total 112, Failed 15, Errors 0, Passed 79, Skipped 18 [error] Failed tests: [error] org.apache.daffodil.IBMTestsThatPass2 [error] org.apache.daffodil.IBMTestsThatPass [error] org.apache.daffodil.TresysTests * Added NACHA to cross-tests (all tests pass). Fixed bug in IBM DFDLToSAXEventAdapter not escaping strings. Now escapes them with Daffodil's escape util, which should allow daffodil-tests that expect daffodil-style XML-escaping (such as NUL to  to work. This illustrates how to add more DFDLSchemas to this cross-test rig. Setup source and resource folders for eclipse, so that you can test daffodil-test-ibm1 tests from here locally under eclipse, as well as run NACHA tests from here locally under eclipse. Also works from sbt. * Remap PUA back to XML Illegal chars on unparsing. This is symmetric with changes that map XML-illegals to the Unicode Private Use Area on parsing. Ex: On parse NUL (aka U+0) becomes U+E000. On unparse U+E000 becomes 0, that is NUL again. This should allow daffodil TDML tests that depend on this escaping and substitution convention to be portable to IBM DFDL as well. * Add our TraceListener that is used for user and service trace listener. This is added whenver trace is on. * Updated to be a library that is used by tests. Must be published i.e., 'sbt publishLocal' so test modules can load it as a managed dependency. A plugin will auto-edit classpaths to use the IBM DFDL tdml processor. See README.md for instructions on the plugin. * Update README.md * Rewrite DFDLReader1 in scala to avoid exhibiting IBM's code.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Dynamically loaded by Daffodil if found on the class path.
Using DFDLCatalogResolver from Daffodil to insure include/imports
resolve the same as on Daffodil.
There are test-rig-tests here, and also the sample that comes with
IBM DFDL is here as a TDML test.
All the tests pass on Daffodil and IBM DFDL.
See the README.txt file for how to setup and run.