Merge pom_dlineage.xml into pom.xml, and make CI run the jar it builds - #48
Merged
Conversation
Closes #46 and #47. Both were reported against the standalone data-lineage build, and both come from the same place: it was a second POM that no workflow ever *ran*. #46 the documented `java -cp` put external_lib/* on the classpath, a directory only the Windows .bat route creates. Anyone following the Maven instructions in order got NoClassDefFoundError on the first command in the section. The jar Maven had actually downloaded sat in ~/.m2 under a filename carrying a version number that changes every release, so it could not be guessed either. #47 pom_dlineage.xml lacked the JAXB dependencies pom.xml had carried for ages. It compiled fine and died at runtime on every JDK after 8. Since pom.xml *excluded* DataFlowAnalyzer, that second POM was the only thing that ever compiled the class, so no other build path could catch it. That is the third and fourth time this file broke unnoticed, after the pinned 3.1.1.0 parser jar and the <parent> published nowhere. Fixing the two instances would leave the category, so the file is gone. One POM ------- DataFlowAnalyzer's exclusion was circular: excluded because a separate POM compiled it, and that POM existed to produce a standalone jar. maven-shade-plugin produces the standalone jar from the one build, so nothing is excluded from pom.xml any more: target/gsp_demo_java-1.0-SNAPSHOT.jar the demos target/gsp_demo_java-1.0-SNAPSHOT-dlineage.jar executable, self-contained java -jar target/gsp_demo_java-1.0-SNAPSHOT-dlineage.jar /f x.sql /o out.json /json No classpath, no version number. #39 goes with it -- the second POM's incremental cleanup deleting the root build's target/classes, since both shared ${project.basedir} -- because one POM cannot collide with itself. target-dlineage/ is gone. Dependencies by coordinate, not by file --------------------------------------- lib/ held three <scope>system</scope> jars. Two were on Maven Central all along and byte-for-byte identical to the committed copies: sha256 7a43d2d5...f4e429f9 simple-xml-2.7.1.jar sha256 641a4d65...5fe692631d fastjson-1.2.83.jar so they are ordinary dependencies now: same bytes, resolved instead of committed. expr4j genuinely has no public coordinate and moved to lib-repo/, a file-based Maven repository in the checkout, as org.boris:expr:0.0.0-vendored with a hand-written .pom. lib/ is deleted. system scope is not merely deprecated here, it was load-bearing for the bug: maven-shade-plugin, maven-assembly-plugin and dependency:copy-dependencies all skip system-scope artifacts, so no packaged jar or generated classpath could ever contain them. That is most of why #46 needed a hand-assembled classpath. The switch also restored simple-xml's transitive stax/xpp3, which system scope had been silently dropping, and -Dexec.classpathScope=compile is no longer needed by any demo. jaxb-runtime 2.3.3 -> 2.3.9 --------------------------- Adding the JAXB dependencies as #47 suggests is not sufficient. 2.3.3's Injector calls sun.misc.Unsafe.defineClass, removed in JDK 11, so XML output still fails on JDK 9+ -- writing a 0-byte file and exiting 0. The API stays at 2.3.3 because the parser calls javax.xml.bind.JAXBContext and that is the last javax-namespace release; the runtime must not be "aligned" to match it. CI runs it now -------------- .github/scripts/smoke-dlineage-jar.sh executes the jar in both workflows on JDK 8 and 21. It asserts on the output, not the exit status: in #47 the process created the file, wrote nothing into it, and exited 0, so both an existence check and an exit-code check would have called that a pass. It checks XML as well as JSON because those are separate code paths and JAXB only breaks one -- verified by reverting to 2.3.3, where the guard goes red. Also ---- - setenv/fetch-parser.bat runs dependency:copy-dependencies instead of copying the parser alone, since lib/ no longer supplies the rest. external_lib\* is the whole CLASSPATH now. This also gives the .bat route JAXB, which it never had -- run_DataFlowAnalyzer.bat hit #47 too. - buildJar.sh, buildJar.bat and dlineage/MANIFEST.MF deleted: a fourth way to build the same jar, all naming files that no longer exist, none run by CI. - set-parser-version.sh drops to four files. Verified: mvn clean package green, 144 tests pass with nothing skipped, build from a completely empty local repository resolves expr from lib-repo and creates no gudusoft/ group directory, run-all-demos.sh 77/77, run-demo-cases.sh 22/22, and a Linux simulation of the .bat compile+run flow against external_lib alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012qPRpoD8exYRrUmbfXXWXj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #46. Closes #47.
Both issues were filed against the standalone data-lineage build, and both trace to the same place: it was a second POM that no workflow ever ran.
java -cpputexternal_lib/*on the classpath — a directory only the Windows.batroute creates. Anyone following the Maven instructions in order hitNoClassDefFoundErroron the first command in the section. The jar Maven had actually downloaded sat in~/.m2under a filename carrying a version number that changes every release, so it couldn't be guessed either.pom_dlineage.xmllacked the JAXB dependenciespom.xmlhad carried for ages. Compiles perfectly; dies at runtime on every JDK after 8. Sincepom.xmlexcludedDataFlowAnalyzer, that second POM was the only thing that ever compiled the class, so no other build path could have caught it.That's the third and fourth time this file broke unnoticed, after the pinned 3.1.1.0 parser jar and the
<parent>published nowhere. Fixing the two instances would leave the category, so the file is gone.One POM
DataFlowAnalyzer's exclusion was circular — excluded because a separate POM compiled it, and that POM existed to produce a standalone jar.maven-shade-pluginproduces the standalone jar from the one build, so nothing is excluded frompom.xmlany more:No classpath, no version number — so #46 can't recur through documentation drift. #39 goes with it: the second POM's incremental cleanup deleting the root build's
target/classes(both shared${project.basedir}) can't happen when one POM cannot collide with itself.target-dlineage/is gone.Dependencies by coordinate, not by file
lib/held three<scope>system</scope>jars. Two were on Maven Central all along, byte-for-byte identical to the committed copies:They're ordinary dependencies now — the same bytes, resolved instead of committed.
expr4jgenuinely has no public coordinate and moved tolib-repo/, a file-based Maven repository in the checkout, asorg.boris:expr:0.0.0-vendoredwith a hand-written.pom.lib/is deleted.systemscope wasn't just deprecated here, it was load-bearing for the bug:maven-shade-plugin,maven-assembly-pluginanddependency:copy-dependenciesall skip system-scope artifacts, so no packaged jar or generated classpath could ever contain them. That's most of why #46 needed a hand-assembled classpath. The switch also restoredsimple-xml's transitivestax/xpp3, which system scope had been silently dropping, and-Dexec.classpathScope=compileis no longer needed by any demo.jaxb-runtime2.3.3 → 2.3.9Adding the JAXB dependencies as #47 suggests is not sufficient. 2.3.3's
Injectorcallssun.misc.Unsafe.defineClass, removed in JDK 11, so XML output still fails on JDK 9+ — writing a 0-byte file and exiting 0. The API stays at 2.3.3 because the parser callsjavax.xml.bind.JAXBContextand that's the last javax-namespace release; the runtime must not be "aligned" to match it. There's a comment inpom.xmlsaying so.CI runs it now
.github/scripts/smoke-dlineage-jar.shexecutes the jar in both workflows on JDK 8 and 21. It asserts on the output, not the exit status — in #47 the process created the file, wrote nothing into it, and exited0, so both an existence check and an exit-code check would have called that a pass. It checks XML as well as JSON because those are separate code paths and JAXB only breaks one.Verified by reverting to 2.3.3, where the guard goes red.
Also in here
setenv/fetch-parser.batrunsdependency:copy-dependenciesinstead of copying the parser alone, sincelib/no longer supplies the rest.external_lib\*is the wholeCLASSPATH. This also gives the.batroute JAXB, which it never had —run_DataFlowAnalyzer.bathit pom_dlineage.xml is missing the JAXB dependency pom.xml already added for the same JDK-9+ removal, so DataFlowAnalyzer dies with NoClassDefFoundError on Java 9+ #47 too.buildJar.sh,buildJar.batanddlineage/MANIFEST.MFdeleted: a fourth way to build the same jar, all naming files that no longer exist, none run by CI.set-parser-version.shdrops from five files to four.Verification
mvn clean packageexprfromlib-repo/, creates nogudusoft/group dir, no checksum warningssmoke-dlineage-jar.shrun-all-demos.shDataFlowAnalyzer)run-demo-cases.sh.batclasspath, simulated on Linuxchecksyntax,antiSQLInjection,DataFlowAnalyzerall compile and run, 0 exceptionsWindows
.batcompilation itself is only exercised by thewindows-batjob — worth a look when CI reports.Follow-up, not in this PR
fastjsonis now a real coordinate, so Dependabot will start reporting its 1.x CVEs.1.2.83is the final 1.x and where the deserialization advisories were fixed; the real move is tofastjson2. Worth its own issue.🤖 Generated with Claude Code
https://claude.ai/code/session_012qPRpoD8exYRrUmbfXXWXj