Skip to content

Merge pom_dlineage.xml into pom.xml, and make CI run the jar it builds - #48

Merged
sqlparser merged 1 commit into
masterfrom
merge-dlineage-pom
Jul 28, 2026
Merged

Merge pom_dlineage.xml into pom.xml, and make CI run the jar it builds#48
sqlparser merged 1 commit into
masterfrom
merge-dlineage-pom

Conversation

@sqlparser

Copy link
Copy Markdown
Owner

Closes #46. Closes #47.

Both issues were filed against the standalone data-lineage build, and both trace to the same place: it was a second POM that no workflow ever ran.

#46 The documented java -cp put external_lib/* on the classpath — a directory only the Windows .bat route creates. Anyone following the Maven instructions in order hit NoClassDefFoundError on the first command in the section. The jar Maven had actually downloaded sat in ~/.m2 under a filename carrying a version number that changes every release, so it couldn't be guessed either.
#47 pom_dlineage.xml lacked the JAXB dependencies pom.xml had carried for ages. Compiles perfectly; dies at runtime on every JDK after 8. Since pom.xml excluded DataFlowAnalyzer, that second POM was the only thing that ever compiled the class, so no other build path could have caught it.

That's the third and fourth time this file broke unnoticed, after the pinned 3.1.1.0 parser jar and the <parent> published nowhere. Fixing the two instances would leave the category, so the file is gone.

One POM

DataFlowAnalyzer's exclusion was circular — excluded because a separate POM compiled it, and that POM existed to produce a standalone jar. maven-shade-plugin produces the standalone jar from the one build, so nothing is excluded from pom.xml any more:

target/gsp_demo_java-1.0-SNAPSHOT.jar            the demos
target/gsp_demo_java-1.0-SNAPSHOT-dlineage.jar   executable, self-contained
java -jar target/gsp_demo_java-1.0-SNAPSHOT-dlineage.jar /f x.sql /o out.json /json

No classpath, no version number — so #46 can't recur through documentation drift. #39 goes with it: the second POM's incremental cleanup deleting the root build's target/classes (both shared ${project.basedir}) can't happen when one POM cannot collide with itself. target-dlineage/ is gone.

Dependencies by coordinate, not by file

lib/ held three <scope>system</scope> jars. Two were on Maven Central all along, byte-for-byte identical to the committed copies:

sha256 7a43d2d5…f4e429f9   simple-xml-2.7.1.jar
sha256 641a4d65…5fe692631d fastjson-1.2.83.jar

They're ordinary dependencies now — the same bytes, resolved instead of committed. expr4j genuinely has no public coordinate and moved to lib-repo/, a file-based Maven repository in the checkout, as org.boris:expr:0.0.0-vendored with a hand-written .pom. lib/ is deleted.

system scope wasn't just deprecated here, it was load-bearing for the bug: maven-shade-plugin, maven-assembly-plugin and dependency:copy-dependencies all skip system-scope artifacts, so no packaged jar or generated classpath could ever contain them. That's most of why #46 needed a hand-assembled classpath. The switch also restored simple-xml's transitive stax/xpp3, which system scope had been silently dropping, and -Dexec.classpathScope=compile is no longer needed by any demo.

jaxb-runtime 2.3.3 → 2.3.9

Adding the JAXB dependencies as #47 suggests is not sufficient. 2.3.3's Injector calls sun.misc.Unsafe.defineClass, removed in JDK 11, so XML output still fails on JDK 9+ — writing a 0-byte file and exiting 0. The API stays at 2.3.3 because the parser calls javax.xml.bind.JAXBContext and that's the last javax-namespace release; the runtime must not be "aligned" to match it. There's a comment in pom.xml saying so.

CI runs it now

.github/scripts/smoke-dlineage-jar.sh executes the jar in both workflows on JDK 8 and 21. It asserts on the output, not the exit status — in #47 the process created the file, wrote nothing into it, and exited 0, so both an existence check and an exit-code check would have called that a pass. It checks XML as well as JSON because those are separate code paths and JAXB only breaks one.

Verified by reverting to 2.3.3, where the guard goes red.

Also in here

Verification

check result
mvn clean package green, both jars
test suite 144 pass, 0 failed, 0 skipped
build from an empty local repository resolves expr from lib-repo/, creates no gudusoft/ group dir, no checksum warnings
smoke-dlineage-jar.sh JSON 14 relationships, XML 24 relationships
run-all-demos.sh 77/77 start (incl. newly-built DataFlowAnalyzer)
run-demo-cases.sh 22/22
.bat classpath, simulated on Linux checksyntax, antiSQLInjection, DataFlowAnalyzer all compile and run, 0 exceptions

Windows .bat compilation itself is only exercised by the windows-bat job — worth a look when CI reports.

Follow-up, not in this PR

fastjson is now a real coordinate, so Dependabot will start reporting its 1.x CVEs. 1.2.83 is the final 1.x and where the deserialization advisories were fixed; the real move is to fastjson2. Worth its own issue.

🤖 Generated with Claude Code

https://claude.ai/code/session_012qPRpoD8exYRrUmbfXXWXj

Closes #46 and #47. Both were reported against the standalone data-lineage
build, and both come from the same place: it was a second POM that no workflow
ever *ran*.

  #46  the documented `java -cp` put external_lib/* on the classpath, a
       directory only the Windows .bat route creates. Anyone following the
       Maven instructions in order got NoClassDefFoundError on the first
       command in the section. The jar Maven had actually downloaded sat in
       ~/.m2 under a filename carrying a version number that changes every
       release, so it could not be guessed either.

  #47  pom_dlineage.xml lacked the JAXB dependencies pom.xml had carried for
       ages. It compiled fine and died at runtime on every JDK after 8. Since
       pom.xml *excluded* DataFlowAnalyzer, that second POM was the only thing
       that ever compiled the class, so no other build path could catch it.

That is the third and fourth time this file broke unnoticed, after the pinned
3.1.1.0 parser jar and the <parent> published nowhere. Fixing the two instances
would leave the category, so the file is gone.

One POM
-------
DataFlowAnalyzer's exclusion was circular: excluded because a separate POM
compiled it, and that POM existed to produce a standalone jar.
maven-shade-plugin produces the standalone jar from the one build, so nothing
is excluded from pom.xml any more:

    target/gsp_demo_java-1.0-SNAPSHOT.jar            the demos
    target/gsp_demo_java-1.0-SNAPSHOT-dlineage.jar   executable, self-contained

    java -jar target/gsp_demo_java-1.0-SNAPSHOT-dlineage.jar /f x.sql /o out.json /json

No classpath, no version number. #39 goes with it -- the second POM's
incremental cleanup deleting the root build's target/classes, since both shared
${project.basedir} -- because one POM cannot collide with itself.
target-dlineage/ is gone.

Dependencies by coordinate, not by file
---------------------------------------
lib/ held three <scope>system</scope> jars. Two were on Maven Central all along
and byte-for-byte identical to the committed copies:

    sha256 7a43d2d5...f4e429f9  simple-xml-2.7.1.jar
    sha256 641a4d65...5fe692631d fastjson-1.2.83.jar

so they are ordinary dependencies now: same bytes, resolved instead of
committed. expr4j genuinely has no public coordinate and moved to lib-repo/, a
file-based Maven repository in the checkout, as org.boris:expr:0.0.0-vendored
with a hand-written .pom. lib/ is deleted.

system scope is not merely deprecated here, it was load-bearing for the bug:
maven-shade-plugin, maven-assembly-plugin and dependency:copy-dependencies all
skip system-scope artifacts, so no packaged jar or generated classpath could
ever contain them. That is most of why #46 needed a hand-assembled classpath.
The switch also restored simple-xml's transitive stax/xpp3, which system scope
had been silently dropping, and -Dexec.classpathScope=compile is no longer
needed by any demo.

jaxb-runtime 2.3.3 -> 2.3.9
---------------------------
Adding the JAXB dependencies as #47 suggests is not sufficient. 2.3.3's
Injector calls sun.misc.Unsafe.defineClass, removed in JDK 11, so XML output
still fails on JDK 9+ -- writing a 0-byte file and exiting 0. The API stays at
2.3.3 because the parser calls javax.xml.bind.JAXBContext and that is the last
javax-namespace release; the runtime must not be "aligned" to match it.

CI runs it now
--------------
.github/scripts/smoke-dlineage-jar.sh executes the jar in both workflows on JDK
8 and 21. It asserts on the output, not the exit status: in #47 the process
created the file, wrote nothing into it, and exited 0, so both an existence
check and an exit-code check would have called that a pass. It checks XML as
well as JSON because those are separate code paths and JAXB only breaks one --
verified by reverting to 2.3.3, where the guard goes red.

Also
----
- setenv/fetch-parser.bat runs dependency:copy-dependencies instead of copying
  the parser alone, since lib/ no longer supplies the rest. external_lib\* is
  the whole CLASSPATH now. This also gives the .bat route JAXB, which it never
  had -- run_DataFlowAnalyzer.bat hit #47 too.
- buildJar.sh, buildJar.bat and dlineage/MANIFEST.MF deleted: a fourth way to
  build the same jar, all naming files that no longer exist, none run by CI.
- set-parser-version.sh drops to four files.

Verified: mvn clean package green, 144 tests pass with nothing skipped, build
from a completely empty local repository resolves expr from lib-repo and
creates no gudusoft/ group directory, run-all-demos.sh 77/77, run-demo-cases.sh
22/22, and a Linux simulation of the .bat compile+run flow against external_lib
alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012qPRpoD8exYRrUmbfXXWXj
@sqlparser
sqlparser merged commit e2e272b into master Jul 28, 2026
3 checks passed
@sqlparser
sqlparser deleted the merge-dlineage-pom branch July 28, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant