chore: remove stale upstream snapshot + port spark_dialect from upstream #150#188
Merged
Conversation
…eam #150 ## Stale artifact removal (182 files, 3 MB) `AdaWorldAPI-lance-graph-d9df43b/` was a committed snapshot of an older upstream version (48 .rs files vs our 98). Full audit confirmed: - ZERO files exist only in the artifact (every file has a counterpart) - Every differing file: ours >= artifact in LOC (ours is strictly ahead) - All upstream features (#125 parameter_substitution, #140 lance_vector_search) are already in our src tree The directory created GitHub path confusion — duplicate navigation paths for datafusion_planner, spo, blasgraph, neighborhood, arigraph. Removing it eliminates that confusion with zero content loss. ## Cherry-pick: spark_dialect.rs from upstream PR #150 The ONE file upstream has that we didn't: - `crates/lance-graph/src/spark_dialect.rs` (107 LOC) Spark SQL dialect for DataFusion unparser: backtick quoting, STRING type casting, EXTRACT for dates, BIGINT/INT types, LENGTH(), derived table aliases. - `crates/lance-graph/tests/test_to_spark_sql.rs` (293 LOC) Full test suite for Spark SQL output. - `pub mod spark_dialect;` added to lib.rs Adapted from upstream's DF 50.3 to our DF 51 — same API surface, no changes needed. ## Upstream audit result (for the record) Upstream (lance-format/lance-graph) is at v0.5.4. Our fork is at v0.5.3 with newer deps (arrow 57 vs 56.2, datafusion 51 vs 50.3). Other than spark_dialect, every upstream feature and fix is already present in our source tree — parameter_substitution (#125), lance_vector_search (#140), complex RETURN clauses (#142), duplicate columns fix (#128) are all in `crates/lance-graph/src/`. Their deleted `simple_executor` was a prototype cold-path executor we never had. Our `ExecutionStrategy::DataFusion` path (6K LOC planner) + `ExecutionStrategy::BlasGraph` (semiring algebra) subsume it. The user has flagged adding a deliberate `ExecutionStrategy::Simple` cold path as a 4th strategy for trivial queries — that's a separate PR per the documented matrix of execution strategies. https://claude.ai/code/session_01NYGrxVopyszZYgLBxe4hgj
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 11fe3e21e1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| .with_config(config); | ||
|
|
||
| let sql = query | ||
| .to_sql(datasets, Some(SqlDialect::Spark)) |
There was a problem hiding this comment.
These new integration tests call CypherQuery::to_sql with a second SqlDialect argument, but the current API in crates/lance-graph/src/query.rs only accepts datasets and the crate does not export a SqlDialect type. That makes this test target fail at compile time (before any assertions run), so cargo test cannot complete after this commit unless the API changes are added in the same patch.
Useful? React with 👍 / 👎.
AdaWorldAPI
pushed a commit
that referenced
this pull request
Apr 17, 2026
Codifies 7 anti-patterns (AP1-AP7) learned from PRs #176-#188 into an agent card that fires flags when the session repeats them: AP1: "225/225 feels like success" without gate 2 (#178) AP2: Projecting quality from docs instead of measuring (#177) AP3: Building new codec before benching existing ones (#184) AP4: Centroid-residual framing on near-orthogonal data (#177/#183) AP5: Python in the inference hot path AP6: Chained score multiplication without chain-collapse check (P5) AP7: Modifying ndarray without explicit permission (#176) Invoked by adk-coordinator when pattern repetition is suspected, or by human directly. Output: list of fired flags, max 7 lines. Also audited all 29 agent cards across both repos: - All pin model: opus or model: sonnet (no hardcoded versions) - opus → Opus 4.7 automatically, sonnet → Sonnet 4.6 - 3 ndarray agents on sonnet (l3-strategist, migration-tracker, product-engineer) — intentional for speed-over-depth roles - adk-coordinator missing Bash tool (by design — delegates) - sentinel-qa missing Edit/Write (by design — audit-only) No agent changes needed for Opus 4.7 compatibility — model: opus resolves correctly. https://claude.ai/code/session_01NYGrxVopyszZYgLBxe4hgj
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Upstream sync: remove stale snapshot artifact + cherry-pick the one upstream feature we were missing.
Stale artifact removal (182 files, 3 MB)
AdaWorldAPI-lance-graph-d9df43b/was a committed copy of an older upstream version sitting inside our repo. Created duplicate navigation paths on GitHub fordatafusion_planner/,spo/,blasgraph/,neighborhood/,arigraph/— every directory appeared twice.Full audit before deletion:
crates/tree)Cherry-pick:
spark_dialect.rsfrom upstream PR #150The one file upstream (lance-format/lance-graph v0.5.4) ships that we didn't:
crates/lance-graph/src/spark_dialect.rsSTRINGtype,EXTRACTdates,BIGINT/INT,LENGTH(), derived table aliasescrates/lance-graph/tests/test_to_spark_sql.rslib.rspub mod spark_dialect;Upstream audit (for the record)
simple_executorTest plan
spark_dialect.rssyntactically correct (datafusion_sql imports resolve at crate level)pub mod spark_dialect;wired in lib.rscargo test -p lance-graph(blocked by pre-existing protoc issue on this VM — not introduced by this PR)https://claude.ai/code/session_01NYGrxVopyszZYgLBxe4hgj