Commit c148393
fix: ParquetImporter detects duplicate column names before VortexWriter
Previously the first signal a caller got was VortexWriter's internal
uniqueness check, several stack frames removed from the real cause and
with no mention of which import or source file triggered it:
java.lang.IllegalArgumentException: duplicate field name: A
at io.github.dfa1.vortex.writer.VortexWriter.<init>(VortexWriter.java:164)
at io.github.dfa1.vortex.parquet.ParquetImporter.importParquet(...)
checkNoDuplicateNames now runs right after the schema is built from the
Parquet columns, throwing a ParquetImporter-specific message naming the
duplicate(s) and the source file. Root cause is always the source
Parquet file's own schema (DType.Struct legitimately permits duplicate
names in memory; VortexWriter correctly enforces the file format's
uniqueness requirement) — this only improves error quality at the
import boundary, no auto-disambiguation.
Verified against the real repro (Raincloud's uk-price-paid slug, whose
source CSV lacked a header row so the Parquet conversion tool used the
first data row's values as column names):
IllegalArgumentException: Parquet schema has duplicate column
name(s): [A]; source file: .../uk-price-paid.parquet
Fixes #280.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>1 parent 85e7d4e commit c148393
3 files changed
Lines changed: 54 additions & 0 deletions
File tree
- parquet/src
- main/java/io/github/dfa1/vortex/parquet
- test/java/io/github/dfa1/vortex/parquet
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
8 | 14 | | |
9 | 15 | | |
10 | 16 | | |
| |||
Lines changed: 18 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| 25 | + | |
25 | 26 | | |
| 27 | + | |
26 | 28 | | |
27 | 29 | | |
| 30 | + | |
28 | 31 | | |
29 | 32 | | |
30 | 33 | | |
| |||
71 | 74 | | |
72 | 75 | | |
73 | 76 | | |
| 77 | + | |
74 | 78 | | |
75 | 79 | | |
76 | 80 | | |
| |||
115 | 119 | | |
116 | 120 | | |
117 | 121 | | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
118 | 136 | | |
119 | 137 | | |
120 | 138 | | |
| |||
Lines changed: 30 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| 28 | + | |
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
| |||
181 | 182 | | |
182 | 183 | | |
183 | 184 | | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
184 | 214 | | |
185 | 215 | | |
186 | 216 | | |
| |||
0 commit comments