"Error: invalid names array" when creating database from polylines #127
Comments
|
hi @kylebarron it sounds like there is a problem with the the I'll also change that fatal error to a warning, if it's only 1 or 2 streets that are missing their names then we can probably just skip over those and warn the user they are malformed, rather than fatally erroring. |
|
Thanks @missinglink; sorry, which is the |
|
Sorry, I mean the |
|
Closing issue, still interested to hear what happened. I suspect the You can also run the import again now with the latest code from |
|
I read my initial bug report again, and I should've specified that the error arises from a specific line around ~128000. Above I cut out some of the output to be concise; the full output to console is below. The file size of |
|
I updated from master and it did spit out the warning: It seems to be progressing fine otherwise. To clarify since I couldn't find this in the documentation, is there a way to append to the sqlite db, or should I be doing to get a full US interpolation dataset. |
|
ok cool, looks like the commit I pushed is now allowing the importer to skip over the offending line(s). could you please copy->paste the output of this grep command so I can get a copy of the exact line that's causing it? grep -aC5 'oee}mAthsagDN' *.polylinesI don't think the syntax you posted is doing what you expect (try https://explainshell.com). My understanding is that the first (and only the first) filename after the What you can do instead is to use the |
|
you could also write it like this (assuming you want to import all the or AFAIK these two syntaxes are equivalent |
|
Yes you're right. I use |
|
@kylebarron I'm still very confused what's happening, there should be a nullbyte ( For example here is another $ head -n1 file.0sv | sed 's/\x0/___/g'
kgionAlvsegCdb@oJpaAkU___10th Avenuebut when I try this on the output you posted there is no null byte: $ head -n1 error.0sv | sed 's/\x0/___/g'
m~v~iAzvchrDQpuAm@lfC?tPCjtABj~@Lvb@I~yBI`m@\fZ?z_C?~ESjr@?`ZQpy@Lfm@]n_@Efq@]npAIxo@MnEm@pE{@pEoAvEqBzEkCjFqj@|dAcYbh@yPbXkEzF{KlMwQ|RgIlKaFzJaCbJsAnLwDbi@�West 126th Streetso either there is a copy->paste error or something on your computer is removing them? edit: I do see a |
|
The format is: |
|
|
|
And that screenshot in text form: |
|
Just FYI I'm not sure that actually uses all files. At the least, on the following step, When I do I get an error |
|
heya, so...
$ sqlite3 street.db
SQLite version 3.22.0 2018-01-22 18:45:57
Enter ".help" for usage hints.
sqlite> .tables
names rtree rtree_parent
polyline rtree_node rtree_rowid
sqlite> select count(*) from polyline;
10
sqlite> select count(*) from names;
10
sqlite> .exit
I have included a script in this repo to handle concatenation of openaddresses csv files: try this: export OAPATH=../../data/openaddresses/us
./script/concat_oa.shyou should get a stream of all the files combined, deduplicated and only containing the header line once.
|
|
yep, re: the same thing happens if you take all the data away and just do: /dev/null < $(cat /dev/urandom)you'll see your system memory filling up until eventually your shell runs out of heap the same doesn't apply to: cat /dev/urandom > /dev/nullor cat /dev/urandom | cat > /dev/null |
I got an error "invalid names array" when making the
street.dbfile from theus-midwest-latest.polylinesfile, which was created withpbf streets us-midwest-latest.osm.pbf > us-midwest-latest.polylines, which was downloaded from the Geofabrik mirror.Is this most likely an issue with the original
.osm.pbffile, with thepbfconversion, or with the interpolation script?The text was updated successfully, but these errors were encountered: