-
-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WKT-->spatial data conversion problems #4183
Comments
@mkoo it's actually that MOST WKT (I'm only looking at geography right now) isn't valid. https://arctos.database.museum/media/10598522 ferexample starts/ends with
and it turns out "close enough" isn't close enough for PGIS. (I can't find the "hand grenade mode" setting....) What would be required to regenerate those data (as postgis If that isn't practical, I could probably - at least for straight POLYGON data - fake it, and just make a straight line from wherever the shape ends back to where it begins (which might be a step left out of some conversion, or it might just be complete garbage). Starting over may or may not be less-evil than making up the last segment, and at this point I have no idea if the last segment is the entirety of the problem (for 3,167 records in test) or just the beginning. |
The spatial (datatype I was able to fake valid WKT in ways that aren't visually obvious to me for most geography WKTs; unless someone stops me now, I'm going to proceed with this and we can sort out whatever falls through the cracks later. In test, that's these:
I'm not yet sure how loading geography data is going to work, if someone has any of those in whatever format and wants to pass it on I can experiment with that. Also: who knew Kentucky has a weird thing? |
i was offline/out of town so ignoring GH issues.... all the WKTs can be easily regenerated into other formats (no guarantees about speediness however we batch these since I havent thought out the process) so not a probably. So what's the preferred format? Anything useful in our Berkeleymapper repo? https://github.com/BNHM/spatial-layers eg. |
not sure if that's the only weird thing about Kentucky... |
You tell me, I'm new at this! I'm storing as MOST everything in test converted with just a little script-hammer-beating-magic, so I'm rebuilding UIs to use postgis. I'll make a list of the failures when this gets to production and we can deal with them after the fact. Maybe we'll want to update other stuff to use data with less jagged edges, IDK, but I think I have enough data to get the system in place without losing anything so I'm gonna rock on. The locality failures (very few in test) will be bigger problems, they are "data" where geog is more metadata. I'll hopefully have a UI this week, then we can figure out how to feed it better data. |
These geography records will lose their (invalid) spatial data in production.
|
These localities will lose their not-WKT spatial data in production. |
This is done, except the conversion failures above. Leaving it open in case someone wants to investigate further. |
Thanks for the list -- we can restore/fix these.
…On Tue, Jan 18, 2022 at 6:31 PM dustymc ***@***.***> wrote:
This is done, except the conversion failures above. Leaving it open in
case someone wants to investigate further.
—
Reply to this email directly, view it on GitHub
<#4183 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AATH7UJ4S35RWFEP4P65GY3UWYO77ANCNFSM5JZYQFLQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you were assigned.Message ID:
***@***.***>
|
Maybe we should try as geojson? or other format? |
The docs say that should work. I definitely don't know enough about this to say much of anything, other than I'm willing to experiment with anything I can convert to geography.
You (mostly) can't, see #4263
No, it's columns of datatype
|
Anyone who was going to act probably has by now, tentatively closing. |
From @mkoo
WKT is also a pain for me and now we (will soon, I hope...) have spatial tools, so
polygon(geography)
to geog_auth_rec and locality, move the WKT data in, lose the media_idsI think the biggest drawback is backups - this will make them larger. Seems a price well worth paying.
This will probably mean it's not possible to use media IDs in the catalog record bulkloader, but I don't think anyone ever has and it's possible (and introduces much less chance for confusion) to pre-create localities and use them.
Originally posted by @dustymc in #2660 (comment)
Splitting this off to a dedicated issue.
I played with this a tiny bit, a fair number of WKTs aren't valid for one reason or another, Google seems to ignore that, PGIS rightly complains. Another compelling reason to bring them into a GIS system rather than something text-based.
One possibility for backups and passing text data around would be to store the spatial data in a dedicated table and reference it; needs more investigation, input most welcome.
The text was updated successfully, but these errors were encountered: