New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Geolocation/Spatial types support #1798
Comments
@teebot is there a workaround for this? |
Yes!
|
I would love to see and example @teebot :) |
Hi all. First time poster here, coming to Prisma from the Redwood community. Thanks for the work you all are doing -- I've really enjoyed working with Prisma 2 and Prisma migrate so far. Is there a place I can check to see where geo support falls on the Prisma roadmap or, if not, is that something you could speak to here? (Apologies if I missed this in my initial search.) |
@tctrautman once thats in im hoping that you dont have to fight against migrate that its trying to remove columns thats not in the schema etc. running raw queries is really not an issue for me personally |
Thanks @jaywalklabs -- this is really helpful. |
Hi, |
@thebiglabasky My only real issue with unsupported GIS types is being unable to use the migration tool. Are you planning to / is it feasible to implement a type or function to flag a field as any type so the migration tool can ignore it? |
@bkiac Interesting, that's good context, thanks. |
I believe supporting GIS types is extremely important, especially given the fact that all the databases that prisma support/plan to support has a way to work with GIS :
@thebiglabasky While I understand that prioritising things can be hard, what I would really like to see in the meantime is improvements when it comes to unhandled cases like this one. //CustomMigration.ts
export const up = async () => {
//Raw SQL logic that creates tables/columns/install extensions etc
}
export const down = async () => {
//Raw SQL logic that tear down things created by "up"
} This would generate a migration folder with a README.md, schema.prisma and steps.json, and would add the migration to the lockfile. In the data-model the created columns with unsupported types would simply appear with a special annotation/flag. While this is a little bit rough, at least it would allow the handling of a lot of edge cases and make prisma migrate a much more versatile tool. This would make prisma + prisma migrate a much better standalone tool (no need to manually run sql scripts/introspect anymore). Having first class geometry/postGIS support would be much better in this example, but implementing something like this in the meantime seems much easier, and would be quite valuable (for example it gives you a way to add postgres functions using a similar manual migration approach). |
Thanks for these details! |
+1 |
1 similar comment
+1 |
Will this be rolling out soon? |
You can now try
Release Notes: Also check: Daniel created a PostGIS example using Truncated example:
We are looking for feedback and are planning to release Prisma Migrate out of Preview soon |
Hi! Thanks for the example using the unsupported types. Is it possible to use a middleware to handle these unsupported types? A common use case would be:
Can this be done with the current middleware API, or do we have to resort to raw queries ? |
Hi @benjamintd So you'll need to use raw queries since |
I have a Is there any simple way or workaround that would allow prisma to at least fetch this field in regular queries? |
May you attention to our request, please. |
we need this too! |
Do you store them as decimal numbers, or as string? And is there any technique available to filter points by coordinates+radius, using that storage type? Also for storing simple GPS points - there is a separate issue #5984 - so maybe implementing that feature will be the good first step forward to this issue? |
Stored as decimals. Migration file will look something like this: CREATE EXTENSION IF NOT EXISTS "cube";
CREATE EXTENSION IF NOT EXISTS "earthdistance";
-- ...
CREATE TABLE "thing" (
-- ...
"latitude" DOUBLE PRECISION NOT NULL,
"longitude" DOUBLE PRECISION NOT NULL,
-- ...
); Then in my query I have to do some funky stuff. // This is a hack to force the Prisma to see the lat and long as real numbers and not whole numbers.
const FIX = 0.9999999999
// ...
// Getting the lat and long from the REST query params
const { lat, long } = params
// Running a raw query
const results = await this.db.$queryRaw<any[]>`
SELECT *, earth_distance(
ll_to_earth(${+lat * FIX}, ${+long * FIX}),
ll_to_earth(s.latitude, s.longitude)
) AS distance
FROM thing
ORDER BY distance ASC
` So the I had to add the Not sure if that helps your use case, but that will give me, for example, a list of "things" in order of proximity to my current location. |
+1 |
I don't know how much this would affect the folks in this thread, but #10634 did introduce the Gist index to Prisma. In my application, that was enough to work with spatial data pretty comfortably - Prisma's migrations now work fine with custom geometry columns and geo indexes, I do custom sql when i need to use geo queries, etc. |
@tmcw Sorry to ping you but I'm wondering what you schema looks like to use a geometry column with Prisma? I understand you still need to write raw SQL queries. I don't quite understand what the Gist index brings though. |
Here's an example snippet: model Blockgroup {
id String @id
geom Unsupported("geometry(MultiPolygon, 4326)")
water Float
Area Area[]
@@index([geom], name: "blockgroup_idx", type: Gist)
} PostGIS is more or less three components - a column type called geometry, an index type that works with that column type (in this example, a Gist index), and geometry functions. If you use the geometry column but don't use the Gist index, the other index types will complain about being too large and they won't perform well. To do geo stuff like queries using intersection, you basically need to use the Gist index. |
Thanks @tmcw that's really helpful. |
Hello everyone We are kicking off design of GIS support for Prisma! If you would like to help, please let us learn about your GIS needs! Via survey: Via email:
Thank you! |
[just to note: I was asked to leave a comment here by Nick: https://twitter.com/nikolasburk/status/1568143931199692800?s=20&t=EjAdvQpt2amoLTxIeVxIXQ ] It's September 2022. I wanted to try Prisma, but not being able to search MongoDB by proximity means that I will stick with Mongoose for now. A snippet from my Node.js, where I am building up a query (and note: this was working fine in Mongoose in 2017): retVal = {
"$near": {
"$geometry": {
"type": "Point",
"coordinates": [ longitudeFloat, latitudeFloat ]
},
"$maxDistance": 2000
}
}; Mapping apps often need to search by proximity. Prisma needs to support this. |
@floelhoeffel Hi! Do you have any update on this? I am right on the edge of choosing an ORM for my next project where I need a lot of GIS stuff. Could you share anything? |
Is there a plan to map it to a specific Release? Working with GeoLocation data has been increasingly common and powerful nowadays. |
@murbanowicz we are wrapping up discovery as we speak. We hope to pickup implementation in Q4 but can only tell after planning (which happens early December). Thank you for your patience |
It's amazing that this is taking so long (over 2.5 years...) Prisma is not someone's hobby project - it's a company with over $50M in financing. And..and.. this is very much a solved problem (Mongoose for > 5 years now, and others). Kick it up a notch. |
@floelhoeffel I'm planning to use Prisma for my current work projects, and support for geospatial data types will be a requirement, I'd love to get involved in the implementation process - I think I have a pretty good knowledge of the issues you might encounter, as my previous work at Directus specifically involved building an GeoJSON IO layer compatible with Postgres, MySQL, Oracle, SQLite and SQL Server, and supporting all specific geometric types ((multi) points, lines, polygons). Please get in touch if you're interested in getting me involved. |
That sounds wonderful. How can we contact you? You are hard to google |
Thanks Jan, I sent you an email |
I see that on the roadmap this has moved from work in progress back to planned (after being WIP for about 2months). Does this mean that we won't be getting this anytime soon? I think a lot of us were hopeful that this would be in the December or January release. Can we get a rough eta, like will this be soon or months? |
If you define "soon" as "less than a month or two months", then yes we will not get this done in that timeframe. With the complexity we uncovered, we unfortunately needed to revisit our prioritization and reflected that in our roadmap. We still want to and will build this - but not in the timeframe we initially hoped. |
Leave a comment there in the issue how you would imagine this would work, what new types and API methods would be useful or required for that standalone. Thanks. |
Hi @janpio, I didn't get an answer to the email I sent. Did you (or me) not receive it, or are you not interested anymore in collaborating on this specific issue ? |
@janpio I appreciate the update, as you still seem to be in the api design phase I would think looking at something like entity framework from c# would be useful as they have the best implementation/support I've seen for spatial queries and operations and is currently what I use. |
Do we have support for this yet or should i use another ORM |
Currently prisma2 does not support spatial types. This is not exactly an edge case since according to google 9% of sql files on github carry out spatial queries.
I was thinking how that feature could work but it’s probably a bit naive given I don’t know the implementation details of prisma2 :
The rationale behind this is that you basically want migrate to create the right spatial type in your db, currently schema/code-first does not work and introspect detects spatial as a string. Yuo also want client to perform CRUD on tables containing spatial types, currently CRUD fails if the spatial type is in the select so a
.raw
request is the only workaround.The text was updated successfully, but these errors were encountered: