Postgres ltree support / Open Source Contributions

[halfmatthalfcat]

Hey all, not sure if this is the right forum for this but wanted to start a conversation around adding Postgres ltree support into Prisma.

I've started the process of updating the chain of libraries in order for Prisma to fully support this new datatype.

1. rust-postgres PR
2. quaint working PR
3. prisma-engines PR is currently a work in progress (query, migration, etc)
4. Any further work in prisma/prisma for this to be officially supported (documentation, etc)

I'd like to gauge the Prisma team's (especially @janpio) at the feasibility of this being accepted into Prisma in the short term. I understand Prisma has it's own priorities but I'm sinking a lot of time into this and want to know whether it would be seriously considered for merge when it is ready (hopefully by the end of March).

Appreciate any advice or guidance on how you've managed larger open source contributions to Prisma and what I should expect going forward.

edit: There is an open issue for this #2568, however wanted to have more of a discussion on how this looks for me personally as an open source contributor.

[janpio]

Thanks for your work @halfmatthalfcat ❤️

Saw your email and was planning to answer next week, but here is fine as well. Actually better, as others can also chime in (and my "no work on the weekend rule is much laxer about spending time on GitHub 😆). I will open 2 "threads" here now: One to continue working on the feature, and another one to discuss the topic of Open Source contributions to Prisma.

This one is for discussing the feature:

The big open question we usually start with internally (after we decided it is worth spending our resources on a specific feature vs. on others), is how a database feature would look like in Prisma Schema Language (PSL) and how support across databases looks like. Anything that only works in one database is harder to integrate, unless it presents itself as a "native type" (where we can do @db.Foo, and just have to find a fitting parent type). So to have a conversation on how probable it is that we add something to Prisma, we should start there.

I know very little about ltree, so will now read a bit to learn the basics, and then come back to post my thoughts as a comment on this thread. Maybe we can come up with a description together how this would look like in PSL.

[halfmatthalfcat]

As far as extensions go, I think it would be great to be able to auto-include them, potentially within the datasource block. The issue is, with Postgres at least, a ton of useful extensions must be installed manually and then the extension created via query. ltree natively ships with Postgres, so all you need to do is enable it via query which can be done in the initial migration if specified somewhere within Prisma but for others, it's more manual.

There is already precedent though, in Prisma, for relying on the user to make sure they have their db setup correctly; example being with citext: https://www.prisma.io/docs/concepts/components/prisma-schema/features-without-psl-equivalent#enable-extensions-for-native-database-functions

[halfmatthalfcat]

So I've finally got ltree, lquery and ltxtquery merged into rust-postgres, which means we can now update @pimeys's pgbouncer-mode fork/branch to bring those changes in quaint.

@pimeys I remember you mentioning earlier that you hope on maintaining a complete fork (or at least, control the postgres lib for Prisma's sake long term) but I'm hoping we can at least update pgbouncer-mode in the interim to keep the ltree initiative moving.

I have a PR open in your fork: pimeys/rust-postgres#4

[janpio]

Nice, seems you already managed with Julius to make progress here and Quaint can now support these three.

Now it gets more interesting: How can we use this in Prisma already? What do you think is the absolute minimum that would make sense for you to surface this? Because it became quite obvious that the full package is a huge thing we really need to understand and then design properly. But maybe something smaller...

[halfmatthalfcat]

So yeah, I've been aiming to support ancestor_of, descendant_of, matches and matches_fulltext as filters, if ltree is a native type on a given field:

• New filters
• Added only if native type is ltree

I think these 4 concepts, largely, make up the desired functionality of using the ltree type. All of the functions and extra things Postgres provides are much more advanced and I would argue, more niche, that can be added at a later time if the demand is there.

Per the Postgres docs too, ltree can support =, <>, <, >, <=, >=, so those will be included as well.

I think this is a good first foray into supporting ltrees in Prisma proper. I'm still working on tests and have to go back and update my quaint PR to include the rust-postgres bump and add some more documentation.

[halfmatthalfcat]

Quaint PR now up: prisma/quaint#352

[janpio]

Appreciate any advice or guidance on how you've managed larger open source contributions to Prisma and what I should expect going forward.

Prisma is very early on our "accept contributions from the community" journey.

Until now those contributions were mainly in dependent projects lik quaint or tiberius, or on the smaller scale across our TS codebase in prisma/prisma or content changes in prisma/docs. The reason is what I described above - a) deciding we want to spend time on something as a company and b) then designing a feature that fits the long term strategy is very hard in our current state.

Prisma did not start as a Open Source project building on community contributions, and as such does not have any of the mechanisms required to make that work. Product Management is internal (Roadmap, Opportunity documents, Solution design, etc), as it Engineering generally (although we work in the open in GitHub, all communication and planning is still behind the scenes in private.

[halfmatthalfcat]

Cool, thank you for the context. I completely understand where you're coming from. I can see where it would get hard as, speaking personally, I'm not sure architecturally what the internal standards or plans are for the various codebases. I've been trying to confirm to patterns and conventions currently established (I'm also newer to Rust, so that is another wrinkle) as well as add ample testing to ease the friction/give some reassurance that external code isn't breaking.

I'm not sure I'm going to be very useful in adding suggestions since I've never been in the position of maintaining large OSS codebases but in anyway I can help, I'd be down to lend my opinions/time.

[janpio]

You did the best thing someone can do in such a situation already: Reach out to the maintainers early and look for feedback and context. We have on our agenda for this year to understand our situation better and work on ceating a first strategy to (potentially) open up a bit more. Fortunately these things happen around the same time, so we can figure it out together.

[halfmatthalfcat]

Cool, so I guess I'll be somewhat of a guinea pig. I'll continue on as-is and we can address process as things progress.

[ludralph]

Hi there,

To keep our discussions organized and focused on the most relevant topics, we’re reviewing and tidying up our backlog. As part of this process, we’re closing discussions that haven’t had any recent activity and appear to be outdated.

If this discussion is still important to you or unresolved, we’d love to hear from you! Feel free to reopen it or start a new one with updated details.

For more details about our priorities and vision for the future of Prisma ORM, check out our latest blog post: https://www.prisma.io/blog/prisma-orm-manifesto.

Thank you for your understanding and being part of the community!