[Integration] Use PRQL to query log data

[nitisht]

Hi there!

Parseable is an open source, log storage and query platform written in Rust. Parseable support SQL as the query interface.

PRQL seems to be a great way to get insight from log data. Is there a way we could integrate and may be even have a hosted playground that sends PRQL to SQL query to the demo parseable server (login with parseable, parseable)

[max-sixty]

Hi @nitisht

Parseable looks great! I'd heard of it on the grapevine though haven't used it yet.

I think this could be quite a good fit, and I'd be happy to work together.

I can see a few ways of doing this:

• Separate: because Parseable consumes PostgresQL, it'd be possible to have an app with the PRQL binding which did the translation and sent SQL to a server
• Integrated: add a feature to the Parseable server so it can compile PRQL to SQL internally, and so could accept PRQL as a lang. That would be (optimistically) a dozen lines of code given we're both in rust, and could be an optional cargo feature.
• Native: (I wouldn't propose we do this quite yet...) Parseable could directly operate on PRQL / the PRQL AST, which would give it a bunch of benefits — e.g. it could aggressively cache earlier steps in a query, so changing a filter further down the query could use an intermediate result of a previous query's result. Or it could give keystroke-level feedback on results, since appending a transformation can't change the prior result. But it would require translating PRQL into the Parseable AST without the current SQL layer (arguably easier than translating the SQL, but now we have the SQL...).

I haven't looked at the server yet, but I imagine this would be possible with any of those.

Let us know your thoughts. If it would be helpful, happy to speak at the end of next week.

[nitisht]

Hey @max-sixty thank you so much for the detailed response. All great ideas :)

IMHO

Integrated: add a feature to the Parseable server so it can compile PRQL to SQL internally, and so could accept PRQL as a lang. That would be (optimistically) a dozen lines of code given we're both in rust, and could be an optional cargo feature.

Seems the best approach in the longer run. To start with, I was thinking taking the first approach, - A simple cli tool to write PRQL in the terminal to be executed against a Parseable server instance.

If it would be helpful, happy to speak at the end of next week.

It would be great to talk and discuss further. Would you please send a note to nitish at parseable dot io. We can setup a time..

[snth]

To start with, I was thinking taking the first approach, - A simple cli tool to write PRQL in the terminal to be executed against a Parseable server instance.

I haven't followed this discussion in detail but just a quick note that prql-query has a fairly modular backend architecture so perhaps you just provide a Parseable backend for it?

Logs are a more specific use case so it might make sense to have a more specialised tool for that but in case that prql-query might be enough I just wanted to make you aware of it.

[nitisht]

Thanks @snth . Checking it out now.