Question: Async API?

[rocketraman]

I apologize for asking a question here, but I couldn't find any mailing list or other place to do so.

I came across pgjdbc-ng referenced from HikariCP, and it looks great. I see that your interface to the server is Netty-based.

Given that, is there any plans for / interest in a non-JDBC async API that would allow users to better leverage the async nature of the underlying driver?

[brettwooldridge]

I also think this is an interesting idea, though I would prefer to see an asynchronous JDBC API driven through the standards process for OpenJDK. There are a number of asynchronous driver attempts out there, but each employes its own API. Rather than adding more confusion to the mix, it would be better if everyone could agree on a specification. Otherwise your code is locked into not only a single database, but also a single driver vendor.

[rocketraman]

though I would prefer to see an asynchronous JDBC API driven through the standards process for OpenJDK.

Agreed. But a successful standards process is often based on existing real-world experience and experimentation. I don't think having another async API -- especially if it is marked clearly as experimental and subject to change -- would add too much confusion.

[mkurz]

+1

[rocketraman]

Looks like the upstream work on this has begun: https://twitter.com/brunoborges/status/915302682939711488

Awesome!

[kdubb]

@rocketraman Great news... the driver is essentially an async engine mapped to sync JDBC. This could be a fun area to explore and hopefully with minimal development effort.

[kdubb]

@rocketraman Quite a bit of work has been done to ready the core to host an implementation of ADBA (what its currently being called). I'm closing this as a new specific issue is forthcoming to implement ADBA.

[rocketraman]

@kdubb I didn't see any new issue for ADBA. Is this still something being pursued?

[kdubb]

Yes... but ADBA is dead see here

My current thought is to use their exploration as a base... it is quite polished. I've implemented a bit of it in a local branch but nothing usable yet.

[davecramer]

It's pretty much dead and there are a number of other competing options. Seems like a useless distraction to me

[kdubb]

@davecramer I agree a bit and the downside of ADBA is it abandons all of JDBC

What I find myself usually wanting is an asynchronous row processor but keep the JDBC API (because other code is written for it). A ResultSet representing a single row or a small batch of rows would probably do it. It's a bit of an overload but again I can pass a ResultSet around and not have to rewrite anything.

For example:

public interface Statement {
    public void query(String sql, Consumer<ResultSet> resultSetProcessor);
}

Something like this would get me most of want I want with asynchronous stuff.

[davecramer]

@kdubb Agreed.

The sad thing is that there are probably a dozen async protocol implementations now that work 80% of the time but fail 20% of the time and aren't getting the exposure they need to fix the corner cases. Anyways, I'm not going down this rabbit hole.

[cretz]

@kdubb - I had similar needs, i.e. jdbc worked for most of my things, but I needed to do more such as async row processing. I came to the conclusion that you just can't finagle JDBC for async use cases or wait on a vendor-agnostic interfaces. At the moment you either choose JDBC or choose an async lib (I ended up writing my own impl of the latter because I needed much more control).

[mkurz]

Checkout https://r2dbc.io/

[cretz]

Wasn't there when I was looking, but w/out support for some copy things, multidimensional arrays, some data types, and in general just really low level pg protocol stuff (like needing to end queries early, reuse them in different ways, choosing when to "describe" and when not, etc) it wouldn't work for my needs (https://github.com/cretz/pgnio was my solution, but may be too low level and too pg-specific for most use cases).

Would definitely agree that project solves 99% of most people's the jdbc-like-but-async needs 👍

[davecramer]

@cretz cool, thanks for sharing

[rocketraman]

Thanks for the references. Looks like ADBA is being abandoned in favor of the same synchronous JDBC API, but made non-blocking and scaled via fibers. Somewhat ironically, the reason I wanted an async API in the first place was to use with Kotlin coroutines (which are very similar conceptually to fibers, and may actually take advantage of fibers in the future, at least on the JVM). A non-blocking, but synchronous, JDBC API works just as well for that use case.