Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support R2DBC #1454

Open
VaughnVernon opened this issue Feb 7, 2019 · 14 comments

Comments

Projects
None yet
7 participants
@VaughnVernon
Copy link

commented Feb 7, 2019

Implement support for R2DBC. This may be something such as jdbi-r2dbc. FYI:

https://r2dbc.io/
https://github.com/r2dbc

Also, we support Jdbi in the vlingo-symbio which is part of our reactive platform. Check it out. We'd love to get your team up to speed on our full platform.

https://github.com/vlingo/vlingo-symbio-jdbc/tree/master/src/main/java/io/vlingo/symbio/store/object/jdbc/jdbi

@stevenschlansker

This comment has been minimized.

Copy link
Member

commented Feb 7, 2019

Hi Vaughn, thanks for the pointer. Reactive is very new and exciting but my understanding is that the current implementation of JDBC, and in fact the database server software itself, makes it a poor candidate for reactive concepts at the lower layers. The database server has worker processes that take queries and produce results, and each process will work on its query until completion. There's no ability to asynchronously submit 10 queries and await their results.

This means that as you implement JDBC reactive, you eventually find that you must implement an old style thread pool that holds your connections / handles, services queries, and then emits results into the reactive framework. This means that you can have some of the benefits of reactive, in particular you can write reactive code, but it seriously limits the scalability benefits that reactive attempts to achieve.

So we'd definitely welcome API integrations, but unless the state of the JDBC world has moved on since I last investigated, it'll be a somewhat limited wrapper that just hides the fact that the code underneath is still old school threaded code.

I appreciate the offer to get up to speed on your platform, but our library prides itself on being almost entirely standalone and not part of any platform or framework. So while we might look for inspiration and are always willing to collaborate towards shared goals, we're a fiercely independent project :)

@VaughnVernon

This comment has been minimized.

Copy link
Author

commented Feb 7, 2019

Hey, Steven! Thanks for your comments. Currently we solve the reactive-over-JDBC by serializing requests within actors in our vlingo-symbio implementations. You can create as many JDBCObjectStoreActor as makes sense for your connection limitations, but the actors themselves access JDBC synchronously. It's more that the client of the ObjectStore implementations running on actors don't block while queries are being fulfilled.

I don't have a thorough understanding of R2DBC, but it is touted as being a full JDBC replacement and works completely asynchronously as a database driver. There are currently implementations for Postgres and MS SQL Server only, but probably will grow in time. I was wondering what it would be like for Jdbi to implement over R2DBC. I understand if this doesn't make sense for you now, but the reference is here in case it does at some future time. No stress :)

@stevenschlansker

This comment has been minimized.

Copy link
Member

commented Feb 7, 2019

Interesting, they actually do provide their own drivers. So maybe there is some benefit to be had here!

I'm curious to explore this further: benchmarks, proof of concept code, etc. There seem to be some limitations: their r2dbc driver for postgres advertises that e.g. BLOB and CLOB types don't work yet, and there's no mention at all of stored procedures or CALL.

But at least my focus personally right now is elsewhere so this may need some community input and attention to move forward, as I don't have time right now to open an exploration here, and I don't imagine other core members are looking to jump on it either.

Looking forward to hearing more, even if just votes from other members of the community that this is a needed feature.

@mp911de

This comment has been minimized.

Copy link

commented Feb 8, 2019

Great to see this ticket.

I wanted to quickly drop two things to clarify the state of R2DBC:

  • It's still a young initiative and things like BLOB/CLOB and Stored Procedure support are to be implemented before we go GA.
  • Drivers are fully reactive and non-blocking meaning the whole I/O part is non-blocking. There's no offloading of blocking work into thread pools.

In any case, R2DBC aims for an SPI to be picked up by client implementations such as Spring Data R2DBC, R2DBC Client (a tiny wrapper on top of R2DBC SPI that lend some ideas from jdbi) and more to come.

@nebhale

This comment has been minimized.

Copy link

commented Feb 8, 2019

Just wanted to chime in and mention that my original prototype/demonstration client layer, r2dbc-client was designed based on Jdbi's APIs and ethos. If you do choose to investigate further, I think you'll find it familiar 😄.

@brianm

This comment has been minimized.

Copy link
Member

commented Apr 24, 2019

I have seen folks expose Rx stuff out of JDBI ( @hgschmie for example ). I am not sure what "support" for R2DBC would mean.

Are you talking about implementing R2DBC on top of JDBI, having JDBi consume R2DBC Connections, swapping JDBI to focus on Rx interfaces, or... something else?

I think it would be "support R2DBC drivers to be used by JDBI, exposing the JDBI interfaces" ?

@VaughnVernon

This comment has been minimized.

Copy link
Author

commented Apr 24, 2019

R2DBC is an async replacement for JDBC.

@mp911de

This comment has been minimized.

Copy link

commented Apr 25, 2019

@brianm R2DBC isn't JDBC on ThreadPools. R2DBC means two things:

  1. R2DBC drivers are implementations using a non-blocking transport layer returning org.reactivestreams.Publisher types for each I/O-bound operation. They do not use JDBC drivers underneath but implement wire protocols from scratch.
  2. R2DBC is a standardized (vendor-independent) SPI allowing to build client libraries on top. Driver implementations can be interchanged like it is done today for JDBC.

Are you talking about implementing R2DBC on top of JDBI, having JDBi consume R2DBC Connections

Yes, this is what it basically means. Having an R2DBC-specific JDBI module returning Project Reactor/RxJava2/Zerodep types.

@brianm

This comment has been minimized.

Copy link
Member

commented Apr 25, 2019

I'm pretty sure I get what R2DBC is, and I think it is a Very Good Thing.

I am trying to figure out what "JDBI support" for it is!

@TheRealMarnes

This comment has been minimized.

Copy link
Contributor

commented Apr 25, 2019

Maybe write a code sample of what you expect to do via JDBI's API?

@mp911de

This comment has been minimized.

Copy link

commented Apr 25, 2019

How about supporting the following for starters:

Jdbi jdbi = Jdbi.create("r2dbc:h2:mem:///test"); // (H2 in-memory database), obtain ConnectionFactory via ConnectionFactories.get(String url)

// Reactor style:
Flux<User> users = jdbi.withHandle(handle -> {
    
    return handle.execute("CREATE TABLE user (id INTEGER PRIMARY KEY, name VARCHAR)")
    
        .then(handle.execute("INSERT INTO user(id, name) VALUES (?0, ?1)", 0, "Alice"))
    
        .then(handle.createUpdate("INSERT INTO user(id, name) VALUES (?0, ?1)")
            .bind(0, 1)
            .bind(1, "Bob")
            .execute())
         .then(handle.createQuery("SELECT * FROM user ORDER BY name")
              .mapToBean(User.class).many());
});

// RxJava style
Flowable<User> users = jdbi.withHandle(handle -> { … });

The interface-based support could look like:

public interface UserDao {
    @SqlUpdate("CREATE TABLE user (id INTEGER PRIMARY KEY, name VARCHAR)")
    Completable createTableRxJava();
    
    @SqlUpdate("CREATE TABLE user (id INTEGER PRIMARY KEY, name VARCHAR)")
    Mono<Void> createTableProjectReactor();

    @SqlQuery("SELECT * FROM user ORDER BY name")
    @RegisterBeanMapper(User.class)
    Flowable<User> listUsersRxJava();
    
    @SqlQuery("SELECT * FROM user ORDER BY name")
    @RegisterBeanMapper(User.class)
    Flux<User> listUsersProjectReactor();
}

I'm not opinionated about RxJava vs. Project Reactor, therefore I tried to list variants using both APIs.

@qualidafial

This comment has been minimized.

Copy link
Member

commented May 6, 2019

Could somebody clarify what exactly is being proposed here?

Is R2DBC a layer on top of JDBC, or its own thing? Because Jdbi is coupled to JDBC in the extreme.

@mp911de

This comment has been minimized.

Copy link

commented May 6, 2019

R2DBC is not a layer on top of JDBC. R2DBC is a non-blocking API to access SQL databases and it is its own thing (R2DBC drivers are typically written from scratch implementing vendor-specific wire-protocols using a non-blocking I/O layer). If you will, R2DBC is the reactive specification of how to integrate with SQL databases.

The proposal here would be having an API that looks and works like Jdbi but uses underneath R2DBC drivers. Instead of returning scalar objects, the API would return Reactive Streams types.

@stevenschlansker

This comment has been minimized.

Copy link
Member

commented May 6, 2019

Thanks for the clarification. I expect the first step here would be to prototype a Jdbi fork or branch that serves minimally as a PoC showing how to implement the above examples on top of R2DBC. From there we would have to determine a path towards integration in parallel to existing JDBC support, or establish that it would be better as a hard fork. As Matthew points out above we are tightly coupled to JDBC currently and changing that is likely to both be a lot of hard work as well as potentially require breaking changes.

I am excited to have this on the project's long-term radar but I don't expect it to move quickly without significant community involvement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.