Skip to content

Conversation

@DrDet
Copy link
Contributor

@DrDet DrDet commented Dec 19, 2021

The main goal of the PR is to introduce internal resumable-friendly API for embedding asynchronous database drivers (MySQL, PostgreSQL, Redis etc.)

Resumable-friendly means that any network communication based on this API will be considered as a suspendable point by KPHP resumable functions (a.k.a. coroutines). To achieve this the API deeply interacts with KPHP web server network internals and epoll reactor.

The API provides the following abstractions:

  • database_drivers::Connector -- class representing connection to database.
    Specific database driver is supposed to extend this class and implement all pure virtual methods.
    The most important of them:

    • Obtaining file descriptor
      /**
      * @brief Gets file descriptor number of underlying connection.
      * @return File descriptor number.
      */
      virtual int get_fd() const noexcept = 0;
    • Performing connection itself
      /**
       * @brief Attempts to establish underlying connection asynchronously.
       * @return Status of operation: in progress, completed or error.
       */
      virtual AsyncOperationStatus connect_async() noexcept = 0;
    • Methods for handling epoll events of different types:
      virtual void handle_write() noexcept = 0;
      virtual void handle_read() noexcept = 0;
      virtual void handle_special() noexcept = 0;

    See all such methods at server/database-drivers/connector.h

    Note: It's only for not persistent connections, e.g. bound to script memory. All such connections will be closed at the end of script execution.

  • database_drivers::Request -- class representing request to database. It has the only method for sending itself asynchronously:

    /**
     * @brief Sends this requests asynchronously.
     * @return Status of operation: in progress, completed or error.
     */
    virtual AsyncOperationStatus send_async() noexcept = 0;

    Should be called from database_drivers::Connector::handle_write()

  • database_drivers::Response -- class representing response from database. It has the only method for fetching itself asynchronously:

    /**
     * @brief Fetches this response asynchronously.
     * @return Status of operation: in progress, completed or error.
     */
    virtual AsyncOperationStatus fetch_async() noexcept = 0;

    Should be called from database_drivers::Connector::handle_read()

And special singleton class to manipulate these abstractions - database_drivers::Adaptor. With it you can:

  • Connect to database:
    /**
     * @brief Registers @a connector and embeds it into event loop.
     * @param connector
     * @return ID of registered connector.
     */
    int initiate_connect(std::unique_ptr<Connector> &&connector) noexcept;
  • Send request asynchronously and get waitable reumable id:
    /**
     * @brief Launches @a request resumable (coroutine), that will send @a request asynchronously.
     * @param request
     * @return ID of launched resumable (coroutine), which can be awaited with wait_request_resumable().
     *
     * Creates forked resumable as if we call `fork(launch_request_resumable())`, @see fork_resumable().
     * Moves @a request to Connector::push_async_request().
     */
    int launch_request_resumable(std::unique_ptr<Request> &&request) noexcept
  • Wait sent request by resumable id:
    /**
     * @brief Waits request resumable with @a resumable_id.
     * @param resumable_id
     * @return Response for Request associated with @a resumable_id.
     */
    std::unique_ptr<Response> wait_request_resumable(int resumable_id) const noexcept;
  • And more. See the full methods list at server/database-drivers/adaptor.h

As an example of usage this API, basic resumable PDO::MySQL version is implemented in this PR as well.
It supports only basic read and modify operations with default behaviour. But these operations are truly resumable thanks to new API.

Note: This API can be used only with drivers that are more or less asynchronous and expose the way to get file descriptor of connection. Basically, It can be said that if some database driver is compatible with libevent or similar lib, then it can be embedded to KPHP preserving asynchrony via this API.

@DrDet DrDet added enhancement New feature or request help wanted Extra attention is needed labels Dec 19, 2021
@DrDet DrDet changed the title WIP: Resumable-friendly external database drivers embedding API & PDO::MySQL basic version WIP: Resumable-friendly external database drivers embedding API & PDO::MySQL resumable basic version Dec 19, 2021
@DrDet DrDet force-pushed the dvaksman/mysql-resumable branch from 1eaecff to 0ce100f Compare December 19, 2021 15:10
@DrDet DrDet force-pushed the dvaksman/mysql-resumable branch 3 times, most recently from 36a7cd1 to 7e4c4ae Compare December 23, 2021 18:54
@DrDet DrDet force-pushed the dvaksman/mysql-resumable branch 2 times, most recently from 24345e2 to ed41a39 Compare December 30, 2021 19:45
@DrDet DrDet force-pushed the dvaksman/mysql-resumable branch 6 times, most recently from 8cda7c7 to 56033de Compare January 17, 2022 14:06
@DrDet DrDet force-pushed the dvaksman/mysql-resumable branch 3 times, most recently from d8b5af8 to 8980f64 Compare January 25, 2022 12:12
@DrDet DrDet force-pushed the dvaksman/mysql-resumable branch 10 times, most recently from 939c1de to 033e906 Compare January 27, 2022 17:11
@DrDet DrDet changed the title WIP: Resumable-friendly external database drivers embedding API & PDO::MySQL resumable basic version Resumable-friendly external database drivers embedding API & PDO::MySQL resumable basic version Jan 28, 2022
@DrDet DrDet force-pushed the dvaksman/mysql-resumable branch 5 times, most recently from f61e257 to 101bd86 Compare February 3, 2022 16:49
@DrDet DrDet requested review from drdzyk and troy4eg February 15, 2022 11:11
@DrDet DrDet force-pushed the dvaksman/mysql-resumable branch 4 times, most recently from 6bd7e02 to 21b79ab Compare February 15, 2022 17:10
drdzyk
drdzyk previously approved these changes Feb 16, 2022
@DrDet DrDet force-pushed the dvaksman/mysql-resumable branch 2 times, most recently from 905a82d to 83d2d65 Compare February 16, 2022 12:22
drdzyk
drdzyk previously approved these changes Feb 17, 2022
@DrDet DrDet merged commit 1ee498e into master Feb 17, 2022
@DrDet DrDet deleted the dvaksman/mysql-resumable branch February 17, 2022 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request help wanted Extra attention is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants