Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(driver-adapters): enable Wasm on request-handlers #4455

Merged
merged 51 commits into from
Nov 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
61ef25b
feat(quaint): allow wasm32-unknown-unknown compilation; currently fai…
jkomyno Nov 10, 2023
055e696
feat(quaint): split postgres connector into native and wasm submodules
jkomyno Nov 10, 2023
12c6ebb
feat(quaint): split mysql connector into native and wasm submodules
jkomyno Nov 10, 2023
060486d
feat(quaint): recover wasm error for mysql
jkomyno Nov 10, 2023
5de1dc0
feat(quaint): split mssql connector into native and wasm submodules
jkomyno Nov 10, 2023
8ecbc5c
feat(quaint): split sqlite connector into native and wasm submodules
jkomyno Nov 13, 2023
45df24f
chore(quaint): fix clippy when compiling natively
jkomyno Nov 13, 2023
6a1f733
chore(quaint): fix clippy when compiling to wasm32-unknown-unknown
jkomyno Nov 13, 2023
e61bf75
chore(quaint): update README
jkomyno Nov 13, 2023
257c4c8
chore(quaint): rename "*-connector" feature flag to "*-native"
jkomyno Nov 14, 2023
5ab6d96
feat(quaint): enable pure Wasm SqliteError
jkomyno Nov 14, 2023
ab65c95
feat(query-connect): allow wasm32-unknown-unknown compilation
jkomyno Nov 14, 2023
485f6dc
Merge branch 'main' into feat/quaint-on-wasm32-unknown-unknown
jkomyno Nov 14, 2023
cfb5507
feat(sql-query-connector): allow wasm32-unknown-unknown compilation
jkomyno Nov 14, 2023
e7df5a3
chore(query-engine-wasm): add currently unused local crates to test w…
jkomyno Nov 14, 2023
8c5d3dc
chore: update Cargo.lock
jkomyno Nov 14, 2023
6648a88
chore: remove leftover comments
jkomyno Nov 14, 2023
754746e
feat(query-core): allow wasm32-unknown-unknown compilation
jkomyno Nov 14, 2023
fe2fb8b
chore(sql-query-connector): fix clipppy on wasm32
jkomyno Nov 14, 2023
2ffe394
Merge branch 'feat/sql-query-connector-on-wasm32-unknown-unknown' of …
jkomyno Nov 14, 2023
e66fb65
Merge branch 'feat/sql-query-connector-on-wasm32-unknown-unknown' int…
jkomyno Nov 14, 2023
9c41dc1
chore: remove leftover comment
jkomyno Nov 14, 2023
b69bb84
feat(driver-adapters): enable Wasm on request-handlers
jkomyno Nov 15, 2023
c987dce
WIP: refactor mysql module to flatten its structure
miguelff Nov 15, 2023
626bc1e
feat(quaint): flatten mssql connector module
jkomyno Nov 15, 2023
a9f8ba8
feat(quaint): flatten postgres connector module
jkomyno Nov 15, 2023
3c1a100
feat(quaint): flatten sqlite connector module
jkomyno Nov 15, 2023
7f4c8f9
chore(quaint): export all public definitions in connector "url" modules
jkomyno Nov 15, 2023
95a4e28
chore(quaint): refactor tests for connectors, addressing feedback
jkomyno Nov 15, 2023
bacb635
chore: add comment on MysqlAsyncError
jkomyno Nov 15, 2023
263bab0
chore: add comment on ffi.rs for sqlite
jkomyno Nov 15, 2023
414ae2b
Merge branch 'feat/quaint-on-wasm32-unknown-unknown' into feat/sql-qu…
jkomyno Nov 15, 2023
d0f783d
Merge branch 'feat/sql-query-connector-on-wasm32-unknown-unknown' int…
jkomyno Nov 15, 2023
76816fd
chore: replace awkward "super::super::" with "crate::..."
jkomyno Nov 15, 2023
7359a08
Merge branch 'feat/sql-query-connector-on-wasm32-unknown-unknown' of …
jkomyno Nov 15, 2023
f61665a
Merge branch 'feat/sql-query-connector-on-wasm32-unknown-unknown' int…
jkomyno Nov 15, 2023
5126a75
chore: add comments around "query_core::executor::task"
jkomyno Nov 15, 2023
de39d9e
chore: add "request-handlers" to "query-engine-wasm"
jkomyno Nov 15, 2023
4ed297e
Merge branch 'feat/query-core-on-wasm32-unknown-unknown' into feat/re…
jkomyno Nov 15, 2023
33cdf77
Merge branch 'main' into feat/sql-query-connector-on-wasm32-unknown-u…
jkomyno Nov 17, 2023
2dd3f02
Merge branch 'main' into feat/query-core-on-wasm32-unknown-unknown
jkomyno Nov 17, 2023
2afeddc
Merge branch 'feat/sql-query-connector-on-wasm32-unknown-unknown' int…
jkomyno Nov 17, 2023
2339b31
chore: move "task" module into its own file
jkomyno Nov 17, 2023
edf0f4e
Merge branch 'main' into feat/query-core-on-wasm32-unknown-unknown
jkomyno Nov 17, 2023
5976ab1
Merge branch 'feat/query-core-on-wasm32-unknown-unknown' into feat/re…
jkomyno Nov 17, 2023
96cd8ca
fix(driver-adapters): ci for "request-handlers"
jkomyno Nov 17, 2023
3541054
fix(driver-adapters): ci for "request-handlers"
jkomyno Nov 17, 2023
e79cb1f
Merge branch 'main' into feat/sql-query-connector-on-wasm32-unknown-u…
jkomyno Nov 17, 2023
e52500f
Merge branch 'feat/sql-query-connector-on-wasm32-unknown-unknown' int…
jkomyno Nov 17, 2023
2c70d12
Merge branch 'feat/query-core-on-wasm32-unknown-unknown' into feat/re…
jkomyno Nov 17, 2023
a82584a
chore: merge main, fix conflicts
jkomyno Nov 20, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions query-engine/query-engine-wasm/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ user-facing-errors = { path = "../../libs/user-facing-errors" }
psl.workspace = true
prisma-models = { path = "../prisma-models" }
quaint = { path = "../../quaint" }
request-handlers = { path = "../request-handlers", default-features = false, features = ["sql", "driver-adapters"] }
connector = { path = "../connectors/query-connector", package = "query-connector" }
sql-query-connector = { path = "../connectors/sql-query-connector" }
query-core = { path = "../core" }
Expand Down
7 changes: 4 additions & 3 deletions query-engine/request-handlers/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ edition = "2021"
prisma-models = { path = "../prisma-models" }
query-core = { path = "../core", features = ["metrics"] }
user-facing-errors = { path = "../../libs/user-facing-errors" }
quaint = { path = "../../quaint" }
SevInf marked this conversation as resolved.
Show resolved Hide resolved
psl.workspace = true
dmmf_crate = { path = "../dmmf", package = "dmmf" }
itertools = "0.10"
Expand All @@ -20,7 +21,6 @@ thiserror = "1"
tracing = "0.1"
url = "2"
connection-string.workspace = true
quaint.workspace = true
once_cell = "1.15"

mongodb-query-connector = { path = "../connectors/mongodb-query-connector", optional = true }
Expand All @@ -32,10 +32,11 @@ schema = { path = "../schema" }
codspeed-criterion-compat = "1.1.0"

[features]
default = ["mongodb", "sql"]
default = ["sql", "mongodb", "native"]
mongodb = ["mongodb-query-connector"]
sql = ["sql-query-connector"]
driver-adapters = ["sql-query-connector"]
driver-adapters = ["sql-query-connector/driver-adapters"]
native = ["mongodb", "sql-query-connector", "quaint/native", "query-core/metrics"]

[[bench]]
name = "query_planning_bench"
Expand Down
1 change: 1 addition & 0 deletions query-engine/request-handlers/src/connector_mode.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#[derive(Copy, Clone, PartialEq, Eq)]
pub enum ConnectorMode {
/// Indicates that Rust drivers are used in Query Engine.
#[cfg(feature = "native")]
Rust,

/// Indicates that JS drivers are used in Query Engine.
Expand Down
162 changes: 84 additions & 78 deletions query-engine/request-handlers/src/load_executor.rs
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
#![allow(unused_imports)]

use psl::{builtin_connectors::*, Datasource, PreviewFeatures};
use query_core::{executor::InterpretingExecutor, Connector, QueryExecutor};
use sql_query_connector::*;
use std::collections::HashMap;
use std::env;
use tracing::trace;
use url::Url;

#[cfg(feature = "mongodb")]
use mongodb_query_connector::MongoDb;

use super::ConnectorMode;

/// Loads a query executor based on the parsed Prisma schema (datasource).
Expand All @@ -27,6 +25,7 @@ pub async fn load(
driver_adapter(source, url, features).await
}

#[cfg(feature = "native")]
ConnectorMode::Rust => {
if let Ok(value) = env::var("PRISMA_DISABLE_QUAINT_EXECUTORS") {
let disable = value.to_uppercase();
Expand All @@ -36,14 +35,14 @@ pub async fn load(
}

match source.active_provider {
p if SQLITE.is_provider(p) => sqlite(source, url, features).await,
p if MYSQL.is_provider(p) => mysql(source, url, features).await,
p if POSTGRES.is_provider(p) => postgres(source, url, features).await,
p if MSSQL.is_provider(p) => mssql(source, url, features).await,
p if COCKROACH.is_provider(p) => postgres(source, url, features).await,
p if SQLITE.is_provider(p) => native::sqlite(source, url, features).await,
p if MYSQL.is_provider(p) => native::mysql(source, url, features).await,
p if POSTGRES.is_provider(p) => native::postgres(source, url, features).await,
p if MSSQL.is_provider(p) => native::mssql(source, url, features).await,
p if COCKROACH.is_provider(p) => native::postgres(source, url, features).await,

#[cfg(feature = "mongodb")]
p if MONGODB.is_provider(p) => mongodb(source, url, features).await,
p if MONGODB.is_provider(p) => native::mongodb(source, url, features).await,

x => Err(query_core::CoreError::ConfigurationError(format!(
"Unsupported connector type: {x}"
Expand All @@ -53,57 +52,88 @@ pub async fn load(
}
}

async fn sqlite(
#[cfg(feature = "driver-adapters")]
async fn driver_adapter(
source: &Datasource,
url: &str,
features: PreviewFeatures,
) -> query_core::Result<Box<dyn QueryExecutor + Send + Sync>> {
trace!("Loading SQLite query connector...");
let sqlite = Sqlite::from_source(source, url, features).await?;
trace!("Loaded SQLite query connector.");
Ok(executor_for(sqlite, false))
) -> Result<Box<dyn QueryExecutor + Send + Sync>, query_core::CoreError> {
let js = Js::from_source(source, url, features).await?;
Ok(executor_for(js, false))
}

async fn postgres(
source: &Datasource,
url: &str,
features: PreviewFeatures,
) -> query_core::Result<Box<dyn QueryExecutor + Send + Sync>> {
trace!("Loading Postgres query connector...");
let database_str = url;
let psql = PostgreSql::from_source(source, url, features).await?;

let url = Url::parse(database_str)
.map_err(|err| query_core::CoreError::ConfigurationError(format!("Error parsing connection string: {err}")))?;
let params: HashMap<String, String> = url.query_pairs().into_owned().collect();

let force_transactions = params
.get("pgbouncer")
.and_then(|flag| flag.parse().ok())
.unwrap_or(false);
trace!("Loaded Postgres query connector.");
Ok(executor_for(psql, force_transactions))
}
#[cfg(feature = "native")]
mod native {
use super::*;
use tracing::trace;

pub(crate) async fn sqlite(
source: &Datasource,
url: &str,
features: PreviewFeatures,
) -> query_core::Result<Box<dyn QueryExecutor + Send + Sync>> {
trace!("Loading SQLite query connector...");
let sqlite = Sqlite::from_source(source, url, features).await?;
trace!("Loaded SQLite query connector.");
Ok(executor_for(sqlite, false))
}

async fn mysql(
source: &Datasource,
url: &str,
features: PreviewFeatures,
) -> query_core::Result<Box<dyn QueryExecutor + Send + Sync>> {
let mysql = Mysql::from_source(source, url, features).await?;
trace!("Loaded MySQL query connector.");
Ok(executor_for(mysql, false))
}
pub(crate) async fn postgres(
source: &Datasource,
url: &str,
features: PreviewFeatures,
) -> query_core::Result<Box<dyn QueryExecutor + Send + Sync>> {
trace!("Loading Postgres query connector...");
let database_str = url;
let psql = PostgreSql::from_source(source, url, features).await?;

let url = Url::parse(database_str).map_err(|err| {
query_core::CoreError::ConfigurationError(format!("Error parsing connection string: {err}"))
})?;
let params: HashMap<String, String> = url.query_pairs().into_owned().collect();

let force_transactions = params
.get("pgbouncer")
.and_then(|flag| flag.parse().ok())
.unwrap_or(false);
trace!("Loaded Postgres query connector.");
Ok(executor_for(psql, force_transactions))
}

async fn mssql(
source: &Datasource,
url: &str,
features: PreviewFeatures,
) -> query_core::Result<Box<dyn QueryExecutor + Send + Sync>> {
trace!("Loading SQL Server query connector...");
let mssql = Mssql::from_source(source, url, features).await?;
trace!("Loaded SQL Server query connector.");
Ok(executor_for(mssql, false))
pub(crate) async fn mysql(
source: &Datasource,
url: &str,
features: PreviewFeatures,
) -> query_core::Result<Box<dyn QueryExecutor + Send + Sync>> {
let mysql = Mysql::from_source(source, url, features).await?;
trace!("Loaded MySQL query connector.");
Ok(executor_for(mysql, false))
}

pub(crate) async fn mssql(
source: &Datasource,
url: &str,
features: PreviewFeatures,
) -> query_core::Result<Box<dyn QueryExecutor + Send + Sync>> {
trace!("Loading SQL Server query connector...");
let mssql = Mssql::from_source(source, url, features).await?;
trace!("Loaded SQL Server query connector.");
Ok(executor_for(mssql, false))
}

#[cfg(feature = "mongodb")]
pub(crate) async fn mongodb(
source: &Datasource,
url: &str,
_features: PreviewFeatures,
) -> query_core::Result<Box<dyn QueryExecutor + Send + Sync>> {
use mongodb_query_connector::MongoDb;

trace!("Loading MongoDB query connector...");
let mongo = MongoDb::new(source, url).await?;
trace!("Loaded MongoDB query connector.");
Ok(executor_for(mongo, false))
}
}

fn executor_for<T>(connector: T, force_transactions: bool) -> Box<dyn QueryExecutor + Send + Sync>
Expand All @@ -112,27 +142,3 @@ where
{
Box::new(InterpretingExecutor::new(connector, force_transactions))
}

#[cfg(feature = "mongodb")]
async fn mongodb(
source: &Datasource,
url: &str,
_features: PreviewFeatures,
) -> query_core::Result<Box<dyn QueryExecutor + Send + Sync>> {
trace!("Loading MongoDB query connector...");
let mongo = MongoDb::new(source, url).await?;
trace!("Loaded MongoDB query connector.");
Ok(executor_for(mongo, false))
}

#[cfg(feature = "driver-adapters")]
async fn driver_adapter(
source: &Datasource,
url: &str,
features: PreviewFeatures,
) -> Result<Box<dyn QueryExecutor + Send + Sync>, query_core::CoreError> {
trace!("Loading driver adapter...");
let js = Js::from_source(source, url, features).await?;
trace!("Loaded driver adapter...");
Ok(executor_for(js, false))
}
Loading