Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hide public usages of bigdecimal::BigDecimal behind a crate feature #922

Merged
merged 7 commits into from Feb 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/rust.yml
Expand Up @@ -44,6 +44,8 @@ jobs:
run: cargo check --all-targets --manifest-path "scylla/Cargo.toml" --features "num-bigint-03"
- name: Cargo check with num-bigint-04 feature
run: cargo check --all-targets --manifest-path "scylla/Cargo.toml" --features "num-bigint-04"
- name: Cargo check with bigdecimal-04 feature
run: cargo check --all-targets --manifest-path "scylla/Cargo.toml" --features "bigdecimal-04"
- name: Build scylla-cql
run: cargo build --verbose --all-targets --manifest-path "scylla-cql/Cargo.toml" --features "full-serialization"
- name: Build
Expand Down
14 changes: 11 additions & 3 deletions Cargo.lock.msrv

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/source/data-types/data-types.md
Expand Up @@ -26,7 +26,7 @@ Database types and their Rust equivalents:
* `Time` <----> `value::CqlTime`, `chrono::NaiveTime`, `time::Time`
* `Timestamp` <----> `value::CqlTimestamp`, `chrono::DateTime<Utc>`, `time::OffsetDateTime`
* `Duration` <----> `value::CqlDuration`
* `Decimal` <----> `bigdecimal::Decimal`
* `Decimal` <----> `value::CqlDecimal`, `bigdecimal::Decimal`
* `Varint` <----> `value::CqlVarint`, `num_bigint::BigInt` (v0.3 and v0.4)
* `List` <----> `Vec<T>`
* `Set` <----> `Vec<T>`
Expand Down
36 changes: 35 additions & 1 deletion docs/source/data-types/decimal.md
@@ -1,5 +1,39 @@
# Decimal
`Decimal` is represented as [`bigdecimal::BigDecimal`](https://docs.rs/bigdecimal/0.2.0/bigdecimal/struct.BigDecimal.html)
`Decimal` is represented as `value::CqlDecimal` or [`bigdecimal::BigDecimal`](https://docs.rs/bigdecimal/latest/bigdecimal/struct.BigDecimal.html)

## value::CqlDecimal

Without any feature flags, the user can interact with `decimal` type by making use of `value::CqlDecimal` which is a very simple wrapper representing the value as signed binary number in big-endian order with a 32-bit scale.

```rust
# extern crate scylla;
# use scylla::Session;
# use std::error::Error;
# async fn check_only_compiles(session: &Session) -> Result<(), Box<dyn Error>> {
use scylla::IntoTypedRows;
use scylla::frame::value::CqlDecimal;
use std::str::FromStr;

// Insert a decimal (123.456) into the table
let to_insert: CqlDecimal =
CqlDecimal::from_signed_be_bytes_and_exponent(vec![0x01, 0xE2, 0x40], 3);
session
.query("INSERT INTO keyspace.table (a) VALUES(?)", (to_insert,))
.await?;

// Read a decimal from the table
if let Some(rows) = session.query("SELECT a FROM keyspace.table", &[]).await?.rows {
for row in rows.into_typed::<(CqlDecimal,)>() {
let (decimal_value,): (CqlDecimal,) = row?;
}
}
# Ok(())
# }
```

## bigdecimal::BigDecimal

To make use of `bigdecimal::Bigdecimal` type, user should enable `bigdecimal-04` crate feature.

```rust
# extern crate scylla;
Expand Down
2 changes: 1 addition & 1 deletion docs/source/quickstart/create-project.md
Expand Up @@ -12,7 +12,7 @@ scylla = "0.11"
tokio = { version = "1.12", features = ["full"] }
futures = "0.3.6"
uuid = "1.0"
bigdecimal = "0.2.0"
bigdecimal = "0.4"
num-bigint = "0.3"
tracing = "0.1.36"
tracing-subscriber = { version = "0.3.14", features = ["env-filter"] }
Expand Down
2 changes: 1 addition & 1 deletion examples/Cargo.toml
Expand Up @@ -10,7 +10,7 @@ futures = "0.3.6"
openssl = "0.10.32"
rustyline = "9"
rustyline-derive = "0.6"
scylla = {path = "../scylla", features = ["ssl", "cloud", "chrono", "time", "num-bigint-03", "num-bigint-04"]}
scylla = {path = "../scylla", features = ["ssl", "cloud", "chrono", "time", "num-bigint-03", "num-bigint-04", "bigdecimal-04"]}
tokio = {version = "1.1.0", features = ["full"]}
tracing = "0.1.25"
tracing-subscriber = { version = "0.3.14", features = ["env-filter"] }
Expand Down
5 changes: 3 additions & 2 deletions scylla-cql/Cargo.toml
Expand Up @@ -19,9 +19,9 @@ secrecy = { version = "0.7.0", optional = true }
snap = "1.0"
uuid = "1.0"
thiserror = "1.0"
bigdecimal = "0.2.0"
num-bigint-03 = { package = "num-bigint", version = "0.3", optional = true }
num-bigint-04 = { package = "num-bigint", version = "0.4", optional = true }
bigdecimal-04 = { package = "bigdecimal", version = "0.4", optional = true }
chrono = { version = "0.4.27", default-features = false, optional = true }
lz4_flex = { version = "0.11.1" }
async-trait = "0.1.57"
Expand All @@ -43,4 +43,5 @@ time = ["dep:time"]
chrono = ["dep:chrono"]
num-bigint-03 = ["dep:num-bigint-03"]
num-bigint-04 = ["dep:num-bigint-04"]
full-serialization = ["chrono", "time", "secret", "num-bigint-03", "num-bigint-04"]
bigdecimal-04 = ["dep:bigdecimal-04"]
full-serialization = ["chrono", "time", "secret", "num-bigint-03", "num-bigint-04", "bigdecimal-04"]
21 changes: 15 additions & 6 deletions scylla-cql/src/frame/response/cql_to_rust.rs
@@ -1,8 +1,7 @@
use super::result::{CqlValue, Row};
use crate::frame::value::{
Counter, CqlDate, CqlDuration, CqlTime, CqlTimestamp, CqlTimeuuid, CqlVarint,
Counter, CqlDate, CqlDecimal, CqlDuration, CqlTime, CqlTimestamp, CqlTimeuuid, CqlVarint,
};
use bigdecimal::BigDecimal;
use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet};
use std::hash::{BuildHasher, Hash};
use std::net::IpAddr;
Expand Down Expand Up @@ -136,7 +135,7 @@ impl_from_cql_value_from_method!(Vec<u8>, into_blob); // Vec<u8>::from_cql<CqlVa
impl_from_cql_value_from_method!(IpAddr, as_inet); // IpAddr::from_cql<CqlValue>
impl_from_cql_value_from_method!(Uuid, as_uuid); // Uuid::from_cql<CqlValue>
impl_from_cql_value_from_method!(CqlTimeuuid, as_timeuuid); // CqlTimeuuid::from_cql<CqlValue>
impl_from_cql_value_from_method!(BigDecimal, into_decimal); // BigDecimal::from_cql<CqlValue>
impl_from_cql_value_from_method!(CqlDecimal, into_cql_decimal); // CqlDecimal::from_cql<CqlValue>
impl_from_cql_value_from_method!(CqlDuration, as_cql_duration); // CqlDuration::from_cql<CqlValue>
impl_from_cql_value_from_method!(CqlDate, as_cql_date); // CqlDate::from_cql<CqlValue>
impl_from_cql_value_from_method!(CqlTime, as_cql_time); // CqlTime::from_cql<CqlValue>
Expand Down Expand Up @@ -169,6 +168,16 @@ impl FromCqlVal<CqlValue> for num_bigint_04::BigInt {
}
}

#[cfg(feature = "bigdecimal-04")]
impl FromCqlVal<CqlValue> for bigdecimal_04::BigDecimal {
fn from_cql(cql_val: CqlValue) -> Result<Self, FromCqlValError> {
match cql_val {
CqlValue::Decimal(cql_decimal) => Ok(cql_decimal.into()),
_ => Err(FromCqlValError::BadCqlType),
}
}
}

#[cfg(feature = "chrono")]
impl FromCqlVal<CqlValue> for NaiveDate {
fn from_cql(cql_val: CqlValue) -> Result<Self, FromCqlValError> {
Expand Down Expand Up @@ -414,7 +423,6 @@ mod tests {
use crate as scylla;
use crate::frame::value::{Counter, CqlDate, CqlDuration, CqlTime, CqlTimestamp, CqlTimeuuid};
use crate::macros::FromRow;
use bigdecimal::BigDecimal;
use std::collections::HashSet;
use std::net::{IpAddr, Ipv4Addr};
use std::str::FromStr;
Expand Down Expand Up @@ -502,12 +510,13 @@ mod tests {
);
}

#[cfg(feature = "bigdecimal-04")]
#[test]
fn decimal_from_cql() {
let decimal = BigDecimal::from_str("123.4").unwrap();
let decimal = bigdecimal_04::BigDecimal::from_str("123.4").unwrap();
assert_eq!(
Ok(decimal.clone()),
BigDecimal::from_cql(CqlValue::Decimal(decimal))
bigdecimal_04::BigDecimal::from_cql(CqlValue::Decimal(decimal.try_into().unwrap()))
);
}

Expand Down
22 changes: 13 additions & 9 deletions scylla-cql/src/frame/response/result.rs
Expand Up @@ -2,10 +2,9 @@ use crate::cql_to_rust::{FromRow, FromRowError};
use crate::frame::response::event::SchemaChangeEvent;
use crate::frame::types::vint_decode;
use crate::frame::value::{
Counter, CqlDate, CqlDuration, CqlTime, CqlTimestamp, CqlTimeuuid, CqlVarint,
Counter, CqlDate, CqlDecimal, CqlDuration, CqlTime, CqlTimestamp, CqlTimeuuid, CqlVarint,
};
use crate::frame::{frame_errors::ParseError, types};
use bigdecimal::BigDecimal;
use byteorder::{BigEndian, ReadBytesExt};
use bytes::{Buf, Bytes};
use std::{
Expand Down Expand Up @@ -82,7 +81,7 @@ pub enum CqlValue {
Boolean(bool),
Blob(Vec<u8>),
Counter(Counter),
Decimal(BigDecimal),
Decimal(CqlDecimal),
/// Days since -5877641-06-23 i.e. 2^31 days before unix epoch
/// Can be converted to chrono::NaiveDate (-262145-1-1 to 262143-12-31) using as_date
Date(CqlDate),
Expand Down Expand Up @@ -371,7 +370,7 @@ impl CqlValue {
}
}

pub fn into_decimal(self) -> Option<BigDecimal> {
pub fn into_cql_decimal(self) -> Option<CqlDecimal> {
match self {
Self::Decimal(i) => Some(i),
_ => None,
Expand Down Expand Up @@ -671,9 +670,10 @@ pub fn deser_cql_value(typ: &ColumnType, buf: &mut &[u8]) -> StdResult<CqlValue,
CqlValue::Counter(crate::frame::value::Counter(buf.read_i64::<BigEndian>()?))
}
Decimal => {
let scale = types::read_int(buf)? as i64;
let int_value = bigdecimal::num_bigint::BigInt::from_signed_bytes_be(buf);
let big_decimal: BigDecimal = BigDecimal::from((int_value, scale));
let scale = types::read_int(buf)?;
let bytes = buf.to_vec();
let big_decimal: CqlDecimal =
CqlDecimal::from_signed_be_bytes_and_exponent(bytes, scale);

CqlValue::Decimal(big_decimal)
}
Expand Down Expand Up @@ -967,7 +967,6 @@ pub fn deserialize(buf: &mut &[u8]) -> StdResult<Result, ParseError> {
mod tests {
use crate as scylla;
use crate::frame::value::{Counter, CqlDate, CqlDuration, CqlTime, CqlTimestamp, CqlTimeuuid};
use bigdecimal::BigDecimal;
use scylla::frame::response::result::{ColumnType, CqlValue};
use std::str::FromStr;
use uuid::Uuid;
Expand Down Expand Up @@ -1111,8 +1110,10 @@ mod tests {
}
}

#[cfg(feature = "bigdecimal-04")]
#[test]
fn test_decimal() {
use bigdecimal_04::BigDecimal;
struct Test<'a> {
value: BigDecimal,
encoding: &'a [u8],
Expand All @@ -1139,7 +1140,10 @@ mod tests {

for t in tests.iter() {
let value = super::deser_cql_value(&ColumnType::Decimal, &mut &*t.encoding).unwrap();
assert_eq!(CqlValue::Decimal(t.value.clone()), value);
assert_eq!(
CqlValue::Decimal(t.value.clone().try_into().unwrap()),
value
);
}
}

Expand Down
114 changes: 110 additions & 4 deletions scylla-cql/src/frame/value.rs
@@ -1,6 +1,5 @@
use crate::frame::frame_errors::ParseError;
use crate::frame::types;
use bigdecimal::BigDecimal;
use bytes::BufMut;
use std::borrow::Cow;
use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet};
Expand Down Expand Up @@ -365,6 +364,91 @@ impl std::hash::Hash for CqlVarint {
}
}

/// Native CQL `decimal` representation.
///
/// Represented as a pair:
/// - a [`CqlVarint`] value
/// - 32-bit integer which determines the position of the decimal point
///
/// The type is not very useful in most use cases.
/// However, users can make use of more complex types
/// such as `bigdecimal::BigDecimal` (v0.4).
/// The library support (e.g. conversion from [`CqlValue`]) for the type is
/// enabled via `bigdecimal-04` crate feature.
///
/// # DB data format
/// Notice that [constructors](CqlDecimal#impl-CqlDecimal)
/// don't perform any normalization on the provided data.
/// For more details, see [`CqlVarint`] documentation.
muzarski marked this conversation as resolved.
Show resolved Hide resolved
#[derive(Clone, PartialEq, Eq, Debug)]
pub struct CqlDecimal {
int_val: CqlVarint,
scale: i32,
}

/// Constructors
impl CqlDecimal {
/// Creates a [`CqlDecimal`] from an array of bytes
/// representing [`CqlVarint`] and a 32-bit scale.
///
/// See: disclaimer about [non-normalized values](CqlVarint#db-data-format).
pub fn from_signed_be_bytes_and_exponent(bytes: Vec<u8>, scale: i32) -> Self {
Self {
int_val: CqlVarint::from_signed_bytes_be(bytes),
scale,
}
}

/// Creates a [`CqlDecimal`] from a slice of bytes
/// representing [`CqlVarint`] and a 32-bit scale.
///
/// See: disclaimer about [non-normalized values](CqlVarint#db-data-format).
pub fn from_signed_be_bytes_slice_and_exponent(bytes: &[u8], scale: i32) -> Self {
Self::from_signed_be_bytes_and_exponent(bytes.to_vec(), scale)
}
}

/// Conversion to raw bytes
impl CqlDecimal {
/// Returns a slice of bytes in two's complement
/// binary big-endian representation and a scale.
pub fn as_signed_be_bytes_slice_and_exponent(&self) -> (&[u8], i32) {
(self.int_val.as_signed_bytes_be_slice(), self.scale)
}

/// Converts [`CqlDecimal`] to an array of bytes in two's
/// complement binary big-endian representation and a scale.
pub fn into_signed_be_bytes_and_exponent(self) -> (Vec<u8>, i32) {
(self.int_val.into_signed_bytes_be(), self.scale)
}
}

#[cfg(feature = "bigdecimal-04")]
impl From<CqlDecimal> for bigdecimal_04::BigDecimal {
fn from(value: CqlDecimal) -> Self {
Self::from((
bigdecimal_04::num_bigint::BigInt::from_signed_bytes_be(
value.int_val.as_signed_bytes_be_slice(),
),
value.scale as i64,
))
}
}

#[cfg(feature = "bigdecimal-04")]
impl TryFrom<bigdecimal_04::BigDecimal> for CqlDecimal {
type Error = <i64 as TryInto<i32>>::Error;

fn try_from(value: bigdecimal_04::BigDecimal) -> Result<Self, Self::Error> {
let (bigint, scale) = value.into_bigint_and_exponent();
let bytes = bigint.to_signed_bytes_be();
Ok(Self::from_signed_be_bytes_and_exponent(
bytes,
scale.try_into()?,
))
}
}

/// Native CQL date representation that allows for a bigger range of dates (-262145-1-1 to 262143-12-31).
///
/// Represented as number of days since -5877641-06-23 i.e. 2^31 days before unix epoch.
Expand Down Expand Up @@ -881,14 +965,36 @@ impl Value for i64 {
}
}

impl Value for BigDecimal {
impl Value for CqlDecimal {
fn serialize(&self, buf: &mut Vec<u8>) -> Result<(), ValueTooBig> {
let (bytes, scale) = self.as_signed_be_bytes_slice_and_exponent();

if bytes.len() > (i32::MAX - 4) as usize {
return Err(ValueTooBig);
}
let serialized_len: i32 = bytes.len() as i32 + 4;

buf.put_i32(serialized_len);
buf.put_i32(scale);
buf.extend_from_slice(bytes);

Ok(())
}
}

#[cfg(feature = "bigdecimal-04")]
impl Value for bigdecimal_04::BigDecimal {
fn serialize(&self, buf: &mut Vec<u8>) -> Result<(), ValueTooBig> {
let (value, scale) = self.as_bigint_and_exponent();

let serialized = value.to_signed_bytes_be();
let serialized_len: i32 = serialized.len().try_into().map_err(|_| ValueTooBig)?;

buf.put_i32(serialized_len + 4);
if serialized.len() > (i32::MAX - 4) as usize {
muzarski marked this conversation as resolved.
Show resolved Hide resolved
return Err(ValueTooBig);
}
let serialized_len: i32 = serialized.len() as i32 + 4;

buf.put_i32(serialized_len);
buf.put_i32(scale.try_into().map_err(|_| ValueTooBig)?);
buf.extend_from_slice(&serialized);

Expand Down