Skip to content
Alexandr edited this page Jan 14, 2024 · 5 revisions

Otterbrix: Columnar storage (Datafraem) is entirely based on Arrow data types and backed by Arrow memory arrays. This makes data processing cache-efficient and well-supported for Inter Process Communication. Most data types follow the exact implementation from Arrow, with exception of Utf8 (this is actually LargeUtf8), Categorical, and Object (support is limited). The data types are:

SQL C++ DataFrame
Small integer SMALLINT short (signed/unsigned) Int16
Tiny integer - short (signed/unsigned) Int8
Medium integer - int Int16
Integer INTEGER int Int32
Big integer BIGINT long long Int64
Autoincrement integer SERIAL - -
Autoincrement big integer BIGSERIAL - -
Decimal number DECIMAL, NUMERIC no direct equivalent Decimal
Real number REAL float Float
Money MONEY no direct equivalent no direct equivalent
Double precision floating point DOUBLE PRECISION double Double
Character or string CHAR(N), CHARACTER(N) char, char[N] String, LargeString
Variable length string VARCHAR(N), CHARACTER VARYING(N) std::string String, LargeString
Large text TEXT std::string String, LargeString
Boolean value BOOLEAN bool Boolean
Date DATE no direct equivalent Date32, Date64
Time TIME no direct equivalent Time32, Time64
Date and time TIMESTAMP no direct equivalent Timestamp
Arrays ARRAY std::array, std::vector List, LargeList
JSON data structures JSON, JSONB no direct equivalent no direct equivalent
Universal Unique Identifier UUID no direct equivalent no direct equivalent
Bit string BIT(N) std::bitset no direct equivalent
Binary data BYTEA std::vector Binary, LargeBinary
Unsigned integers of various bit widths - - UInt8, UInt16, UInt32, UInt64
Half precision floating point number - - HalfFloat
Fixed size binary data - - FixedSizeBinary
Fixed size lists - - FixedSizeList
Key-value pair data structure - - Map
Structure containing a set of fields with various data types - - Struct
Data type that can hold values of different types - - Union
Dictionary for efficient encoding of repeating values - - Dictionary
Special data type for representing NULL values - - Null

To learn more about the internal representation of these data types, check the Arrow columnar format.

Clone this wiki locally