Skip to content
Permalink
Browse files
IGNITE-16605 Actualize row layout description in README.md - Fixes #730.
Signed-off-by: zstan <stanilovsky@gmail.com>
  • Loading branch information
zstan committed Mar 21, 2022
1 parent 953c531 commit b86cdc9d2233f27657737c6d65f535a89f1169b7
Show file tree
Hide file tree
Showing 3 changed files with 24 additions and 37 deletions.
@@ -34,7 +34,7 @@ safely compared instead of keys themselves avoiding unnecessary deserialization.
at a time of table created, and key columns can't be added or removed. All the key columns values must be provided
for a table operation, to resolve unambiguity of 'null or absent' column.

Data layout is documentation can be found [here](src/main/java/org/apache/ignite/internal/schema/README.md)
Row layout documentation can be found [here](src/main/java/org/apache/ignite/internal/schema/README.md)

## Object-to-schema mapping

@@ -94,17 +94,4 @@ public interface BinaryRow {
* Get byte array of the row.
*/
byte[] bytes();

/**
* Row flags.
*/
final class RowFlags {
/** First two flag bits reserved for format code. */
public static final int VARTABLE_FORMAT_MASK = 0x03;

/** Stub. */
private RowFlags() {
// No-op.
}
}
}
@@ -3,8 +3,8 @@ schema-defined rows.

### Schema definition

Schema is defined as a set of columns which are split into key columns chunk and value columns chunk.
Each column defined by a name, nullability flag, and a `org.apache.ignite.internal.schema.NativeType`.
Schema is defined as a set of columns which are split into keys and values columns chunks.
Each column is defined by a name, nullability flag (for value columns only), and a `org.apache.ignite.internal.schema.NativeType`.
Type is a thin wrapper over the `org.apache.ignite.internal.schema.NativeTypeSpec` to provide differentiation
between types of one kind with different size (an example of such differentiation is bitmask(n) or number(n)).
`org.apache.ignite.internal.schema.NativeTypeSpec` provides necessary indirection to read a column as a
@@ -16,13 +16,13 @@ Type | Size | Description
---- | ---- | -----------
Bitmask(n)|⌈n/8⌉ bytes|A fixed-length bitmask of n bits
Int8|1 byte|1-byte signed integer
Uint8|1 byte|1-byte unsigned integer
UInt8|1 byte|1-byte unsigned integer
Int16|2 bytes|2-byte signed integer
Uint16|2 bytes|2-byte unsigned integer
UInt16|2 bytes|2-byte unsigned integer
Int32|4 bytes|4-byte signed integer
Uint32|4 bytes|4-byte unsigned integer
UInt32|4 bytes|4-byte unsigned integer
Int64|8 bytes|8-byte signed integer
Uint64|8 bytes|8-byte unsigned integer
UInt64|8 bytes|8-byte unsigned integer
Float|4 bytes|4-byte floating-point number
Double|8 bytes|8-byte floating-point number
Number([n])|Variable|Variable-length number (optionally bound by n bytes in size)
@@ -45,26 +45,26 @@ lookup. In a row, key columns and value columns are separated and written to chu

Row structure has the following format:

┌─────────────────────────────┬─────────────────────┐
Header │ Data │
├─────────┬─────────┬─────────┼──────────┬──────────┤
│ Schema │ Flags │ Key │ Key │ Value │
│ Version │ Hash │ Chunk │ Chunk │
├─────────┼─────────┼─────────┼──────────┼──────────┤
│ 2 Bytes │ 2 Bytes │ 4 Bytes │ Variable │ Variable │
└─────────┴─────────┴─────────┴──────────┴──────────┘
┌───────────────────┬─────────────────────┐
│ Header │ Data │
├─────────┬─────────┼──────────┬──────────┤
│ Schema │ Key │ Key │ Value │
│ Version │ Hash │ Chunk │ Chunk │
├─────────┼─────────┼──────────┼──────────┤
│ 2 Bytes │ 4 Bytes │ Variable │ Variable │
└─────────┴─────────┴──────────┴──────────┘


Each chunk section has the following structure:

┌────────────────────────┐
│ │
┌─────────┬─────────────────────────┬─────────────────────────┬────────┴────────┬──────────┬────⌄─────┐
│ Full │ Null-Defaults │ Varsize Columns Offsets │ Varsize Columns │ Fixsize │ Varsize │
│ Size │ Map │ Table Size │ Offsets Table │ Columns │ Columns │
├─────────┼─────────────────────────┼─────────────────────────┼─────────────────┼──────────┼──────────┤
│ 4 Bytes │ ⌈Number of columns / 8⌉ │ 2 Bytes │ Variable │ Variable │ Variable │
└─────────┴─────────────────────────┴─────────────────────────┴─────────────────┴──────────┴──────────┘
┌────────────────────────┐
│ │
┌─────────┬───────┬──────────────────────────┬─────────────────────────┬────────┴────────┬──────────┬────⌄─────┐
│ Full │ Flags │ Null-Defaults mapping │ Varsize Columns Offsets │ Varsize Columns │ Fixsize │ Varsize │
│ Size │ Size │ (only for Value chunk) │ Table Size │ Offsets Table │ Columns │ Columns │
├─────────┼───────┼──────────────────────────┼─────────────────────────┼─────────────────┼──────────┼──────────┤
│ 4 Bytes │ 1 Byte│ ⌈Number of columns / 8⌉ │ 2 Bytes │ Variable │ Variable │ Variable │
└─────────┴───────┴──────────────────────────┴─────────────────────────┴─────────────────┴──────────┴──────────┘
All columns within a group are split into groups of fixed-size columns and variable-size columns. Withing the group of
fixsize columns, the columns are sorted by size, then by column name. Within the group of varsize columns, the columns
are sorted by column name. Inside a row default values and nulls are omitted and encoded in the null-defaults map
@@ -79,7 +79,7 @@ To assemble a row with some schema, an instance of `org.apache.ignite.internal.s
must be used which provides the low-level API for building rows. When using the row assembler, the
columns must be passed to the assembler in the internal schema sort order. Additionally, when constructing
the instance of the assembler, the user should pre-calculate the size of the row to avoid extra array copies,
and the number of non-null varlen columns for key and value chunks. Less restrictive building techniques
and the number of non-null varlen columns for a value chunks. Less restrictive building techniques
are provided by class (de)serializers and tuple builder, which take care of sizing and column order.

To read column values of a row, one needs to construct a subclass of

0 comments on commit b86cdc9

Please sign in to comment.