Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 13 additions & 14 deletions docs/guide/java/row-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Row format is a cache-friendly binary random access format that supports:
- **Zero-copy access**: Read fields directly from binary without allocating objects
- **Partial deserialization**: Access only the fields you need
- **Skipping serialization**: Skip serialization of fields you don't need
- **Cross-language compatibility**: Works across Python, Java, C++, and other languages
- **Cross-language compatibility**: Standard rows work across Python, Java, C++, and Rust
- **Column format conversion**: Can convert to Apache Arrow columnar format automatically

## Basic Usage
Expand Down Expand Up @@ -62,7 +62,7 @@ for (int i = 0; i < 1_000_000; i++) {
}
foo.f4 = bars;

// Encode to row format (cross-language compatible with Python/C++)
// Encode to row format (cross-language compatible with Python/C++/Rust)
BinaryRow binaryRow = encoder.toRow(foo);

// Zero-copy random access without full deserialization
Expand Down Expand Up @@ -95,13 +95,13 @@ straight to the ordinal row getter without another schema map lookup or typed ha

## Key Benefits

| Feature | Description |
| ----------------------- | ------------------------------------------------------ |
| Zero-Copy Access | Read nested fields without deserializing entire object |
| Memory Efficiency | Memory-map large datasets directly from disk |
| Cross-Language | Binary format compatible between Java, Python, C++ |
| Partial Deserialization | Deserialize only specific elements you need |
| High Performance | Skip unnecessary data parsing for analytics workloads |
| Feature | Description |
| ----------------------- | -------------------------------------------------------- |
| Zero-Copy Access | Read nested fields without deserializing entire object |
| Memory Efficiency | Memory-map large datasets directly from disk |
| Cross-Language | Binary format compatible between Java, Python, C++, Rust |
| Partial Deserialization | Deserialize only specific elements you need |
| High Performance | Skip unnecessary data parsing for analytics workloads |

## When to Use Row Format

Expand All @@ -121,20 +121,19 @@ Row format works seamlessly across languages. The same binary data can be access

```python
import pyfory
import pyarrow as pa
from dataclasses import dataclass
from typing import List, Dict

@dataclass
class Bar:
f1: str
f2: List[pa.int64]
f2: List[pyfory.Int64]

@dataclass
class Foo:
f1: pa.int32
f2: List[pa.int32]
f3: Dict[str, pa.int32]
f1: pyfory.Int32
f2: List[pyfory.Int32]
f3: Dict[str, pyfory.Int32]
f4: List[Bar]

encoder = pyfory.encoder(Foo)
Expand Down
16 changes: 8 additions & 8 deletions docs/guide/python/row-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,13 @@ Row format drastically reduces overhead when working with large objects where on

**Key Benefits:**

| Feature | Description |
| ----------------------- | ------------------------------------------------------ |
| Zero-Copy Access | Read nested fields without deserializing entire object |
| Memory Efficiency | Memory-map large datasets directly from disk |
| Cross-Language | Binary format compatible between Python, Java, C++ |
| Partial Deserialization | Deserialize only specific elements you need |
| High Performance | Skip unnecessary data parsing for analytics workloads |
| Feature | Description |
| ----------------------- | -------------------------------------------------------- |
| Zero-Copy Access | Read nested fields without deserializing entire object |
| Memory Efficiency | Memory-map large datasets directly from disk |
| Cross-Language | Binary format compatible between Python, Java, C++, Rust |
| Partial Deserialization | Deserialize only specific elements you need |
| High Performance | Skip unnecessary data parsing for analytics workloads |

## Basic Usage

Expand Down Expand Up @@ -102,7 +102,7 @@ standard-library Python `array.array` carrier, not PyArrow.

## Cross-Language Compatibility

Row format works seamlessly across languages. The same binary data can be accessed from Java and C++.
Row format works seamlessly across languages. The same binary data can be accessed from Java, C++, and Rust.

### Java

Expand Down
13 changes: 8 additions & 5 deletions docs/guide/rust/basic-serialization.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,10 +152,12 @@ fory = { version = "1.5.0", features = ["chrono"] }

### Custom Types

| Macro | Description |
| ----------------------- | -------------------------- |
| `#[derive(ForyStruct)]` | Object graph serialization |
| `#[derive(ForyRow)]` | Row-based serialization |
| Macro | Description |
| ----------------------- | ------------------------------------- |
| `#[derive(ForyStruct)]` | Object graph serialization |
| `#[derive(ForyRow)]` | Standard Row Format for named structs |

`ForyRow` has a separate type set and returns borrowed field views. Row reads and field access use `Result` to report invalid row data. See [Row Format](row-format.md) for supported types, nullability, and examples.

## Serialization APIs

Expand Down Expand Up @@ -205,7 +207,7 @@ all supported carriers, and registration.

## Performance Tips

- **Zero-Copy Deserialization**: Row format enables direct memory access without copying
- **Selective Zero-Copy Access**: Row Format returns borrowed views for direct field and element access
- **Buffer Pre-allocation**: Minimizes memory allocations during serialization
- **Compact Encoding**: Variable-length encoding for space efficiency
- **Little-Endian**: Optimized for modern CPU architectures
Expand All @@ -217,3 +219,4 @@ all supported carriers, and registration.
- [References](references.md) - Shared and circular references
- [Custom Serializers](custom-serializers.md) - Custom serialization
- [External-Type Serialization](external-types.md) - Third-party values and carrier roots
- [Row Format](row-format.md) - Standard Row Format and zero-copy borrowed views
35 changes: 6 additions & 29 deletions docs/guide/rust/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ license: |
limitations under the License.
---

**Apache Fory™** is a blazing fast multi-language serialization framework powered by **JIT compilation** and **zero-copy** techniques, providing up to **ultra-fast performance** while maintaining ease of use and safety.
**Apache Fory™** is a high-performance multi-language serialization framework. The Rust implementation uses compile-time code generation for object serialization and borrowed views for zero-copy Row Format access.

The Rust implementation provides versatile and high-performance serialization with automatic memory management and compile-time type safety. It supports both xlang mode for cross-language payloads and native mode for Rust-only payloads.

Expand All @@ -32,7 +32,7 @@ The Rust implementation provides versatile and high-performance serialization wi
- **Circular references**: Automatic tracking of shared and circular references with `Rc`/`Arc` and weak pointers
- **Polymorphic**: Serialize trait objects with `Box<dyn Trait>`, `Rc<dyn Trait>`, and `Arc<dyn Trait>`
- **Schema evolution**: Compatible mode for independent schema changes
- **Two formats**: Object graph serialization and zero-copy row-based format
- **Two formats**: Object graph serialization and the Standard Row Format shared with Java, C++, and Python

## Crates

Expand Down Expand Up @@ -146,30 +146,6 @@ fn main() -> Result<(), Error> {

**Tip:** Perform registrations (such as `fory.register::<T>(id)`) before spawning threads so every worker sees the same metadata. Once configured, wrapping the instance in `Arc` is enough to fan out serialization and deserialization tasks safely.

## Architecture

The Rust implementation consists of three main crates:

```
fory/ # High-level API
├── src/lib.rs # Public API exports

fory-core/ # Core serialization engine
├── src/
│ ├── fory.rs # Main serialization entry point
│ ├── buffer.rs # Binary buffer management
│ ├── serializer/ # Type-specific serializers
│ ├── resolver/ # Type resolution and metadata
│ ├── meta/ # Meta string compression
│ ├── row/ # Row format implementation
│ └── types.rs # Type definitions

fory-derive/ # Procedural macros
├── src/
│ ├── object/ # ForyStruct macro
│ └── fory_row.rs # ForyRow macro
```

## Use Cases

### Object Serialization
Expand All @@ -180,13 +156,14 @@ fory-derive/ # Procedural macros
- Schema evolution with compatible mode
- Graph-like data structures with circular references

### Row-Based Serialization
### Standard Row Format

- High-throughput data processing
- Analytics workloads requiring fast field access
- Memory-constrained environments
- Real-time data streaming applications
- Zero-copy scenarios
- Zero-copy field and collection access
- Standard Row Format interchange with Java, C++, and Python

## Next Steps

Expand All @@ -199,5 +176,5 @@ fory-derive/ # Procedural macros
- [Custom Serializers](custom-serializers.md) - Implement custom serialization behavior
- [External-Type Serialization](external-types.md) - External structural and custom serializers
plus carrier composition
- [Row Format](row-format.md) - Zero-copy row-based format
- [Row Format](row-format.md) - Standard Row Format with borrowed views
- [gRPC Support](grpc-support.md) - Fory payloads over tonic
Loading
Loading