Skip to content

Commit

Permalink
WIP: readme: remove status and roadmap, fold synopsis
Browse files Browse the repository at this point in the history
  • Loading branch information
drmingdrmer committed Jun 28, 2022
1 parent 8e13563 commit e34f481
Show file tree
Hide file tree
Showing 3 changed files with 60 additions and 60 deletions.
62 changes: 30 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,17 +24,12 @@ corresponding serialization APIs to persisting them on-disk or for transport.

- [Why slim](#why-slim)
- [Memory overhead](#memory-overhead)
- [Performance](#performance)
- [False Positive Rate](#false-positive-rate)
- [Status](#status)
- [Roadmap](#roadmap)
- [Change-log](#change-log)
- [Performance benchmark](#performance-benchmark)
- [Synopsis](#synopsis)
- [Index keys, get by key](#index-keys-get-by-key)
- [Index key ranges, get by key](#index-key-ranges-get-by-key)
- [Scan](#scan)
- [Use SlimIndex to index external data](#use-slimindex-to-index-external-data)
- [Getting started](#getting-started)
- [Who are using slim](#who-are-using-slim)
- [Roadmap](#roadmap)
- [Feedback and contributions](#feedback-and-contributions)
- [Authors](#authors)
- [License](#license)
Expand Down Expand Up @@ -146,34 +141,19 @@ Time(in nano second) spent on a `Get()` with different key count(`n`) and key le
See: [trie/report/](trie/report/)


## Status

- `SlimTrie` APIs are stable, and has been used in a production env.

Meanwhile we focus on optimizing memory usage and query performance.

- Internal data structure are promised to be backward compatible for ever.
No data migration issue!


## Roadmap

- [x] **2021-01-15** v0.5.11 Query by range
- [x] **2019-09-18** v0.5.10 Reduce false positive rate to less than 0.05%
- [x] **2019-06-03** v0.5.9 Large key set benchmark
- [x] **2019-05-29** v0.5.6 Support up to 2 billion keys
- [x] **2019-05-18** v0.5.4 Reduce memory usage from 40 to 14 bits/key
- [x] **2019-04-20** v0.4.3 Range index: many keys share one index item
- [x] **2019-04-18** v0.4.1 Marshaling support
- [x] **2019-03-08** v0.1.0 SlimIndex SlimTrie

## Change-log

[Change-log](docs/change-log.yaml)

## Synopsis

### Index keys, get by key
### 1. Index on-disk key-values

One of the typical usages of slim is to index serialized data on disk(e.g., key value records in a SSTable).
By keeping a slim in memory, one can quickly find the on-disk offset of the record by a key.

<details>
<summary>Show me the code ...</summary>

```go
package index_test
Expand Down Expand Up @@ -241,7 +221,9 @@ func Example() {
}
```

### Index key ranges, get by key
</details>

### 2. Sparse index

Create an index item for every 4(or more as you wish) keys.

Expand All @@ -250,6 +232,9 @@ cost if there are huge amount keys in external data.
Such as to index billions of 4KB objects on a 4TB disk(because one disk IO
costs 20ms for either reading 4KB or reading 1MB).

<details>
<summary>Show me the code ...</summary>

```go
package index_test

Expand Down Expand Up @@ -331,9 +316,20 @@ func Example_indexRanges() {
}
```

</details>

### Scan

### 3. Range scan

Slim can also be used as a traditional in-memory kv-store:
Building a slim with `Opt{ Complete: Bool(true) }`,
it won't strip out any information(e.g., it won't eliminate single-branch labels)
and it will functions the same as a `btree`.

This snippet shows how to iterate key values.

<details>
<summary>Show me the code ...</summary>

```go
package trie
Expand Down Expand Up @@ -418,6 +414,8 @@ func ExampleSlimTrie_ScanFrom() {
}
```

</details>

<!-- ## FAQ -->

## Getting started
Expand Down
52 changes: 27 additions & 25 deletions docs/README.md.j2
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ corresponding serialization APIs to persisting them on-disk or for transport.
- [Why slim](#why-slim)
- [Memory overhead](#memory-overhead)
- [Performance benchmark](#performance-benchmark)
- [Status](#status)
- [Synopsis](#synopsis)
- [Use SlimIndex to index external data](#use-slimindex-to-index-external-data)
- [Getting started](#getting-started)
Expand Down Expand Up @@ -134,40 +133,27 @@ Time(in nano second) spent on a `Get()` with different key count(`n`) and key le
See: [trie/report/](trie/report/)


## Status

- `SlimTrie` APIs are stable, and has been used in a production env.

Meanwhile we focus on optimizing memory usage and query performance.

- Internal data structure are promised to be backward compatible for ever.
No data migration issue!


## Roadmap

- [x] **2021-01-15** v0.5.11 Query by range
- [x] **2019-09-18** v0.5.10 Reduce false positive rate to less than 0.05%
- [x] **2019-06-03** v0.5.9 Large key set benchmark
- [x] **2019-05-29** v0.5.6 Support up to 2 billion keys
- [x] **2019-05-18** v0.5.4 Reduce memory usage from 40 to 14 bits/key
- [x] **2019-04-20** v0.4.3 Range index: many keys share one index item
- [x] **2019-04-18** v0.4.1 Marshaling support
- [x] **2019-03-08** v0.1.0 SlimIndex SlimTrie

## Change-log

[Change-log](docs/change-log.yaml)

## Synopsis

### Index keys, get by key
### 1. Index on-disk key-values

One of the typical usages of slim is to index serialized data on disk(e.g., key value records in a SSTable).
By keeping a slim in memory, one can quickly find the on-disk offset of the record by a key.

<details>
<summary>Show me the code ...</summary>

```go
{% include 'index/example_test.go' %}
```

### Index key ranges, get by key
</details>

### 2. Sparse index

Create an index item for every 4(or more as you wish) keys.

Expand All @@ -176,18 +162,34 @@ cost if there are huge amount keys in external data.
Such as to index billions of 4KB objects on a 4TB disk(because one disk IO
costs 20ms for either reading 4KB or reading 1MB).

<details>
<summary>Show me the code ...</summary>

```go
{% include 'index/example_range_test.go' %}
```

</details>


### Scan
### 3. Range scan

Slim can also be used as a traditional in-memory kv-store:
Building a slim with `Opt{ Complete: Bool(true) }`,
it won't strip out any information(e.g., it won't eliminate single-branch labels)
and it will functions the same as a `btree`.

This snippet shows how to iterate key values.

<details>
<summary>Show me the code ...</summary>

```go
{% include 'trie/example_scan_test.go' %}
```

</details>

<!-- ## FAQ -->

## Getting started
Expand Down
6 changes: 3 additions & 3 deletions scripts/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
semantic_version~=2.6.0
jinja2~=2.11.0
PyYAML~=5.4.0
semantic_version>=2.6.0
jinja2>=2.11.0
PyYAML>=5.3.0

0 comments on commit e34f481

Please sign in to comment.