Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/sdk readonly client #536

Merged
merged 26 commits into from
Jul 2, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
3c03fd0
feat: add doc store
imotai Jun 13, 2023
bf6b69b
Merge remote-tracking branch 'origin/main'
imotai Jun 14, 2023
50dd93a
Merge remote-tracking branch 'origin/main'
imotai Jun 19, 2023
99d8aad
Merge remote-tracking branch 'origin/main'
imotai Jun 21, 2023
ea84758
Merge remote-tracking branch 'origin/main'
imotai Jun 23, 2023
3dc6d3a
Merge remote-tracking branch 'origin/main'
imotai Jun 25, 2023
954d205
Merge remote-tracking branch 'origin/main'
imotai Jun 26, 2023
a70c835
Merge remote-tracking branch 'origin/main'
imotai Jun 29, 2023
9aa7c6f
Merge remote-tracking branch 'origin/main' into feat/database_data_im…
imotai Jun 29, 2023
da7e0ae
feat: support readonly client
imotai Jun 29, 2023
c512346
fix: fix the comment
imotai Jun 29, 2023
2e3e63c
feat: add database counter
imotai Jun 30, 2023
4d0e21f
feat: update the typescript sdk
imotai Jun 30, 2023
5de8bed
fix: fix test case
imotai Jun 30, 2023
d09c716
fix: fix the test case
imotai Jun 30, 2023
b8f4342
feat: fix the project bug
imotai Jul 1, 2023
02ea6df
test: add more query test
imotai Jul 1, 2023
66fbc92
fix: fix query test case
imotai Jul 1, 2023
26cca45
fix: fix test case
imotai Jul 1, 2023
8715084
fix: fix sdk compile error
imotai Jul 1, 2023
3ec024d
Merge branch 'main' into feat/database_data_improvement
imotai Jul 1, 2023
c97e21d
feat: add cloud sandbox
imotai Jul 2, 2023
46f1adf
feat: merge from main
imotai Jul 2, 2023
d2cbb03
feat: merge from origin
imotai Jul 2, 2023
7bf6cce
feat: update readme
imotai Jul 2, 2023
13be2b5
feat:add collection persistence
imotai Jul 2, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
16 changes: 15 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,21 @@ const {id} = await addDoc(collection, {
// query the document
const resultSet = await queryDoc<Book>(collection, "/[author=Cixin-Liu]")
```
if you have any questions, please feel free to ask us for help

if you have any questions, please feel free to ask us for help and you can find more detail about the example
* the doc for [createClient](https://docs.db3.network/functions/createClient.html)
* the doc about [queryDoc](https://docs.db3.network/functions/queryDoc.html)


# Try Our Cloud Sandbox

* [Console](https://console.cloud.db3.network/console/home)
* Data Rollup Node: https://rollup.cloud.db3.network
* Data Index Node: https://index.cloud.db3.network

You can connect to the Data Rollup Node and Data Index Node with db3.js

Note: the cloud sandbox is just for testing and unstable

# How it works

Expand Down
24 changes: 15 additions & 9 deletions sdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,32 +6,38 @@
db3.js is the [db3 network](https://github.com/dbpunk-labs/db3) javascript API and you can use it to write and query JSON documents against the db3 network.
and you can build fully decentralized applications combining [web3.js](https://github.com/web3/web3.js) and db3.js

# Play with db3.js
# Getting Started

## Install db3.js

```
yarn add db3.js
```

## Use db3.js in action
## Quick Started

```typescript
// create a account
const account = createRandomAccount()
// create the client
const client = createClient('http://127.0.0.1:26619',
'http://127.0.0.1:26639',
// create the client from db3 cloud sandbox
const client = createClient('https://rollup.cloud.db3.network',
'https://index.cloud.db3.network',
account)

await syncAccountNonce(client)
// get the collection
const collection = await getCollection("0xF7..79", "book", client)

const collection = await getCollection("0x6ef32f0d8fc6bc872ffa977eb80920a0a75d0206", "book", client)
// add a document
const {id} = await addDoc(collection, {
name:"The Three-Body Problem"
name:"The Three-Body Problem",
author:"Cixin-Liu",
rate:"4.8"} as Book)
console.log(id)
// query the document
const resultSet = await queryDoc<Book>(collection, "/[author=Cixin-Liu]")
```

<img width="500px" src="./statics/quick_started.png" align="center"/>

[Try on codesandbox](https://codesandbox.io/s/db3-js-helloword-447nxd?file=/src/App.tsx:673-710)


8 changes: 4 additions & 4 deletions sdk/src/store/database_v2.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ import {
EventDatabaseMutation,
} from '../proto/db3_mutation_v2'

import { Client } from '../client/types'
import { Client, ReadClient } from '../client/types'
import { toHEX, fromHEX } from '../crypto/crypto_utils'
import { Index } from '../proto/db3_database_v2'

Expand Down Expand Up @@ -172,7 +172,7 @@ export async function createDocumentDatabase(client: Client, desc: string) {
export async function getCollection(
addr: string,
name: string,
client: Client
client: Client | ReadClient
) {
const db = await getDatabase(addr, client)
const collections = await showCollection(db)
Expand All @@ -197,7 +197,7 @@ export async function getCollection(
* @returns the {@link Database}[]
*
**/
export async function getDatabase(addr: string, client: Client) {
export async function getDatabase(addr: string, client: Client | ReadClient) {
const response = await client.provider.getDatabase(addr)
const db = response.database
if (!db) {
Expand All @@ -223,7 +223,7 @@ export async function getDatabase(addr: string, client: Client) {
* @returns the {@link Database}[]
*
**/
export async function showDatabase(owner: string, client: Client) {
export async function showDatabase(owner: string, client: Client | ReadClient) {
const response = await client.provider.getDatabaseOfOwner(owner)
return response.databases
.filter((item) => item.database.oneofKind != undefined)
Expand Down
4 changes: 2 additions & 2 deletions sdk/src/store/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
// limitations under the License.
//

import { Client } from '../client/types'
import { Client, ReadClient } from '../client/types'
import { DocumentEntry, DocumentData } from '../client/base'
import {
DatabaseMessage as InternalDatabase,
Expand All @@ -37,7 +37,7 @@ export type CreateCollectionResult = {

export type Database = {
addr: string
client: Client
client: Client | ReadClient
internal: InternalDatabase | undefined
state: DatabaseState | undefined
}
Expand Down
Binary file added sdk/statics/quick_started.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 1 addition & 2 deletions sdk/tests/query.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ describe('test db3.js document_v2 module', () => {
],
}
await addDoc(collection, doc1)
await new Promise((r) => setTimeout(r, 1000))
await new Promise((r) => setTimeout(r, 2000))
return collection
}

Expand All @@ -125,7 +125,6 @@ describe('test db3.js document_v2 module', () => {
const queryStr = '/* | count '
const resultSet = await queryDoc(collection, queryStr)
expect('1').toBe(resultSet.count)

})

test('test query with projection', async () => {
Expand Down
6 changes: 6 additions & 0 deletions src/proto/proto/db3_database_v2.proto
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,12 @@ message DatabaseState {
uint64 total_col_count = 3;
}

message DatabaseStatePersistence {
string addr = 1;
uint64 total_doc_count = 2;
uint64 total_col_count = 3;
}

message EventTable {
string name = 1;
repeated Index index_fields = 2;
Expand Down
6 changes: 6 additions & 0 deletions src/storage/src/db_store_v2.rs
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ use tracing::info;
type StorageEngine = DBWithThreadMode<MultiThreaded>;
type DBRawIterator<'a> = DBRawIteratorWithThreadMode<'a, StorageEngine>;

const STATE_CF: &str = "DB_STATE_CF";

#[derive(Clone)]
pub struct DBStoreV2Config {
pub db_path: String,
Expand All @@ -58,11 +60,13 @@ pub struct DBStoreV2Config {
pub doc_store_conf: DocStoreConfig,
}

#[derive(Clone)]
struct CollectionState {
// the total doc count
pub total_doc_count: u64,
}

#[derive(Clone)]
struct DatabaseState {
pub doc_order: i64,
pub collection_state: HashMap<String, CollectionState>,
Expand Down Expand Up @@ -95,6 +99,7 @@ impl DBStoreV2 {
config.index_store_cf_name.as_str(),
config.doc_owner_store_cf_name.as_str(),
config.db_owner_store_cf_name.as_str(),
STATE_CF,
],
)
.map_err(|e| {
Expand All @@ -113,6 +118,7 @@ impl DBStoreV2 {
})
}

pub fn flush_state(&self) {}
fn update_db_state_for_add_db(&self, db_addr: &str) {
self.db_state.insert(
db_addr.to_string(),
Expand Down