Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions docs/namespaces/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -53,17 +53,34 @@ directory:

As a user of LanceDB OSS, you might never notice namespaces at first, because LanceDB exposes the single-level hierarchy shown above, with the data stored in the `data/` directory, where the root namespace is implicit. Connecting to this namespace is as simple as connecting to the catalog root:

<CodeGroup>
```python Python icon="python"
import lancedb

# Connect to the directory namespace root
db = lancedb.connect("./local_lancedb")
```

```typescript TypeScript icon="square-js"
import * as lancedb from "@lancedb/lancedb";

// Connect to the directory namespace root
const db = await lancedb.connect("./local_lancedb");
```

```rust Rust icon="rust"
use lancedb::connect;

// Connect to the directory namespace root
let db = connect("./local_lancedb").execute().await?;
```
</CodeGroup>

This creates the default namespace directory (`data/`) under the specified root path.

You can also explicitly connect to a namespace using `lancedb.connect_namespace(...)` with the directory namespace implementation:

<CodeGroup>
```python Python icon="python"
import lancedb

Expand All @@ -79,6 +96,47 @@ print(f"Created table: {table.name}")
# Created table: user
```

```typescript TypeScript icon="square-js"
import * as lancedb from "@lancedb/lancedb";

// Local namespace-backed catalog root (DirectoryNamespace)
// See https://lance.org/format/namespace/dir/catalog-spec/
const db = await lancedb.connectNamespace("dir", { root: "./local_lancedb" });

const table = await db.createTable(
"user",
[{ id: 1, vector: [0.1, 0.2], name: "alice" }],
{ mode: "create" },
);
console.log(`Created table: ${table.name}`);
// Created table: user
```

```rust Rust icon="rust"
use std::collections::HashMap;
use std::sync::Arc;
use arrow_schema::{DataType, Field, Schema};

// Local namespace-backed catalog root (DirectoryNamespace)
// See https://lance.org/format/namespace/dir/catalog-spec/
let mut properties = HashMap::new();
properties.insert("root".to_string(), "./local_lancedb".to_string());
let db = lancedb::connect_namespace("dir", properties)
.execute()
.await?;

let schema = Arc::new(Schema::new(vec![
Field::new("id", DataType::Int64, false),
]));
let table = db
.create_empty_table("user", schema)
.execute()
.await?;
println!("Created table: {}", table.name());
// Created table: user
```
</CodeGroup>

<Info>
- For simple use cases in LanceDB OSS, you don't need to go too deep into namespaces.
- To integrate LanceDB with external catalogs and to use it as a true **multimodal lakehouse**, it's useful to understand the different namespace implementations and how to use them in your organization's setup.
Expand All @@ -98,6 +156,7 @@ LanceDB Enterprise REST requests use the `x-api-key` header for API-key authenti
that route multiple databases through the same endpoint can also use headers such as
`x-lancedb-database` or `x-lancedb-database-prefix` for database context.

<CodeGroup>
```python Python icon="python"
import os
import lancedb
Expand All @@ -114,6 +173,46 @@ db = lancedb.connect_namespace(
},
)
```

```typescript TypeScript icon="square-js"
import * as lancedb from "@lancedb/lancedb";

// Remote namespace-backed catalog root (RestNamespace)
// See https://lance.org/format/namespace/rest/catalog-spec/
const db = await lancedb.connectNamespace("rest", {
uri: "https://<your_catalog>.internal.<your_org>.com",
headers: {
"x-api-key": process.env.API_KEY ?? "",
// or:
// Authorization: `Bearer ${process.env.REST_AUTH_TOKEN}`,
},
});
```

```rust Rust icon="rust"
use std::collections::HashMap;

// Remote namespace-backed catalog root (RestNamespace)
// See https://lance.org/format/namespace/rest/catalog-spec/
let mut properties = HashMap::new();
properties.insert(
"uri".to_string(),
"https://<your_catalog>.internal.<your_org>.com".to_string(),
);
properties.insert(
"headers.x-api-key".to_string(),
std::env::var("API_KEY")?,
);
// or:
// properties.insert(
// "headers.Authorization".to_string(),
// format!("Bearer {}", std::env::var("REST_AUTH_TOKEN")?),
// );
let db = lancedb::connect_namespace("rest", properties)
.execute()
.await?;
```
</CodeGroup>
[LanceDB Enterprise](/enterprise) operates a REST namespace server on top of the Lance format, so any REST client that can speak the REST namespace API
contract can be used to interact with it. For authentication examples in LanceDB Enterprise, visit
the [Namespaces in SDKs](/namespaces/usage#namespaces-in-lancedb-enterprise) page.
Expand Down
31 changes: 15 additions & 16 deletions docs/namespaces/usage.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ keywords: ["namespace", "create_table", "open_table", "list_tables", "catalog"]
import {
PyNamespaceTableOps,
PyNamespaceAdminOps,
TsNamespaceTableOps,
TsNamespaceAdminOps,
RsNamespaceAdminOps,
RsNamespaceTableOps,
} from '/snippets/connection.mdx';
Expand Down Expand Up @@ -46,6 +48,10 @@ The SDK methods expose the namespace path in the idiom of each language:
{PyNamespaceTableOps}
</CodeBlock>

<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsNamespaceTableOps}
</CodeBlock>

<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsNamespaceTableOps}
</CodeBlock>
Expand All @@ -57,19 +63,14 @@ An empty namespace (`[]`), which is the default, means "root namespace", and the
the `data/` directory under the specified root path.
</Note>

In TypeScript, namespace-scoped table operations are available on `Connection` even though namespace
lifecycle operations are not exposed there. These calls use the same namespace path array:

- Create a table: `await db.createTable("user", rows, ["prod", "search"], { mode: "create" })`
- List tables: `await db.tableNames(["prod", "search"])`
- Drop a table: `await db.dropTable("user", ["prod", "search"])`

## Namespace management APIs

You can open/create/drop tables inside a namespace path (like `["prod", "search"]`).
The Python and Rust SDKs expose namespace lifecycle operations directly.
All three SDKs expose namespace lifecycle operations directly.
In Python, use `lancedb.connect_namespace(...)` when calling namespace lifecycle methods such as
`create_namespace`, `list_namespaces`, `describe_namespace`, and `drop_namespace`.
In TypeScript, use `lancedb.connectNamespace(...)` and call `createNamespace`, `listNamespaces`,
`describeNamespace`, and `dropNamespace` on the returned `Connection`.
In Rust, use `lancedb::connect_namespace(...)` and call `create_namespace`, `list_namespaces`,
and `drop_namespace`.

Expand All @@ -78,23 +79,21 @@ and `drop_namespace`.
{PyNamespaceAdminOps}
</CodeBlock>

<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsNamespaceAdminOps}
</CodeBlock>

<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsNamespaceAdminOps}
</CodeBlock>
</CodeGroup>

<Note>
In TypeScript, namespace lifecycle operations such as creating, listing, describing, and dropping
namespaces are not currently exposed on `Connection`. Use a namespace-aware admin surface, such as
[REST](/api-reference/rest/index), when you need to manage namespace lifecycle from TypeScript.
</Note>

Namespace creation and deletion have modes that control what happens when the target already exists,
doesn't exist, or contains data:

- Create mode: `create` fails if the namespace already exists, `exist_ok` keeps the existing namespace, and `overwrite` replaces it.
- Drop mode: `FAIL` reports an error when the namespace doesn't exist, and `SKIP` treats a missing namespace as a successful no-op.
- Drop behavior: `RESTRICT` keeps non-empty namespaces from being dropped, and `CASCADE` drops child namespaces and tables first.
- Drop mode: `fail` reports an error when the namespace doesn't exist, and `skip` treats a missing namespace as a successful no-op.
- Drop behavior: `restrict` keeps non-empty namespaces from being dropped, and `cascade` drops child namespaces and tables first.

Namespace path components can't be empty. Each component can contain only letters, numbers,
underscores, hyphens, and periods.
Expand Down
4 changes: 4 additions & 0 deletions docs/snippets/connection.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ export const TsConnectEnterpriseQuickstart = "const uri = \"db://your-database-u

export const TsConnectObjectStorage = "async function connectObjectStorageExample() {\n const uri = \"s3://your-bucket/path\";\n // You can also use \"gs://your-bucket/path\" or \"az://your-container/path\".\n const db = await lancedb.connect(uri);\n return db;\n}\n";

export const TsNamespaceAdminOps = "const db = await lancedb.connectNamespace(\"dir\", { root: \"./local_lancedb\" });\nconst namespace = [\"prod\", \"search\"];\n\nawait db.createNamespace([\"prod\"]);\nawait db.createNamespace([\"prod\", \"search\"]);\n\nconst childNamespaces = (await db.listNamespaces([\"prod\"])).namespaces;\nconsole.log(`Child namespaces under ${JSON.stringify(namespace)}:`, childNamespaces);\n// Child namespaces under [\"prod\",\"search\"]: [ 'search' ]\n\nconst metadata = await db.describeNamespace([\"prod\", \"search\"]);\nconsole.log(`Metadata for namespace ${JSON.stringify(namespace)}:`, metadata);\n\nawait db.dropNamespace([\"prod\", \"search\"], { mode: \"skip\" });\nawait db.dropNamespace([\"prod\"], { mode: \"skip\" });\n";

export const TsNamespaceTableOps = "const db = await lancedb.connectNamespace(\"dir\", { root: \"./local_lancedb\" });\n\n// Create namespace tree: prod/search and prod/recommendations\nawait db.createNamespace([\"prod\"], { mode: \"exist_ok\" });\nawait db.createNamespace([\"prod\", \"search\"], { mode: \"exist_ok\" });\nawait db.createNamespace([\"prod\", \"recommendations\"], { mode: \"exist_ok\" });\n\nawait db.createTable(\n \"user\",\n [{ id: 1, vector: [0.1, 0.2], name: \"alice\" }],\n [\"prod\", \"search\"],\n { mode: \"create\" }, // use \"overwrite\" only if you want to replace existing table\n);\n\nawait db.createTable(\n \"user\",\n [{ id: 2, vector: [0.3, 0.4], name: \"bob\" }],\n [\"prod\", \"recommendations\"],\n { mode: \"create\" },\n);\n\n// Verify\nconsole.log((await db.listNamespaces()).namespaces); // [\"prod\"]\nconsole.log((await db.listNamespaces([\"prod\"])).namespaces); // [\"recommendations\", \"search\"]\nconsole.log(await db.tableNames([\"prod\", \"search\"])); // [\"user\"]\nconsole.log(await db.tableNames([\"prod\", \"recommendations\"])); // [\"user\"]\n";

export const RsConnect = "async fn connect_example(uri: &str) {\n let db = connect(uri).execute().await.unwrap();\n let _ = db;\n}\n";

export const RsConnectEnterprise = "let uri = \"db://your-database-uri\";\nlet api_key = \"your-api-key\";\nlet region = \"us-east-1\";\n";
Expand Down
62 changes: 61 additions & 1 deletion tests/ts/connection.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -67,4 +67,64 @@ async function connectObjectStorageExample() {
}
// --8<-- [end:connect_object_storage]

void [uri, apiKey, region, connectObjectStorageExample, connectEnterpriseQuickstart];
async function namespaceTableOpsExample() {
// --8<-- [start:namespace_table_ops]
const db = await lancedb.connectNamespace("dir", { root: "./local_lancedb" });

// Create namespace tree: prod/search and prod/recommendations
await db.createNamespace(["prod"], { mode: "exist_ok" });
await db.createNamespace(["prod", "search"], { mode: "exist_ok" });
await db.createNamespace(["prod", "recommendations"], { mode: "exist_ok" });

await db.createTable(
"user",
[{ id: 1, vector: [0.1, 0.2], name: "alice" }],
["prod", "search"],
{ mode: "create" }, // use "overwrite" only if you want to replace existing table
);

await db.createTable(
"user",
[{ id: 2, vector: [0.3, 0.4], name: "bob" }],
["prod", "recommendations"],
{ mode: "create" },
);

// Verify
console.log((await db.listNamespaces()).namespaces); // ["prod"]
console.log((await db.listNamespaces(["prod"])).namespaces); // ["recommendations", "search"]
console.log(await db.tableNames(["prod", "search"])); // ["user"]
console.log(await db.tableNames(["prod", "recommendations"])); // ["user"]
// --8<-- [end:namespace_table_ops]
}

async function namespaceAdminOpsExample() {
// --8<-- [start:namespace_admin_ops]
const db = await lancedb.connectNamespace("dir", { root: "./local_lancedb" });
const namespace = ["prod", "search"];

await db.createNamespace(["prod"]);
await db.createNamespace(["prod", "search"]);

const childNamespaces = (await db.listNamespaces(["prod"])).namespaces;
console.log(`Child namespaces under ${JSON.stringify(namespace)}:`, childNamespaces);
// Child namespaces under ["prod","search"]: [ 'search' ]

const metadata = await db.describeNamespace(["prod", "search"]);
console.log(`Metadata for namespace ${JSON.stringify(namespace)}:`, metadata);

await db.dropNamespace(["prod", "search"], { mode: "skip" });
await db.dropNamespace(["prod"], { mode: "skip" });
// --8<-- [end:namespace_admin_ops]
return { childNamespaces, metadata };
}

void [
uri,
apiKey,
region,
connectObjectStorageExample,
connectEnterpriseQuickstart,
namespaceTableOpsExample,
namespaceAdminOpsExample,
];
Loading