diff --git a/docs/api/branches.mdx b/docs/api/branches.mdx new file mode 100644 index 0000000..ebccdd0 --- /dev/null +++ b/docs/api/branches.mdx @@ -0,0 +1,192 @@ +--- +title: Branches API +description: Manage database branches and their settings +--- + +The Branches API allows you to manage your database branches, including listing, creating, updating, and deleting branches. + +## List branches + +Retrieves a list of all branches within a project. + +```bash +GET /organizations/{organizationID}/projects/{projectID}/branches +``` + +### Path Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| organizationID | string | Unique identifier for the organization | +| projectID | string | Unique identifier for the project | + +### Response + +```json +{ + "branches": [ + { + "id": "br_123", + "name": "main", + "createdAt": "2024-01-01T00:00:00Z", + "updatedAt": "2024-01-01T00:00:00Z", + "region": "us-east-1", + "publicAccess": false, + "description": "Main branch" + } + ] +} +``` + +## Create branch + +Creates a new branch within a project. Branches can be created from scratch or derived from an existing parent branch. + +```bash +POST /organizations/{organizationID}/projects/{projectID}/branches +``` + +### Path Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| organizationID | string | Unique identifier for the organization | +| projectID | string | Unique identifier for the project | + +### Request Body + +```json +{ + "name": "new-branch", + "description": "Development branch", + "parentID": "br_123", + "configuration": { + "image": "postgresql:16", + "instanceType": "standard", + "instances": 1, + "region": "us-east-1", + "storage": 10 + } +} +``` + +### Response + +```json +{ + "id": "br_456", + "name": "new-branch", + "createdAt": "2024-01-01T00:00:00Z", + "updatedAt": "2024-01-01T00:00:00Z", + "region": "us-east-1", + "publicAccess": false, + "description": "Development branch", + "connectionString": "postgresql://..." +} +``` + +## Get branch + +Retrieves detailed information about a specific branch. + +```bash +GET /organizations/{organizationID}/projects/{projectID}/branches/{branchID} +``` + +### Path Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| organizationID | string | Unique identifier for the organization | +| projectID | string | Unique identifier for the project | +| branchID | string | Unique identifier for the branch | + +### Response + +```json +{ + "id": "br_123", + "name": "main", + "createdAt": "2024-01-01T00:00:00Z", + "updatedAt": "2024-01-01T00:00:00Z", + "region": "us-east-1", + "publicAccess": false, + "description": "Main branch", + "connectionString": "postgresql://...", + "status": { + "status": "healthy", + "instanceCount": 1, + "instanceReadyCount": 1, + "instances": [ + { + "id": "inst_123", + "status": "ready" + } + ], + "lifecycle": { + "phase": "running" + } + } +} +``` + +## Update branch + +Updates the configuration of a branch. + +```bash +PATCH /organizations/{organizationID}/projects/{projectID}/branches/{branchID} +``` + +### Path Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| organizationID | string | Unique identifier for the organization | +| projectID | string | Unique identifier for the project | +| branchID | string | Unique identifier for the branch | + +### Request Body + +```json +{ + "name": "updated-branch", + "description": "Updated description", + "instances": 2, + "storage": 20 +} +``` + +### Response + +```json +{ + "id": "br_123", + "name": "updated-branch", + "createdAt": "2024-01-01T00:00:00Z", + "updatedAt": "2024-01-01T00:00:00Z", + "region": "us-east-1", + "publicAccess": false, + "description": "Updated description" +} +``` + +## Delete branch + +Permanently deletes a branch and all its associated data. This action cannot be undone. + +```bash +DELETE /organizations/{organizationID}/projects/{projectID}/branches/{branchID} +``` + +### Path Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| organizationID | string | Unique identifier for the organization | +| projectID | string | Unique identifier for the project | +| branchID | string | Unique identifier for the branch | + +### Response + +Returns a 204 status code with no content on success. \ No newline at end of file diff --git a/docs/api/index.mdx b/docs/api/index.mdx new file mode 100644 index 0000000..7222121 --- /dev/null +++ b/docs/api/index.mdx @@ -0,0 +1,74 @@ +--- +title: REST API +description: Interact with Xata programmatically using our REST API +--- + +The Xata REST API allows you to programmatically manage your databases, branches, and data. All API endpoints are available at `https://api.xata.tech`. + +## Authentication + +All API requests require authentication using an API key. You can create API keys in the [Xata dashboard](https://app.xata.io). Include your API key in the `Authorization` header: + +```bash +curl -H "Authorization: Bearer YOUR_API_KEY" https://api.xata.tech/... +``` + +## Available Endpoints + +The API is organized into the following sections: + +### Organizations + +- [List organizations](/api/organizations#list-organizations) +- [Get organization details](/api/organizations#get-organization) +- [Update organization](/api/organizations#update-organization) +- [Delete organization](/api/organizations#delete-organization) + +### Projects + +- [List projects](/api/projects#list-projects) +- [Create project](/api/projects#create-project) +- [Get project details](/api/projects#get-project) +- [Update project](/api/projects#update-project) +- [Delete project](/api/projects#delete-project) +- [Get available regions](/api/projects#get-regions) + +### Branches + +- [List branches](/api/branches#list-branches) +- [Create branch](/api/branches#create-branch) +- [Get branch details](/api/branches#get-branch) +- [Update branch](/api/branches#update-branch) +- [Delete branch](/api/branches#delete-branch) + +## Error Handling + +The API uses standard HTTP status codes to indicate the success or failure of requests: + +- `200 OK`: The request was successful +- `201 Created`: A new resource was successfully created +- `400 Bad Request`: The request was malformed or contained invalid parameters +- `401 Unauthorized`: Authentication failed or is missing +- `404 Not Found`: The requested resource does not exist +- `5XX`: Server error + +Error responses include a JSON object with a `message` field describing the error: + +```json +{ + "message": "Error description" +} +``` + +## Rate Limiting + +API requests are subject to rate limiting. The current limits are: + +- 100 requests per minute per API key +- 1000 requests per hour per API key + +Rate limit headers are included in all responses: + +- `X-RateLimit-Limit`: The maximum number of requests allowed per time window +- `X-RateLimit-Remaining`: The number of requests remaining in the current time window +- `X-RateLimit-Reset`: The time when the current rate limit window resets (Unix timestamp) \ No newline at end of file diff --git a/docs/api/organizations.mdx b/docs/api/organizations.mdx new file mode 100644 index 0000000..cce0122 --- /dev/null +++ b/docs/api/organizations.mdx @@ -0,0 +1,105 @@ +--- +title: Organizations API +description: Manage organizations and their settings +--- + +The Organizations API allows you to manage your Xata organizations, including listing, creating, updating, and deleting organizations. + +## List organizations + +Retrieves a list of all organizations you have access to. + +```bash +GET /organizations +``` + +### Response + +```json +{ + "organizations": [ + { + "id": "org_123", + "name": "My Organization", + "createdAt": "2024-01-01T00:00:00Z", + "updatedAt": "2024-01-01T00:00:00Z" + } + ] +} +``` + +## Get organization + +Retrieves detailed information about a specific organization. + +```bash +GET /organizations/{organizationID} +``` + +### Path Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| organizationID | string | Unique identifier for the organization | + +### Response + +```json +{ + "id": "org_123", + "name": "My Organization", + "createdAt": "2024-01-01T00:00:00Z", + "updatedAt": "2024-01-01T00:00:00Z" +} +``` + +## Update organization + +Updates the name of an organization. + +```bash +PATCH /organizations/{organizationID} +``` + +### Path Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| organizationID | string | Unique identifier for the organization | + +### Request Body + +```json +{ + "name": "New Organization Name" +} +``` + +### Response + +```json +{ + "id": "org_123", + "name": "New Organization Name", + "createdAt": "2024-01-01T00:00:00Z", + "updatedAt": "2024-01-01T00:00:00Z" +} +``` + +## Delete organization + +Permanently deletes an organization and all its associated data. This action cannot be undone. + +```bash +DELETE /organizations/{organizationID} +``` + +### Path Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| organizationID | string | Unique identifier for the organization | + +### Response + +Returns a 204 status code with no content on success. \ No newline at end of file diff --git a/docs/api/projects.mdx b/docs/api/projects.mdx new file mode 100644 index 0000000..57adbd7 --- /dev/null +++ b/docs/api/projects.mdx @@ -0,0 +1,174 @@ +--- +title: Projects API +description: Manage projects and their settings +--- + +The Projects API allows you to manage your Xata projects, including listing, creating, updating, and deleting projects. + +## List projects + +Retrieves a list of all projects within an organization. + +```bash +GET /organizations/{organizationID}/projects +``` + +### Path Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| organizationID | string | Unique identifier for the organization | + +### Response + +```json +{ + "projects": [ + { + "id": "proj_123", + "name": "My Project", + "createdAt": "2024-01-01T00:00:00Z", + "updatedAt": "2024-01-01T00:00:00Z" + } + ] +} +``` + +## Create project + +Creates a new project within an organization. + +```bash +POST /organizations/{organizationID}/projects +``` + +### Path Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| organizationID | string | Unique identifier for the organization | + +### Request Body + +```json +{ + "name": "New Project" +} +``` + +### Response + +```json +{ + "id": "proj_123", + "name": "New Project", + "createdAt": "2024-01-01T00:00:00Z", + "updatedAt": "2024-01-01T00:00:00Z" +} +``` + +## Get project + +Retrieves detailed information about a specific project. + +```bash +GET /organizations/{organizationID}/projects/{projectID} +``` + +### Path Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| organizationID | string | Unique identifier for the organization | +| projectID | string | Unique identifier for the project | + +### Response + +```json +{ + "id": "proj_123", + "name": "My Project", + "createdAt": "2024-01-01T00:00:00Z", + "updatedAt": "2024-01-01T00:00:00Z" +} +``` + +## Update project + +Updates the name of a project. + +```bash +PATCH /organizations/{organizationID}/projects/{projectID} +``` + +### Path Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| organizationID | string | Unique identifier for the organization | +| projectID | string | Unique identifier for the project | + +### Request Body + +```json +{ + "name": "Updated Project Name" +} +``` + +### Response + +```json +{ + "id": "proj_123", + "name": "Updated Project Name", + "createdAt": "2024-01-01T00:00:00Z", + "updatedAt": "2024-01-01T00:00:00Z" +} +``` + +## Delete project + +Permanently deletes a project and all its associated data. This action cannot be undone. + +```bash +DELETE /organizations/{organizationID}/projects/{projectID} +``` + +### Path Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| organizationID | string | Unique identifier for the organization | +| projectID | string | Unique identifier for the project | + +### Response + +Returns a 204 status code with no content on success. + +## Get available regions + +Retrieves a list of all regions where new branches can be deployed for the specified organization. + +```bash +GET /organizations/{organizationID}/regions +``` + +### Path Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| organizationID | string | Unique identifier for the organization | + +### Response + +```json +{ + "regions": [ + { + "id": "us-east-1", + "publicAccess": true + } + ] +} +``` \ No newline at end of file diff --git a/docs/architecture.mdx b/docs/architecture.mdx new file mode 100644 index 0000000..2498a2b --- /dev/null +++ b/docs/architecture.mdx @@ -0,0 +1,74 @@ +--- +title: Architecture +description: Xata's PostgreSQL platform architecture +--- + +Xata delivers "Postgres at scale" by combining a globally hosted control-plane with a data-plane that can be either hosted by Xata or deployed in your own cloud. The platform brings together three core building blocks: + +1. **A cloud-native storage layer** that separates storage from compute and enables instant copy-on-write (CoW) branching. +2. **Vanilla PostgreSQL on Kubernetes** orchestrated by **CloudNativePG (CNPG)** for high availability, rolling upgrades, and self-healing. +3. **A developer-centric workflow** powered by open-source projects [**pgstream**](https://github.com/xataio/pgstream) (replication & data masking) and [**pgroll**](https://github.com/xataio/pgroll) (zero-downtime schema changes). + +## 1. High-level architecture + +| Layer | Responsibilities | Runs in | +|-------|------------------|---------| +| **Control plane** | Web UI, API, auth, organizations & projects metadata | Xata cloud (multi-region) | +| **Data plane – compute** | PostgreSQL instances, read replicas, backups, pgroll workers | Xata cloud (standard) or customer cloud (BYOC) | +| **Data plane – storage** | Distributed NVMe/TCP storage cluster (Simplyblock) | Xata cloud (standard) or customer cloud (BYOC) | + +In the standard Xata architecture, both the control plane and data plane are hosted and managed by Xata in our cloud infrastructure. This provides a fully managed experience with automatic scaling, backups, and maintenance. + +## 2. Storage layer – distributed, bottomless, NVMe/TCP + +![Compute Kubernetes cluster and storage cluster](/assets/images/k8s-and-storage.png) + +Behind every Xata instance sits a [**Simplyblock**](https://www.simplyblock.io/)-powered, software-defined storage cluster that presents logical NVMe volumes to the Postgres pods over NVMe-oF. Because the block device lives in a separate tier, you can grow compute and storage independently: scaling a database is as easy as changing the pod size, and you never have to migrate data to a larger disk. +The cluster protects data with node-level erasure coding rather than traditional RAID, so even if several nodes disappear your database keeps running without losing redundancy. All I/O is handled entirely in user space via **SPDK** and **DPDK**, shaving unnecessary kernel context switches and keeping latencies low even under heavy parallel workloads. +Finally, storage is billed on the bytes that actually exist on disk—no pre-allocating big volumes—so costs stay proportional to real usage. + +Under the hood, Simplyblock detaches raw NVMe devices from the Linux kernel and hands them over to **SPDK** (Storage Performance Development Kit) and **DPDK** (Data Plane Development Kit) in user-space. Bypassing the kernel eliminates expensive context switches and lets the I/O stack talk directly to the hardware queues. The result is single-digit-microsecond latency even when dozens of Postgres instances hammer the same storage pool in parallel. + +Durability is handled by **erasure coding at the node level**. Instead of mirroring every block multiple times—wasting 2× or 3× the space—Simplyblock stripes data and parity chunks across the cluster so that any two nodes can vanish without losing information. Because the parity lives at cluster scope, the system tolerates not just drive failures but full-node outages and even zone-level disruptions. + +On top of that sits **NVMe-oF multipathing**. Each logical volume is presented to the Postgres pods with several active paths; if a primary storage node is drained for maintenance or suffers a failure, I/O is automatically rerouted through a secondary path with zero application impact. All of this orchestration happens transparently to the database, so developers and operators never have to think about RAID sets, fail-over scripts, or hot spares. + +From a total-cost-of-ownership perspective, the combination of thin-provisioned capacity, efficient parity and just-in-time scaling means the same workload can run at a fraction of the price you would pay for monolithic EBS volumes or pre-allocated local SSDs in a managed database service. + +![Redundancy with Simplyblock](/assets/images/redundancy-with-simplyblock.png) + +### 3. Copy-on-write branching + +![Copy-on-write branch (no writes yet)](/assets/images/copy-on-write-branching-1.png) + +Instant branching in Xata is implemented at the storage layer itself. When you create a new branch the platform simply clones the metadata index that maps logical blocks to their physical location; no data is copied, so the operation finishes in milliseconds whether the database is megabytes or terabytes in size. +As soon as either the parent or the child branch writes to a page, the storage layer allocates a fresh block for the modified data, leaving the original block untouched. The two branches therefore share all unchanged pages and diverge only where changes occur, which keeps additional disk usage minimal and enables hundreds of simultaneous branches without exploding storage costs. +Because branching happens below Postgres, the feature is completely transparent to the database engine and works with any extension. + +To make this more concrete, imagine the storage system as a set of immutable **data chunks** and a lightweight **metadata index** that records which chunk belongs to which logical address. Creating a branch duplicates the index—not the chunks—so it takes the same amount of time no matter how large the database is. When the first write lands on, say, chunk `3`, that chunk is copied, the write is applied to the new copy, and the child branch's index is updated to point to it. All other chunks continue to be shared. +This model is a perfect match for developer workflows: feature branches usually touch only a small fraction of the total dataset, so the storage overhead is tiny. It is equally powerful in staging and CI pipelines where dozens of short-lived branches can exist in parallel without additional storage planning. + +Copy-on-write also underpins the **merge-back** story. Because each modified chunk is tracked precisely, tools such as [**pgroll**](https://github.com/xataio/pgroll) can compare the branch with its parent, compute an idempotent migration plan, and apply it to production with zero downtime. Combined with [**pgstream**](https://github.com/xataio/pgstream) for continuous, masked replication, the result is a feedback loop where developers experiment freely and ship schema changes safely. + +![Copy-on-write branch (chunks 3 and 6 duplicated)](/assets/images/copy-on-write-branching-2.png) + +## 4. Compute layer – Kubernetes & CloudNativePG + +PostgreSQL itself runs unmodified inside Kubernetes pods managed by the **CloudNativePG** operator. CNPG takes care of high-availability via synchronous replication, automated fail-over, rolling minor version upgrades and continuous, incremental backups to object storage. +Running vanilla Postgres means you retain 100 % extension compatibility—if it works upstream, it works on Xata. Scaling up or down is a matter of adjusting the pod spec; the decoupled storage layer makes the change immediate and risk-free. + +### 5. Developer workflow components + +The storage and compute layers come together in a developer experience powered by two open-source projects from Xata. [**pgstream**](https://github.com/xataio/pgstream) taps into the logical replication stream, applies masking transforms on the fly and keeps an always-fresh, anonymised replica of production data. [**pgroll**](https://github.com/xataio/pgroll) builds on top of that replica to generate reversible, zero-downtime migration plans. +In practice a developer calls `xata clone` to obtain an anonymised copy of production, uses `xata branch create` to spin up an isolated branch, and later runs `xata roll migrate` (or `xata roll rollback`) to move the validated schema changes into production without downtime. + +## 6. Bring Your Own Cloud (BYOC) + +![BYOC control plane and data plane](/assets/images/byoc-architecture.png) + +While the default deployment keeps both control-plane and data-plane in Xata's multi-region cloud, organisations with strict governance or latency requirements can choose **Bring Your Own Cloud**. In BYOC the full data plane—Postgres pods, storage cluster, [pgstream](https://github.com/xataio/pgstream) and [pgroll](https://github.com/xataio/pgroll) workers—runs inside your own AWS, GCP, Azure or on-prem Kubernetes cluster. +The control plane stays with Xata and communicates over outbound-only gRPC, so no inbound ports have to be opened. All customer data remains inside the VPC, allowing teams to leverage existing cloud credits, satisfy residency constraints and keep network latency to an absolute minimum. + +--- + +**Need help?** Reach out to the Xata team at [info@xata.io](mailto:info@xata.io) or join our Discord community. diff --git a/docs/assets/images/byoc-architecture.png b/docs/assets/images/byoc-architecture.png new file mode 100644 index 0000000..8865386 Binary files /dev/null and b/docs/assets/images/byoc-architecture.png differ diff --git a/docs/assets/images/copy-on-write-branching-2.png b/docs/assets/images/copy-on-write-branching-2.png new file mode 100644 index 0000000..d220a8d Binary files /dev/null and b/docs/assets/images/copy-on-write-branching-2.png differ diff --git a/docs/assets/images/copy-on-write-branching-3.png b/docs/assets/images/copy-on-write-branching-3.png new file mode 100644 index 0000000..d6d7bb1 Binary files /dev/null and b/docs/assets/images/copy-on-write-branching-3.png differ diff --git a/docs/assets/images/copy-on-write-diagram-1.png b/docs/assets/images/copy-on-write-diagram-1.png new file mode 100644 index 0000000..ae3e79b Binary files /dev/null and b/docs/assets/images/copy-on-write-diagram-1.png differ diff --git a/docs/assets/images/cow-branching.png b/docs/assets/images/cow-branching.png new file mode 100644 index 0000000..dcf3e4b Binary files /dev/null and b/docs/assets/images/cow-branching.png differ diff --git a/docs/assets/images/k8s-and-storage.png b/docs/assets/images/k8s-and-storage.png new file mode 100644 index 0000000..30daf22 Binary files /dev/null and b/docs/assets/images/k8s-and-storage.png differ diff --git a/docs/assets/images/redundancy-with-simplyblock.png b/docs/assets/images/redundancy-with-simplyblock.png new file mode 100644 index 0000000..3317abb Binary files /dev/null and b/docs/assets/images/redundancy-with-simplyblock.png differ diff --git a/docs/assets/llm-evals-with-vercel-ai-and-vitest-og@2x.png b/docs/assets/llm-evals-with-vercel-ai-and-vitest-og@2x.png deleted file mode 100644 index f30b0d3..0000000 Binary files a/docs/assets/llm-evals-with-vercel-ai-and-vitest-og@2x.png and /dev/null differ diff --git a/docs/cli/auth.mdx b/docs/cli/auth.mdx new file mode 100644 index 0000000..912df2e --- /dev/null +++ b/docs/cli/auth.mdx @@ -0,0 +1,76 @@ +--- +title: Authentication Commands +description: Commands for managing Xata authentication +--- + +The `auth` commands help you manage your Xata authentication state, including logging in, switching accounts, and managing tokens. + +## login + +Log in to your Xata account. + +```bash +xata auth login +``` + +This command will open your default web browser to authenticate with Xata. After successful authentication, your credentials will be stored locally. + +## logout + +Log out of the current account. + +```bash +xata auth logout +``` + +This command removes the stored credentials for the current account. + +## status + +Display active account and authentication state. + +```bash +xata auth status +``` + +Shows information about your current authentication state, including: +- Currently logged-in account +- Authentication status +- Token expiration (if applicable) + +## switch + +Switch to a different Xata account profile. + +```bash +xata auth switch +``` + +Arguments: +- `profile` - The name of the profile to switch to + +This command allows you to quickly switch between different Xata accounts that you've previously logged into. + +## access-token + +Print the current access token. + +```bash +xata auth access-token +``` + +This command outputs your current access token, which can be used for API authentication. + +## refresh-token + +Print the current refresh token. + +```bash +xata auth refresh-token +``` + +This command outputs your current refresh token, which can be used to obtain new access tokens. + +## Global Flags + +- `-h, --help` - Print help information and exit \ No newline at end of file diff --git a/docs/cli/branch.mdx b/docs/cli/branch.mdx new file mode 100644 index 0000000..7212b9b --- /dev/null +++ b/docs/cli/branch.mdx @@ -0,0 +1,115 @@ +--- +title: Branch Commands +description: Commands for managing Xata database branches +--- + +The `branch` commands help you manage your Xata database branches, including creating, switching, and managing branch configurations. + +## list + +List all branches in the current project. + +```bash +xata branch list +``` + +This command displays all branches in your project, including: +- Branch name +- Parent branch +- Creation date +- Status + +## describe + +Describe a branch in detail. + +```bash +xata branch describe +``` + +This command shows detailed information about the current branch, including: +- Branch configuration +- Schema version +- Migration status +- Connection details + +## create + +Create a new branch. + +```bash +xata branch create +``` + +This command creates a new branch from the current branch, allowing you to make isolated changes to your database schema. + +## delete + +Delete a branch. + +```bash +xata branch delete +``` + +This command permanently deletes the current branch and all its data. This action cannot be undone. + +## url + +Print the connection URL for a branch. + +```bash +xata branch url +``` + +This command outputs the connection string (DSN) for the current branch, which can be used to connect your application to the database. + +## checkout + +Switch to a different branch. + +```bash +xata branch checkout +``` + +Arguments: +- `branch` - The name of the branch to switch to + +This command switches your working context to the specified branch. + +## tree + +List all branches as a tree. + +```bash +xata branch tree +``` + +This command displays the branch hierarchy in a tree format, showing the relationships between branches. + +## get + +Get a field from a branch description. + +```bash +xata branch get +``` + +Arguments: +- `field` - The field to retrieve from the branch description + +This command retrieves specific information about the current branch. + +## wait-ready + +Wait for a branch to be ready. + +```bash +xata branch wait-ready +``` + +This command waits until the current branch is fully provisioned and ready to accept connections. + +## Global Flags + +- `-h, --help` - Print help information and exit +- `--json` - Output in JSON format \ No newline at end of file diff --git a/docs/cli/cli.mdx b/docs/cli/cli.mdx new file mode 100644 index 0000000..ea43fc0 --- /dev/null +++ b/docs/cli/cli.mdx @@ -0,0 +1,87 @@ +--- +title: Command-line Interface +description: Complete reference for the Xata CLI +--- + +The Xata CLI is a powerful tool for managing your Xata databases, projects, and organizations from the command line. It provides commands for authentication, project management, database operations, and more. + +## Installation + +```bash +# Install using npm +npm install -g @xata.io/cli + +# Or using Homebrew +brew install xataio/tap/xata +``` + +## Basic Usage + +```bash +xata [command] [subcommand] [options] +``` + +## Available Commands + +### Authentication +- [auth login](/reference/cli/auth#login) - Log in to your Xata account +- [auth logout](/reference/cli/auth#logout) - Log out of the current account +- [auth status](/reference/cli/auth#status) - Display active account and authentication state +- [auth switch](/reference/cli/auth#switch) - Switch to a different Xata account profile +- [auth access-token](/reference/cli/auth#access-token) - Print the current access token +- [auth refresh-token](/reference/cli/auth#refresh-token) - Print the current refresh token + +### Project Management +- [init](/reference/cli/init#init) - Link a project to the current folder +- [project list](/reference/cli/project#list) - List all projects +- [project describe](/reference/cli/project#describe) - Describe a project +- [project create](/reference/cli/project#create) - Create a new project +- [project delete](/reference/cli/project#delete) - Delete a project +- [project init](/reference/cli/project#init) - Link a project to the folder +- [project get](/reference/cli/project#get) - Get a field from a project description + +### Organization Management +- [organization list](/reference/cli/organization#list) - List all organizations +- [organization describe](/reference/cli/organization#describe) - Describe an organization +- [organization create](/reference/cli/organization#create) - Create a new organization +- [organization delete](/reference/cli/organization#delete) - Delete an organization +- [organization get](/reference/cli/organization#get) - Get a field from an organization description + +### Branch Management +- [branch list](/reference/cli/branch#list) - List all branches +- [branch describe](/reference/cli/branch#describe) - Describe a branch +- [branch create](/reference/cli/branch#create) - Create a new branch +- [branch delete](/reference/cli/branch#delete) - Delete a branch +- [branch url](/reference/cli/branch#url) - Print URL (connection string) for a branch +- [branch checkout](/reference/cli/branch#checkout) - Checkout a branch +- [branch tree](/reference/cli/branch#tree) - List all branches as a tree +- [branch get](/reference/cli/branch#get) - Get a field from a branch description +- [branch wait-ready](/reference/cli/branch#wait-ready) - Wait for a branch to be ready + +### Schema Migrations +- [roll complete](/reference/cli/roll#complete) - Complete an ongoing migration +- [roll init](/reference/cli/roll#init) - Initialize pgroll in the target database +- [roll latest](/reference/cli/roll#latest) - Print the name of the latest schema version +- [roll migrate](/reference/cli/roll#migrate) - Apply outstanding migrations +- [roll pull](/reference/cli/roll#pull) - Pull migration history from the target database +- [roll rollback](/reference/cli/roll#rollback) - Roll back an ongoing migration +- [roll start](/reference/cli/roll#start) - Start a migration +- [roll status](/reference/cli/roll#status) - Show pgroll status +- [roll convert](/reference/cli/roll#convert) - Convert SQL statements to a pgroll migration + +### Database Synchronization +- [clone start](/reference/cli/clone#start) - Clone a PostgreSQL database with anonymization +- [clone config](/reference/cli/clone#config) - Configure transforms for the clone command + +### Utility Commands +- [status](/reference/cli/status#status) - Get the status of the Xata CLI +- [version](/reference/cli/version#version) - Get the version of the Xata CLI and pgroll +- [upgrade](/reference/cli/upgrade#upgrade) - Upgrade the Xata CLI +- [completions install](/reference/cli/completions#install) - Install shell completions +- [completions uninstall](/reference/cli/completions#uninstall) - Uninstall shell completions + +## Global Flags + +- `-h, --help` - Print help information and exit +- `-v, --version` - Print version information and exit +- `--json` - Output in JSON format (where applicable) diff --git a/docs/cli/clone.mdx b/docs/cli/clone.mdx new file mode 100644 index 0000000..5284752 --- /dev/null +++ b/docs/cli/clone.mdx @@ -0,0 +1,24 @@ +--- +title: Clone Command +description: Command for cloning Xata databases +--- + +The `clone` command helps you create a copy of your Xata database. + +## clone + +Create a clone of your Xata database. + +```bash +xata clone +``` + +This command: +- Creates a new database branch +- Copies all data from the source branch +- Sets up the new branch for development + +## Global Flags + +- `-h, --help` - Print help information and exit +- `--json` - Output in JSON format \ No newline at end of file diff --git a/docs/cli/completions.mdx b/docs/cli/completions.mdx new file mode 100644 index 0000000..0dc8c28 --- /dev/null +++ b/docs/cli/completions.mdx @@ -0,0 +1,33 @@ +--- +title: Shell Completions Commands +description: Commands for managing shell completions for the Xata CLI +--- + +The `completions` commands help you set up and manage shell autocompletion for the Xata CLI. + +## install + +Install shell autocompletion support. + +```bash +xata completions install +``` + +This command installs autocompletion support for the Xata CLI in your shell, enabling: +- Command autocompletion +- Subcommand suggestions +- Flag and argument completion + +## uninstall + +Uninstall shell autocompletion support. + +```bash +xata completions uninstall +``` + +This command removes the autocompletion configuration for the Xata CLI from your shell. + +## Global Flags + +- `-h, --help` - Print help information and exit \ No newline at end of file diff --git a/docs/cli/index.mdx b/docs/cli/index.mdx deleted file mode 100644 index 82fa699..0000000 --- a/docs/cli/index.mdx +++ /dev/null @@ -1,6 +0,0 @@ -# CLI index - -```bash -echo "hello world" -``` -Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here.Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. diff --git a/docs/cli/init.mdx b/docs/cli/init.mdx index cca68f3..04351d4 100644 --- a/docs/cli/init.mdx +++ b/docs/cli/init.mdx @@ -1,16 +1,23 @@ +--- +title: Init Command +description: Command for initializing a Xata project +--- -# CLI init +The `init` command helps you set up a new Xata project in your current directory. -```bash -echo "xata init" -``` +## init -Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here.Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. +Initialize a new Xata project. -Richard +```bash +xata init +``` -![Picture](https://raw.githubusercontent.com/xataio/mdx-blog/main/images/llm-evals-with-vercel-ai-and-vitest@2x.png) +This command: +- Creates a new project configuration +- Sets up the project structure +- Configures your local environment -Local Picture: +## Global Flags -![Picture](assets/llm-evals-with-vercel-ai-and-vitest-og@2x.png) +- `-h, --help` - Print help information and exit \ No newline at end of file diff --git a/docs/cli/organization.mdx b/docs/cli/organization.mdx new file mode 100644 index 0000000..b2f6a40 --- /dev/null +++ b/docs/cli/organization.mdx @@ -0,0 +1,72 @@ +--- +title: Organization Commands +description: Commands for managing Xata organizations +--- + +The `organization` commands help you manage your Xata organizations, including creating, listing, and configuring organizations. + +## list + +List all organizations. + +```bash +xata organization list +``` + +This command displays all organizations you have access to, including: +- Organization name +- Organization ID +- Creation date +- Status + +## describe + +Describe an organization. + +```bash +xata organization describe +``` + +This command shows detailed information about the current organization, including: +- Organization configuration +- Member information +- Project settings +- Billing details + +## create + +Create a new organization. + +```bash +xata organization create +``` + +This command creates a new organization. + +## delete + +Delete an organization. + +```bash +xata organization delete +``` + +This command permanently deletes the current organization and all its data. + +## get + +Get a field from an organization description. + +```bash +xata organization get +``` + +Arguments: +- `field` - The field to retrieve from the organization description + +This command retrieves specific information about the current organization. + +## Global Flags + +- `-h, --help` - Print help information and exit +- `--json` - Output in JSON format \ No newline at end of file diff --git a/docs/cli/project.mdx b/docs/cli/project.mdx new file mode 100644 index 0000000..81b0174 --- /dev/null +++ b/docs/cli/project.mdx @@ -0,0 +1,82 @@ +--- +title: Project Commands +description: Commands for managing Xata projects +--- + +The `project` commands help you manage your Xata projects, including creating, listing, and configuring projects. + +## list + +List all projects. + +```bash +xata project list +``` + +This command displays all projects in your organization, including: +- Project name +- Project ID +- Creation date +- Status + +## describe + +Describe a project. + +```bash +xata project describe +``` + +This command shows detailed information about the current project, including: +- Project configuration +- Database settings +- Branch information +- Connection details + +## create + +Create a new project. + +```bash +xata project create +``` + +This command creates a new project in your organization. + +## delete + +Delete a project. + +```bash +xata project delete +``` + +This command permanently deletes the current project and all its data. + +## init + +Link a project to the folder. + +```bash +xata project init +``` + +This command links an existing project to your current directory. + +## get + +Get a field from a project description. + +```bash +xata project get +``` + +Arguments: +- `field` - The field to retrieve from the project description + +This command retrieves specific information about the current project. + +## Global Flags + +- `-h, --help` - Print help information and exit +- `--json` - Output in JSON format \ No newline at end of file diff --git a/docs/cli/roll.mdx b/docs/cli/roll.mdx new file mode 100644 index 0000000..07c03a9 --- /dev/null +++ b/docs/cli/roll.mdx @@ -0,0 +1,101 @@ +--- +title: Roll Commands +description: Commands for managing database migrations +--- + +The `roll` commands help you manage database migrations using pgroll. + +## complete + +Complete an ongoing migration. + +```bash +xata roll complete +``` + +This command finalizes a migration that has been started but not yet completed. + +## init + +Initialize pgroll in the target database. + +```bash +xata roll init +``` + +This command sets up pgroll in your database, creating necessary tables and functions. + +## latest + +Print the name of the latest schema version. + +```bash +xata roll latest +``` + +This command shows the most recent schema version in your database. + +## migrate + +Apply outstanding migrations. + +```bash +xata roll migrate +``` + +This command applies any pending migrations to your database. + +## pull + +Pull migration history from the target database. + +```bash +xata roll pull +``` + +This command synchronizes your local migration history with the database. + +## rollback + +Roll back an ongoing migration. + +```bash +xata roll rollback +``` + +This command reverts a migration that has been started but not yet completed. + +## start + +Start a migration. + +```bash +xata roll start +``` + +This command begins a new migration process. + +## status + +Show pgroll status. + +```bash +xata roll status +``` + +This command displays the current state of migrations in your database. + +## convert + +Convert SQL statements to a pgroll migration. + +```bash +xata roll convert +``` + +This command helps you convert raw SQL into a pgroll migration. + +## Global Flags + +- `-h, --help` - Print help information and exit +- `--json` - Output in JSON format \ No newline at end of file diff --git a/docs/cli/status.mdx b/docs/cli/status.mdx new file mode 100644 index 0000000..6d1ae77 --- /dev/null +++ b/docs/cli/status.mdx @@ -0,0 +1,25 @@ +--- +title: Status Command +description: Command for checking Xata CLI status +--- + +The `status` command helps you check the current state of your Xata CLI configuration. + +## status + +Check the status of your Xata CLI configuration. + +```bash +xata status +``` + +This command displays information about your current configuration, including: +- Authentication status +- Current project +- Current branch +- Environment settings + +## Global Flags + +- `-h, --help` - Print help information and exit +- `--json` - Output in JSON format \ No newline at end of file diff --git a/docs/cli/upgrade.mdx b/docs/cli/upgrade.mdx new file mode 100644 index 0000000..5b72470 --- /dev/null +++ b/docs/cli/upgrade.mdx @@ -0,0 +1,28 @@ +--- +title: Upgrade Command +description: Command for upgrading the Xata CLI +--- + +The `upgrade` command helps you update the Xata CLI to the latest version. + +## upgrade + +Upgrade the Xata CLI. + +```bash +xata upgrade [--channel ] [--version ] +``` + +Arguments: +- `--channel` - The channel to upgrade from (dev|next|latest, default = latest) +- `--version` - The specific version to upgrade to + +This command: +- Checks for available updates +- Downloads the latest version +- Installs the update +- Verifies the installation + +## Global Flags + +- `-h, --help` - Print help information and exit \ No newline at end of file diff --git a/docs/cli/version.mdx b/docs/cli/version.mdx new file mode 100644 index 0000000..eba226c --- /dev/null +++ b/docs/cli/version.mdx @@ -0,0 +1,23 @@ +--- +title: Version Command +description: Command for checking Xata CLI version +--- + +The `version` command helps you check the current version of your Xata CLI. + +## version + +Get the version of the Xata CLI. + +```bash +xata version +``` + +This command displays: +- CLI version +- Build information +- Release date + +## Global Flags + +- `-h, --help` - Print help information and exit \ No newline at end of file diff --git a/docs/config.json b/docs/config.json index b7ab16a..7d042fa 100644 --- a/docs/config.json +++ b/docs/config.json @@ -6,14 +6,206 @@ "file": "docs/getting-started.mdx" }, { - "title": "Command-line interface", + "title": "Architecture", + "href": "/architecture", + "file": "docs/architecture.mdx" + }, + { + "title": "Core concepts", + "href": "/core-concepts", + "file": "docs/core-concepts/core-concepts.mdx", + "items": [ + { + "title": "Instant branching", + "href": "/core-concepts/branching", + "file": "docs/core-concepts/branching.mdx" + }, + { + "title": "Data anonymization", + "href": "/core-concepts/anonymization", + "file": "docs/core-concepts/anonymization.mdx" + }, + { + "title": "Schema changes", + "href": "/core-concepts/schema-changes", + "file": "docs/core-concepts/schema-changes.mdx" + }, + { + "title": "Bring Your Own Cloud", + "href": "/core-concepts/byoc", + "file": "docs/core-concepts/byoc.mdx" + } + ] + }, + { + "title": "Connect", + "href": "/connect", + "file": "docs/connect/connect.mdx", + "items": [ + { + "title": "Languages", + "href": "/connect/languages", + "file": "docs/connect/languages.mdx" + }, + { + "title": "Frameworks", + "href": "/connect/frameworks", + "file": "docs/connect/frameworks.mdx" + }, + { + "title": "ORMs", + "href": "/connect/orms", + "file": "docs/connect/orms.mdx" + } + ] + }, + { + "title": "Tutorials", + "href": "/tutorials", + "file": "docs/tutorials/tutorials.mdx", + "items": [ + { + "title": "Developer workflow", + "href": "/tutorials/developer-workflow", + "file": "docs/tutorials/developer-workflow.mdx" + }, + { + "title": "Migrate to Xata", + "href": "/tutorials/migrate-to-xata", + "file": "docs/tutorials/migrate-to-xata.mdx" + } + ] + }, + { + "title": "Command-line Interface", "href": "/cli", - "file": "docs/cli/index.mdx", + "file": "docs/cli/cli.mdx", "items": [ { - "title": "Init", + "title": "auth", + "href": "/cli/auth", + "file": "docs/cli/auth.mdx" + }, + { + "title": "branch", + "href": "/cli/branch", + "file": "docs/cli/branch.mdx" + }, + { + "title": "clone", + "href": "/cli/clone", + "file": "docs/cli/clone.mdx" + }, + { + "title": "completions", + "href": "/cli/completions", + "file": "docs/cli/completions.mdx" + }, + { + "title": "init", "href": "/cli/init", "file": "docs/cli/init.mdx" + }, + { + "title": "organization", + "href": "/cli/organization", + "file": "docs/cli/organization.mdx" + }, + { + "title": "project", + "href": "/cli/project", + "file": "docs/cli/project.mdx" + }, + { + "title": "roll", + "href": "/cli/roll", + "file": "docs/cli/roll.mdx" + }, + { + "title": "status", + "href": "/cli/status", + "file": "docs/cli/status.mdx" + }, + { + "title": "upgrade", + "href": "/cli/upgrade", + "file": "docs/cli/upgrade.mdx" + }, + { + "title": "version", + "href": "/cli/version", + "file": "docs/cli/version.mdx" + } + ] + }, + { + "title": "REST API", + "href": "/api", + "file": "docs/api/index.mdx", + "items": [ + { + "title": "Organizations", + "href": "/api/organizations", + "file": "docs/api/organizations.mdx" + }, + { + "title": "Projects", + "href": "/api/projects", + "file": "docs/api/projects.mdx" + }, + { + "title": "Branches", + "href": "/api/branches", + "file": "docs/api/branches.mdx" + } + ] + }, + { + "title": "GitHub Actions", + "href": "/github-actions", + "file": "docs/github-actions/github-actions.mdx", + "items": [ + { + "title": "Check merge readiness", + "href": "/github-actions/check", + "file": "docs/github-actions/check.mdx" + }, + { + "title": "Create Xata dev branch", + "href": "/github-actions/pr", + "file": "docs/github-actions/pr.mdx" + }, + { + "title": "Xata clone (daily, on-demand)", + "href": "/github-actions/clone", + "file": "docs/github-actions/clone.mdx" + } + ] + }, + { + "title": "Resources", + "href": "/resources", + "file": "docs/resources/resources.mdx", + "items": [ + { + "title": "Troubleshooting & FAQ", + "href": "/resources/troubleshooting", + "file": "docs/resources/troubleshooting.mdx" + }, + { + "title": "Compliance", + "href": "/resources/compliance", + "file": "docs/resources/compliance.mdx" + }, + { + "title": "Roadmap", + "href": "/resources/roadmap", + "file": "docs/resources/roadmap.mdx" + }, + { + "title": "Pricing", + "href": "/resources/pricing", + "file": "docs/resources/pricing.mdx" } ] } diff --git a/docs/connect/connect.mdx b/docs/connect/connect.mdx new file mode 100644 index 0000000..ada20ef --- /dev/null +++ b/docs/connect/connect.mdx @@ -0,0 +1,74 @@ +--- +title: Connect +description: Connect to Xata using your preferred language, framework, or ORM +--- + +# Connect to Xata + +Every connection to Xata ultimately uses the standard PostgreSQL wire protocol, so you can reuse the drivers you already know. The high-level flow is always the same: + +1. **Get a connection string for the branch you want to work with** + ```bash + export DATABASE_URL=$(xata branch url --branch main) + ``` + The generated URL already embeds SSL settings and a passwordless access token if you are logged-in via the CLI. +2. **Pass `DATABASE_URL` to your language, framework or ORM** + Most drivers read the variable automatically; others take it as a constructor argument. + +> **Tip:** Xata enforces TLS. Make sure `sslmode=require` (or `verify-full`) is present when using raw libpq strings. + +Xata provides multiple ways to connect to your databases, whether you're using a specific programming language, framework, or ORM. + +## Languages + +Connect to Xata using your preferred programming language: + +- [JavaScript/TypeScript](/connect/languages#javascript) +- [Python](/connect/languages#python) +- [PHP](/connect/languages#php) +- [Java](/connect/languages#java) +- [Rust](/connect/languages#rust) + +## Frameworks + +Integrate Xata with popular web frameworks: + +- [Next.js](/connect/frameworks#nextjs) +- [Remix](/connect/frameworks#remix) +- [SvelteKit](/connect/frameworks#sveltekit) +- [Nuxt](/connect/frameworks#nuxt) +- [Solid](/connect/frameworks#solid) + +## ORMs + +Use Xata with your favorite ORM: + +- [Prisma](/connect/orms#prisma) +- [Drizzle](/connect/orms#drizzle) +- [TypeORM](/connect/orms#typeorm) +- [SQLAlchemy](/connect/orms#sqlalchemy) +- [Django ORM](/connect/orms#django) + +## Direct PostgreSQL Connection + +You can also connect directly to Xata using the PostgreSQL wire protocol: + +```bash +psql $(xata branch url) +``` + +This is useful for: +- Running ad-hoc queries +- Using database management tools +- Importing/exporting data +- Running migrations + +## Authentication + +All connections to Xata require authentication. You can authenticate using: + +- API keys (recommended for production) +- OAuth tokens (for user-specific access) +- CLI authentication (for development) + +[Learn more about authentication](/cli/auth) \ No newline at end of file diff --git a/docs/connect/frameworks.mdx b/docs/connect/frameworks.mdx new file mode 100644 index 0000000..a54c234 --- /dev/null +++ b/docs/connect/frameworks.mdx @@ -0,0 +1,44 @@ +--- +title: Connect from web frameworks +description: Connect to Xata from popular web frameworks +--- + +Most modern frameworks read the `DATABASE_URL` environment variable automatically when you run migrations or start the dev server. Export it first, then follow the framework-specific command. + +## Next.js / Prisma + +```bash +export DATABASE_URL=$(xata branch url) +prisma generate && prisma migrate deploy +``` + +## Django + +Set `DATABASE_URL` and install `dj-database-url`: + +```bash +export DATABASE_URL=$(xata branch url) +``` + +In `settings.py`: + +```python +import dj_database_url +DATABASES = { 'default': dj_database_url.config() } +``` + +## Rails + +```bash +export DATABASE_URL=$(xata branch url) +bin/rails db:create db:migrate +``` + +## SvelteKit (Drizzle) + +```bash +export DATABASE_URL=$(xata branch url) +DRIZZLE_DB_URL=$DATABASE_URL npm run db:push +``` + + diff --git a/docs/connect/languages.mdx b/docs/connect/languages.mdx new file mode 100644 index 0000000..ddda43e --- /dev/null +++ b/docs/connect/languages.mdx @@ -0,0 +1,39 @@ +--- +title: Connect from programming languages +description: Connect to Xata from various programming languages +--- + +Below are minimal examples that assume you exported `DATABASE_URL` via `xata branch url`. + +## Node.js (pg) + +```js +import { Pool } from 'pg'; +const pool = new Pool({ connectionString: process.env.DATABASE_URL }); +const res = await pool.query('SELECT 1'); +``` + +## Python (psycopg3) + +```python +import psycopg +conn = psycopg.connect(conninfo=os.environ['DATABASE_URL'], autocommit=True) +``` + +## Go (pgx) + +```go +conn, _ := pgx.Connect(context.Background(), os.Getenv("DATABASE_URL")) +``` + +## Java (JDBC) + +```java +Connection c = DriverManager.getConnection(System.getenv("DATABASE_URL")); +``` + +## Rust (tokio-postgres) + +```rust +let (client, connection) = tokio_postgres::connect(&env::var("DATABASE_URL")?, NoTls).await?; +``` diff --git a/docs/connect/orms.mdx b/docs/connect/orms.mdx new file mode 100644 index 0000000..870bf35 --- /dev/null +++ b/docs/connect/orms.mdx @@ -0,0 +1,48 @@ +--- +title: ORM quick-starts +description: Connect to Xata using popular ORMs +--- + +Export `DATABASE_URL` via `xata branch url` and then use the snippets below. + +## Prisma + +```prisma +datasource db { + provider = "postgresql" + url = env("DATABASE_URL") +} +``` + +> **Note:** Use `shadowDatabaseUrl` for pgroll migrations + +## Drizzle + +```ts +import { drizzle } from 'drizzle-orm/postgres-js'; +const db = drizzle(process.env.DATABASE_URL); +``` + +## TypeORM + +```ts +DataSource({ + url: process.env.DATABASE_URL, + ssl: true +}) +``` + +## SQLAlchemy + +```python +engine = create_engine(os.environ['DATABASE_URL'], pool_pre_ping=True) +``` + +## Django ORM + +```python +DATABASE_URL = os.getenv('DATABASE_URL') +DATABASES = {'default': dj_database_url.parse(DATABASE_URL)} +``` + + diff --git a/docs/core-concepts/anonymization.mdx b/docs/core-concepts/anonymization.mdx new file mode 100644 index 0000000..5b293ac --- /dev/null +++ b/docs/core-concepts/anonymization.mdx @@ -0,0 +1,68 @@ +--- +title: Data anonymization +description: Secure data handling with automatic PII detection and anonymization for development environments +--- + +Xata's data anonymization system automatically detects and masks sensitive information during database replication, enabling secure development and testing environments. + +When a staging replica is built with `xata clone --anonymize`, the replication pipeline passes every row through a streaming transformation layer powered by [**pgstream**](https://github.com/xataio/pgstream). This layer inspects column names, data types and actual values to decide whether the field contains personally-identifiable information (PII) or other sensitive content. Out-of-the-box detectors recognise emails, phone numbers, names, street addresses, credit-card numbers and more. You can also register custom detectors for domain-specific patterns. + +Once a value is flagged as sensitive, a *transformer* replaces it on-the-fly. The most common transformer is **consistent hashing**—the same input always yields the same opaque token—so application logic that relies on equality checks continues to work. Other transformers preserve formatting (for example keeping the `@domain.com` part of an e-mail) or maintain referential integrity by applying the same hash across foreign-key relationships. Because the masking happens inside the replication stream, no unmasked data ever lands in the staging database. + +## Configuring rules + +Anonymisation behaviour is controlled by JSON rules stored in the project configuration. A typical snippet looks like this: + +```json +{ + "rules": { + "users.email": { "type": "email", "preserve_domain": true }, + "users.phone": { "type": "phone", "preserve_country": true }, + "orders.cc_number": { "type": "credit_card", "format": "****-****-****-####" } + } +} +``` + +If a column is not covered by a rule but matches a built-in detector, the platform applies a sensible default transformer (usually consistent hashing). You can opt-out by explicitly setting `"action": "ignore"`. + +## Workflow integration + +A typical CI job that prepares an anonymised test environment looks like this: + +```yaml +steps: + - name: Checkout repo + uses: actions/checkout@v4 + - name: Install Xata CLI + run: curl -fsSL https://xata.io/install.sh | bash + - name: Create anonymised replica + run: xata clone --anonymize --branch ci-${{ github.sha }} + - name: Run integration tests + run: npm test +``` + +The staging branch behaves exactly like production—but with all sensitive data masked—so you can run load tests, debug migrations or train machine-learning models without violating compliance policies. + +## Why deterministic anonymisation matters + +Because hashing is consistent, the same e-mail address in two tables maps to the same anonymised value, preserving JOIN semantics. Foreign keys are never broken, and applications that rely on lookups or unique constraints continue to work unmodified. At the same time, the hash is irreversible without the secret key held by the masking service, so the original data cannot be reconstructed. + +## Security and compliance + +Anonymisation happens before the data leaves the production network segment, ensuring PII never traverses the wire in clear form. The process is compatible with GDPR, HIPAA and SOC-2 requirements and can be tuned further for region-specific regulations such as Schrems II or LGPD. + +## Getting started + +```bash +# Enable anonymisation for future clones +xata config set anonymization.enabled true + +# Create the first anonymised replica +xata clone --anonymize staging +``` + +If you want to experiment locally, you can preview the transformation result: + +```bash +echo 'john.doe@example.com' | xata anonymize --type email +``` \ No newline at end of file diff --git a/docs/core-concepts/branching.mdx b/docs/core-concepts/branching.mdx new file mode 100644 index 0000000..fb68e7a --- /dev/null +++ b/docs/core-concepts/branching.mdx @@ -0,0 +1,90 @@ +--- +title: Instant branching +description: Copy-on-Write branching system for PostgreSQL databases with instant branch creation and efficient storage +--- + +Xata's branching system uses Copy-on-Write (CoW) at the storage layer to create instant database branches. This enables developers to create isolated environments for testing and development without duplicating data unnecessarily. + +Copy-on-Write works by splitting the physical data into immutable *chunks* that are shared by every branch. A lightweight metadata index keeps track of which chunk belongs to which logical page of the database. Creating a branch therefore means cloning only that index—a metadata-only operation that finishes in milliseconds, regardless of whether the database holds a few megabytes or several terabytes. + +![Copy-on-Write branch — no writes yet](/assets/images/copy-on-write-diagram-1.png) + +After the branch exists, reads from either the parent or the child still hit the same chunks. The first time one of the branches writes to a page, the storage layer allocates a brand-new chunk, applies the change there and updates *only* that branch's index to reference it. Every other chunk continues to be shared, so additional disk usage grows in proportion to the amount of data that actually changes. + +![Copy-on-Write branch — chunks diverged](/assets/images/copy-on-write-branching-2.png) + +## How it works + +### Storage-level implementation + +Because the mechanism lives entirely below PostgreSQL, it is compatible with vanilla Postgres and every extension you might load. There is no performance penalty on reads—unchanged chunks are served directly—and the write-amplification is limited to a single additional chunk copy per modified page. + +### Developer workflow + +A typical flow looks like this: + +1. A staging replica kept in sync by [**pgstream**](https://github.com/xataio/pgstream) holds an anonymised copy of production. +2. `xata branch create feature-xyz` clones the metadata index and gives the developer a fully isolated environment that behaves like production. +3. The developer makes schema edits, runs tests and benchmarks against realistic data. +4. Once validated, [**pgroll**](https://github.com/xataio/pgroll) compares the branch with its parent, generates a reversible migration plan and applies it to production with zero-downtime. + +Because CoW tracks every changed chunk, the merge-back step knows *exactly* which pages were touched, enabling deterministic, audit-ready schema migrations. + +## Benefits at scale + +* **Speed** – branch creation takes milliseconds, enabling per-PR ephemeral environments. +* **Storage efficiency** – only the modified chunks consume additional space; dozens of branches cost little more than a single copy. +* **Safety** – experiments happen against realistic data without risking production state. + +### Integration with any Postgres provider + +The storage layer is provider-agnostic: you can run instant branches on AWS RDS, Aurora, Google Cloud SQL, Azure Database or even self-hosted Postgres. Xata improves rather than replaces your existing production setup. + +## Technical Benefits + +### Performance + +- **Instant Branch Creation**: No data copying means branches are created in seconds +- **Storage Efficiency**: Only modified data is duplicated +- **Write Performance**: CoW operations are optimized at the storage layer + +### Integration + +Works with any PostgreSQL cloud provider: +- AWS RDS +- Amazon Aurora +- Google Cloud SQL +- Azure Database +- Self-hosted PostgreSQL + +## Use Cases + +### Development Workflow + +1. Create a branch from production +2. Make schema changes +3. Test with real data +4. Merge changes back to production + +### Testing + +1. Create multiple test branches +2. Run parallel test suites +3. Compare results across branches +4. No storage overhead for read-only operations + +## Getting Started + +To create a branch: + +```bash +xata branch create my-feature-branch +``` + +To switch between branches: + +```bash +xata branch checkout my-feature-branch +``` + +[Learn more about branch management in the CLI documentation](/cli/branch) \ No newline at end of file diff --git a/docs/core-concepts/byoc.mdx b/docs/core-concepts/byoc.mdx new file mode 100644 index 0000000..4fa38fa --- /dev/null +++ b/docs/core-concepts/byoc.mdx @@ -0,0 +1,136 @@ +--- +title: Bring Your Own Cloud +description: Deploy Xata's PostgreSQL platform in your own cloud environment +--- + +Deploy Xata's PostgreSQL platform in your own AWS, Azure, or GCP account while maintaining full control over your infrastructure and data. + +Running Xata in BYOC mode gives you all the benefits of the managed service—control-plane UI, CLI, GitHub Actions, automated migrations—while keeping every byte of customer data inside your private VPC. The deployment ships as a set of Kubernetes manifests (or a Terraform module) that spin up the Postgres pods, the Simplyblock storage cluster and the [pgstream](https://github.com/xataio/pgstream) / [pgroll](https://github.com/xataio/pgroll) workers alongside your existing workloads. + +Because the **control plane** remains in Xata's multi-region cloud, upgrades to the SaaS UI or API surface are rolled out instantly without touching your cluster. Communication flows outbound-only over mutual-TLS gRPC, so you never have to open an inbound port. All telemetry is aggregated and anonymised before it leaves your VPC, satisfying even the strictest compliance departments. + +From an economics point of view, BYOC lets you pay for compute with the reservations and discounts you already have from your cloud provider. Network traffic between your application pods and the database never crosses the public Internet, reducing both latency and egress costs. + +## Architecture + +![BYOC Architecture](/assets/images/byoc-architecture.png) + +*BYOC Architecture: Control plane managed by Xata, data plane deployed in your cloud account* + +### Control Plane + +The Xata control plane manages: +- User authentication and authorization +- Project and organization management +- Monitoring and alerting +- Backup and restore operations +- Schema migration management + +### Data Plane + +The data plane runs in your cloud account: +- PostgreSQL instances on Kubernetes +- Distributed storage system +- Network and security configuration +- Monitoring and logging +- Backup storage + +## Implementation + +### Kubernetes Integration + +Xata uses CloudNativePG operator for PostgreSQL management: +- High availability via synchronous replication +- Read replicas for scaling +- Automatic failover +- Rolling updates +- Backup management + +### Storage System + +The platform uses a distributed storage system: +- NVMe over Fabrics (NVMe-oF) for high performance +- Erasure coding for data resilience +- Node-level redundancy +- Automatic failover +- Unlimited storage growth + +## Deployment Options + +### Cloud Providers + +- **AWS**: EKS, EC2, EBS +- **Azure**: AKS, VMs, Managed Disks +- **GCP**: GKE, Compute Engine, Persistent Disks +- **Hetzner**: Self-managed Kubernetes + +### Infrastructure Requirements + +- Kubernetes cluster (1.20+) +- 3+ nodes for high availability +- NVMe storage support +- Network connectivity to control plane +- IAM/Service Principal configuration + +## Security + +### Data Protection + +- Data never leaves your cloud account +- Encryption at rest and in transit +- Network isolation via VPC/VNet +- IAM integration for access control +- Audit logging + +### Compliance + +- GDPR compliance +- HIPAA compliance +- SOC 2 compliance +- Custom compliance requirements +- Data residency controls + +## Security posture + +* **Data residency** – choose the exact regions (or on-prem data centres) where data is stored and processed. +* **No shared tenancy** – storage and compute nodes belong exclusively to your account. +* **Key management** – integrate with KMS / HSM services for at-rest encryption keys. +* **Auditability** – logs and metrics stay inside your logging stack (CloudWatch, Stackdriver, etc.). + +## Typical rollout + +1. Provision a Kubernetes cluster of three or more nodes with NVMe disks. +2. Install the CloudNativePG operator and the Simplyblock storage Helm chart. +3. Run `xata byoc init` to register the cluster; the control plane issues an agent token. +4. Apply the generated manifests; Postgres and storage pods come online and self-register. +5. Migrate data from the previous provider using `xata clone --target byoc-prod` or start with a fresh database. + +The entire process is idempotent, so you can treat the cluster as cattle: tear it down and recreate it from Git at any time. + +> **Tip:** BYOC pairs well with multi-region fail-over. You can deploy a second BYOC data plane in another region and let Xata orchestrate asynchronous replication. + +## Getting Started + +1. Set up Kubernetes cluster: +```bash +# AWS +eksctl create cluster --name xata-cluster + +# Azure +az aks create --name xata-cluster + +# GCP +gcloud container clusters create xata-cluster +``` + +2. Configure Xata deployment: +```bash +xata init --byoc +``` + +3. Deploy Xata platform: +```bash +xata deploy +``` + +[Learn more about project management](/cli/project) \ No newline at end of file diff --git a/docs/core-concepts/core-concepts.mdx b/docs/core-concepts/core-concepts.mdx new file mode 100644 index 0000000..d4ae0cb --- /dev/null +++ b/docs/core-concepts/core-concepts.mdx @@ -0,0 +1,30 @@ +--- +title: Core concepts +description: Learn about Xata's core features for PostgreSQL development and operations +--- + +Xata provides a comprehensive set of features for PostgreSQL development and operations. This section covers the key concepts that make Xata unique: + +## Instant branching + +Create Copy-on-Write branches for development and testing in seconds. Works with any PostgreSQL cloud provider, including AWS RDS, Amazon Aurora, Google Cloud SQL, and more. + +[Learn more about instant branching](/core-concepts/branching) + +## Data anonymization + +Protect sensitive data in your development and testing environments. Automatically anonymize PII and other sensitive information while maintaining data relationships. + +[Learn more about data anonymization](/core-concepts/anonymization) + +## Schema changes + +Update your PostgreSQL schema online and with safe, instant rollbacks. Zero-downtime migrations that work with any PostgreSQL cloud provider. + +[Learn more about schema changes](/core-concepts/schema-changes) + +## Bring Your Own Cloud + +Deploy Xata in your own AWS, Azure, or GCP account. You get full control over your data, infrastructure and costs. + +[Learn more about BYOC](/core-concepts/byoc) \ No newline at end of file diff --git a/docs/core-concepts/schema-changes.mdx b/docs/core-concepts/schema-changes.mdx new file mode 100644 index 0000000..93c641b --- /dev/null +++ b/docs/core-concepts/schema-changes.mdx @@ -0,0 +1,113 @@ +--- +title: Schema Changes +description: Zero-downtime schema migrations with automatic rollback support +--- + +Xata's schema migration system enables zero-downtime database changes with automatic rollback support, powered by the open-source [**pgroll**](https://github.com/xataio/pgroll) project. Instead of locking tables or waiting for long-running `ALTER TABLE` commands to finish, [pgroll](https://github.com/xataio/pgroll) creates a *new* schema version next to the live one and moves traffic over when everything is ready. + +## How online migrations work + +When you run `xata roll migrate`, [pgroll](https://github.com/xataio/pgroll) first generates a migration plan that can always be reversed. The plan creates a *shadow schema*—for example `v2`—that contains the target structure. Triggers and logical decoding keep the data between the two versions consistent while the migration is in progress. As soon as the shadow schema has caught up, [pgroll](https://github.com/xataio/pgroll) performs an atomic *switchover* by updating a search-path alias. Because the alias switch is a metadata operation, it takes microseconds and never blocks reads or writes. + +If something goes wrong—constraint violation, performance regression, unexpected app error—you can issue `xata roll rollback` and [pgroll](https://github.com/xataio/pgroll) flips the alias back to the original schema. No data is lost, and the cut-over is again instantaneous. + +### Behind the scenes + +1. **Version tracking** – A dedicated table stores the current schema version and the planned next version. +2. **Dual execution** – During the catch-up phase both versions accept writes; triggers forward changes to keep them in sync. +3. **Progress monitoring** – [pgroll](https://github.com/xataio/pgroll) streams progress metrics so you can observe long-running back-fills. +4. **Automatic cleanup** – After a successful switchover the old version is kept for a configurable grace period and then dropped. + +## Operational benefits + +* **No query interruption** – Clients continue to read and write while the migration runs in the background. +* **No table locks** – Large `ADD COLUMN` or `ALTER TYPE` operations no longer freeze traffic. +* **Safe experimentation** – Because every plan is reversible you can abort at any point without risking data divergence. + +## Typical workflow + +```bash +# Initialise pgroll once per database +gxata roll init + +# Generate a new migration skeleton +xata roll create add_user_status + +# Edit `migrations/add_user_status.sql` (DDL only) + +# Test locally against a branch +xata branch create schema-test --from main +xata roll migrate --branch schema-test + +# Apply to staging / production +xata roll migrate # runs against current branch (main) +``` + +In a CI pipeline you would usually call `xata roll migrate --non-interactive` during the *deploy* phase and `xata roll status` to wait until the catch-up is complete before flipping traffic. + +## Best practices + +* Keep individual migrations small; several tiny steps are safer than one large one. +* Avoid back-filling huge columns inside the migration; use incremental jobs where possible. +* Clean up old schema versions to prevent catalog bloat. + +For a GitHub Actions reference see the [check merge readiness workflow](/github-actions/check) that blocks a pull-request until all pending migrations have completed. + +## Features + +### Zero-Downtime Operations + +- **No Table Locks**: Operations don't block reads or writes +- **Atomic Switches**: Instant schema version changes +- **Background Processing**: Data sync happens in background +- **Consistent Views**: Applications see consistent data + +### Safety Mechanisms + +- **Automatic Rollback**: Reverts on validation failure +- **Data Integrity**: Maintains constraints and relationships +- **Version Control**: Tracks all schema changes +- **Audit Trail**: Logs all migration operations + +## Supported operations + +Xata migrations, powered by pgroll, support a wide range of schema evolution tasks. Here are the currently supported operations: + +- [Add column](https://pgroll.com/docs/latest/operations/add_column) +- [Alter column](https://pgroll.com/docs/latest/operations/alter_column) +- [Change type](https://pgroll.com/docs/latest/operations/alter_column_type) +- [Change default](https://pgroll.com/docs/latest/operations/alter_column_default) +- [Change comment](https://pgroll.com/docs/latest/operations/alter_column_comment) +- [Add check constraint](https://pgroll.com/docs/latest/operations/add_check_constraint) +- [Add foreign key](https://pgroll.com/docs/latest/operations/add_foreign_key) +- [Add not null constraint](https://pgroll.com/docs/latest/operations/add_not_null_constraint) +- [Drop not null constraint](https://pgroll.com/docs/latest/operations/drop_not_null_constraint) +- [Add unique constraint](https://pgroll.com/docs/latest/operations/add_unique_constraint) +- [Create index](https://pgroll.com/docs/latest/operations/create_index) +- [Create table](https://pgroll.com/docs/latest/operations/create_table) +- [Create constraint](https://pgroll.com/docs/latest/operations/create_constraint) +- [Drop column](https://pgroll.com/docs/latest/operations/drop_column) +- [Drop multi column constraint](https://pgroll.com/docs/latest/operations/drop_multi_column_constraint) +- [Drop index](https://pgroll.com/docs/latest/operations/drop_index) +- [Drop table](https://pgroll.com/docs/latest/operations/drop_table) +- [Raw SQL](https://pgroll.com/docs/latest/operations/raw_sql) +- [Rename table](https://pgroll.com/docs/latest/operations/rename_table) +- [Rename column](https://pgroll.com/docs/latest/operations/rename_column) +- [Rename constraint](https://pgroll.com/docs/latest/operations/rename_constraint) +- [Add sequence](https://pgroll.com/docs/latest/operations/add_sequence) +- [Drop sequence](https://pgroll.com/docs/latest/operations/drop_sequence) +- [Alter sequence](https://pgroll.com/docs/latest/operations/alter_sequence) +- [Add view](https://pgroll.com/docs/latest/operations/add_view) +- [Drop view](https://pgroll.com/docs/latest/operations/drop_view) +- [Rename view](https://pgroll.com/docs/latest/operations/rename_view) +- [Add trigger](https://pgroll.com/docs/latest/operations/add_trigger) +- [Drop trigger](https://pgroll.com/docs/latest/operations/drop_trigger) +- [Add function](https://pgroll.com/docs/latest/operations/add_function) +- [Drop function](https://pgroll.com/docs/latest/operations/drop_function) +- [Rename function](https://pgroll.com/docs/latest/operations/rename_function) +- [Alter function](https://pgroll.com/docs/latest/operations/alter_function) +- [Add schema](https://pgroll.com/docs/latest/operations/add_schema) +- [Drop schema](https://pgroll.com/docs/latest/operations/drop_schema) +- [Rename schema](https://pgroll.com/docs/latest/operations/rename_schema) + +For the full syntax, advanced options, and the latest supported operations, see the [pgroll documentation](https://pgroll.com/docs/latest/) and the [CLI reference for roll commands](/cli/roll). \ No newline at end of file diff --git a/docs/getting-started.mdx b/docs/getting-started.mdx index d4a9f7e..7363cd0 100644 --- a/docs/getting-started.mdx +++ b/docs/getting-started.mdx @@ -1,7 +1,42 @@ -# Getting started +--- +title: Getting started +description: Quick start guide for Xata's PostgreSQL platform +--- -## Another heading +Xata is a PostgreSQL platform that provides instant Copy-on-Write branching, data masking, and separation of storage from compute. It's designed for modern teams running PostgreSQL at scale, with features like zero-downtime schema changes, realistic staging environments, and cloud-agnostic deployment options. -Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here.Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. Content goes here. +## Quick start +After signing up for your account, you can get started by installing the CLI and intializing your project. +1. **Install the CLI** + ```bash + curl -fsSL https://xata.io/install.sh | bash + ``` +2. **Authenticate with Xata** + ```bash + xata auth login + ``` + +3. **Initialize your project** + ```bash + xata init + ``` + +## Next steps + +There are two primary workflows Xata supports natively. + +- **Developer workflow**: Set up staging and development environments with our [developer workflow](/tutorials/developer-workflow) +- **Production migration**: Move your existing PostgreSQL database to Xata by [migrating to Xata](/tutorials/migrate-to-xata) + +## Additional resources + +Want to dive in further? Check out these pages. + +- [Architecture overview](/architecture) - Deep dive into Xata's technical architecture +- [Core concepts](/core-concepts) - Learn about branching, staging, and data anonymization +- [Command line interface](/cli) - Complete CLI reference +- [GitHub actions](/github-actions) - Automate your development workflow + +**Need help?** Reach out to the Xata team at [info@xata.io](mailto:info@xata.io) or join our Discord community. \ No newline at end of file diff --git a/docs/github-actions/check.mdx b/docs/github-actions/check.mdx new file mode 100644 index 0000000..64a955f --- /dev/null +++ b/docs/github-actions/check.mdx @@ -0,0 +1,127 @@ +--- +title: Check merge readiness +description: GitHub Actions workflow for validating main branch status and migration readiness +--- + +This workflow validates that the main branch is in a ready state for merging pull requests. It checks the migration status and ensures no pending migrations are running that could affect the merge process. + +## Workflow configuration + +```yaml +name: Check merge readiness + +on: + pull_request: + types: [opened, synchronize, reopened] + +jobs: + check: + runs-on: ubuntu-latest + name: Xata dev branch + steps: + - uses: actions/checkout@v3 + + - name: Install Xata CLI + run: | + curl -fsSL https://xata.io/install.sh | bash + + - name: Check Xata status + run: | + xata status + xata roll status + xata branch view + + - name: Check Xata main branch pgroll status + id: xata-main-branch-roll-status + run: | + xata roll status | jq -r '.status' + echo XATA_MAIN_BRANCH_ROLL_STATUS=`xata roll status | jq -r '.status'` >> $GITHUB_OUTPUT + + - name: Complete any pending migrations (inherited from main) + if: ${{ steps.xata-main-branch-roll-status.outputs.XATA_MAIN_BRANCH_ROLL_STATUS == 'In progress' }} + run: | + echo "Warning: A migration is currently running on the main branch. Please wait for it to complete before merging this PR." + exit 1 +``` + +## Authentication and environment variables + +The workflow requires several environment variables to be set: + +- `XATA_API_REFRESH_TOKEN`: Your Xata API refresh token (set as a GitHub secret) +- `XATA_API_ENVIRONMENT`: The Xata environment to use (e.g., staging) +- `XATA_ORGANIZATIONID`: Your Xata organization ID +- `XATA_PROJECTID`: Your Xata project ID +- `XATA_BRANCHID`: The ID of the main branch +- `XATA_BRANCHNAME`: The name of the main branch +- `XATA_DATABASENAME`: Your database name + +## Workflow steps + +### 1. Install Xata CLI + +```yaml +- name: Install Xata CLI + run: | + curl -fsSL https://xata.io/install.sh | bash +``` + +This step installs the Xata CLI using the official installer script and adds it to the GitHub Actions PATH. + +### 2. Check Xata status + +```yaml +- name: Check Xata status + run: | + xata status + xata roll status + xata branch view +``` + +This step performs three checks: +- Verifies CLI and connection status +- Checks for any pending migrations +- Views the current branch status + +## Concurrency control + +The workflow uses GitHub Actions concurrency to prevent multiple runs from interfering with each other: + +```yaml +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true +``` + +This ensures that: +- Only one workflow run per branch is active at a time +- New runs cancel any in-progress runs +- Prevents race conditions during status checks + +## When to use + +Use this workflow when you want to: +- Ensure main branch is ready for merging +- Prevent merging during active migrations +- Validate branch and migration status +- Maintain database consistency + +## Best practices + +1. **Set appropriate timeout** + - The workflow has a 10-minute timeout + - Adjust based on your typical migration duration + +2. **Use environment variables** + - Store sensitive information in GitHub secrets + - Use environment variables for configuration + +3. **Handle migration states** + - Check for in-progress migrations + - Prevent merging during active migrations + - Provide clear error messages + +## Related workflows + +- [Clone workflow](./clone.mdx) - For cloning databases in CI/CD +- [PR workflow](./pr.mdx) - For managing pull request environments \ No newline at end of file diff --git a/docs/github-actions/clone.mdx b/docs/github-actions/clone.mdx new file mode 100644 index 0000000..aa7c34e --- /dev/null +++ b/docs/github-actions/clone.mdx @@ -0,0 +1,178 @@ +--- +title: Xata clone (daily, on-demand) +description: GitHub Actions workflow for cloning databases on a schedule or on-demand +--- + +This workflow helps you clone your database on a schedule or manually trigger it. It's useful for creating regular backups or setting up test environments. + +## Workflow configuration + +```yaml +name: Xata clone (daily, on-demand) + +on: + schedule: + - cron: '0 0 * * *' # Run daily at midnight + workflow_dispatch: # Allow manual triggers + +jobs: + clone: + name: Xata dev branch + runs-on: ubuntu-latest + steps: + - name: Install Xata CLI + run: | + curl -fsSL https://xata.io/install.sh | bash + + - name: Check Xata status + run: | + xata status + xata branch view + + - name: Checkout Xata dev branch + run: | + echo XATA_BRANCHID=`xata branch checkout clone_demo --json | jq -r '.id'` >> $GITHUB_ENV + + - name: Check Xata status + run: | + xata status + xata branch view + + - name: Xata clone + run: | + xata clone start --source-url ${{ env.XATA_CLI_SOURCE_POSTGRES_URL }} +``` + +## Authentication and environment variables + +The workflow requires several environment variables to be set: + +- `XATA_API_REFRESH_TOKEN`: Your Xata API refresh token (set as a GitHub secret) +- `XATA_API_ENVIRONMENT`: The Xata environment to use (e.g., staging) +- `XATA_ORGANIZATIONID`: Your Xata organization ID +- `XATA_PROJECTID`: Your Xata project ID +- `XATA_BRANCHID`: The ID of the main branch +- `XATA_BRANCHNAME`: The name of the main branch +- `XATA_DATABASENAME`: Your database name +- `XATA_CLI_SOURCE_POSTGRES_URL`: The source PostgreSQL database URL (set as a GitHub secret) + +## Workflow steps + +### 1. Checkout code + +```yaml +- uses: actions/checkout@v3 + with: + fetch-depth: 0 +``` + +This step checks out your repository code with full history and proper credentials configuration. + +### 2. Install Xata CLI + +```yaml +- name: Install Xata CLI + run: | + curl -fsSL https://xata.io/install.sh | bash +``` + +This step installs the Xata CLI using the official installer script and adds it to the GitHub Actions PATH. + +### 3. Check Xata status + +```yaml +- name: Check Xata status + run: | + xata status + xata branch view +``` + +This step verifies that your Xata CLI is properly configured and can connect to your database. + +For more details, see: +- [status command documentation](../cli/status.mdx) +- [branch view command documentation](../cli/branch.mdx#view) + +### 4. Checkout clone branch + +```yaml +- name: Checkout Xata dev branch + run: | + echo XATA_BRANCHID=`xata branch checkout clone_demo --json | jq -r '.id'` >> $GITHUB_ENV +``` + +This step: +- Checks out the clone_demo branch +- Extracts the branch ID using `jq` +- Stores it in an environment variable + +### 5. Start clone process + +```yaml +- name: Xata clone + run: | + xata clone start --source-url ${{ env.XATA_CLI_SOURCE_POSTGRES_URL }} +``` + +This step initiates the cloning process from your source PostgreSQL database. + +For more details, see the [clone start command documentation](../cli/clone.mdx#start). + +## Triggering the workflow + +The workflow can be triggered in two ways: + +1. **Scheduled run** + ```yaml + schedule: + - cron: "0 8 * * *" + ``` + Runs daily at 8:00 AM UTC + +2. **Manual trigger** + ```yaml + workflow_dispatch: + ``` + Can be triggered manually from the GitHub Actions UI + +## Concurrency control + +The workflow uses GitHub Actions concurrency to prevent multiple runs from interfering with each other: + +```yaml +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true +``` + +This ensures that: +- Only one workflow run is active at a time +- New runs cancel any in-progress runs +- Prevents race conditions during cloning + +## When to use + +Use this workflow when you want to: +- Create regular database backups +- Set up test environments +- Clone production data for development +- Maintain a separate clone of your database + +## Best practices + +1. **Set appropriate timeout** + - The workflow has a 10-minute timeout + - Adjust based on your database size and cloning duration + +2. **Use environment variables** + - Store sensitive information in GitHub secrets + - Use environment variables for configuration + +3. **Schedule regular clones** + - Set up a schedule that matches your backup needs + - Consider your database size and update frequency + +## Related workflows + +- [Check workflow](./check.mdx) - For validating main branch status +- [PR workflow](./pr.mdx) - For managing pull request environments \ No newline at end of file diff --git a/docs/github-actions/github-actions.mdx b/docs/github-actions/github-actions.mdx new file mode 100644 index 0000000..10841f1 --- /dev/null +++ b/docs/github-actions/github-actions.mdx @@ -0,0 +1,100 @@ +--- +title: GitHub Actions +description: Reference documentation for Xata GitHub Actions workflows +--- + +Xata provides several GitHub Actions workflows to help you automate database operations in your CI/CD pipeline. These workflows help you manage database branches, migrations, and deployments in a safe and automated way. + +## Available workflows + +### [Ensure main is ready for merge](./github-actions/check.mdx) + +This workflow validates that the main branch is in a ready state for merging pull requests. It checks the migration status and ensures no pending migrations are running that could affect the merge process. + +Key features: +- Validates main branch status +- Checks migration state +- Prevents merging during active migrations +- Provides clear status messages + +### [Create Xata dev branch](./github-actions/pr.mdx) + +This workflow creates a dedicated Xata branch for each pull request, allowing you to test database changes in isolation. It also manages PR comments to provide branch information and status updates. + +Key features: +- Creates isolated database environments for PRs +- Manages branch lifecycle +- Handles migrations automatically +- Provides connection details in PR comments + +### [Xata clone workflow](./github-actions/clone.mdx) + +This workflow helps you clone your database on a schedule or manually trigger it. It's useful for creating regular backups or setting up test environments. + +Key features: +- Scheduled database cloning +- Manual trigger support +- Backup management +- Test environment setup + +## Common features + +All workflows share some common features: + +### Authentication + +Each workflow requires a Xata API refresh token, which should be stored as a GitHub secret: + +```yaml +env: +XATA_API_REFRESH_TOKEN: ${{ secrets.XATA_API_REFRESH_TOKEN }} +``` + +### Environment variables + +Common environment variables used across workflows: + +- `XATA_API_ENVIRONMENT`: The Xata environment to use (e.g., staging) +- `XATA_ORGANIZATIONID`: Your Xata organization ID +- `XATA_PROJECTID`: Your Xata project ID +- `XATA_BRANCHID`: The ID of the main branch +- `XATA_BRANCHNAME`: The name of the main branch +- `XATA_DATABASENAME`: Your database name + +### Concurrency Control + +All workflows use GitHub Actions concurrency to prevent multiple runs from interfering with each other: + +```yaml +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true +``` + +## Best Practices + +1. **Security** + - Store sensitive information in GitHub secrets + - Use environment variables for configuration + - Follow the principle of least privilege + +2. **Performance** + - Set appropriate timeouts + - Use concurrency control + - Optimize workflow triggers + +3. **Maintenance** + - Keep workflows up to date + - Monitor workflow runs + - Review and update dependencies + +4. **Documentation** + - Document workflow changes + - Keep README files updated + - Include clear error messages + +## Related Documentation + +- [CLI Reference](../cli.mdx) - For details about the Xata CLI commands used in workflows +- [Branch Management](../cli/branch.mdx) - For information about database branches +- [Migrations](../cli/roll.mdx) - For details about database migrations \ No newline at end of file diff --git a/docs/github-actions/pr.mdx b/docs/github-actions/pr.mdx new file mode 100644 index 0000000..ed3039d --- /dev/null +++ b/docs/github-actions/pr.mdx @@ -0,0 +1,261 @@ +--- +title: Create Xata dev branch +description: GitHub Actions workflow for creating and managing development branches for pull requests +--- + +This workflow creates a dedicated Xata branch for each pull request, allowing you to test database changes in isolation. It also manages PR comments to provide branch information and status updates. + +## Workflow Configuration + +```yaml +name: Create Xata dev branch + +on: + pull_request: + ignore-branches: + - main + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +env: + FORCE_COLOR: 3 + XATA_API_REFRESH_TOKEN: ${{ secrets.XATA_API_REFRESH_TOKEN }} + XATA_API_ENVIRONMENT: staging + XATA_ORGANIZATIONID: 123xyz + XATA_PROJECTID: prj_i3qsq56h3p3d3crnk7htnc75d0 + XATA_BRANCHID: s99l2grl6d3nr1cqcbaaa5piic + XATA_BRANCHNAME: main + XATA_DATABASENAME: app + +permissions: + id-token: write + contents: write + packages: write + pages: write + pull-requests: write +``` + +## Authentication and Environment Variables + +The workflow requires several environment variables to be set: + +- `XATA_API_REFRESH_TOKEN`: Your Xata API refresh token (set as a GitHub secret) +- `XATA_API_ENVIRONMENT`: The Xata environment to use (e.g., staging) +- `XATA_ORGANIZATIONID`: Your Xata organization ID +- `XATA_PROJECTID`: Your Xata project ID +- `XATA_BRANCHID`: The ID of the main branch +- `XATA_BRANCHNAME`: The name of the main branch +- `XATA_DATABASENAME`: Your database name + +## Required Permissions + +The workflow requires the following GitHub permissions: + +```yaml +permissions: + id-token: write + contents: write + packages: write + pages: write + pull-requests: write +``` + +## Workflow Steps + +### 1. Find Previous Comment + +```yaml +- name: Find Previous Comment + id: find-comment + uses: peter-evans/find-comment@v1 + with: + token: ${{ secrets.GITHUB_TOKEN }} + issue-number: ${{ github.event.pull_request.number }} + comment-author: "github-actions[bot]" + body-includes: "Xata Dev Branch" +``` + +This step searches for any existing comments from the workflow to update them later. + +### 2. Checkout Code + +```yaml +- uses: actions/checkout@v4 + with: + fetch-depth: 0 + persist-credentials: false +``` + +This step checks out your repository code with full history and proper credentials configuration. + +### 3. Install Dependencies + +```yaml +- name: Install pnpm + uses: pnpm/action-setup@v2 + with: + version: 10 + +- name: Install Xata CLI + run: | + curl -fsSL https://xata.io/install.sh | bash + echo "/home/runner/.config/xata/bin" >> $GITHUB_PATH +``` + +These steps install the required dependencies: +- pnpm package manager +- Xata CLI + +### 4. Check Status and Migration State + +```yaml +- name: Check Xata Status + run: | + xata status + xata roll status + xata branch view + +- name: Check Xata main branch pgroll status + id: xata-main-branch-roll-status + run: | + echo XATA_MAIN_BRANCH_ROLL_STATUS=`xata roll status | jq -r '.status'` >> $GITHUB_OUTPUT +``` + +These steps verify the current state of your Xata environment and check for any pending migrations. + +### 5. Create or Update Comment + +```yaml +- name: Create or Update Comment + id: create_comment + uses: peter-evans/create-or-update-comment@v1 + with: + token: ${{ secrets.GITHUB_TOKEN }} + comment-id: ${{ steps.find-comment.outputs.comment-id }} + issue-number: ${{ github.event.pull_request.number }} + edit-mode: replace + body: | + ### Xata Dev Branch + + Creating Xata Dev Branch for commit: ${{ github.sha }} + Branch name: ${{ github.head_ref }}_${{ github.event.pull_request.number }} + + *Please wait for the comment to be updated with branch details.* +``` + +### 6. Branch Management + +```yaml +- name: Delete Any Existing Xata Dev Branch + run: xata branch delete ${{ github.head_ref }}_${{ github.event.pull_request.number }} --yes + continue-on-error: true + +- name: Create Xata Dev Branch + run: xata branch create --name ${{ github.head_ref }}_${{ github.event.pull_request.number }} + +- name: Checkout Xata Dev Branch + run: | + echo XATA_BRANCHID=`xata branch checkout ${{ github.head_ref }}_${{ github.event.pull_request.number }} --json | jq -r '.id'` >> $GITHUB_ENV +``` + +### 7. Migration Handling + +```yaml +- name: Complete Any Pending Migrations + if: ${{ steps.xata-main-branch-roll-status.outputs.XATA_MAIN_BRANCH_ROLL_STATUS == 'In progress' }} + run: | + xata roll complete + +- name: Apply Xata Roll Migrations + run: xata roll migrate +``` + +### 8. Environment Setup + +```yaml +- name: Get Latest Schema + id: xata-roll-get-latest-schema + run: | + echo LATEST_SCHEMA=`xata roll latest --with-schema=true` >> $GITHUB_OUTPUT + +- name: Setup New Preview Environment + run: | + echo DATABASE_URL=`xata branch url` >> $GITHUB_ENV + echo SEARCH_PATH=${{ steps.xata-roll-get-latest-schema.outputs.LATEST_SCHEMA }} >> $GITHUB_ENV +``` + +### 9. Final Comment Update + +```yaml +- name: Update Comment with Branch Details + uses: peter-evans/create-or-update-comment@v1 + with: + token: ${{ secrets.GITHUB_TOKEN }} + comment-id: ${{ steps.find-comment.outputs.comment-id }} + issue-number: ${{ github.event.pull_request.number }} + edit-mode: replace + body: | + ### Xata Dev Branch + + Created Xata Dev Branch for commit: ${{ github.sha }} + Branch name: ${{ github.head_ref }}_${{ github.event.pull_request.number }} + + Set the following schemas as the search_path: + ${{ steps.xata-roll-get-latest-schema.outputs.LATEST_SCHEMA }} + + You can drop into the psql shell for this branch with: + xata branch checkout ${{ github.head_ref }}_${{ github.event.pull_request.number }} + psql `xata branch url` + + Branch URL: [View in Console](https://console.xata.io/organizations/123xyz/projects/prj_i3qsq56h3p3d3crnk7htnc75d0/branches/s99l2grl6d3nr1cqcbaaa5piic) +``` + +## Concurrency Control + +The workflow uses GitHub Actions concurrency to prevent multiple runs from interfering with each other: + +```yaml +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true +``` + +This ensures that: +- Only one workflow run per PR is active at a time +- New runs cancel any in-progress runs +- Prevents race conditions during branch creation + +## When to Use + +Use this workflow when you want to: +- Create isolated database environments for PRs +- Test database changes in isolation +- Provide preview environments for PR reviewers +- Automate database branch management + +## Best Practices + +1. **Set Appropriate Timeout** + - The workflow has a 10-minute timeout + - Adjust based on your typical migration duration + +2. **Use Environment Variables** + - Store sensitive information in GitHub secrets + - Use environment variables for configuration + +3. **Handle Migration States** + - Check for in-progress migrations + - Complete pending migrations + - Apply new migrations + +4. **PR Communication** + - Keep PR comments updated + - Provide clear connection instructions + - Include relevant warnings + +## Related Workflows + +- [Check Workflow](./check.mdx) - For validating main branch status +- [Clone Workflow](./clone.mdx) - For cloning databases in CI/CD \ No newline at end of file diff --git a/docs/resources/compliance.mdx b/docs/resources/compliance.mdx new file mode 100644 index 0000000..bb71135 --- /dev/null +++ b/docs/resources/compliance.mdx @@ -0,0 +1,66 @@ +--- +title: Compliance +description: Security certifications and compliance standards supported by Xata +--- + +Xata maintains several security certifications and follows industry best practices to ensure the highest level of data protection and privacy. + +## Certifications + +Xata is currently certified for: + +- **SOC-2**: Service Organization Control 2 compliance for security, availability, and confidentiality +- **HIPAA**: Health Insurance Portability and Accountability Act compliance for healthcare data +- **GDPR**: General Data Protection Regulation compliance for EU data protection + +Additional compliance certifications are being pursued. If you require a specific certification to use Xata, please contact our team. + +## Security Practices + +Xata implements comprehensive security measures: + +### Data Protection + +- Data encryption at rest for all data stores +- Data encryption in transit for all network communication +- Clear separation between control plane and data plane +- PII information retention limited to 2 weeks in observability systems + +### Access Control + +- Minimal scope credentials with periodic rotation +- SSO with MFA required for critical services +- Strict controls on production system access +- Regular third-party risk assessments + +### Software Security + +- Automatic vulnerability scanning for dependencies +- Weekly cadence for dependency upgrades +- Regular security assessments +- Continuous monitoring of security news and threat intelligence + +## Compliance Reports + +SOC-2 and HIPAA compliance reports are available upon request. Please contact support to obtain these reports for your compliance needs. + +## Responsible Disclosure + +Xata maintains a responsible vulnerability disclosure program: + +1. Report vulnerabilities to security@xata.io +2. We respond within 3 business days +3. We maintain confidentiality throughout the process +4. We provide credit for discovered vulnerabilities +5. We offer bounties for medium severity or higher issues + +### Out of Scope + +The following are not eligible for the disclosure program: +- Non-security related bugs +- Email spoofing +- Social engineering +- DoS/DDoS attacks +- Vulnerabilities on feedback.xata.io + +[Learn more about our security practices](/security) \ No newline at end of file diff --git a/docs/resources/pricing.mdx b/docs/resources/pricing.mdx new file mode 100644 index 0000000..29e45d2 --- /dev/null +++ b/docs/resources/pricing.mdx @@ -0,0 +1,86 @@ +--- +title: Pricing +description: Simple, transparent pricing for Xata's PostgreSQL platform +--- + +Xata offers simple, transparent pricing with two main plans: Pay As You Go and Bring Your Own Cloud (BYOC). Save on infrastructure costs through better resource utilization. + +## Pay As You Go + +Starting at $0.012 / hour + +### Features +- Postgres hosting on our platform +- Data anonymization / masking +- Instant Copy-on-Write branches +- Unlimited projects / databases / branches +- Premium support + +### Instance Types + +| Instance | vCPUs | RAM (GB) | Hourly Rate | Monthly Cost* | +| ------------ | ----- | -------- | ----------- | ------------- | +| xata.micro | 2 | 1 | $0.012 | $9 | +| xata.small | 2 | 2 | $0.024 | $18 | +| xata.medium | 2 | 4 | $0.048 | $35 | +| xata.large | 2 | 8 | $0.096 | $70 | +| xata.xlarge | 4 | 16 | $0.192 | $140 | +| xata.2xlarge | 8 | 32 | $0.384 | $280 | +| xata.4xlarge | 16 | 64 | $0.768 | $561 | +| xata.8xlarge | 32 | 128 | $1.536 | $1121 | + +*Monthly cost is approximate and based on 730 hours of usage. Contact us for custom instance sizes. + +### Storage + +Postgres storage is charged at **$0.30 / GB / month**. Only the actually used storage is charged, so you don't have to worry about allocating disk space. Xata uses an advanced distributed storage system for separating storage from compute. + +[Learn more about storage](/core-concepts/storage) + +## Bring Your Own Cloud (BYOC) + +$0.59 / node / hour + +### Features +- Everything included in Pay As You Go +- Managed Postgres instances on your account +- Runs on AWS, GCP, Azure, or your own data center +- Custom extensions support + +[Learn more about BYOC](/core-concepts/byoc) + +## Enterprise Plan + +Contact us to discuss: +- Custom plans +- Support options +- White-labeling +- Professional services + +## Frequently Asked Questions + +### Do you have a free tier? +Yes, we offer a free tier for development and testing. Contact us for details. + +### What about a free trial? +We offer a free trial period for all paid plans. Contact us to get started. + +### Are all anonymization features and zero-downtime schema changes free? +Yes, these features are included in both Pay As You Go and BYOC plans. + +### I want to host production on your platform. Is Xata production ready? +Yes, Xata is production-ready with features like high availability, automatic backups, and monitoring. + +### Is there a minimal number of nodes for the BYOC model? +Yes, BYOC requires a minimum of 3 nodes for high availability. Contact us for specific requirements. + +### Do you offer instances with a different CPU / RAM ratio? +Yes, we can customize instance configurations. Contact us to discuss your specific needs. + +### Do you charge for bandwidth? +No, we don't charge for bandwidth. You only pay for compute and storage. + +### We are a PaaS and we'd like to offer a Postgres service to our customers. Can we use Xata to do that? +Yes, we offer white-labeling options for PaaS providers. Contact us to discuss partnership opportunities. + +[Contact us](mailto:info@xata.io) for custom pricing or to discuss your specific needs. diff --git a/docs/resources/resources.mdx b/docs/resources/resources.mdx new file mode 100644 index 0000000..e2bd1ea --- /dev/null +++ b/docs/resources/resources.mdx @@ -0,0 +1,73 @@ +--- +title: Resources +description: Additional resources and information for Xata users +--- + +# Resources + +Additional resources and information to help you get the most out of Xata. + +## Troubleshooting & FAQ + +Find answers to common questions and solutions to typical issues in our [Troubleshooting & FAQ](/resources/troubleshooting) guide. Topics include: + +- General questions about Xata +- Staging and development workflows +- Data privacy and security +- Performance and storage +- Deployment and integration +- Pricing and plans +- Technical details + +## Compliance + +Learn about Xata's security certifications and compliance standards in our [Compliance](/resources/compliance) documentation: + +- SOC-2 compliance +- HIPAA compliance +- GDPR compliance +- Security practices +- Compliance reports +- Responsible disclosure + +## Roadmap + +Check out our [Roadmap](/resources/roadmap) to see what's coming next: + +- Multi-region active-active clusters +- Point-in-time branch rewinds +- Automatic index recommendations +- Performance improvements +- New features and integrations + +## Pricing + +Review our [Pricing](/resources/pricing) information: + +- Pay As You Go plan +- Bring Your Own Cloud (BYOC) +- Enterprise plans +- Instance types and costs +- Storage pricing +- Frequently asked questions + +## Additional Resources + +### Documentation + +- [Architecture Overview](/architecture) +- [Core Concepts](/core-concepts) +- [Command Line Interface](/cli) +- [GitHub Actions](/github-actions) + +### Community + +- [Discord Community](https://discord.gg/xata) +- [GitHub Repository](https://github.com/xataio) +- [Blog](https://xata.io/blog) + +### Support + +- [Contact Support](mailto:info@xata.io) +- [Feature Requests](https://github.com/xataio/xata/issues) +- [Security Issues](mailto:security@xata.io) \ No newline at end of file diff --git a/docs/resources/roadmap.mdx b/docs/resources/roadmap.mdx new file mode 100644 index 0000000..9541a51 --- /dev/null +++ b/docs/resources/roadmap.mdx @@ -0,0 +1,6 @@ +--- +title: Roadmap +--- + +TODO: +Add roadmap \ No newline at end of file diff --git a/docs/resources/troubleshooting.mdx b/docs/resources/troubleshooting.mdx new file mode 100644 index 0000000..765fb15 --- /dev/null +++ b/docs/resources/troubleshooting.mdx @@ -0,0 +1,139 @@ +--- +title: Troubleshooting & FAQ +description: Frequently asked questions and troubleshooting guides for Xata's PostgreSQL platform +--- + +## General Questions + +### What is Xata? +Xata is a PostgreSQL platform that provides instant Copy-on-Write branching, data masking, and separation of storage from compute. It's designed for teams running PostgreSQL at scale, offering features like zero-downtime schema changes, realistic staging environments, and cloud-agnostic deployment options. + +### Is Xata production ready? +Yes, Xata is production-ready with features like high availability, automatic backups, and monitoring. The platform is built on proven technologies including CloudNativePG for PostgreSQL management and Simplyblock for distributed storage. + +### Do you offer a free tier? +Yes, we offer a free tier for development and testing. Contact us for details about the current free tier offerings. + +### What about a free trial? +We offer a free trial period for all paid plans. Contact us to get started with your trial. + +## Staging & Development + +### How does Xata handle staging environments? +Xata creates a "staging replica" that combines Copy-on-Write branching with data anonymization. This allows you to: +- Create instant dev branches from the staging replica +- Work with realistic, anonymized versions of your production data +- Test schema changes safely before applying them to production + +### Will Xata alter my production database? +No, your production database stays exactly as it is. Xata works with your existing PostgreSQL service, whether it's AWS RDS, Amazon Aurora, GCP CloudSQL, Azure Database, or self-hosted Postgres. + +### How do ephemeral environments improve development speed? +By providing instant Copy-on-Write branches, developers can: +- Work in isolation without waiting for others +- Test changes against realistic data +- Avoid conflicts with other developers +- Quickly create and destroy environments as needed + +### Can I test complex schema changes with Xata? +Yes, you can use `xata roll` (powered by [pgroll](https://github.com/xataio/pgroll)) to test schema changes and safely apply them with zero-downtime to production. This allows you to test complex changes in isolation before applying them to production. + +## Data Privacy & Security + +### How does data anonymization work? +Xata uses [pgstream](https://github.com/xataio/pgstream) to copy and anonymize data on the fly. The system: +- Detects and masks PII/PHI and sensitive data +- Maintains referential integrity +- Preserves data relationships +- Allows configurable transformers for different data types + +### Is my data secure in staging environments? +Yes, Xata ensures data security by: +- Automatically anonymizing sensitive data +- Maintaining strict access controls +- Providing audit trails +- Supporting compliance requirements (SOC-2, HIPAA, GDPR) + +### Can I keep my data within my own infrastructure? +Yes, through our Bring Your Own Cloud (BYOC) deployment model, you can run the full Xata platform in your own cloud account or on-premises, ensuring data never leaves your infrastructure. + +## Performance & Storage + +### How does Xata's storage system work? +Xata uses a distributed storage system that: +- Separates storage from compute +- Uses NVMe/TCP for high performance +- Implements erasure coding for data resilience +- Provides automatic storage growth +- Charges only for actual storage used + +### Can Xata help with performance tuning? +Yes, Xata provides the Xata Agent, an LLM-powered tool that helps: +- Monitor PostgreSQL instances +- Diagnose performance issues +- Optimize queries +- Tune PostgreSQL settings +- Analyze active queries and statistics + +### Do you charge for bandwidth? +No, we don't charge for bandwidth. You only pay for compute and storage resources. + +## Deployment & Integration + +### Can I bring my own cloud infrastructure? +Yes, Xata offers a Bring Your Own Cloud (BYOC) deployment model that allows you to: +- Deploy in your own AWS, GCP, or Azure account +- Run on-premises if needed +- Maintain full control over your infrastructure +- Use your cloud provider's credits and discounts + +### Can Xata integrate with CI/CD pipelines? +Yes, Xata integrates with: +- GitHub Actions +- Jenkins +- Other CI/CD systems +- Custom automation workflows + +### Do I need to modify my existing database schema for Xata? +No, Xata works with your existing PostgreSQL schema. You can start using Xata's features without any schema modifications. + +## Pricing & Plans + +### Are all anonymization features and zero-downtime schema changes free? +Yes, these features are included in both Pay As You Go and BYOC plans. + +### Is there a minimal number of nodes for the BYOC model? +Yes, BYOC requires a minimum of 3 nodes for high availability. Contact us for specific requirements. + +### Do you offer instances with different CPU/RAM ratios? +Yes, we can customize instance configurations. Contact us to discuss your specific needs. + +### Can I use Xata for white-labeling? +Yes, we offer white-labeling options for PaaS providers. Contact us to discuss partnership opportunities. + +## Technical Details + +### How does Copy-on-Write branching work? +Xata's Copy-on-Write branching: +- Creates instant branches without copying data +- Only copies modified chunks when writes occur +- Maintains data integrity across branches +- Provides significant disk space savings +- Enables fast branch creation and deletion + +### What PostgreSQL extensions are supported? +Because we run vanilla PostgreSQL, we support: +- All standard PostgreSQL extensions +- Custom extensions +- Community extensions +- We can quickly approve and deploy new extensions as needed + +### How does the storage system achieve high performance? +The storage system: +- Uses SPDK and DDPK for user-space operations +- Implements NVMe-oF multipathing +- Provides distributed parity data +- Enables fault tolerance across the cluster +- Minimizes latency and context switches + +[Contact us](mailto:info@xata.io) if you have any other questions or need assistance. \ No newline at end of file diff --git a/docs/tutorials/developer-workflow.mdx b/docs/tutorials/developer-workflow.mdx new file mode 100644 index 0000000..e247300 --- /dev/null +++ b/docs/tutorials/developer-workflow.mdx @@ -0,0 +1,140 @@ +--- +title: Developer workflow +description: Learn how to use Xata's branching and migration features for zero-downtime deployments +--- + +This guide walks through a typical developer workflow using Xata's branching and migration features. We'll cover: + +- Setting up your development environment +- Creating and managing branches +- Making schema changes +- Testing changes in isolation +- Deploying to production + +## Prerequisites + +Before you begin, make sure you have: + +1. A Xata account (sign up at [console.xata.io](https://console.xata.io)) +2. The Xata CLI installed: + ```bash + curl -fsSL https://xata.io/install.sh | bash + ``` +3. Authenticated with Xata: + ```bash + xata auth login + ``` + +## Initial setup + +1. Create a new project: + ```bash + xata init + ``` + +2. Create your first branch: + ```bash + xata branch create main + ``` + +3. Initialize your database schema: + ```bash + xata init + ``` + +## Development workflow + +### 1. Create a feature branch + +Create a new branch for your feature: + ```bash + xata branch create feature/new-column + ``` + +This creates an isolated copy of your database where you can safely make changes. + +### 2. Make schema changes + +Edit your schema file (e.g., `schema.json`) to add your changes. For example, adding a new column: + +```json +{ + "tables": [ + { + "name": "users", + "columns": [ + { + "name": "email", + "type": "string" + }, + { + "name": "name", + "type": "string" + }, + { + "name": "age", + "type": "integer" + } + ] + } + ] +} +``` + +### 3. Apply changes + +Apply your schema changes to the feature branch: + ```bash + xata branch apply + ``` + +This will: +- Validate your schema changes +- Apply the changes to your feature branch +- Show you a preview of the changes + +### 4. Test your changes + +Your application can now connect to the feature branch and test the changes: + ```bash + xata branch url + ``` + +Use the URL to connect your application to the feature branch. + +### 5. Deploy to production + +When you're ready to deploy: + +1. Merge your changes to main: + ```bash + xata branch merge main + ``` + +2. Apply the changes to production: + ```bash + xata branch apply + ``` + +## Best practices + +1. **Create small, focused branches**: Each branch should represent a single feature or fix +2. **Test thoroughly**: Use the feature branch to test your changes before merging +3. **Keep branches up to date**: Regularly merge changes from main to avoid conflicts +4. **Use meaningful branch names**: Make it clear what each branch is for +5. **Clean up old branches**: Delete branches after they're merged + +## Common commands + +- **Delete branches**: `xata branch delete ` +- **Get branch URL**: `xata branch url` +- **List branches**: `xata branch list` +- **Switch branches**: `xata branch checkout ` +- **View branch status**: `xata branch view` + +## Next steps + +- Learn about [data masking](/core-concepts/anonymization) +- Explore [GitHub Actions integration](/github-actions) +- Read about [zero-downtime migrations](/core-concepts/migrations) + diff --git a/docs/tutorials/migrate-to-xata.mdx b/docs/tutorials/migrate-to-xata.mdx new file mode 100644 index 0000000..b83ffea --- /dev/null +++ b/docs/tutorials/migrate-to-xata.mdx @@ -0,0 +1,122 @@ +--- +title: Tutorial – migrate an existing database to Xata +description: Step-by-step guide for moving data from any PostgreSQL service to Xata with minimal downtime +--- + +Migrating to Xata is straightforward because the platform speaks pure PostgreSQL wire-protocol. You can start with a one-time dump/restore and, if you need *near-zero downtime*, finish the cut-over with real-time replication powered by our open-source project [**pgstream**](https://github.com/xataio/pgstream). + +--- + +## 1. Choose a migration strategy + +| Strategy | Downtime | When to use | +| --- | --- | --- | +| **Snapshot** – `pg_dump` → `pg_restore` | requires a maintenance window equal to dump + upload time | personal projects, non-critical services, small (< 50 GB) databases | +| **Continuous streaming** – snapshot + [**pgstream**](https://github.com/xataio/pgstream) logical replication | seconds | production workloads that cannot tolerate extended read-only periods | + +Regardless of the option you pick, the first step is identical: create a database branch in Xata that will receive the data. + +```bash +# create a dedicated migration branch so you can test safely +xata branch create migrate-prod + +# grab the connection URL +export XATA_MIGRATE_URL=$(xata branch url --branch migrate-prod) +``` + +--- + +## 2. One-time snapshot with `pg_dump` / `pg_restore` + +1. Dump your existing database (source Postgres, RDS, CloudSQL, etc.): + ```bash + pg_dump \ + --format=directory \ + --jobs=4 \ + --no-owner \ + --dbname=$SOURCE_DATABASE_URL \ + --file=/tmp/pgdump + ``` +2. Restore into the Xata branch: + ```bash + pg_restore \ + --jobs=4 \ + --clean \ + --if-exists \ + --dbname=$XATA_MIGRATE_URL \ + /tmp/pgdump + ``` +3. Validate data counts and run application smoke-tests against the branch. +4. Promote the branch to **main** (or switch your app's `DATABASE_URL`) once satisfied. + +> **Tip:** Large objects & extensions are included automatically because Xata runs unmodified Postgres. + +--- + +## 3. Near-zero-downtime migration with [**pgstream**](https://github.com/xataio/pgstream) + +If you run a 24×7 service or the dump window is too long, use [**pgstream**](https://github.com/xataio/pgstream) to replicate changes while users keep writing to the source database. + +### 3.1. Install pgstream + +```bash +go install github.com/xataio/pgstream/cmd/pgstream@latest +``` + +### 3.2. Prepare the source database + +Ensure **logical replication** is enabled: + +```sql +ALTER SYSTEM SET wal_level = logical; +ALTER SYSTEM SET max_replication_slots = 10; +SELECT pg_reload_conf(); +``` + +Create a replication user: + +```sql +CREATE ROLE pgstream REPLICATION LOGIN PASSWORD 'secret'; +``` + +### 3.3. Run the initial snapshot + streaming + +```bash +pgstream \ + --source $SOURCE_DATABASE_URL \ + --target $XATA_MIGRATE_URL \ + --slot pgstream_slot \ + --create-snapshot +``` + +The tool first copies a consistent snapshot (similar to `pg_basebackup`) and then switches to logical decoding to stream ongoing changes. + +You can monitor progress with: + +```bash +pgstream status --slot pgstream_slot --source $SOURCE_DATABASE_URL +``` + +### 3.4. Cut over + +1. Stop writes on the source (e.g. put the app in maintenance mode). +2. Wait for `pgstream` lag to reach **0 bytes**. +3. Point the application to the Xata branch URL and resume traffic. +4. Shut down `pgstream` and decommission the old cluster. + +Downtime is limited to the seconds it takes to restart the application. + +--- + +## 4. Post-migration housekeeping + +* **Apply [pgroll](https://github.com/xataio/pgroll) migrations** – initialise roll on the new database with `xata roll init`. +* **Enable anonymised staging branches** – `xata clone --anonymize` for preview environments. +* **Set up GitHub Actions** – see the [check merge readiness workflow](/github-actions/check). + +--- + +## 5. Need help? + +Our team is happy to review migration plans and help tune `pg_dump`, `pg_restore` or [**pgstream**](https://github.com/xataio/pgstream) settings. +Reach us at [support@xata.io](mailto:support@xata.io) or join the community Discord. diff --git a/docs/tutorials/tutorials.mdx b/docs/tutorials/tutorials.mdx new file mode 100644 index 0000000..74d62fd --- /dev/null +++ b/docs/tutorials/tutorials.mdx @@ -0,0 +1,57 @@ +--- +title: Tutorials +description: Step-by-step guides for common Xata workflows and use cases +--- + +Learn how to use Xata with step-by-step tutorials for common workflows and use cases. + +## Developer workflow + +[Developer workflow](/tutorials/developer-workflow) guides you through: +- Setting up development environments +- Creating and managing branches +- Testing schema changes +- Using GitHub Actions for automation + +## Migration guide + +[Migrate to Xata](/tutorials/migrate-to-xata) covers: +- Planning your migration +- Setting up your Xata environment +- Migrating your data +- Updating your application +- Validating the migration + +## Best practices + +### Development workflow +- Use branches for feature development +- Test schema changes in isolation +- Keep staging data anonymized +- Automate your workflow with GitHub Actions + +### Production deployment +- Use BYOC for production workloads +- Implement proper monitoring +- Set up automated backups +- Follow security best practices + +## Common use cases + +### Web applications +- Building a searchable blog +- Implementing user authentication +- Managing file uploads +- Real-time data updates + +### AI/ML projects +- Storing and querying embeddings +- Managing training data +- Implementing vector search +- Data anonymization for ML + +## Getting help + +- Join our [Discord community](https://discord.gg/xata) +- Check the [Troubleshooting & FAQ](/resources/troubleshooting) +- Contact [support](mailto:info@xata.io) \ No newline at end of file