Go/Xorm examples with separate go package for helper functions

.gitignore

@@ -105,6 +105,7 @@ web_modules/
 .env.test.local
 .env.production.local
 .env.local
+.envrc
 
 # parcel-bundler cache (https://parceljs.org/)
 

GETTINGSTARTED.md

@@ -0,0 +1,139 @@
+## Getting started
+
+The following guide assumes you have the prerequisites installed and running, and are running the SQL statements through your CipherStash Proxy instance.
+
+### Prerequisites
+
+- [PostgreSQL 14+](https://www.postgresql.org/download/)
+- [Cipherstash Proxy guide](https://github.com/cipherstash/encrypt-query-language/tree/main/cipherstash-proxy)
+
+EQL relies on [Cipherstash Proxy](https://cipherstash.com/docs/getting-started/cipherstash-proxy) for low-latency encryption & decryption.
+We plan to support direct language integration in the future.
+
+> This guide will use raw SQL statements to demonstrate how to use the EQL extension. See the `languages` directory for more information on using specific language specific ORMs.
+
+### Installation
+
+In order to use EQL, you must first install the EQL extension in your PostgreSQL database.
+You can do this by running the following command, which will execute the SQL from the `src/install.sql` file:
+
+Update the database credentials based on your environment.
+
+```bash
+psql -U postgres -d postgres -f src/install.sql
+```
+
+> Note: We also have direct language specific ORM support for installing the extension. See the `languages` directory for more information.
+
+### Add a table with encrypted columns
+
+Create a table with encrypted columns.
+For this example, we'll use the `users` table, with a plaintext `email` column and an encrypted `email_encrypted` column.
+
+```sql
+CREATE TABLE IF NOT EXISTS "users" (
+	"id" serial PRIMARY KEY NOT NULL,
+	"email" varchar,
+	"email_encrypted" "cs_encrypted_v1"
+);
+```
+
+Some language specific ORMs don't support custom types, so EQL also supports `jsonb` rather than the `cs_encrypted_v1` domain type.
+
+### Inserting data
+
+When inserting data into the encrypted column, you must wrap the plaintext in the appropriate EQL payload.
+These statements must be run through the CipherStash Proxy in order to **encrypt** the data.
+
+```sql
+INSERT INTO users (email_encrypted) VALUES ('{"v":1,"k":"pt","p":"test@test.com","i":{"t":"users","c":"email_encrypted"}}');
+```
+
+For reference, the EQL payload is defined as a `jsonb` with a specific schema:
+
+```json
+{
+  "v": 1,
+  "k": "pt",
+  "p": "test@test.com",
+  "i": {
+    "t": "users",
+    "c": "email_encrypted"
+  }
+}
+```
+
+### Querying data
+
+When querying data, you must wrap the encrypted column in the appropriate EQL payload.
+These statements must be run through the CipherStash Proxy in order to **decrypt** the data.
+
+```sql
+SELECT email_encrypted FROM users;
+```
+
+The returned value is a `json` object with the decrypted value located in the `p` field.
+
+```json
+{
+  "v": 1,
+  "k": "ct",
+  "p": "test@test.com",
+  "i": {
+    "t": "users",
+    "c": "email_encrypted"
+  }
+}
+```
+
+### Adding an index and enable encryption
+
+To add an index to the encrypted column, you must run the `cs_add_index_v1` function.
+This function takes the following parameters:
+
+- `table_name`: The name of the table containing the encrypted column.
+- `column_name`: The name of the encrypted column.
+- `index_name`: The name of the index.
+- `cast_as`: The type of the index (text, int, small_int, big_int, real, double, boolean, date, jsonb).
+- `opts`: An optional JSON object containing additional index options.
+
+For the example above, and using a match index, the following statement would be used:
+
+```sql
+SELECT cs_add_index_v1('users', 'email_encrypted', 'match', 'text', '{"token_filters": [{"kind": "downcase"}], "tokenizer": { "kind": "ngram", "token_length": 3 }}');
+```
+
+Once you have added an index, you must enable encryption.
+This will update the encryption configuration to include the new index.
+
+```sql
+SELECT cs_encrypt_v1();
+SELECT cs_activate_v1();
+```
+
+In this example `cs_encrypt_v1` and `cs_activate_v1` are called to immediately set the new Encrypt config to **active** for demonstration purposes.
+In a production environment, you will need to consider a migration strategy to ensure the encryption config is updated based on the current state of the database.
+
+See the [reference guide on migrations](https://cipherstash.com/docs/getting-started/cipherstash-encrypt#migrations) for more information.
+
+### Removing an index and disabling encryption
+
+To remove an index from the encrypted column, you must run the `cs_remove_index_v1` function.
+This function takes the following parameters:
+
+- `table_name`: The name of the table containing the encrypted column.
+- `column_name`: The name of the encrypted column.
+- `index_name`: The name of the index.
+
+For the example above, and using a match index, the following statement would be used:
+
+```sql
+SELECT cs_remove_index_v1('users', 'email_encrypted', 'match');
+```
+
+Once you have removed an index, you must disable encryption.
+This will update the encryption configuration to exclude the removed index.
+
+```sql
+SELECT cs_encrypt_v1();
+```

PROXY.md

@@ -0,0 +1,90 @@
+# CipherStash Proxy
+
+The CipherStash Proxy is a lightweight proxy that can be used to encrypt and decrypt data in your database.
+
+## Table of Contents
+
+- [Getting Started](#getting-started)
+- [Uploading an empty dataset](#uploading-an-empty-dataset)
+- [Configuring CipherStash Proxy](#configuring-cipherstash-proxy)
+- [Running the Proxy](#running-the-proxy)
+- [Using the Proxy](#using-the-proxy)
+- [How EQL works with CipherStash Proxy](#how-eql-works-with-cipherstash-proxy)
+  - [Writes](#writes)
+  - [Reads](#reads)
+
+## Getting Started
+
+To get started, you'll need to sign up for a free account at [https://dashboard.cipherstash.com](https://dashboard.cipherstash.com).
+
+Once you've signed up, you can create an access key from your default workspace.
+
+## Uploading an empty dataset
+
+Before you can start using the proxy, you'll need to upload an empty dataset. You can do this using the [CipherStash CLI](https://cipherstash.com/docs/reference/cli)
+
+1. [Create a dataset.](https://cipherstash.com/docs/how-to/creating-datasets)
+1. [Create a client key for cryptographic operations.](https://cipherstash.com/docs/how-to/creating-clients)
+
+```bash
+stash datasets config upload --file dataset.yml --client-id client_id --client-key client_key
+```
+
+> Note: We plan to remove the need for this step in the future.
+
+This will upload an empty dataset to your default workspace.
+
+## Configuring CipherStash Proxy
+
+You can then create a `cipherstash-proxy.toml` file in the root of this directory. You can use the `cipherstash-proxy.toml.example` file as a starting point.
+
+Populate the following fields with your values:
+
+- `workspace_id`: The ID of your workspace.
+- `client_access_key`: The access key for your client.
+- `client_id`: The ID of your client.`
+- `client_key`: The key of your client.
+- `database.name`: The name of your database.
+- `database.username`: The username for your database.
+- `database.password`: The password for your database.
+- `database.host`: The host for your database.
+- `database.port`: The port for your database.
+
+## Running the Proxy
+
+To run the proxy, you can use the `start.sh` script in this directory. This script will start the proxy using the configuration in the `cipherstash-proxy.toml` file.
+
+```bash
+./start.sh
+```
+
+## Using the Proxy
+
+Once the proxy is running, you can use the different language examples to test the proxy and EQL.
+
+## How EQL works with CipherStash Proxy
+
+EQL uses **CipherStash Proxy** to mediate access to your PostgreSQL database and provide low-latency encryption & decryption.
+
+At a high level:
+
+- encrypted data is stored as `jsonb`
+- references to the column in sql statements are wrapped in a helper function
+- Cipherstash Proxy transparently encrypts and indexes data
+
+### Writes
+
+1. Database client sends `plaintext` data encoded as `jsonb`
+1. CipherStash Proxy encrypts the `plaintext` and encodes the `ciphertext` value and associated indexes into the `jsonb` payload
+1. The data is written to the encrypted column
+
+![Insert](../diagrams/overview-insert.drawio.svg)
+
+### Reads
+
+1. Wrap references to the encrypted column in the appropriate EQL function
+1. CipherStash Proxy encrypts the `plaintext`
+1. PostgreSQL executes the SQL statement
+1. CipherStash Proxy decrypts any returned `ciphertext` data and returns to client
+
+![Select](../diagrams/overview-select.drawio.svg)