Zero 0.18

Author: aboodman

contents/docs/auth/index.mdx renamed to contents/docs/auth.mdx

@@ -0,0 +1,64 @@
+---
+title: Inspector API
+---
+
+The Zero instance provides an API to gather information about the client's current state, such as:
+
+* All active queries
+* Query TTL
+* Active clients
+* Client database contents
+
+This can help figuring out why you hit loading states, how many queries are active at a time, if you have any resource leaks due to failing to clean up queries or if expected data is missing on the client.
+
+## Creating an Inspector
+
+Each `Zero` instance has an `inspect` method that will return the inspector.
+
+```ts
+const z = new Zero({
+  /*your zero options*/
+});
+const inspector = await z.inspect();
+```
+
+Once you have an inspector you can inspect the current client and client group.
+
+<Note heading="Clients and Client Groups">
+  The client group represents a web browser profile. A client represents an
+  instance of `Zero` within that profile. If a user has Chrome open with 5
+  different tabs, each with one `Zero` instance, there will be 1 client
+  group and 5 clients.
+</Note>
+
+For example to see active queries for the current client:
+
+```ts
+console.table(await inspector.client.queries());
+```
+
+To inspect other clients within the group:
+
+```ts
+const allClients = await inspector.clients();
+```
+
+## Dumping Data
+
+In addition to information about queries, you can see the contents of the client side database.
+
+```ts
+const inspector = await zero.inspect();
+const client = inspector.client;
+
+// All raw k/v data currently synced to client
+console.log('client map:');
+console.log(await client.map());
+
+// kv table extracted into tables
+// This is same info that is in z.query[tableName].run()
+for (const tableName of Object.keys(schema.tables)) {
+  console.log(`table ${tableName}:`);
+  console.table(await client.rows(tableName));
+}
+```

contents/docs/community/index.mdx renamed to contents/docs/community.mdx

@@ -0,0 +1,35 @@
+---
+title: Debugging Permissions
+---
+
+Given that permissions are defined in their own file and internally applied to queries, it might be hard to figure out if or why a permission check is failing.
+
+## Read Permissions
+
+The `transform-query` utility is provided to transform a query by applying permissions to it. As of today you'll need to provide the hash of the query you want to transform. You can find this in server logs, websocket network inspector, or in the CVR database. In a future release, you'll be able to write ZQL directly.
+
+```ts
+npx transform-query --hash=2i81bazy03a00 --schema=./shared/schema.ts
+```
+
+The output will be the ZQL query with permissions applied as well as the AST of that query.
+
+<Note type="note">
+  The printed AST is slightly different than the source ZQL string as it
+  leverages internal APIs to prevent syncing rows that are used strictly for
+  permission checks.
+</Note>
+
+## Write Permissions
+
+Look for a `WARN` level log in the output from `zero-cache` like this:
+
+```
+Permission check failed for {"op":"update","tableName":"message",...}, action update, phase preMutation, authData: {...}, rowPolicies: [...], cellPolicies: []
+```
+
+Zero prints the row, auth data, and permission policies that was applied to any failed writes.
+
+<Note>
+The ZQL query is printed in AST format. See [Query ASTs](./query-asts) to convert it to a more readable format.
+</Note>

contents/docs/connecting-to-postgres/index.mdx renamed to contents/docs/connecting-to-postgres.mdx

@@ -0,0 +1,26 @@
+---
+title: Query ASTs
+---
+
+An AST (Abstract Syntax Tree) is a representation of a query that is used internally by Zero. It is not meant to be human readable, but it sometimes shows up in logs and other places.
+
+If you need to read one of these, save the AST to a json file. Then run the following command:
+
+```bash
+cat ast.json | npx ast-to-zql
+```
+
+The returned ZQL query will be using server names, rather than client names, to identify columns and tables.
+If you provide the schema file as an option you will get mapped back to client names:
+
+```bash
+cat ast.json | npx ast-to-zql --schema schema.ts
+```
+
+This comes into play if, in your schema.ts, you use the `from` feature to have different names on the client than your backend DB.
+
+<Note type="note">
+  The `ast-to-zql` process is a de-compilation of sorts. Given that, the ZQL
+  string you get back will not be identical to the one you wrote in your
+  application. Regardless, the queries will be semantically equivalent.
+</Note>

contents/docs/custom-mutators.mdx

@@ -0,0 +1,43 @@
+---
+title: Replication
+---
+
+## Resetting
+
+During development we all do strange things (unsafely changing schemas, removing files, etc.). If the replica ever gets wedged (stops replicating, acts strange) you can wipe it and start over.
+
+- If you copied your setup from `hello-zero` or `hello-zero-solid`, you can also run `npm run dev:clean`
+- Otherwise you can run `rm /tmp/my-zero-replica.db*` (see your `.env` file for the replica file location) to clear the contents of the replica.
+
+It is always safe to wipe the replica. Wiping will have no impact on your upstream database. Downstream zero-clients will get re-synced when they connect.
+
+## Inspecting
+
+For data to be synced to the client it must first be replicated to `zero-cache`. You can check the contents of `zero-cache` via:
+
+```bash
+$ npx zero-sqlite3 /tmp/my-zero-replica.db
+```
+
+This will drop you into a `sqlite3` shell with which you can use to explore the contents of the replica.
+
+```sql
+sqlite> .tables
+_zero.changeLog          emoji                    viewState
+_zero.replicationConfig  issue                    zero.permissions
+_zero.replicationState   issueLabel               zero.schemaVersions
+_zero.runtimeEvents      label                    zero_0.clients
+_zero.versionHistory     user
+comment                  userPref
+sqlite> .mode qbox
+sqlite> SELECT * FROM label;
+┌─────────────────────────┬──────────────────────────┬────────────┐
+│           id            │           name           │ _0_version │
+├─────────────────────────┼──────────────────────────┼────────────┤
+│ 'ic_g-DZTYDApZR_v7Cdcy' │ 'bug'                    │ '4ehreg'   │
+...
+```
+
+## Miscellaneous
+
+If you see `FATAL:  sorry, too many clients already` in logs, it’s because you have two zero-cache instances running against dev. One is probably in a background tab somewhere. In production, `zero-cache` can run horizontally scaled but on dev it doesn’t run in the config that allows that.

contents/docs/debug/inspector.mdx

@@ -0,0 +1,68 @@
+---
+title: Slow Queries (zero-cache)
+---
+
+In the `zero-cache` logs, you may see statements indicating a query is slow:
+
+```shell
+{
+    "level": "DEBUG",
+    "worker": "syncer",
+    "component": "view-syncer",
+    "hydrationTimeMs": 1339,
+    "message": "Total rows considered: 146"
+  },
+```
+
+or:
+
+```shell
+hash=3rhuw19xt9vry transformationHash=1nv7ot74gxfl7
+Slow query materialization 325.46865100000286
+```
+
+Here are some tips to help debug such slow queries.
+
+## Check Storage
+
+`zero-cache` is effectively a database. It requires fast (low latency and high bandwidth) disk access to perform well. If you're running on network attached storage with high latency, or on AWS with low IOPS, then this is the most likely culprit.
+
+The default deployment of Zero currently uses Fargate which scales IOPS with vCPU. Increasing the vCPU will increase storage throughput and likely resolve the issue.
+
+Fly.io provides physically attached SSDs, even for their smallest VMs. Deploying zero-cache there (or any other provider that offers physically attached SSDs) is another option.
+
+## Locality
+
+If you see log lines like:
+
+```shell
+flushed cvr ... (124ms)
+```
+
+this indicates that `zero-cache` is likely deployed too far away from your [CVR database](../deployment#architecture). If you did not configure a CVR database URL then this will be your product's Postgres DB. A slow CVR flush can slow down Zero, since it must complete the flush before sending query result(s) to clients.
+
+Try moving `zero-cache` to be deployed as close as possible to the CVR database.
+
+## Query Plan
+
+If neither (1) nor (2) is a problem, then the query itself is the most likely culprit. The `@rocicorp/zero` package ships with a query analyzer to help debug this.
+
+The analyzer should be run in the directory that contains the `.env` file for `zero-cache` as it will use the `.env` file to find your replica.
+
+Example:
+
+```shell
+npx analyze-query \
+  --schema=./shared/schema.ts \
+  --query='issue.related("comments")'
+```
+
+This will output the query plan and time to execute each phase of that plan.
+
+If you're unsure which query is slow, or you do not know what it looks like after permissions have been applied, you can obtain the query from the hash that is output in the server logs (e.g., `hash=3rhuw19xt9vry`):
+
+```shell
+npx transform-query --hash=2i81bazy03a00 --schema=./shared/schema.ts
+```
+
+This will find the query with that hash in the CVR DB and output the ZQL for that query, with all read permissions applied. You can then feed this output into `analyze-query`.