From 92f45830507dde9b59e7668ab94eaf9db77236ec Mon Sep 17 00:00:00 2001
From: Andy Gayton <andy@thecablelounge.com>
Date: Wed, 8 Oct 2025 01:50:17 +0000
Subject: [PATCH 1/9] .

---
 book/metadata.md | 36 ++++++++++++++++++++++++++++++++++++
 cookbook/http.md | 29 +++++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)

diff --git a/book/metadata.md b/book/metadata.md
index 950a050e7ed..36f43a6e57d 100644
--- a/book/metadata.md
+++ b/book/metadata.md
@@ -37,3 +37,39 @@ metadata (open Cargo.toml) | get span
 ```
 
 The span "start" and "end" here refer to where the underline will be in the line. If you count over 5, and then count up to 15, you'll see it lines up with the "Cargo.toml" filename. This is how the error we saw earlier knew what to underline.
+
+## Custom Metadata
+
+You can attach arbitrary metadata to pipeline data using the [`metadata set`](/commands/docs/metadata_set.md) command with the `--merge` flag:
+
+```nu
+"data" | metadata set --merge {custom_key: "custom_value"}
+```
+
+This allows you to attach any key-value pairs to data flowing through the pipeline. To avoid key collisions between different commands and plugins, it's recommended to use namespaced keys with an underscore separator:
+
+- `"http_response"` - HTTP response metadata
+- `"polars_schema"` - DataFrame schema information
+- `"custom_plugin_field"` - Plugin-specific metadata
+
+This naming convention helps ensure different parts of the system don't overwrite each other's metadata.
+
+## HTTP Response Metadata
+
+All HTTP commands (`http get`, `http post`, etc.) automatically attach response metadata under the `"http_response"` key. This includes the status code, headers, and redirect history.
+
+You can access this metadata using the [`metadata access`](/commands/docs/metadata_access.md) command, which is especially useful for streaming large responses:
+
+```nu
+# Check status code while streaming the response
+http get https://api.example.com/large-file
+| metadata access {|meta|
+    if $meta.http_response.status != 200 {
+        error make {msg: "Request failed"}
+    } else { }
+  }
+| lines
+| each {|line| process $line }
+```
+
+The `else { }` clause passes the input through when the check succeeds, allowing the response body to stream through the pipeline. This pattern lets you fail fast on error responses without downloading the entire body.
diff --git a/cookbook/http.md b/cookbook/http.md
index 944f553641b..892d31348fb 100644
--- a/cookbook/http.md
+++ b/cookbook/http.md
@@ -255,3 +255,32 @@ http post https://httpbin.org/post --content-type "multipart/form-data" {
 # => │ url     │ https://httpbin.org/post                                                                                 │
 # => ╰─────────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────╯
 ```
+
+---
+
+### Checking HTTP Response Status While Streaming
+
+When working with large HTTP responses, you often want to stream the response body while still checking the HTTP status code. All HTTP commands automatically attach response metadata under the `"http_response"` key, which you can access using `metadata access`.
+
+```nu
+# Stream a large response while checking the status code
+http get https://api.example.com/large-stream
+| metadata access {|meta|
+    if $meta.http_response.status != 200 {
+        error make {msg: $"Request failed with status ($meta.http_response.status)"}
+    } else { }
+  }
+| lines
+| each {|line| process $line }
+```
+
+The `metadata access` command allows you to inspect metadata before the response body streams through. The `else { }` clause is required to pass the input through when the status check succeeds. This pattern is useful for:
+
+- Failing fast on error responses without downloading the entire body
+- Handling large payloads efficiently (the body still streams)
+- Checking response metadata without blocking the pipeline
+
+The response metadata includes:
+- `status` - HTTP status code (e.g., 200, 404, 500)
+- `headers` - Response headers as a list of `{name, value}` records
+- `urls` - Redirect history (list of URLs followed)

From 4fe711c8b63559921e5f313bbfbb481602af5de1 Mon Sep 17 00:00:00 2001
From: Andy Gayton <andy@thecablelounge.com>
Date: Wed, 8 Oct 2025 01:52:24 +0000
Subject: [PATCH 2/9] .

---
 book/metadata.md | 23 ++++++++++++++++++-----
 cookbook/http.md | 35 ++++++++++++++++++++---------------
 2 files changed, 38 insertions(+), 20 deletions(-)

diff --git a/book/metadata.md b/book/metadata.md
index 36f43a6e57d..b92d50a8284 100644
--- a/book/metadata.md
+++ b/book/metadata.md
@@ -56,20 +56,33 @@ This naming convention helps ensure different parts of the system don't overwrit
 
 ## HTTP Response Metadata
 
-All HTTP commands (`http get`, `http post`, etc.) automatically attach response metadata under the `"http_response"` key. This includes the status code, headers, and redirect history.
+All HTTP commands (`http get`, `http post`, etc.) attach response metadata:
 
-You can access this metadata using the [`metadata access`](/commands/docs/metadata_access.md) command, which is especially useful for streaming large responses:
+```nu
+http get https://api.example.com | metadata | get http_response
+# => ╭─────────┬───────────────────╮
+# => │ status  │ 200               │
+# => │ headers │ [list]            │
+# => │ urls    │ [list]            │
+# => ╰─────────┴───────────────────╯
+```
+
+The metadata includes:
+- `status` - HTTP status code (200, 404, 500, etc.)
+- `headers` - Response headers as `[{name, value}, ...]`
+- `urls` - Redirect history
+
+For large responses, use [`metadata access`](/commands/docs/metadata_access.md) to check metadata before the body downloads:
 
 ```nu
-# Check status code while streaming the response
 http get https://api.example.com/large-file
 | metadata access {|meta|
     if $meta.http_response.status != 200 {
         error make {msg: "Request failed"}
     } else { }
   }
-| lines
+| lines  # body streams through only if status is 200
 | each {|line| process $line }
 ```
 
-The `else { }` clause passes the input through when the check succeeds, allowing the response body to stream through the pipeline. This pattern lets you fail fast on error responses without downloading the entire body.
+The `else { }` passes the input through when the check succeeds.
diff --git a/cookbook/http.md b/cookbook/http.md
index 892d31348fb..df141f5ff8b 100644
--- a/cookbook/http.md
+++ b/cookbook/http.md
@@ -258,29 +258,34 @@ http post https://httpbin.org/post --content-type "multipart/form-data" {
 
 ---
 
-### Checking HTTP Response Status While Streaming
+### Accessing HTTP Response Metadata
 
-When working with large HTTP responses, you often want to stream the response body while still checking the HTTP status code. All HTTP commands automatically attach response metadata under the `"http_response"` key, which you can access using `metadata access`.
+All HTTP commands attach response metadata (status, headers, redirect history):
 
 ```nu
-# Stream a large response while checking the status code
-http get https://api.example.com/large-stream
+# After response completes
+http get https://api.example.com/data.json
+| metadata
+| get http_response.status
+# => 200
+```
+
+For large responses, check status *before* downloading the body:
+
+```nu
+# Fail fast without downloading body on error
+http get https://api.example.com/large-file.bin
 | metadata access {|meta|
     if $meta.http_response.status != 200 {
         error make {msg: $"Request failed with status ($meta.http_response.status)"}
     } else { }
   }
-| lines
-| each {|line| process $line }
+| save large-file.bin  # only runs if status is 200
 ```
 
-The `metadata access` command allows you to inspect metadata before the response body streams through. The `else { }` clause is required to pass the input through when the status check succeeds. This pattern is useful for:
-
-- Failing fast on error responses without downloading the entire body
-- Handling large payloads efficiently (the body still streams)
-- Checking response metadata without blocking the pipeline
+This checks metadata before the body streams through. If the status isn't 200, the error occurs immediately—the body is never downloaded.
 
-The response metadata includes:
-- `status` - HTTP status code (e.g., 200, 404, 500)
-- `headers` - Response headers as a list of `{name, value}` records
-- `urls` - Redirect history (list of URLs followed)
+Available metadata:
+- `status` - HTTP status code (200, 404, 500, etc.)
+- `headers` - `[{name, value}, ...]`
+- `urls` - Redirect history

From 2e6d59229f240b64278701c4a575b6c4bf3233c1 Mon Sep 17 00:00:00 2001
From: Andy Gayton <andy@thecablelounge.com>
Date: Wed, 8 Oct 2025 01:56:00 +0000
Subject: [PATCH 3/9] .

---
 book/metadata.md | 27 ++++++++++++---------------
 cookbook/http.md | 26 ++++++++++++++------------
 2 files changed, 26 insertions(+), 27 deletions(-)

diff --git a/book/metadata.md b/book/metadata.md
index b92d50a8284..12eee0bf98b 100644
--- a/book/metadata.md
+++ b/book/metadata.md
@@ -56,33 +56,30 @@ This naming convention helps ensure different parts of the system don't overwrit
 
 ## HTTP Response Metadata
 
-All HTTP commands (`http get`, `http post`, etc.) attach response metadata:
+All HTTP commands attach response metadata (status, headers, redirect history):
 
 ```nu
-http get https://api.example.com | metadata | get http_response
-# => ╭─────────┬───────────────────╮
-# => │ status  │ 200               │
-# => │ headers │ [list]            │
-# => │ urls    │ [list]            │
-# => ╰─────────┴───────────────────╯
+http get https://api.example.com | metadata | get http_response.status
+# => 200
 ```
 
-The metadata includes:
-- `status` - HTTP status code (200, 404, 500, etc.)
-- `headers` - Response headers as `[{name, value}, ...]`
-- `urls` - Redirect history
-
-For large responses, use [`metadata access`](/commands/docs/metadata_access.md) to check metadata before the body downloads:
+To work with metadata while streaming the response body, use [`metadata access`](/commands/docs/metadata_access.md):
 
 ```nu
 http get https://api.example.com/large-file
 | metadata access {|meta|
+    print $"Status: ($meta.http_response.status)"
     if $meta.http_response.status != 200 {
         error make {msg: "Request failed"}
     } else { }
   }
-| lines  # body streams through only if status is 200
+| lines  # body streams through
 | each {|line| process $line }
 ```
 
-The `else { }` passes the input through when the check succeeds.
+Without `metadata access`, you'd need `--full` to get metadata, which consumes the entire response body and prevents streaming. With `metadata access`, the body continues streaming through the pipeline.
+
+Metadata structure:
+- `status` - HTTP status code (200, 404, 500, etc.)
+- `headers` - Response headers as `[{name, value}, ...]`
+- `urls` - Redirect history
diff --git a/cookbook/http.md b/cookbook/http.md
index df141f5ff8b..bdbdd1e76ab 100644
--- a/cookbook/http.md
+++ b/cookbook/http.md
@@ -258,32 +258,34 @@ http post https://httpbin.org/post --content-type "multipart/form-data" {
 
 ---
 
-### Accessing HTTP Response Metadata
+### Accessing HTTP Response Metadata While Streaming
 
-All HTTP commands attach response metadata (status, headers, redirect history):
+All HTTP commands attach response metadata. To access it after the response completes:
 
 ```nu
-# After response completes
-http get https://api.example.com/data.json
-| metadata
-| get http_response.status
+http get https://api.example.com/data.json | metadata | get http_response.status
 # => 200
 ```
 
-For large responses, check status *before* downloading the body:
+To work with metadata while streaming the response body, use `metadata access`. This is useful for handling unexpected responses (errors, redirects, content negotiation) without consuming the entire body:
 
 ```nu
-# Fail fast without downloading body on error
-http get https://api.example.com/large-file.bin
+# Log status and headers while streaming a large file
+http get https://api.example.com/large-dataset.csv
 | metadata access {|meta|
+    print $"Status: ($meta.http_response.status)"
+    print $"Content-Type: ($meta.http_response.headers | where name == content-type | get value.0)"
+
     if $meta.http_response.status != 200 {
-        error make {msg: $"Request failed with status ($meta.http_response.status)"}
+        error make {msg: $"Failed with status ($meta.http_response.status)"}
     } else { }
   }
-| save large-file.bin  # only runs if status is 200
+| lines
+| from csv
+| where amount > 1000
 ```
 
-This checks metadata before the body streams through. If the status isn't 200, the error occurs immediately—the body is never downloaded.
+The response body streams through the pipeline—you can inspect metadata and process the stream simultaneously. Before `metadata access`, you needed `--full` to get metadata, which consumed the entire body and prevented streaming.
 
 Available metadata:
 - `status` - HTTP status code (200, 404, 500, etc.)

From cb2e5ac7932de46fa921bbac64a7d38147b09191 Mon Sep 17 00:00:00 2001
From: Andy Gayton <andy@thecablelounge.com>
Date: Wed, 8 Oct 2025 01:57:30 +0000
Subject: [PATCH 4/9] .

---
 cookbook/http.md | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/cookbook/http.md b/cookbook/http.md
index bdbdd1e76ab..1c6a082cac1 100644
--- a/cookbook/http.md
+++ b/cookbook/http.md
@@ -270,8 +270,8 @@ http get https://api.example.com/data.json | metadata | get http_response.status
 To work with metadata while streaming the response body, use `metadata access`. This is useful for handling unexpected responses (errors, redirects, content negotiation) without consuming the entire body:
 
 ```nu
-# Log status and headers while streaming a large file
-http get https://api.example.com/large-dataset.csv
+# Log status and headers while streaming a large JSONL file
+http get https://api.example.com/events.jsonl
 | metadata access {|meta|
     print $"Status: ($meta.http_response.status)"
     print $"Content-Type: ($meta.http_response.headers | where name == content-type | get value.0)"
@@ -281,8 +281,8 @@ http get https://api.example.com/large-dataset.csv
     } else { }
   }
 | lines
-| from csv
-| where amount > 1000
+| each { from json }
+| where event_type == "error"
 ```
 
 The response body streams through the pipeline—you can inspect metadata and process the stream simultaneously. Before `metadata access`, you needed `--full` to get metadata, which consumed the entire body and prevented streaming.

From 8cf028e9b9e057e4c58347c49ba8f739d08b8d43 Mon Sep 17 00:00:00 2001
From: Andy Gayton <andy@thecablelounge.com>
Date: Wed, 8 Oct 2025 02:00:31 +0000
Subject: [PATCH 5/9] .

---
 book/metadata.md | 8 --------
 cookbook/http.md | 2 +-
 2 files changed, 1 insertion(+), 9 deletions(-)

diff --git a/book/metadata.md b/book/metadata.md
index 12eee0bf98b..212c45298e2 100644
--- a/book/metadata.md
+++ b/book/metadata.md
@@ -46,14 +46,6 @@ You can attach arbitrary metadata to pipeline data using the [`metadata set`](/c
 "data" | metadata set --merge {custom_key: "custom_value"}
 ```
 
-This allows you to attach any key-value pairs to data flowing through the pipeline. To avoid key collisions between different commands and plugins, it's recommended to use namespaced keys with an underscore separator:
-
-- `"http_response"` - HTTP response metadata
-- `"polars_schema"` - DataFrame schema information
-- `"custom_plugin_field"` - Plugin-specific metadata
-
-This naming convention helps ensure different parts of the system don't overwrite each other's metadata.
-
 ## HTTP Response Metadata
 
 All HTTP commands attach response metadata (status, headers, redirect history):
diff --git a/cookbook/http.md b/cookbook/http.md
index 1c6a082cac1..b80f4520427 100644
--- a/cookbook/http.md
+++ b/cookbook/http.md
@@ -267,7 +267,7 @@ http get https://api.example.com/data.json | metadata | get http_response.status
 # => 200
 ```
 
-To work with metadata while streaming the response body, use `metadata access`. This is useful for handling unexpected responses (errors, redirects, content negotiation) without consuming the entire body:
+To work with metadata while streaming the response body, use `metadata access`:
 
 ```nu
 # Log status and headers while streaming a large JSONL file

From 3754efee766a009034aa33cfd9aef7b55d89255a Mon Sep 17 00:00:00 2001
From: Andy Gayton <andy@thecablelounge.com>
Date: Wed, 8 Oct 2025 02:04:01 +0000
Subject: [PATCH 6/9] .

---
 cookbook/http.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cookbook/http.md b/cookbook/http.md
index b80f4520427..af040ef6008 100644
--- a/cookbook/http.md
+++ b/cookbook/http.md
@@ -285,7 +285,7 @@ http get https://api.example.com/events.jsonl
 | where event_type == "error"
 ```
 
-The response body streams through the pipeline—you can inspect metadata and process the stream simultaneously. Before `metadata access`, you needed `--full` to get metadata, which consumed the entire body and prevented streaming.
+The response body streams through the pipeline while you inspect metadata and process the stream simultaneously. Before `metadata access`, you needed `--full` to get metadata, which consumed the entire body and prevented streaming.
 
 Available metadata:
 - `status` - HTTP status code (200, 404, 500, etc.)

From 8ceb1fc721e1d50ff62491cf68e023d30bdd5936 Mon Sep 17 00:00:00 2001
From: Andy Gayton <andy@thecablelounge.com>
Date: Wed, 8 Oct 2025 02:16:40 +0000
Subject: [PATCH 7/9] .

---
 book/metadata.md | 1 +
 cookbook/http.md | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/book/metadata.md b/book/metadata.md
index 212c45298e2..ee4d23a380d 100644
--- a/book/metadata.md
+++ b/book/metadata.md
@@ -72,6 +72,7 @@ http get https://api.example.com/large-file
 Without `metadata access`, you'd need `--full` to get metadata, which consumes the entire response body and prevents streaming. With `metadata access`, the body continues streaming through the pipeline.
 
 Metadata structure:
+
 - `status` - HTTP status code (200, 404, 500, etc.)
 - `headers` - Response headers as `[{name, value}, ...]`
 - `urls` - Redirect history
diff --git a/cookbook/http.md b/cookbook/http.md
index af040ef6008..48cd92db1ae 100644
--- a/cookbook/http.md
+++ b/cookbook/http.md
@@ -186,7 +186,7 @@ To upload a form with a file (think a common file upload form in a browser, wher
 
 1. Specify the content type as `multipart/form-data`
 2. Provide the record as the POST body
-3. Provide the file data in one of the record fields as *binary* data.
+3. Provide the file data in one of the record fields as _binary_ data.
 
 ```nu
 http post https://httpbin.org/post --content-type "multipart/form-data" {
@@ -288,6 +288,7 @@ http get https://api.example.com/events.jsonl
 The response body streams through the pipeline while you inspect metadata and process the stream simultaneously. Before `metadata access`, you needed `--full` to get metadata, which consumed the entire body and prevented streaming.
 
 Available metadata:
+
 - `status` - HTTP status code (200, 404, 500, etc.)
 - `headers` - `[{name, value}, ...]`
 - `urls` - Redirect history

From f8c7724d6cded7dec8d8f653d4431519fad1840e Mon Sep 17 00:00:00 2001
From: Andy Gayton <andy@thecablelounge.com>
Date: Wed, 8 Oct 2025 02:29:17 +0000
Subject: [PATCH 8/9] .

---
 book/metadata.md | 24 ++----------------------
 1 file changed, 2 insertions(+), 22 deletions(-)

diff --git a/book/metadata.md b/book/metadata.md
index ee4d23a380d..f6b5ddcb52d 100644
--- a/book/metadata.md
+++ b/book/metadata.md
@@ -48,31 +48,11 @@ You can attach arbitrary metadata to pipeline data using the [`metadata set`](/c
 
 ## HTTP Response Metadata
 
-All HTTP commands attach response metadata (status, headers, redirect history):
+All HTTP commands attach response metadata:
 
 ```nu
 http get https://api.example.com | metadata | get http_response.status
 # => 200
 ```
 
-To work with metadata while streaming the response body, use [`metadata access`](/commands/docs/metadata_access.md):
-
-```nu
-http get https://api.example.com/large-file
-| metadata access {|meta|
-    print $"Status: ($meta.http_response.status)"
-    if $meta.http_response.status != 200 {
-        error make {msg: "Request failed"}
-    } else { }
-  }
-| lines  # body streams through
-| each {|line| process $line }
-```
-
-Without `metadata access`, you'd need `--full` to get metadata, which consumes the entire response body and prevents streaming. With `metadata access`, the body continues streaming through the pipeline.
-
-Metadata structure:
-
-- `status` - HTTP status code (200, 404, 500, etc.)
-- `headers` - Response headers as `[{name, value}, ...]`
-- `urls` - Redirect history
+For working with metadata while streaming response bodies, see the [HTTP cookbook](/cookbook/http.html#accessing-http-response-metadata-while-streaming).

From 6d587ab2ff9318b2a79fdc7e8ab8a16b0e1b8b6d Mon Sep 17 00:00:00 2001
From: Andy Gayton <andy@thecablelounge.com>
Date: Thu, 9 Oct 2025 14:37:37 +0000
Subject: [PATCH 9/9] Add --allow-errors to http status check example

---
 cookbook/http.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cookbook/http.md b/cookbook/http.md
index 48cd92db1ae..df671aaec1b 100644
--- a/cookbook/http.md
+++ b/cookbook/http.md
@@ -271,7 +271,7 @@ To work with metadata while streaming the response body, use `metadata access`:
 
 ```nu
 # Log status and headers while streaming a large JSONL file
-http get https://api.example.com/events.jsonl
+http get --allow-errors https://api.example.com/events.jsonl
 | metadata access {|meta|
     print $"Status: ($meta.http_response.status)"
     print $"Content-Type: ($meta.http_response.headers | where name == content-type | get value.0)"