Merge branch 'main' into fil/validate-sql-frontmatter

observablehq · Mar 24, 2024 · 885534d · 885534d
2 parents abbe0a4 + 8011884
commit 885534d
Show file tree

Hide file tree

Showing 79 changed files with 451 additions and 300 deletions.
diff --git a/docs/config.md b/docs/config.md
@@ -112,34 +112,41 @@ Whether to show the sidebar. Defaults to true if **pages** is not empty.
 
 ## pages
 
-An array containing pages and/or sections. If not specified, it defaults to all Markdown files found in the source root in directory listing order.
+An array containing pages and sections. If not specified, it defaults to all Markdown files found in the source root in directory listing order.
 
-The following TypeScript interfaces describe pages and sections:
+Both pages and sections have a **name**, which typically corresponds to the page’s title. The name gets displayed in the sidebar. Clicking on a page navigates to the corresponding **path**, which should start with a leading slash and be relative to the root; the path can also be specified as a full URL to navigate to an external site. Clicking on a section header opens or closes that section. Each section must specify an array of **pages**, and optionally whether the section is **open** by default. If **open** is not set, it defaults to true. If **open** is false, the section is closed unless the current page belongs to that section.
 
-```ts run=false
-export interface Page {
-  name: string;
-  path: string;
-}
-```
+For example, here **pages** specifies two sections and a total of four pages:
 
-```ts run=false
-export interface Section {
-  name: string;
-  pages: Page[];
-  open?: boolean;
+```js run=false
+export default {
+  pages: [
+    {
+      name: "Section 1",
+      pages: [
+        {name: "Page 1", path: "/s01/page1"},
+        {name: "Page 2", path: "/s01/page2"}
+      ]
+    },
+    {
+      name: "Section 2",
+      open: false,
+      pages: [
+        {name: "Page 3", path: "/s02/page3"},
+        {name: "Page 4", path: "/s02/page4"}
+      ]
+    }
+  ]
 }
 ```
 
-If a section’s **open** option is not set, it defaults to true.
-
-Projects can have “unlisted” pages that are not included in the pages list. These pages will still be accessible if linked from other pages or visited directly, but they won’t be listed in the sidebar or linked to via the previous & next footer.
+Projects can have “unlisted” pages that are not referenced in **pages**. These pages can still be linked from other pages or visited directly, but they won’t be listed in the sidebar or linked to via the previous & next pager links.
 
-The pages list should _not_ include the root page, `index.md`. Also, we don’t recommend using query strings or anchor fragments, as these will prevent the previous & next footer links from navigating.
+The pages list should _not_ include the home page (`/`) as this is automatically linked at the top of the sidebar. We also do not recommend listing the same page multiple times (say with different query parameters or anchor fragments), as this causes the previous & next pager links to cycle.
 
 ## pager
 
-Whether to show the previous & next footer links; defaults to true.
+Whether to show the previous & next links in the footer; defaults to true. The pages are linked in the same order as they appear in the sidebar.
 
 ## head
 

diff --git a/docs/deploying.md b/docs/deploying.md
@@ -28,6 +28,8 @@ When the deploy command finishes, it prints a link to observablehq.cloud where y
 
 <div class="note">The deploy command creates a file at <code>docs/.observablehq/deploy.json</code> with information on where to deploy the project. This file is required for automated deploys. You will need to commit this file to git to deploy via GitHub Actions. (If you have configured a source root besides <code>docs</code>, the file will be placed there instead.)</div>
 
+<div class="tip">To see more available options when deploying:<pre><code class="language-sh">npm run deploy -- --help</code></pre></div>
+
 ## Automated deploys
 
 To set up automatic deploys (also known as *continuous deployment* or *CD*), we recommend [GitHub Actions](https://github.com/features/actions). In your git repository, create and commit a file at `.github/workflows/deploy.yml`. Here is a starting example:

diff --git a/docs/display-race.md b/docs/display-race.md
diff --git a/docs/getting-started.md b/docs/getting-started.md
@@ -267,7 +267,7 @@ display(1 + 2);
 ```
 ````
 
-To see the new page in the sidebar, you must restart the preview server. In the terminal, use Control-C (⌃C) to kill the preview server. Then use up arrow (↑) to re-run the command to start the preview server (`npm run dev` or `yarn dev`). Lastly, reload your browser. A bit of rigamarole, but you won’t have to do it often… 😓 Upvote <a href="https://github.com/observablehq/framework/issues/645">#645</a> and <a href="https://github.com/observablehq/framework/issues/646">#646</a> if you’d like this to be better.
+To see the new page in the sidebar, reload the page.
 
 If you click on the **Weather report** link in the sidebar, it’ll take you to <http://127.0.0.1:3000/weather>, where you should see:
 

diff --git a/docs/javascript/files.md b/docs/javascript/files.md
@@ -10,7 +10,7 @@ Load files — whether static or generated dynamically by a [data loader](../loa
 import {FileAttachment} from "npm:@observablehq/stdlib";
 ```
 
-The `FileAttachment` function takes a path and returns a file handle. This handle exposes the file’s name, [MIME type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types), and last modification date as the number of milliseconds since UNIX epoch.
+The `FileAttachment` function takes a path and returns a file handle. This handle exposes the file’s name, [MIME type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types), and modification time <a href="https://github.com/observablehq/framework/releases/tag/v1.4.0" target="_blank" class="observablehq-version-badge" data-version="^1.4.0" title="Added in 1.4.0"></a> (represented as the number of milliseconds since UNIX epoch).
 
 ```js echo
 FileAttachment("volcano.json")
@@ -99,10 +99,10 @@ For missing files, `file.lastModified` is undefined. The `file.mimeType` is dete
 
 The contents often dictate the appropriate method — for example, an Apache Arrow file is almost always read with `file.arrow`. When multiple methods are valid, choose based on your needs. For example, you can load a CSV file using `file.text` to implement parsing yourself instead of using D3.
 
-In addition to the above, you can get the resolved absolute URL of the file using `file.url`. This returns a [promise](./promises) to a string:
+In addition to the above, you can get the resolved absolute URL of the file using `file.href`:
 
 ```js echo
-FileAttachment("volcano.json").url()
+FileAttachment("volcano.json").href
 ```
 
 See [file-based routing](../routing#files) for additional details.

diff --git a/docs/lib/duckdb.md b/docs/lib/duckdb.md
@@ -55,7 +55,7 @@ Plot.plot({
 })
 ```
 
-You can also [attach](https://duckdb.org/docs/sql/statements/attach) a complete database saved as DuckDB file, typically using the `.db` file extension (or `.ddb` or `.duckdb`). In this case, the associated name (below `base`) is a _schema_ name rather than a _table_ name.
+You can also [attach](https://duckdb.org/docs/sql/statements/attach) a complete database saved as DuckDB file, <a href="https://github.com/observablehq/framework/releases/tag/v1.4.0" target="_blank" class="observablehq-version-badge" data-version="^1.4.0" title="Added in 1.4.0"></a> typically using the `.db` file extension (or `.ddb` or `.duckdb`). In this case, the associated name (below `base`) is a _schema_ name rather than a _table_ name.
 
 ```js echo
 const db2 = await DuckDBClient.of({base: FileAttachment("quakes.db")});
@@ -92,7 +92,7 @@ db.queryRow("SELECT count() AS count FROM gaia")
 
 See the [DatabaseClient Specification](https://observablehq.com/@observablehq/database-client-specification) for more details on these methods.
 
-Finally, the `DuckDBClient.sql` method takes the same arguments as `DuckDBClient.of` and returns the corresponding `db.sql` tagged template literal. The returned function can be used to redefine the built-in [`sql` tagged template literal](../sql#sql-literals) and thereby change the database used by [SQL code blocks](../sql), allowing you to query dynamically-registered tables (unlike the **sql** front matter option).
+Finally, the `DuckDBClient.sql` method <a href="https://github.com/observablehq/framework/releases/tag/v1.4.0" target="_blank" class="observablehq-version-badge" data-version="^1.4.0" title="Added in 1.4.0"></a> takes the same arguments as `DuckDBClient.of` and returns the corresponding `db.sql` tagged template literal. The returned function can be used to redefine the built-in [`sql` tagged template literal](../sql#sql-literals) and thereby change the database used by [SQL code blocks](../sql), allowing you to query dynamically-registered tables (unlike the **sql** front matter option).
 
 ```js
 const feed = view(Inputs.select(new Map([["M4.5+", "4.5"], ["M2.5+", "2.5"], ["All", "all"]]), {label: "Earthquake feed"}));

diff --git a/docs/loaders.md b/docs/loaders.md
@@ -48,7 +48,7 @@ const quakes = FileAttachment("quakes.csv").csv({typed: true});
 Now you can display the earthquakes in a map using [Observable Plot](./lib/plot):
 
 ```js
-const world = await fetch("https://cdn.jsdelivr.net/npm/world-atlas@1/world/110m.json").then((response) => response.json());
+const world = await fetch(import.meta.resolve("npm:world-atlas@1/world/110m.json")).then((response) => response.json());
 const land = topojson.feature(world, world.objects.land);
 ```
 

diff --git a/docs/markdown.md b/docs/markdown.md
@@ -87,9 +87,12 @@ Cell 1-2   |   Cell 2-2   |    Cell 3-2
 [external link](<https://en.wikipedia.org/wiki/Tar_(computing)>)
 ```
 
+For privacy and convenience, links to external resources are given a default `rel` attribute of [`noreferrer`](https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/rel/noreferrer) and [`noopener`](https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/rel/noopener), and a default `target` attribute of [`_blank`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/a#target). Hence by default an external link will open in a new window and not pass the (potentially sensitive) referrer to the (potentially untrusted) external site. You can override this behavior by specifying the `rel` or `target` attribute explicitly. For example `<a href="https://example.com" target="_self">` will open in the same window, and `<a href="https://acme.com" rel="">` will allow the referrer.
+
 ## Images
 
 ```md
+![A horse](./horse.jpg)
 ![A happy kitten](https://placekitten.com/200/300)
 ```
 
@@ -111,4 +114,28 @@ This produces:
   This text is not visible by default.
 </details>
 
+You can put Markdown inside of HTML by surrounding it with blank lines:
+
+<div class="grid grid-cols-4">
+  <div class="card">
+
+## Card title
+
+This is **Markdown** inside of _HTML_!
+
+  </div>
+</div>
+
+```md run=false
+<div class="grid grid-cols-4">
+  <div class="card">
+
+## Card title
+
+This is **Markdown** inside of _HTML_!
+
+  </div>
+</div>
+```
+
 Also see [Hypertext Literal](./lib/htl) for generating dynamic HTML in JavaScript.
diff --git a/docs/number.md b/docs/number.md
diff --git a/docs/sql.md b/docs/sql.md
@@ -16,7 +16,7 @@ sql:
 ---
 ```
 
-To load externally-hosted data, you can use a full URL:
+To load externally-hosted data, use a full URL: <a href="https://github.com/observablehq/framework/releases/tag/v1.4.0" target="_blank" class="observablehq-version-badge" data-version="^1.4.0" title="Added in 1.4.0"></a>
 
 ```yaml
 ---
@@ -25,7 +25,7 @@ sql:
 ---
 ```
 
-<div class="tip">For performance and reliability, we recommend using local files rather than loading data from external servers at runtime. If needed, you can use a <a href="./loaders">data loader</a> to take a snapshot of a remote data during build.</div>
+<div class="tip">For performance and reliability, we recommend using local files rather than loading data from external servers at runtime. You can use a <a href="./loaders">data loader</a> to take a snapshot of a remote data during build if needed.</div>
 
 You can also register tables via code (say to have sources that are defined dynamically via user input) by defining the `sql` symbol with [DuckDBClient.sql](./lib/duckdb).
 

diff --git a/docs/style.css b/docs/style.css
@@ -21,6 +21,10 @@
   content: "\2197";
 }
 
+#observablehq-header a[target="_blank"][data-decoration]::after {
+  content: attr(data-decoration);
+}
+
 #observablehq-header a[target="_blank"]:not(:hover, :focus)::after {
   color: var(--theme-foreground-muted);
 }

diff --git a/observablehq.config.ts b/observablehq.config.ts
@@ -1,3 +1,13 @@
+import {formatPrefix} from "d3-format";
+
+let stargazers_count: number;
+try {
+  ({stargazers_count} = await github("/repos/observablehq/framework"));
+} catch (error) {
+  if (process.env.CI) throw error;
+  stargazers_count = NaN;
+}
+
 export default {
   output: "docs/.observablehq/dist",
   title: "Observable Framework",
@@ -97,7 +107,7 @@ export default {
   }
 <script type="module">/Win/.test(navigator.platform) || Array.from(document.querySelectorAll(".win"), (e) => e.remove())</script>`,
   header: `<div style="display: flex; align-items: center; gap: 0.5rem; height: 2.2rem; margin: -1.5rem -2rem 2rem -2rem; padding: 0.5rem 2rem; border-bottom: solid 1px var(--theme-foreground-faintest); font: 500 16px var(--sans-serif);">
-  <a href="https://observablehq.com/" style="display: flex; align-items: center;">
+  <a href="https://observablehq.com/" target="_self" style="display: flex; align-items: center;">
     <svg width="22" height="22" viewBox="0 0 21.92930030822754 22.68549919128418" fill="currentColor">
       <path d="M10.9646 18.9046C9.95224 18.9046 9.07507 18.6853 8.33313 18.2467C7.59386 17.8098 7.0028 17.1909 6.62722 16.4604C6.22789 15.7003 5.93558 14.8965 5.75735 14.0684C5.56825 13.1704 5.47613 12.2574 5.48232 11.3427C5.48232 10.6185 5.52984 9.92616 5.62578 9.26408C5.7208 8.60284 5.89715 7.93067 6.15391 7.24843C6.41066 6.56618 6.74143 5.97468 7.14438 5.47308C7.56389 4.9592 8.1063 4.54092 8.72969 4.25059C9.38391 3.93719 10.1277 3.78091 10.9646 3.78091C11.977 3.78091 12.8542 4.00021 13.5962 4.43879C14.3354 4.87564 14.9265 5.49454 15.3021 6.22506C15.6986 6.97704 15.9883 7.7744 16.1719 8.61712C16.3547 9.459 16.447 10.3681 16.447 11.3427C16.447 12.067 16.3995 12.7593 16.3035 13.4214C16.2013 14.1088 16.0206 14.7844 15.7644 15.437C15.4994 16.1193 15.1705 16.7108 14.7739 17.2124C14.3774 17.714 13.8529 18.1215 13.1996 18.4349C12.5463 18.7483 11.8016 18.9046 10.9646 18.9046ZM12.8999 13.3447C13.4242 12.8211 13.7159 12.0966 13.7058 11.3427C13.7058 10.5639 13.4436 9.89654 12.92 9.34074C12.3955 8.78495 11.7441 8.50705 10.9646 8.50705C10.1852 8.50705 9.53376 8.78495 9.00928 9.34074C8.49569 9.87018 8.21207 10.5928 8.22348 11.3427C8.22348 12.1216 8.48572 12.7889 9.00928 13.3447C9.53376 13.9005 10.1852 14.1784 10.9646 14.1784C11.7441 14.1784 12.3891 13.9005 12.8999 13.3447ZM10.9646 22.6855C17.0199 22.6855 21.9293 17.6068 21.9293 11.3427C21.9293 5.07871 17.0199 0 10.9646 0C4.90942 0 0 5.07871 0 11.3427C0 17.6068 4.90942 22.6855 10.9646 22.6855Z"></path>
     </svg>
@@ -106,13 +116,35 @@ export default {
     <a href="/">
       <span class="hide-if-small">Observable</span> Framework
     </a>
-    <span style="display: flex; align-items: baseline; gap: 0.5rem; font-size: 14px;">
-      <a target="_blank" href="https://github.com/observablehq/framework/releases"><span>${process.env.npm_package_version}</span></a>
-      <a target="_blank" href="https://github.com/observablehq/framework"><span>GitHub</span></a>
+    <span style="display: flex; align-items: baseline; gap: 1rem; font-size: 14px;">
+      <a target="_blank" title="${
+        process.env.npm_package_version
+      } release notes" href="https://github.com/observablehq/framework/releases"><span>${
+        process.env.npm_package_version
+      }</span></a>
+      <a target="_blank" data-decoration="★" title="${stargazers_count.toLocaleString(
+        "en-US"
+      )} GitHub stars" href="https://github.com/observablehq/framework"><span>GitHub️ ${
+        stargazers_count ? formatPrefix(".1s", 1000)(stargazers_count) : ""
+      }</span></a>
     </span>
   </div>
 </div>`,
   footer: `© ${new Date().getUTCFullYear()} Observable, Inc.`,
   style: "style.css",
   search: true
 };
+
+async function github(
+  path: string,
+  {
+    authorization = process.env.GITHUB_TOKEN && `token ${process.env.GITHUB_TOKEN}`,
+    accept = "application/vnd.github.v3+json"
+  } = {}
+) {
+  const url = new URL(path, "https://api.github.com");
+  const headers = {...(authorization && {authorization}), accept};
+  const response = await fetch(url, {headers});
+  if (!response.ok) throw new Error(`fetch error: ${response.status} ${url}`);
+  return await response.json();
+}
diff --git a/package.json b/package.json
@@ -1,7 +1,7 @@
 {
   "name": "@observablehq/framework",
   "license": "ISC",
-  "version": "1.3.0",
+  "version": "1.4.0",
   "type": "module",
   "publishConfig": {
     "access": "public"
@@ -90,6 +90,7 @@
   "devDependencies": {
     "@types/cross-spawn": "^6.0.6",
     "@types/d3-array": "^3.2.1",
+    "@types/d3-format": "^3.0.4",
     "@types/he": "^1.2.3",
     "@types/jsdom": "^21.1.6",
     "@types/markdown-it": "^13.0.2",
@@ -107,6 +108,7 @@
     "chai-http": "^4.4.0",
     "concurrently": "^8.2.2",
     "d3-dsv": "^3.0.1",
+    "d3-format": "^3.1.0",
     "eslint": "^8.50.0",
     "eslint-config-prettier": "^9.1.0",
     "eslint-import-resolver-typescript": "^3.6.1",

diff --git a/src/bin/observable.ts b/src/bin/observable.ts
@@ -179,7 +179,8 @@ try {
       const {config, root, host, port, open} = values;
       await import("../preview.js").then(async (preview) =>
         preview.preview({
-          config: await readConfig(config, root),
+          config,
+          root,
           hostname: host!,
           port: port === undefined ? undefined : +port,
           open

diff --git a/src/build.ts b/src/build.ts
@@ -52,7 +52,7 @@ export async function build(
 
   // Make sure all files are readable before starting to write output files.
   let pageCount = 0;
-  for await (const sourceFile of visitMarkdownFiles(root)) {
+  for (const sourceFile of visitMarkdownFiles(root)) {
     await access(join(root, sourceFile), constants.R_OK);
     pageCount++;
   }
@@ -65,7 +65,7 @@ export async function build(
   const localImports = new Set<string>();
   const globalImports = new Set<string>();
   const stylesheets = new Set<string>();
-  for await (const sourceFile of visitMarkdownFiles(root)) {
+  for (const sourceFile of visitMarkdownFiles(root)) {
     const sourcePath = join(root, sourceFile);
     const path = join("/", dirname(sourceFile), basename(sourceFile, ".md"));
     const options = {path, ...config};