From fe3f386116174156d9116b646995e351fd0e386d Mon Sep 17 00:00:00 2001 From: Simon Lydell Date: Sun, 7 Jan 2024 21:34:22 +0100 Subject: [PATCH] Deploy beta docs --- docs/elm-watch.json.md | 4 + docs/https.md | 55 +++++++++- docs/server.md | 244 +++++++++++++++++++++++++++++++++++++++-- 3 files changed, 287 insertions(+), 16 deletions(-) diff --git a/docs/elm-watch.json.md b/docs/elm-watch.json.md index 95aa9a2e..22dda0c6 100644 --- a/docs/elm-watch.json.md +++ b/docs/elm-watch.json.md @@ -17,6 +17,8 @@ type NonEmptyArray = [T, ...Array]; type ElmWatchJson = { postprocess?: NonEmptyArray; port?: number; + webSocketUrl?: string; // ⚠ elm-watch@beta only + serve?: string; // ⚠ elm-watch@beta only targets: { [name: string]: { inputs: NonEmptyArray; @@ -56,6 +58,8 @@ Example: | [targets](#targets) | `Record` | _Required_ | The input Elm files to compile and the output JavaScript files to write to. At least one target is required. | | [postprocess](../postprocess/) | `NonEmptyArray` | No postprocessing. | A command to run after each `elm make` to transform Elm’s JavaScript output. | | port | `number` | An arbitrary available port. Tries to re-use the same port as last time you ran elm-watch. | The port for elm-watch’s HTTP and WebSocket server, used for hot reloading and as a simple file server. In case you _have_ to have the exact same port every time. Note that [some ports cannot be used][port-blocking]. | +| ⚠️ webSocketUrl | `string` | `` `ws://${currentHostname}:${port}/elm-watch` `` (sort of) | **Only available in `elm-watch@beta`.** This lets you customize how the elm-watch client connects its WebSocket for advanced use cases. You can also use the `ELM_WATCH_WEBSOCKET_URL` environment variable for dynamically setting it (the environment variable takes precedence). The value must be a valid URL starting with `ws:` or `wss:`. | +| ⚠ serve | `string` | unset | **Only available in `elm-watch@beta`.** A directory of static files to [serve](../server/). | ## targets diff --git a/docs/https.md b/docs/https.md index cc0cabb0..33927da2 100644 --- a/docs/https.md +++ b/docs/https.md @@ -6,9 +6,9 @@ nav_order: 11 # HTTPS {: .info } -**TL;DR:** I recommend using `http://` for local development. If you really want `https://`, accept elm-watch’s “unsafe” self-signed SSL certificate. +**TL;DR:** I recommend using `http://` for local development. If you really want `https://`, there are ways to set that up yourself. -I’d say it’s the most common to use plain old `http://` when working on `localhost`. One could argue that `https://` would be better even for local development since it’s closer to your production environment (which most likely uses `https://`). To be honest, I’ve tried using `https://` for local development and can’t remember a single time it saved me from a bug. Instead it just complicates things with certificates. +I’d say it’s the most common to use plain old `http://` when working on `localhost`. One could argue that `https://` would be better even for local development since it’s closer to your production environment (which most likely uses `https://`). To be honest, I’ve tried using `https://` for local development and can’t remember a single time it saved me from a bug. Instead it just complicates things with certificates. But there are some niche web features that are only available HTTPS, even on `localhost`. With elm-watch HTTPS causes a new complexity: elm-watch uses WebSockets for hot reloading, which results in the question of `ws://` vs `wss://`. @@ -17,16 +17,63 @@ elm-watch uses: - `ws://` on `http://` pages. - `wss://` on `https://` pages. +elm-watch runs an HTTP server, because WebSockets connect over HTTP before switching to the WebSocket protocol. Now, things differ a little bit depending on the elm-watch version: + +- elm-watch 1.0.2 and older only runs an HTTP server. +- elm-watch 1.1.0 added some support for HTTPS: It runs both and HTTP server and an HTTPS server. +- elm-watch beta removes the HTTPS server, but lets you set that up yourself. + +## elm-watch 1.1 + If you use `https://`, then the first time you visit your page you’ll see how elm-watch’s WebSocket gets stuck in the 🔌 connecting state. In the browser console you might see messages about connection errors due to an invalid certificate. You need to accept the certificate to make it work. Click elm-watch’s [browser UI](../browser-ui/) to expand it. There’s a link there that goes to the WebSocket server. When you click it, your browser will show a scary-looking security screen. That’s because elm-watch uses a self-signed certificate, which isn’t secure. However, there’s no security to worry about here – elm-watch just needs a certificate to be able to use `wss://` (which is basically required on `https://` pages – more on that below). Click a few buttons to proceed to the page anyway. Once you’ve done that once, the browser remembers your choice. Go back to your page (and possibly refresh the page) and now the WebSocket should connect! If you’ve ever created a self-signed certificate yourself for development – that’s exactly what’s happening here. elm-watch ships with a generic self-signed certificate created with `openssl`. -If you’d like to be able to configure the certificate used by elm-watch, let me know! +Using a self-signed certificate isn’t ideal, and cannot be used by everyone. Also, running both HTTP and HTTPS in elm-watch is pretty complicated. This is why `elm-watch@beta` switched to a new approach, where you are in full control over HTTPS. + +## elm-watch beta + +`elm-watch@beta` puts its HTTP server to more use than just connecting WebSockets: It also optionally [serves static files](../server/). That static file server is HTTP, not HTTPS, but the code for choosing between `ws://` and `wss://` based on if you’re on an `https://` page is still there. How can it be `https://` then? That’s if you serve the files yourself on your own HTTPS server (or if you run elm-watch in a certain way – which I’ll get back to). + +Like in elm-watch 1.1, if you use `https://` then you might see how elm-watch’s WebSocket gets stuck in the 🔌 connecting state. That’s because it tries to connect with `wss://` over HTTPS, but elm-watch only runs an HTTP server. In the [browser UI](../browser-ui/), instead of showing a link to a page where you can accept a self-signed certificate, `elm-watch@beta` now just links to this page instead, where you can read up on how to get HTTPS going. + +If you use your own HTTPS server, you can set the `"webSocketUrl"` option in [elm-watch.json](../elm-watch.json/) or the `ELM_WATCH_WEBSOCKET_URL` environment variable to make elm-watch connect to your HTTPS server instead of directly to elm-watch’s HTTP server. In your HTTPS server you need to proxy the WebSocket to elm-watch. Alternatively, you can set up a separate HTTPS proxy server just for elm-watch’s WebSocket if you prefer. + +You can also run elm-watch in an alternate way with a [custom server](../server/#custom-server) to set up HTTPS: + +```js +import * as fs from "node:fs"; +import * as https from "node:https"; +import * as path from "node:path"; +import * as url from "node:url"; +import elmWatch from "elm-watch"; + +const DIRNAME = path.dirname(url.fileURLToPath(import.meta.url)); + +// Deal with certificates and HTTPS options in whatever way you’d like: +const CERTIFICATE = { + key: fs.readFileSync(path.join(DIRNAME, "certificate", "dev.key")), + cert: fs.readFileSync(path.join(DIRNAME, "certificate", "dev.crt")), +}; + +elmWatch(process.argv.slice(2), { + createServer: ({ onRequest, onUpgrade }) => + https.createServer(CERTIFICATE, onRequest).on("upgrade", onUpgrade), +}) + .then((exitCode) => { + process.exit(exitCode); + }) + .catch((error) => { + console.error("Unexpected elm-watch error:", error); + }); +``` + +## Research Here are my findings from testing different combinations of http/s, ws/s, localhost vs not-localhost, and self-signed vs valid certificates: ✅ = Works. -🤕 = Works with workaround: If elm-watch is using port 12345, you need to visit for example https://localhost:12345 once and accept the self-signed certificate. +🤕 = Works with workaround: If the WebSocket connects to port 12345, you need to visit for example https://localhost:12345 once and accept the self-signed certificate. 💥 = `new WebSocket("ws://...")` immediately throws an error (that can be caught using `try-catch`). ❌ = `new WebSocket("ws://...")` throws no error, but the WebSocket never connects. 📢 = A warning is logged to the browser console. It cannot be turned off. diff --git a/docs/server.md b/docs/server.md index 11fb1a3f..570816c2 100644 --- a/docs/server.md +++ b/docs/server.md @@ -20,6 +20,10 @@ For example: "serve": "./public/" ``` +The simple static file server tries to be useful by default through strong conventions, and extensible enough for more advanced use cases, while still being light weight. + +The file server is completely optional. It only serves files. So if you serve your files some other way, that’s totally fine. As long as the generated Elm JS can connect via WebSocket to elm-watch, then Elm hot reloading will work fine. + {: .info } ℹ️ By default, the HTTP server is exposed on the local network (so you can test on your phone on the same Wi-Fi for example). If you are on a public Wi-Fi, you can restrict the server to just your computer by setting an environment variable: `ELM_WATCH_HOST=127.0.0.1`. Otherwise, you could expose the source code of your project if you use `"serve": "."`. @@ -39,24 +43,25 @@ For example: 4. Already being a replacement for `elm make`, this makes elm-watch a more flexible replacement for `elm reactor` as well. -## What it won’t do +## What you can do yourself -Here are some things elm-watch’s simple HTTP server _won’t_ do, by design: +Here are some more advanced dev server needs, that elm-watch simple HTTP server doesn’t do out of the box, but that you can set up yourself: - Routing. - Proxying API requests. TODO: Create example. - HTML templating. - On-the-fly compilation of CSS and TypeScript. +- HTTPS. -The configuration is `"serve": "./directory/"` rather than `"serve": { "directory": "./directory/" }` on purpose to not allow for more options. +You can always create your own little proxy server in front of elm-watch, using whatever technology you prefer. -Just like mentioned in [What elm-watch is _not_](../what-elm-watch-is-not), I don’t want to drown in feature requests for the HTTP server. I want to focus on the Elm aspects. On top of that, I personally think it’s easier and more flexible to write your own little dev server once you need more advanced things. Easier in the form of that you can write plain code instead of reading documentation, and that you can debug why things are not working easily. More flexible since with custom code you can do anything, not just what is supported through configuration. +But there is also another way: [Running elm-watch in an alternative way with a custom server](#custom-server). -So elm-watch’s simple static server focuses on the needs of [Browser.application](#browserapplication). Anything more is outside the scope of Elm and requires a custom solution. +The configuration is `"serve": "./directory/"` rather than `"serve": { "directory": "./directory/" }` on purpose to not allow for more options. -See the [example/] folder for inspiration on how to make your own dev server. +Just like mentioned in [What elm-watch is _not_](../what-elm-watch-is-not), I don’t want to drown in feature requests for the HTTP server. I want to focus on the Elm aspects. On top of that, I personally think it’s easier and more flexible to write a little bit of code when you need more advanced things. Easier in the form of that you can write plain code instead of reading documentation, and that you can debug why things are not working easily. More flexible since with custom code you can do anything, not just what is supported through configuration. -Remember that elm-watch’s HTTP server does _not_ provide any magic for hot reloading of Elm files to work or anything like that. It just serves files. So if you serve your files some other way, that’s totally fine. As long as the generated Elm JS can connect via WebSocket to elm-watch hot reloading works fine. +So elm-watch’s simple static server focuses on the needs of [Browser.application](#browserapplication). Anything more is outside the scope of Elm and requires [custom code](#custom-server). ## index.html @@ -101,19 +106,19 @@ As you can see, the `index.html` conventions lets you have one `Browser.applicat But it also has the side effect of _never getting 404s anymore._ In the two URLs above with typos (which were meant to go to files), you instead got HTML files served. This means that you _can’t look for 404 in the browser devtools Network panel._ There won’t be any 404 requests. Just 200 OK ones with HTML responses. -If you inspect the request for `/main.js`, you can see that elm-watch added some extra response headers to it: +If you inspect the request for `/mani.js` (typo), you can see that elm-watch added some extra response headers to it: - `elm-watch-404: /Users/you/project/public/mani.js`. This header hints that the URL was actually a 404, and shows the absolute file path that couldn’t be found. - `elm-watch-index-html: /Users/you/project/public/index.html`. Shows the `index.html` file that was served instead. - `elm-watch-learn-more: https://lydell.github.io/elm-watch/server/#indexhtml`. A handy link to this page if you need to read up on the details. -elm-watch also adds an HTML comment with `` at the top of the served HTML, which contains the same information as the headers but with more words. The idea is that the headers and the HTML comment should help you when a file path didn’t work. +elm-watch also adds an HTML comment starting with `` at the top of the served HTML, which contains the same information as the headers but with more words. The idea is that the headers and the HTML comment should help you when a file path didn’t work. Here’s a tip for fixing a broken URL to a file: 1. Copy the absolute file path that actually was tried to be read from the `elm-watch-404` header or the HTML comment (or an actual 404 page if there was no closest `index.html`). -2. Open the file you intended to link to in your editor or in a file explorer and copy its absolute path. +2. Open the file you intended to link to, using your editor or a file explorer and copy the absolute path of the file. 3. Paste them next to each other, like so: @@ -126,7 +131,7 @@ Here’s a tip for fixing a broken URL to a file: Note that `index.html` files must be called exactly `index.html`. Not `index.htm` or `INDEX.HTML`. -What would happen if you named `public/admin/index.html` just `public/admin.html` instead? There’s nothing stopping you from doing it. You would need to go to `/admin.html` to access it. Which would probably render a 404-style page in your `Browser.application` program, since you most likely have no route matching `/admin.html`. And if the `Browser.application` program ever changes the URL, refreshing the page won’t work. So stick to `index.html` files for `Browser.application` programs. +What would happen if you named `public/admin/index.html` just `public/admin.html` instead? There’s nothing stopping you from doing it. You would need to go to `/admin.html` to access it. Which would probably render a 404-style page in your `Browser.application` program, since you most likely have no route matching `/admin.html`. And if the `Browser.application` program ever changes the URL, refreshing the page won’t work. So stick to `index.html` files for `Browser.application` programs. Then you get the right page when you Elm app starts, and refreshing the page works. I recommend always creating an `index.html` directly in your static files directory. elm-watch prints a link to the static file server on start up, and if you have a root `index.html` file, that link will take you somewhere useful from the get go. @@ -139,4 +144,219 @@ I recommend always creating an `index.html` directly in your static files direct **Note:** elm-watch’s server is _not_ for production use. If you want to deploy your app somewhere, use any file server of choice. Make sure to set it up to handle serving your HTML file so that reloading the page works. -[example/]: https://github.com/lydell/elm-watch/tree/main/example#readme +## Reacting to changed files + +When `.elm` files change, elm-watch automatically compiles to `.js` and hot reloads the application (regardless of whether you use elm-watch’s static file server or not). + +When `.css` files inside the directory to serve change, elm-watch automatically hot reloads them. + +When other files inside the directory to serve change, elm-watch dispatches a DOM event that you can listen to, if you want to reload for other types of files. You can listen for the event like so: + +```js +window.addEventListener("elm-watch:changed-file-url-paths", (event) => { + // This logs a `Set` of strings. A string can look like this: `"/your/file.js"`. + // The strings are URL paths and always start with a slash. + console.log("Just changed file URL paths:", event.detail); +}); +``` + +For example, if you only have a single application you might want to reload the page whenever a JavaScript file, HTML file or image file etc. changes: + +```js +window.addEventListener("elm-watch:changed-file-url-paths", () => { + // Reload the page whenever a non-Elm and non-CSS file inside the directory to + // be served is changed. + window.location.reload(); +}); +``` + +If you have multiple applications you might want to inspect `event.detail` (which is a set of url paths to changed files) and only reload the page if something related to the current application has changed. + +Why doesn’t elm-watch do that by default? elm-watch only reloads when it can be near perfect, so that you can rely on it always working. + +- Elm files can be reloaded near perfectly, as described in [Hot reloading](../hot-reloading/). +- CSS is stateless and somewhat easy to reload. +- Images _sound_ like they can be hot reloaded if they change, but it’s very difficult to reload images set by CSS `background-image`, so elm-watch does not reload any images. In practice images don’t change that much during development anyway. +- JS is full of state and can’t be hot reloaded in general. It’s possible to instead reload the page. However, due to `import` and other JS loading techniques it’s difficult to know which JS files should reload which pages. Therefore elm-watch does not reload the page by default, but you can do it yourself as mentioned above, given the constraints of your given project. +- HTML could also be refreshed by reloading the page, but I figured it would be easier to remember how hot reloading works if elm-watch never makes full page reloads (except for a few cases for Elm code that makes a lot of sense). + +## Custom server + +An alternative way of running for example `elm-watch hot` is: + +``` +node my-server.mjs +``` + +```js +// my-server.mjs +import elmWatch from "elm-watch"; + +elmWatch(["hot"]) + .then((exitCode) => { + process.exit(exitCode); + }) + .catch((error) => { + console.error("Unexpected elm-watch error:", error); + }); +``` + +Instead of hard coding `hot` you can forward all CLI arguments if you want: + +``` +node my-server.mjs hot +``` + +```diff +-elmWatch(["hot"]) ++elmWatch(process.argv.slice(2)) +``` + +That `elmWatch` function also takes a `createServer` option. Here’s what it looks like if you pass the default for that option in: + +```js +import * as http from "node:http"; +import elmWatch from "elm-watch"; + +elmWatch(process.argv.slice(2), { + createServer: ({ onRequest, onUpgrade }) => + http.createServer(onRequest).on("upgrade", onUpgrade), +}) + .then((exitCode) => { + process.exit(exitCode); + }) + .catch((error) => { + console.error("Unexpected elm-watch error:", error); + }); +``` + +- `onRequest` is not strictly needed for the core of elm-watch to work, but is very interesting because it lets you do a lot of stuff. By default, all `onRequest` is doing is responding to regular HTTP requests (WebSocket connection HTTP requests are handled by `onUpgrade`) with a simple HTML page that tells you how to enable elm-watch’s [static file server](../server/). If you enable the static file server, the `onRequest` instead looks for files in the static file directory and serves them. You can wrap `onRequest` to do whatever you want before letting elm-watch server static files. +- `onUpgrade` is important, but only interesting if your project uses WebSockets too. If you also have your own WebSocket in your project, you can wrap `onUpgrade` to also handle your WebSocket. If you forget `.on("upgrade", onUpgrade)`, then elm-watch’s WebSockets will never connect. +- If you were wondering, you don’t need to add `.on("error", (error) => {...})`. elm-watch does that for you. + +For example, you can proxy URLs starting with `/api/` to your backend server: + +```js +import * as http from "node:http"; +import elmWatch from "elm-watch"; + +elmWatch(process.argv.slice(2), { + createServer: ({ onRequest, onUpgrade }) => + http + .createServer((request, response) => { + if (request.url.startsWith("/api/")) { + // Proxy /api/* to localhost:9000. + localhostProxy(request, response, 9000); + } else { + // Let elm-watch’s server do its thing for all other URLs. + onRequest(request, response); + } + }) + .on("upgrade", onUpgrade), +}) + .then((exitCode) => { + process.exit(exitCode); + }) + .catch((error) => { + console.error("Unexpected elm-watch error:", error); + }); + +function localhostProxy(request, response, port) { + const options = { + hostname: "127.0.0.1", + port, + path: request.url, + method: request.method, + headers: request.headers, + }; + + const proxyRequest = http.request(options, (proxyResponse) => { + response.writeHead(proxyResponse.statusCode, proxyResponse.headers); + proxyResponse.pipe(response, { end: true }); + }); + + proxyRequest.on("error", (error) => { + response.writeHead(503); + response.end( + `Failed to proxy to localhost:${port}. Is nothing running there?\n\n${error.stack}` + ); + }); + + request.pipe(proxyRequest, { end: true }); +} +``` + +Here’s the same `/api/` example again, but using the [http-proxy] npm package. If you don’t mind the extra dependencies, http-proxy can help if you have a lot of different things to proxy to, including remote servers and WebSockets. + +```js +import * as http from "node:http"; +import elmWatch from "elm-watch"; +import httpProxy from "http-proxy"; + +const proxy = new httpProxy.createProxyServer({ + target: { + host: "127.0.0.1", + port: 9000, + }, +}); + +elmWatch(process.argv.slice(2), { + createServer: ({ onRequest, onUpgrade }) => + http + .createServer((request, response) => { + if (request.url.startsWith("/api/")) { + // Proxy /api/* to localhost:9000. + proxy.web(request, response); + } else { + // Let elm-watch’s server do its thing for all other URLs. + onRequest(request, response); + } + }) + .on("upgrade", onUpgrade), +}) + .then((exitCode) => { + process.exit(exitCode); + }) + .catch((error) => { + console.error("Unexpected elm-watch error:", error); + }); +``` + +As you can see, this pretty small API surface from elm-watch gives you full control to do what you need in your project (at the expense of being slightly more verbose for the simplest of cases). + +On top of that, it also lets you set up HTTPS: + +```js +import * as fs from "node:fs"; +import * as https from "node:https"; +import * as path from "node:path"; +import * as url from "node:url"; +import elmWatch from "elm-watch"; + +const DIRNAME = path.dirname(url.fileURLToPath(import.meta.url)); + +// Deal with certificates and HTTPS options in whatever way you’d like: +const CERTIFICATE = { + key: fs.readFileSync(path.join(DIRNAME, "certificate", "dev.key")), + cert: fs.readFileSync(path.join(DIRNAME, "certificate", "dev.crt")), +}; + +elmWatch(process.argv.slice(2), { + createServer: ({ onRequest, onUpgrade }) => + https.createServer(CERTIFICATE, onRequest).on("upgrade", onUpgrade), +}) + .then((exitCode) => { + process.exit(exitCode); + }) + .catch((error) => { + console.error("Unexpected elm-watch error:", error); + }); +``` + +{: .warning } +You can’t really log anything in your custom server code – you’ll compete with the output of elm-watch, which clears the screen and moves the cursor to update parts of the output. If your custom server is complicated enough to need logging, make a separate proxy server. Extending elm-watch’s server is better for smaller, “silent” customizations, such as doing a little bit of proxying or setting up HTTPS. + +It is _not_ recommended to run other compilers as part of the elm-watch server, such as esbuild or Sass, since they also need to print error messages. Run them as separate watchers. [run-pty] is one way of easily starting multiple watchers with one command. + +[http-proxy]: https://github.com/http-party/node-http-proxy +[run-pty]: https://github.com/lydell/run-pty