Remove trailing_slashes option and support URLs with or without HTM…

…L extensions

*TL;DR*: Discharge no longer supports the “directory indexes” format. Configure your static generator to output files however it normally does. Discharge also no longer has an option for configuring “trailing slashes” (a poor name for clean URLs). The option will automatically be removed from your configuration the next time you deploy and Discharge will upload HTML files both with and without extensions—use whichever URL style you prefer!

Right now if you want to use the “no trailing slashes” (clean URLs) option, Discharge expects your static HTML files to be structured and named in the “directory indexes” format.

The directory indexes format is where an HTML file that would normally be named `projects.html` is structured as `projects/index.html`. Then when Discharge deploys your files to S3, it will lop the `/index.html` part of the filename off, leaving just `projects`. This lets S3 serve the `projects` file when a request comes in for the path `/projects`.

In retrospect, requiring that files be generated in the directory indexes format was a mistake. As you might be able to tell, if Discharge is going to transform the files anyways when deploying to S3, it doesn’t really matter how the files are structured. Discharge could take `projects.html` and lop the `.html` off just as well as it could take `projects/index.html` and lop the `/index.html` off. There is no difference.

The reason why this was a mistake is that this places a hard requirement on a static generator to be able to output files in a very specific way, which not all can. Fundamentally, how a URL maps to a static file isn’t really a concern of static site generators, it’s a concern of the static file host. S3 serves files one way, Netlify serves files a different way, etc. Not mandating that static site generators output files in a specific structure will allow Discharge to deploy static sites built by more static site generators.

This change also has some additional, unexpected benefits. If you want to use clean URLs, you can’t _only_ upload the HTML file without the `.html` extension, because there are scenarios in which you might need to have an HTML file in your site that is consumed by a third-party service that you don’t control and it’s specifically requesting the file _with_ the `.html` extension.

That means no matter which clean URLs option you use, Discharge will still have to upload a version of the file with the `.html` extension. That makes the difference between the clean URLs options so small that it obviates the need for that configuration option altogether. It’s easier to just upload two copies of every HTML file and everyone can use whatever URL style they prefer.

README.md

@@ -21,7 +21,7 @@
 * Very little understanding of AWS required
 * Interactive UI for configuring deployment
 * Step-by-step list of what’s happening
-* Support for no trailing slashes in URLs
+* Support for clean URLs (no `.html` extensions)
 * Support for subdomains
 * Use an AWS Profile (named credentials) to authenticate with AWS
 * CDN (CloudFront) and HTTPS/TLS support
@@ -79,7 +79,6 @@ Configuration is done via a `.discharge.json` file located at the root of your a
   "upload_directory": "build",
   "index_key": "index.html",
   "error_key": "404.html",
-  "trailing_slashes": false,
   "cache": 3600,
   "aws_profile": "website-deployment",
   "aws_region": "us-west-1",
@@ -108,26 +107,12 @@ The name of the directory that the `build_command` generated with the static fil
 
 **index_key** `String`
 
-The key of the document to respond with at the root of the website and for URLs that look like folders. `index.html` is almost certainly what you want to use. For example, if `https://example.com` is requested, `https://example.com/index.html` will be returned. And if `https://example.com/some-page/` is requested, `https://example.com/some-page/index.html` will be returned.
-
-If you do not like trailing slashes in your URLs the `trailing_slashes` configuration can remove them.
+The key of the document to respond with at the root of the website. `index.html` is almost certainly what you want to use. For example, if `https://example.com` is requested, `https://example.com/index.html` will be returned.
 
 **error_key** `String`
 
 The key of the document to respond with if the website endpoint responds with a 404 Not Found. For example, `404.html` is pretty common.
 
-Don’t worry about accounting for the `trailing_slashes` configuration. If you disable trailing slashes, the `error_key` will be modified appropriately.
-
-**trailing_slashes** `Boolean`
-
-By default, most static site generators build websites with file extensions in the URL. So a page will look something like `https://example.com/some-page.html`. For a variety of reasons (aesthetics, backwards-compatibility with existing URLs, etc.), you might need something like `https://example.com/some-page` instead.
-
-Amazon S3 has support for “Index Documents”, or what’s commonly called “Directory Indexes”. It’s a feature where if a request is made to what appears to be a folder, like `https://example.com/folder/`, it will look for a file _inside_ that “folder” based on the `index_key`. So if the `index_key` is `index.html`, a request to `https://example.com/folder/` will serve the document at `https://example.com/folder/index.html`. If your static site generator supports Directory Indexes, then you can configure it so that when it builds your site a file named `some-page.html` will be generated as `some-page/index.html`.
-
-S3’s Directory Indexes support will also work without trailing slashes, but not how you might expect. If a request is made to `https://example.com/some-page`, it will first redirect to `https://example.com/some-page/` and then serve the file at `https://example.com/some-page/index.html`. If you don’t like the `html` extensions on your URLs, you probably aren’t going to be happy about the trailing slashes either. 😉
-
-If you set `trailing_slashes` to `false`, when you deploy, your files that look like Directory Indexes will be on-the-fly re-mapped to have no extension. So a file `some-page/index.html` will be uploaded as just `some-page`, which will allow it to be served from `https://example.com/some-page`, without the trailing slash!
-
 **cache** `Number` (optional when `cache_control` is set)
 
 The number of seconds a browser should cache the files of your website for. This is a simplified version of the HTTP `Cache-Control` header. If you set it to `0` the `Cache-Control` will be set to `"no-cache, no-store, must-revalidate"`. If you set it to a positive number, say, `3600`, the `Cache-Control` will be set to `"public, max-age=3600"`.
@@ -213,6 +198,12 @@ After you’ve finished configuring you can run `discharge deploy` to deploy. De
 
 If you change your website configuration (`cache`, `redirects`, etc.) it will be updated. If you change your website content, a diff will be done to figure out what needs to change. New files will be added, changed files will be updated, and deleted files will be removed. The synchronization is one way—that is, if you remove a file from S3 it will just be re-uploaded the next time you deploy.
 
+#### Clean URLs
+
+Clean URLs are when the `.html` extensions are dropped from URLs for aesthetic or functional reasons. The `.html` extensions are now commonly considered superfluous. If you have a file named `/projects.html` it’s now understood and generally preferred that the URL `domain.com/projects` would serve that file.
+
+When you deploy, two copies of each HTML file will be uploaded: one with the `.html` extension and one without. So a file `some-page.html` will be uploaded as `some-page.html` and as `some-page`, which will allow it to be served from `https://example.com/some-page.html`, with the extension, or from `https://example.com/some-page`, without the extension. You are free to use whichever URL style you prefer!
+
 ### Distribute
 
 After you’ve finished deploying you can run `discharge distribute` to distribute your website via a CDN (content delivery network). The command will create a TLS certificate, ensure it’s verified, create a distribution, and ensure it’s deployed. Almost no configuration necessary[1]. This step is completely optional, but if you have a high-traffic website it’s highly recommended, and if you want to secure your website with HTTPS/TLS then you have to do it[2].

lib/configuration.js

@@ -1,5 +1,6 @@
 const fs = require("fs")
 const KnownError = require("./error").error
+const logSymbols = require("log-symbols")
 
 const read = (configurationPath) => {
   if (!fs.existsSync(configurationPath)) {
@@ -9,7 +10,20 @@ const read = (configurationPath) => {
   let file = fs.readFileSync(configurationPath)
 
   try {
-    return JSON.parse(file)
+    let configuration = JSON.parse(file)
+
+    if (configuration.hasOwnProperty("trailing_slashes")) {
+      delete configuration["trailing_slashes"]
+      write(configurationPath, configuration)
+
+      console.log(
+        `\n${logSymbols.warning}`,
+        "Removing unused `trailing_slashes` configuration option",
+        "\n",
+      )
+    }
+
+    return configuration
   } catch (error) {
     throw new KnownError("Configuration file cannot be parsed—ensure the JSON is valid")
   }

lib/configure.js

@@ -75,12 +75,6 @@ module.exports = async () => {
       default: "404.html",
       type: "input",
     },
-    {
-      name: "trailing_slashes",
-      message: "URLs should have trailing slashes?",
-      default: true,
-      type: "confirm",
-    },
     {
       name: "cache",
       message: "Number of seconds to cache pages for?",

lib/deploy/configure-bucket-as-website.js

@@ -6,10 +6,6 @@ module.exports = {
     let indexKey = context.config.index_key
     let errorKey = context.config.error_key
 
-    if (!context.config.trailing_slashes && errorKey != "index.html") {
-      errorKey = errorKey.replace(".html", "")
-    }
-
     let params = {
       Bucket: context.config.domain,
       WebsiteConfiguration: {

lib/deploy/synchronize-website.js

@@ -4,40 +4,50 @@ const CacheControl = require("../cache-control")
 const mime = require("mime")
 const crypto = require("crypto")
 const diff = require("../diff")
+const flatMap = require("lodash.flatmap")
 
 module.exports = {
   title: "Synchronize website",
   task: async (context, task) => {
     let domain = context.config.domain
     let uploadDirectory = context.config.upload_directory
-    let trailingSlashes = context.config.trailing_slashes
-
-    let targetKey = (path) => {
-      if (path != "index.html" && path.endsWith("/index.html") && !trailingSlashes) {
-        let directoryName = path.replace("/index.html", "")
-        return directoryName
-      } else {
-        return path
-      }
-    }
 
     let paths = glob.sync("**/*", {
       cwd: uploadDirectory,
       nodir: true,
     })
 
-    let cacheControl = CacheControl.build(context.config.cache, context.config.cache_control, context.config.cdn)
+    let cacheControl = CacheControl.build(
+      context.config.cache,
+      context.config.cache_control,
+      context.config.cdn,
+    )
 
-    let source = paths.map((path) => {
+    let source = flatMap(paths, (path) => {
       let fullPath = `${uploadDirectory}/${path}`
       let content = fs.readFileSync(fullPath)
-      let md5Hash = `"${crypto.createHash("md5").update(content).digest("hex")}"`
+      let md5Hash = `"${crypto
+        .createHash("md5")
+        .update(content)
+        .digest("hex")}"`
 
-      return {
-        path: path,
-        key: targetKey(path),
-        md5Hash: md5Hash,
+      let files = [
+        {
+          path: path,
+          key: path,
+          md5Hash: md5Hash,
+        },
+      ]
+
+      if (path.endsWith(".html")) {
+        files.push({
+          path: path,
+          key: path.replace(/\.html$/, ""),
+          md5Hash: md5Hash,
+        })
       }
+
+      return files
     })
 
     let response = await context.s3.listObjectsV2({ Bucket: domain })

package.json

@@ -38,6 +38,7 @@
     "inquirer": "^6.2.2",
     "listr": "^0.14.3",
     "lodash.differenceby": "^4.8.0",
+    "lodash.flatmap": "^4.5.0",
     "lodash.intersectionby": "^4.7.0",
     "lodash.intersectionwith": "^4.4.0",
     "log-symbols": "^2.2.0",

yarn.lock

@@ -2261,6 +2261,11 @@ lodash.differenceby@^4.8.0:
   version "4.8.0"
   resolved "https://registry.yarnpkg.com/lodash.differenceby/-/lodash.differenceby-4.8.0.tgz#cfd59e94353af5de51da5d302ca4ebff33faac57"
 
+lodash.flatmap@^4.5.0:
+  version "4.5.0"
+  resolved "https://registry.yarnpkg.com/lodash.flatmap/-/lodash.flatmap-4.5.0.tgz#ef8cbf408f6e48268663345305c6acc0b778702e"
+  integrity sha1-74y/QI9uSCaGYzRTBcaswLd4cC4=
+
 lodash.flatten@^4.2.0:
   version "4.4.0"
   resolved "https://registry.yarnpkg.com/lodash.flatten/-/lodash.flatten-4.4.0.tgz#f31c22225a9632d2bbf8e4addbef240aa765a61f"