Add support for same-site prerendering with Speculation Rules API (#258)

* initial commit to add support for same-origin prerendering with Speculation Rules API

* reset author

* refactor, best practices, and minor logic update

* refactor, best practices, and minor logic update

* avoid prefetching the link that has been prerendered in listen

* create prerenderLimit as a constant to cater for the current Spec Rules API limitations and ease of change once the limitations are updated

* refactor prerender specific checks and compliance with eslint-config-google formatting

* create new tests for prerender with speculation rules

* bug fix: addSpeculationRules does not resolve

* adding prerendering doc in README

* Update README.md

Co-authored-by: Domenic Denicola <d@domenic.me>

* removed reference to outdated OT
removed extra argument in the promise constructor
fixed the inconsistent application of spacing throughout repo

* fixed promise rejection inside catch handler issue

* Update src/index.mjs

Co-authored-by: Domenic Denicola <d@domenic.me>

* Update src/index.mjs

Co-authored-by: Domenic Denicola <d@domenic.me>

* Update src/prerender.mjs

Co-authored-by: Domenic Denicola <d@domenic.me>

* update return value documentation

* remove conn param from prefetch and prerender functions

* updated addSpeculationRules return value

* restored conn param

Co-authored-by: Addy Osmani <addyosmani@gmail.com>
Co-authored-by: Domenic Denicola <d@domenic.me>

README.md

@@ -8,7 +8,7 @@
 </p>
 
 # quicklink
-> Faster subsequent page-loads by prefetching in-viewport links during idle time
+> Faster subsequent page-loads by prefetching or prerendering in-viewport links during idle time
 
 ## How it works
 
@@ -17,11 +17,11 @@ Quicklink attempts to make navigations to subsequent pages load faster. It:
 * **Detects links within the viewport** (using [Intersection Observer](https://developer.mozilla.org/en-US/docs/Web/API/Intersection_Observer_API))
 * **Waits until the browser is idle** (using [requestIdleCallback](https://developer.mozilla.org/en-US/docs/Web/API/Window/requestIdleCallback))
 * **Checks if the user isn't on a slow connection** (using `navigator.connection.effectiveType`) or has data-saver enabled (using `navigator.connection.saveData`)
-* **Prefetches URLs to the links** (using [`<link rel=prefetch>`](https://www.w3.org/TR/resource-hints/#prefetch) or XHR). Provides some control over the request priority (can switch to `fetch()` if supported).
+* **Prefetches** (using [`<link rel=prefetch>`](https://www.w3.org/TR/resource-hints/#prefetch) or XHR) or **prerenders** (using [Speculation Rules API](https://github.com/WICG/nav-speculation/blob/main/triggers.md))  URLs to the links. Provides some control over the request priority (can switch to `fetch()` if supported).
 
 ## Why
 
-This project aims to be a drop-in solution for sites to prefetch links based on what is in the user's viewport. It also aims to be small (**< 1KB minified/gzipped**).
+This project aims to be a drop-in solution for sites to prefetch or prerender links based on what is in the user's viewport. It also aims to be small (**< 1KB minified/gzipped**).
 
 ## Multi page apps
 
@@ -111,7 +111,16 @@ const options = {
 ### quicklink.listen(options)
 Returns: `Function`
 
-A "reset" function is returned, which will empty the active `IntersectionObserver` and the cache of URLs that have already been prefetched. This can be used between page navigations and/or when significant DOM changes have occurred.
+A "reset" function is returned, which will empty the active `IntersectionObserver` and the cache of URLs that have already been prefetched or prerendered. This can be used between page navigations and/or when significant DOM changes have occurred.
+
+#### options.prerender
+Type: `Boolean`<br>
+Default: `false`
+
+Whether to switch from the default prefetching mode to the prerendering mode for the links inside the viewport.
+
+> **Note:** The prerendering mode (when this option is set to true) will fallback to the prefetching mode if the browser does not support prerender.
+
 
 #### options.delay
 Type: `Number`<br>
@@ -226,6 +235,19 @@ By default, calls to `prefetch()` are low priority.
 
 > **Note:** This behaves identically to `listen()`'s `priority` option.
 
+### quicklink.prerender(urls)
+Returns: `Promise`
+
+> **Important:** You much `catch` you own request error(s).
+
+#### urls
+Type: `String` or `Array<String>`<br>
+Required: `true`
+
+One or many URLs to be prerendered.
+
+> **Note:** As prerendering using Speculative Rules API only supports same-origin at this point, only same-origin urls are accepted. Any non same-origin urls will return a rejected Promise.
+
 ## Polyfills
 
 `quicklink`:
@@ -277,6 +299,19 @@ quicklink.prefetch(['2.html', '3.html', '4.js']);
 quicklink.prefetch(['2.html', '3.html', '4.js'], true);
 ```
 
+### Programmatically `prerender()` URLs
+
+If you would prefer to provide a static list of URLs to be prerendered, instead of detecting those in-viewport, customizing URLs is supported.
+
+```js
+// Single URL
+quicklink.prerender('2.html');
+
+// Multiple URLs
+quicklink.prerender(['2.html', '3.html', '4.js']);
+```
+
+
 ### Set the request priority for prefetches while scrolling
 
 Defaults to low-priority (`rel=prefetch` or XHR). For high-priority (`priority: true`), attempts to use `fetch()` or falls back to XHR.
@@ -404,6 +439,7 @@ After installing `quicklink` as a dependency, you can use it as follows:
 * [Using Quicklink in a multi-page site](https://github.com/GoogleChromeLabs/quicklink/tree/master/demos/news)
 * [Using Quicklink with Service Workers (via Workbox)](https://github.com/GoogleChromeLabs/quicklink/tree/master/demos/news-workbox)
 * [Using Quicklink to prefetch API calls instead of `href` attribute](https://github.com/GoogleChromeLabs/quicklink/tree/master/demos/hrefFn)
+* [Using Quicklink to prerender a specific page](https://uskay-prerender2.glitch.me/next.html)
 
 ### Research
 

src/index.mjs

@@ -16,11 +16,17 @@
 import throttle from 'throttles';
 import {priority, supported} from './prefetch.mjs';
 import requestIdleCallback from './request-idle-callback.mjs';
+import {isSameOrigin, addSpeculationRules, hasSpecRulesSupport, isSpecRulesExists} from './prerender.mjs';
 
 // Cache of URLs we've prefetched
 // Its `size` is compared against `opts.limit` value.
 const toPrefetch = new Set();
 
+// Cache of URLs we've prerendered
+const toPrerender = new Set();
+// global var to keep prerenderAndPrefer option
+let shouldPrerenderAndPrefetch = false;
+
 /**
  * Determine if the anchor tag should be prefetched.
  * A filter can be a RegExp, Function, or Array of both.
@@ -36,6 +42,25 @@ function isIgnored(node, filter) {
     : (filter.test || filter).call(filter, node.href, node);
 }
 
+/**
+ * Checks network conditions
+ * @param  {NetworkInformation}  conn    The connection information to be checked
+ * @return {Boolean|Object}  Error Object if the constrainsts are met or boolean otherwise
+ */
+function checkConnection (conn) {
+  if (conn) {
+    // Don't pre* if using 2G or if Save-Data is enabled.
+    if (conn.saveData) {
+      return new Error('Save-Data is enabled');
+    }
+    if (/2g/.test(conn.effectiveType)) {
+      return new Error('network conditions are poor');
+    }
+  }
+
+  return true;
+}
+
 /**
  * Prefetch an array of URLs if the user's effective
  * connection type and data-saver preferences suggests
@@ -56,6 +81,8 @@ function isIgnored(node, filter) {
  * @param {Function} [options.onError] - Error handler for failed `prefetch` requests
  * @param {Function} [options.hrefFn] - Function to use to build the URL to prefetch.
  *                                             If it's not a valid function, then it will use the entry href.
+ * @param {Boolean} [options.prerender] - Option to switch from prefetching and use prerendering only
+ * @param {Boolean} [options.prerenderAndPrefetch] - Option to use both prerendering and prefetching
  * @return {Function}
  */
 export function listen(options) {
@@ -73,7 +100,12 @@ export function listen(options) {
 
   const timeoutFn = options.timeoutFn || requestIdleCallback;
   const hrefFn = typeof options.hrefFn === 'function' && options.hrefFn;
-
+
+  const shouldOnlyPrerender = options.prerender || false;
+  shouldPrerenderAndPrefetch = options.prerenderAndPrefetch || false;
+
+  const prerenderLimit = 1;
+
   const setTimeoutIfDelay = (callback, delay) => {
     if (!delay) {
       callback();
@@ -96,14 +128,32 @@ export function listen(options) {
           if (hrefsInViewport.indexOf(entry.href) === -1) return;
 
           observer.unobserve(entry);
-          // Do not prefetch if will match/exceed limit
-          if (toPrefetch.size < limit) {
+
+          // prerender, if..
+          // either it's the prerender + prefetch mode or it's prerender *only* mode
+          // && no link has been prerendered before (no spec rules defined)
+          if (shouldPrerenderAndPrefetch || shouldOnlyPrerender) {
+            if (toPrerender.size < prerenderLimit) {
+              prerender(hrefFn ? hrefFn(entry) : entry.href).catch(err => {
+                if (options.onError) {
+                  options.onError(err);
+                }else {
+                  throw err;
+                }
+              });
+              return;
+            }
+          }
+
+          // Do not prefetch if will match/exceed limit and user has not switched to shouldOnlyPrerender mode
+          if (toPrefetch.size < limit && !shouldOnlyPrerender) {
             toAdd(() => {
               prefetch(hrefFn ? hrefFn(entry) : entry.href, options.priority).then(isDone).catch(err => {
                 isDone(); if (options.onError) options.onError(err);
               });
             });
           }
+
         }, delay);
       }
       // On exit
@@ -141,7 +191,6 @@ export function listen(options) {
   };
 }
 
-
 /**
 * Prefetch a given URL with an optional preferred fetch priority
 * @param {String} url - the URL to fetch
@@ -150,14 +199,13 @@ export function listen(options) {
 * @return {Object} a Promise
 */
 export function prefetch(url, isPriority, conn) {
-  if (conn = navigator.connection) {
-    // Don't prefetch if using 2G or if Save-Data is enabled.
-    if (conn.saveData) {
-      return Promise.reject(new Error('Cannot prefetch, Save-Data is enabled'));
-    }
-    if (/2g/.test(conn.effectiveType)) {
-      return Promise.reject(new Error('Cannot prefetch, network conditions are poor'));
-    }
+  let chkConn = checkConnection(conn = navigator.connection);
+  if (chkConn instanceof Error) {
+    return Promise.reject(new Error('Cannot prefetch, '+chkConn.message));
+  }
+
+  if(toPrerender.size > 0 && !shouldPrerenderAndPrefetch) {
+    console.warn('[Warning] You are using both prefetching and prerendering on the same document');
   }
 
   // Dev must supply own catch()
@@ -175,3 +223,45 @@ export function prefetch(url, isPriority, conn) {
       })
   );
 }
+
+/**
+* Prerender a given URL
+* @param {String} url - the URL to fetch
+* @param {Object} [conn] - navigator.connection (internal)
+* @return {Object} a Promise
+*/
+export function prerender(urls, conn) {
+  let chkConn = checkConnection(conn = navigator.connection);
+  if (chkConn instanceof Error) {
+    return Promise.reject(new Error('Cannot prerender, '+chkConn.message));
+  }
+
+  // prerendering preconditions:
+  // 1) whether UA supports spec rules.. If not, fallback to prefetch
+  if (!hasSpecRulesSupport()) {
+    prefetch (urls);
+    return Promise.reject(new Error('This browser does not support the speculation rules API. Falling back to prefetch.'));
+  }
+
+  // 2) whether spec rules is already defined (and with this we also covered when we have created spec rules before)
+  if (isSpecRulesExists()) {
+    return Promise.reject(new Error('Speculation Rules is already defined and cannot be altered.'));
+  }
+
+  // 3) whether it's a same origin url,
+  for (const url of [].concat(urls)) {
+    if (!isSameOrigin(url)) {
+      return Promise.reject(new Error('Only same origin URLs are allowed: ' + url));
+    }
+
+    toPrerender.add(url);
+  }
+
+  // check if both prerender and prefetch exists.. throw a warning but still proceed
+  if (toPrefetch.size > 0 && !shouldPrerenderAndPrefetch) {
+    console.warn('[Warning] You are using both prefetching and prerendering on the same document');
+  }
+
+  let addSpecRules = addSpeculationRules(toPrerender);
+  return (addSpecRules === true) ? Promise.resolve() : Promise.reject(addSpecRules);
+}

src/prerender.mjs

@@ -0,0 +1,60 @@
+/**
+ * Portions copyright 2018 Google Inc.
+ * Inspired by Gatsby's prefetching logic, with those portions
+ * remaining MIT. Additions include support for Fetch API,
+ * XHR switching, SaveData and Effective Connection Type checking.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+**/
+/**
+ * Checks if the given string is a same origin url
+ * @param {string} str - the URL to check
+ * @return {Boolean} true for same origin url
+ */
+export function isSameOrigin(str) {
+  return window.location.origin === (new URL(str, window.location.href)).origin;
+}
+
+/**
+ * Add a given set of urls to the speculation rules
+ * @param {Set} toPrerender - the URLs to add to speculation rules
+ * @return {Boolean|Object}  boolean or Error Object
+ */
+export function addSpeculationRules(urlsToPrerender) {
+  let specScript = document.createElement('script');
+  specScript.type = 'speculationrules';
+  specScript.text = '{"prerender":[{"source": "list","urls": ["'+Array.from(urlsToPrerender).join('","')+'"]}]}';  
+  try {
+    document.head.appendChild(specScript);
+  }catch(e) {
+    return e;
+  }
+
+  return true;
+}
+
+/**
+ * Check whether UA supports Speculation Rules API
+ * @return {Boolean} whether UA has support for Speculation Rules API
+ */
+export function hasSpecRulesSupport() {
+  return HTMLScriptElement.supports('speculationrules');
+}
+
+/**
+ * Check whether Spec Rules is already defined in the document
+ * @return {Boolean} whether Spec Rules exists/already defined
+ */
+export function isSpecRulesExists() {
+  return document.querySelector('script[type="speculationrules"]');
+}

test/test-prerender-andPrefetch.html

@@ -0,0 +1,27 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+  <meta charset="utf-8">
+  <meta http-equiv="X-UA-Compatible" content="IE=edge">
+  <title>Prefetch: Basic Usage</title>
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <link rel="stylesheet" media="screen" href="main.css">
+  <script src="https://polyfill.io/v3/polyfill.min.js?features=IntersectionObserver"></script>
+</head>
+
+<body>
+  <a href="1.html">Link 1</a>
+  <a href="2.html">Link 2</a>
+  <a href="3.html">Link 3</a>
+  <section id="stuff">
+    <a href="main.css">CSS</a>
+  </section>
+  <a href="4.html" style="position:absolute;margin-top:900px;">Link 4</a>
+  <script src="../dist/quicklink.umd.js"></script>
+  <script>
+    quicklink.listen({prerenderAndPrefetch: true});
+  </script>
+</body>
+
+</html>

test/test-prerender-only.html

@@ -0,0 +1,27 @@
+<!DOCTYPE html>
+<html>
+
+<head>
+  <meta charset="utf-8">
+  <meta http-equiv="X-UA-Compatible" content="IE=edge">
+  <title>Prefetch: Basic Usage</title>
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <link rel="stylesheet" media="screen" href="main.css">
+  <script src="https://polyfill.io/v3/polyfill.min.js?features=IntersectionObserver"></script>
+</head>
+
+<body>
+  <a href="1.html">Link 1</a>
+  <a href="2.html">Link 2</a>
+  <a href="3.html">Link 3</a>
+  <section id="stuff">
+    <a href="main.css">CSS</a>
+  </section>
+  <a href="4.html" style="position:absolute;margin-top:900px;">Link 4</a>
+  <script src="../dist/quicklink.umd.js"></script>
+  <script>
+    quicklink.listen({prerender: true});
+  </script>
+</body>
+
+</html>