feat(arcgis-rest-request): add demo and code for ability to share ses…

…sion between client and server

* feat(arcgis-rest-request): refresh session and retry with new token for invalid token errors

* chore(arcgis-rest-request): rename to refreshCrednetials and cleanup

* chore(arcgis-rest-js): code review suggestions

Co-authored-by: Noah Mulfinger <nmulfinger@esri.com>

* chore: advanced oauth demo WIP

* chore: add advanced oauth demo

* chore: finish documentation

* chore: remove console.log

* feat(arcgis-rest-request): add updateToken() to ArcGISIdentityManager

Co-authored-by: Noah Mulfinger <nmulfinger@esri.com>

demos/express-oauth-advanced/.env.template

@@ -0,0 +1,4 @@
+CLIENT_ID=S19JZF92TRf2HZit
+REDIRECT_URI=http://localhost:3000/authenticate
+ENCRYPTION_KEY=vf00tpMiGRSCO36yrbNm9jWyAStIJWJ5
+SESSION_SECRET=Sb66uSlcA3RqyIXLOtwK5P1a37FvYqHD

demos/express-oauth-advanced/.gitignore

@@ -0,0 +1,3 @@
+.env
+sessions
+node_modules

demos/express-oauth-advanced/README.md

@@ -0,0 +1,47 @@
+# ArcGIS REST JS Express Advanced oAuth 2.0 Demo
+
+This demo shows some more advanced uses of authentication features in ArcGIS REST JS. Specically:
+
+1. Performing server based oAuth 2.0 to obtain a refresh token store the resulting token and refresh token in a session object on the server that can be looked up via an encrypted cookie.
+2. Sending the short lived token information to hydrate `ArcGISIdentityManager` in a client side environment while keeping the refresh token secure on the server.
+3. Customizing `ArcGISIdentityManager` to ask the server for a new short lived token.
+4. Storing a session id in the in an encrypted cookie.
+
+This demo showcases several security best practices:
+
+- Only the encrypted session id is stored on the client in a cookie.
+- Session information is encrypted when it is stored on disk on the server.
+- Only short lived tokens (30 minutes) are used on the client, new tokens are requested from the server automatically.
+- The refresh token on the server is valid for 2 weeks the default but is extended for another 2 weeks every time the user access the app.
+
+## Setup with provided credentials
+
+These instructions are for setting up the demo using credentials setup by the ArcGIS REST JS team.
+
+1. Make sure you run `npm run build` in the root folder to setup the dependencies
+1. Copy `.env.template` to `.env`
+1. Run `npm run start`
+1. Visit http://localhost:3000 to start.
+
+## Setup with your own credentials
+
+These instructions are for setting up the demo using credentials setup by the ArcGIS REST JS team.
+
+1. Make sure you run `npm run build` in the root folder to setup the dependencies
+1. Copy `.env.template` to `.env`
+1. [Register an app](https://developers.arcgis.com/documentation/mapping-apis-and-services/security/tutorials/register-your-application/) and copy the Client ID into `.env` in the `CLIENT_ID` property.
+1. [Add redirect URIs](https://developers.arcgis.com/documentation/mapping-apis-and-services/security/tutorials/add-redirect-uri/) for `http://localhost:3000/authenticate` to your registered application.
+1. Replace the `ENCRYPTION_KEY` and `SESSION_SECRET` values with new values. You can use a website like https://randomkeygen.com/ to generate new strong keys.
+1. Run `npm run start`
+1. Visit http://localhost:3000 to start.
+
+## How it works
+
+1. At the start of every request received express will check if a secure "session cookie" was sent along with the request.
+2. If a session cookie is found it will contain a session ID that will correspond with an encrypted data file that represents some data stored for that unique session. The contents of this file are passed through the `decode` method in the session configuration an an `ArcGISIdentityManager` instance is created.
+3. The handler for the request runs. You can access the `ArcGISIdentityManager` object from `request.session.arcgis`.
+4. At the end of the request the value of `request.session` is run through the `encode` option on the session configuration and saved back to disk.
+5. In the application route the session is converted to a plain object, the refresh token is removed and the resulting object embedded in the client code.
+6. On the client we can customize the behavior of how `ArcGISIdentityManager` by subclassing it so that it so that it refreshes tokens when they expire by getting a new token from our server.
+7. The client can hydrate the custom subclass of `ArcGISIdentityManager` and use it for requests.
+8. When the client needs to refresh the token it will call the custom `refreshCredentials` method which will call the server. The server will get a new token and send it to the client and the client will update and retry the request with the new token.

demos/express-oauth-advanced/app.html

@@ -0,0 +1,72 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8">
+  <meta http-equiv="X-UA-Compatible" content="IE=edge">
+  <meta name="viewport" content="width=device-width, initial-scale=1.0">
+  <title>Document</title>
+</head>
+<body>
+  <button id="getUserContent">Get Private User Content</button>
+  <button id="refreshSession">Refresh Credentials</button>
+
+  <script type="module">
+    import { ArcGISIdentityManager } from 'https://cdn.skypack.dev/@esri/arcgis-rest-request@beta';
+    import { getUserContent, searchItems, SearchQueryBuilder } from 'https://cdn.skypack.dev/@esri/arcgis-rest-portal@beta';
+
+    let session;
+
+    // We need to customize the behavior of how  ArcGISIdentityManager handles refreshing.
+    class ManagerWithCustomRefresh extends ArcGISIdentityManager {
+      /**
+       * This will be false by default because we do not have a refresh token on the client, we need to
+       * force this to be `true` to internally request will try to refresh tokens.
+       */
+      get canRefresh() {
+        return true;
+      }
+      /**
+       * Now we can override the `refreshCredentials` method to change HOW this will be refreshed.
+       */
+      refreshCredentials() {
+        fetchNewSessionFromServer();
+      }
+    }
+
+    // generic function to get a new session from the server.
+    function fetchNewSessionFromServer () {
+       fetch("/refresh").then(response => {
+        return response.json()
+      }).then(sessionJson => {
+        session.updateToken(sessionJson.token, new Date(session.expires));
+        document.getElementsByTagName("pre")[0].innerText = JSON.stringify(sessionJson, null, 2);
+      })
+    }
+
+
+    // This is where the server inserts serialized session information
+    const serverSessionData = `SESSION_JSON`;
+    session = ManagerWithCustomRefresh.deserialize(serverSessionData);
+
+    // Bind a button to test refreshing from the server
+    const refreshSessionButton = document.getElementById("refreshSession");
+    refreshSessionButton.addEventListener("click", (e)=>{
+      fetchNewSessionFromServer();
+      e.preventDefault();
+    })
+
+    // Bind a button to test getting some private data
+    const getUserContentButton = document.getElementById("getUserContent");
+    getUserContentButton.addEventListener("click", (e) => {
+      getUserContent({
+        authentication: session
+      }).then((items)=>{
+        console.log(items)
+      }).catch(e => {
+        console.log(e)
+      })
+      e.preventDefault();
+    })
+  </script>
+</body>
+</html>

demos/express-oauth-advanced/package.json

@@ -0,0 +1,19 @@
+{
+  "private": true,
+  "name": "@esri/arcgis-rest-demo-express-oauth-advanced",
+  "version": "3.3.0",
+  "description": "Demo of @esri/arcgis-rest-* packages in an Express server",
+  "license": "Apache-2.0",
+  "type": "module",
+  "scripts": {
+    "start": "node server.js"
+  },
+  "dependencies": {
+    "@esri/arcgis-rest-request": "file:../../packages/arcgis-rest-request",
+    "dotenv": "^16.0.0",
+    "express": "^4.16.3",
+    "express-session": "^1.17.2",
+    "session-file-store": "^1.5.0"
+  },
+  "author": ""
+}

demos/express-oauth-advanced/server.js

@@ -0,0 +1,160 @@
+import "dotenv/config";
+import express from "express";
+import { ArcGISIdentityManager } from "@esri/arcgis-rest-request";
+import session from "express-session";
+import SessionFileStore from "session-file-store";
+import { readFile } from "fs/promises";
+
+/**
+ * Create a new express and file store for our sessions
+ */
+const app = express();
+const FileStore = SessionFileStore(session);
+
+/**
+ * Create our oauth 2 options objects
+ */
+const oauthOptions = {
+  clientId: process.env.CLIENT_ID,
+  redirectUri: process.env.REDIRECT_URI
+};
+
+/**
+ * Determine how long we want our session to last we know refresh tokens last 2 weeks so we subtract an hour to provide some padding.
+ */
+const sessionTTL = 60 * 60 * 24 * 7 * 2 - 60 * 60 * 1; // 2 weeks - 1 hours in seconds;
+
+/**
+ * Setup the session middleware
+ */
+app.use(
+  session({
+    name: "arcgis-rest-js-advanced-oauth-demo", // the name of the cookie to store the session id.
+    secret: process.env.SESSION_SECRET, // secret used to sign the session cookie
+    resave: false,
+    saveUninitialized: false,
+    cookie: {
+      maxAge: sessionTTL * 1000 // set the max age on the cookie to match our session duration
+    },
+
+    // store session data in a secure, encrypted file, sessions will be loaded from these files and decrypted
+    // at the end of every request the state of `request.session` will be saved back to disk.
+    store: new FileStore({
+      ttl: sessionTTL, // session duration
+      retries: 1,
+      secret: process.env.ENCRYPTION_KEY, // secret used to encrypt and decrypt sessions on disk
+
+      // define how to encode our session to text for storing in the file. We can use the `serialize()` method on the session for this.
+      encoder: (sessionObj) => {
+        sessionObj.arcgis = sessionObj.arcgis.serialize();
+        return JSON.stringify(sessionObj);
+      },
+
+      // define how to turn the text from the session data back into an object. We can use the `ArcGISIdentityManager.deserialize()` method for this.
+      decoder: (sessionContents) => {
+        const sessionObj = JSON.parse(sessionContents);
+        if (sessionObj.arcgis) {
+          sessionObj.arcgis = ArcGISIdentityManager.deserialize(
+            sessionObj.arcgis
+          );
+        }
+
+        return sessionObj;
+      }
+    })
+  })
+);
+
+// Render a link to the authorization page on the homepage
+app.get("/", (request, response) => {
+  response.send(`<a href="/authorize">Sign in with ArcGIS</a>`);
+});
+
+// When a user visits the authorization page start the oauth 2 process. The `ArcGISIdentityManager.authorize()` method will redirect the user to the authorization page.
+app.get("/authorize", function (request, response) {
+  // send the user to the authorization screen
+  ArcGISIdentityManager.authorize(oauthOptions, response);
+});
+
+// the after authorizing the user is redirected to /authenticate and we can finish the Oauth 2.0 process.
+app.get("/authenticate", async function (request, response) {
+  // exchange the auth code for a an instance of `ArcGISIdentityManager` save that to the session.
+  request.session.arcgis =
+    await ArcGISIdentityManager.exchangeAuthorizationCode(
+      oauthOptions,
+      request.query.code
+    );
+
+  // once we have the session set redirect the user to the /app route so they can use the app.
+  response.redirect("/app");
+});
+
+// The refresh endpoint is used when the client needs to get a new token.
+app.get("/refresh", function (request, response) {
+  // return an error if we cannot find a session.
+  if (!request.session.arcgis) {
+    response.json({ error: "unable to refresh" });
+    return;
+  }
+
+  // refresh the session
+  request.session.arcgis
+    .refreshCredentials()
+    .then((newSession) => {
+      request.session.userSession = newSession;
+
+      // convert the session to an object and remove the refresh token
+      const serializedSession = newSession.toJSON();
+      delete serializedSession.refreshToken;
+      delete serializedSession.refreshTokenExpires;
+
+      // respond with the new session data
+      request.json(serializedSession);
+    })
+    .catch((error) => {
+      response.json({
+        error: error.toString()
+      });
+    });
+});
+
+// This handles the application route
+app.get("/app", (request, response) => {
+  // if there is no session available redirect the user back to the home page
+  if (!request.session.arcgis) {
+    response.redirect("/");
+    return;
+  }
+
+  // next exchange the current refresh token for a new refresh token, this extends the session for another 2 weeks
+  // we also read the template for the app.
+  Promise.all([
+    readFile("app.html", { encoding: "utf-8" }),
+    request.session.arcgis.exchangeRefreshToken()
+  ]).then(([templateContents, freshSession]) => {
+    // update our session object with the new session
+    request.session.arcgis = freshSession;
+
+    // prepare to send the session to the client
+    const serializedSession = freshSession.toJSON();
+    delete serializedSession.refreshToken;
+    delete serializedSession.refreshTokenExpires;
+
+    // insert the session into the HTML
+    templateContents = templateContents.replace(
+      "<body>",
+      `<body><pre>${JSON.stringify(serializedSession, null, 2)}</pre>`
+    );
+    templateContents = templateContents.replace(
+      "SESSION_JSON",
+      JSON.stringify(serializedSession)
+    );
+
+    // send the HTML
+    response.send(templateContents);
+  });
+});
+
+app.listen(3000, function () {
+  console.log("visit http://localhost:3000/authorize to test the application!");
+});

demos/express/config.json.template

@@ -1,3 +1,3 @@
 {
-  "clientId": "QVQNb3XfDzoboWS0" // change me
+  "clientId": "qJv5jgYaCK32K30I" // change me
 }