JavaScript Notebook: React with TypeScript

Building a javascript notebook (think Jupyter notebook) with React and TypeScript

Getting Started

NPM Link

Running Published App

1. Install globally

   npm install -g ghake-jsnote

   • Preferred way

   ghake-jsnote serve

2. Or use npx

   npx ghake-jsnote serve

3. A notebook will be either created or opened depending on where you launched the app for

CLI Options

The main command is serve

If we just run the main command, we will either open or create a file called notebook.js

ghake-jsnote serve

1. Filename

   ghake-jsnote serve test.js

   • Make sure you use a .js file
   • It will create a file in the directory you are in
2. If you already have a file created it will open it
   • You can designate the folder in the serve command
   • Folder/file must exist
3. Port

   ghake-jsnote serve --port 4008
   

   ghake-jsnote serve -p 4008
   

4. Combination
   • You can combine the filename and port in any order

   ghake-jsnote serve test.js --port 4008
   

   ghake-jsnote serve -p 4008 test/test.js
   

Running Locally

1. Clone the repo
2. We are using lerna to launce multiple node projects at once
   • Navigate to jbook folder where lerna.json is
   • Run lerna bootstrap to install packages for all node apps through lerna
3. Stay in same folder
   • Run npm run start which runs a script, but really runs lerna run start --parallel
   • Will start all node projects in the solution
4. You'll need to start the cli manually from the dist folder (index.js)
   • It's annoying since you'll have to restart the cli if you do any changes
   • Navigate to jbook/packages/cli/dist
   • Run node index.js serve to launch the cli
5. This will launch the app on port 4005 so you can navigate to
   • localhost:4005

Our App

1. We want to run something like 'jbook serve'
2. This should start a server on localhost:4005
3. User will write code into an editor
   • js or md in a cell
4. We bundle in the browser
5. We execute the users code in an iframe

Inpsired By

1. codepen.io
2. babeljs.io
3. Jupyter

Transpiling Java Code

The above sites use live transpiling of code. How do we do that in this app?

1. Backend API Server
2. In-Browser Transpiler
   • In React App

From our inspirations above..

1. codepen
   • Uses a backend server to transpile code
   • Running babel
   • Sends js as string in request
   • Gets transpiled js back in response
   • API Server
2. babeljs
   • Uses remote transpiling
   • 'In-Browser' Transpiler
   • In React App

Javascript Modules

A JS file that makes some values available to other files and/or consumes values from other files

Why do we care besides using them in the app?

Our app will allow a user to import js and css files into a code cell

JS Module Systems

1. AMD

   define(['dep'], (dep) => {});

2. common js

   require();
   module.exports;

3. ES Modules

   import a from 'a';
   export default 123;

• Transpiliers will sometimes take one version and convert to another!
• A bundler (webpack) will take all the individual files and combine into a single file

Bundling JS Code

Options

1. Backend API Server
   • NPM Install Plugin
   • Weakness is api server will save a ton of dependencies locally
2. Backend API Server with Custom plugin
   • Makes request to npm registry
   • Bundle code instead of save dependency
3. Do everything inside the React app
   • Instead of API Server

Our Application Approach

We are doing the local, 'in-app', 'in-browser' approach

1. Significantly faster than server approach
2. Big issue, webpack doesn't work in the browser
3. Instead of using webpack and babel (work great locally)
   • We are going to use ESBuild
4. ESBuild can transpile + bundle all in the browser
   • Much faster than webpack (much)

Transpiler

1. ESBuild works out of the box for transpiling
   • 'transform' call

Bundler

1. Not using webpack
2. Using ESBuild
3. ESBuild usually looks on the local filesystem for files to bundle

• Running in the browser...we don't have a local filesystem
• Instead we are pointing ESBuild at a url where the file exists
• NPM registry will be the url...kind of

1. NPM by itself won't work
   • We will use a service called Unpkg
   • NPM registry is configured to block any request not at a specific url
   • Throws CORS error
2. UNPKG
   • Global content delivery network for everything on NPM
3. So, for ESBuild to work, bundling on the web...we have to define our own version of it's build step (where the bundling occurs)

   const result = await ref.current.build({
   	entryPoints: ['index.js'],
   	bundle: true,
   	write: false,
   	plugins: [unpkgPathPlugin()],
   });

   • Define a 'plugin' and override functions that default ESBuild would use
   • 'onResolve' and 'onLoad': Work together
   • 'onResolve' returns an object that is fed to 'onLoad'
   • filter args for onResolve and filter, namespace args are key

Cache Layer

We send lots of requests to unpkg, especially if an includes has a lot of it's own includes

We are going to develop a caching layer to store some of these files to limit the number of requests

1. localforage
   • API over the browser's indexedDB
   • kvp with the path as the key and the loaded package as the value

Loading css Files

1. Need to have a css loader
2. Issue w/ esbuild on the web
   • When building css and js, it'll generate 2 files
     • But we don't have a local file system
   • It only outputs js on the web, so we need a way to include unpackaged css into jss

1. Instead, we will write js to take the content of the css and put it in the head tag
   • Something of a hack

Code Execution

Considerations

These are all big issues we have to solve

1. User-provided code might throw errors and cause our program to crash

   console.loooog('Error');

2. User-provided code might mutate the DOM, causing our program to crash

   document.body.innerHTML = '';

3. A user might accidentally run code provided by another malicious user
4. Infinite loops (or really big loops)
   • Future work to fix

What's the solution? IFrames

IFrames

1. Used to embed one element (html element) inside another

   <iframe src="/test.html" title="test"></iframe>

2. Allows you to run js in separate contexts
   • Separate execution contexts
3. You can connect the parent and child contexts to share info
   • In child element, reference 'parent'
   • In parent, reference:

   document.querySelector('iframe').contentWindow;

4. You can also dissalow communication b/t the two
5. Our considerations above are solved by using iframes
   • It crashes the iframe context, not the app's
6. We are using a few properties in IFrames
   • sandbox="allow-scripts"
     • This keeps our parent and children from being able to access each other
     • Obviously can also run scripts w/i the iframe
   • srcDoc={html}
     • Allows us to generate source for the iframe w/o sending a request
     • Much faster
     • Drawback, can't use localStorage on IFrame
7. To allow using localStorage
   • You have to host the iframe on a separate port
   • Lots more work, but a more 'complete' task
8. We can't communicate directly b/t the parent and child, but we can do something...
   • Add Event Listeners!
   • Have an event listener on the child (code changed) and have parent send the events

This completes the backbone of the app

Code Editor

• Right now we are writing code in a textarea
• We want something like a code editor
  • Line numbers
  • Linter
  • Intellisense?

Options

1. CodeMirror
   • Super easy to use
   • Doesn't have as many out-of-the-box features
2. Ace Editor
   • Moderately easy to use
   • Widely used
3. Monaco Editor (What this app uses)
   • Same editor VS Code uses
   • Hardest to setup
     • React component makes it easy though
   • Gives an almost-perfect editing experience immediately

Monaco Editor as React Component

1. Monaco Editor/React
   • React component around the real Monaco Editor
   • Three different 'Editors' in the props of the component
   • We are using the default editor which is the uncontrolled editor
2. Installation

   npm install --save-exact @monaco-editor/react@3.7.5
   npm install --save-exact monaco-editor

   • First is the actual component
   • Second allows us to see type defs for settings
3. Add Prettier to our code editor

   npm install prettier @types/prettier

App Styling

1. Using bulmaswatch

   npm install bulmaswatch
   

2. Bulma, but with some themes

Syntax Highlighting in Monaco (JSX)

We will be using packages that aren't super well tested, so potential for breaking

1. monaco-jsx-highlighter
   • Does the actual highlighting, but doesn't know how to get the jsx code
2. jscodeshift
   • Will get jsx code to the highlighter

npm install --save-exact monaco-jsx-highlighter@0.0.15 jscodeshift@0.11.0 @types/jscodeshift@0.7.2

Resizing Components

1. Vertical spot between code editor and previe
2. Horizontal spot between code cells
3. React Resizable

   npm install --save-exact react-resizable@1.11.1 @types/react-resizable@1.7.2

4. Resizing w/ an IFrame
   • So the issue w/ IFrames is that it has a different context than the rest of the app
   • So the resizing would 'freeze' if you hovered over the iframe
   • The way to fix is to, when we are hovering over the iframe, to 'draw' a div element
   • The div element is in the app's context
   • Look at the preview.tsx and preview.css files to see how this is implemented

Markdown Editor

We are using a pre-built react component

1. React MD Editor
2. Installation

   npm install --save-exact @uiw/react-md-editor@2.1.1

Redux

1. Installation

   npm install --save-exact @types/react-redux@7.1.15 react-redux@7.2.2 redux@4.0.5 redux-thunk@2.3.0 axios@0.21.1

Immer: Simple State Update

1. Immer
   • Allows you to do immutable state updates in a simple way
2. Installation

   npm install immer

Without Immer

const { id, content } = action.payload;

return {
	...state,
	data: {
		...state.data,
		[id]: {
			...state.data[id],
			content,
		},
	},
};

With Immer

const { id, content } = action.payload;

state.data[id].content = content;

Connecting Bundles with Redux

1. Bundle state is technically derived from cell state
   • Which is usually where a 'selector' comes in
   • But we have async functions in bundling
2. Should we use a bundles reducer?
   • Or should we use a selector?
3. We want to avoid using selectors w/ any async functions
   • Use w/ synchronous calcs
   • selectors do caching type calcs for optimization, which can have weird results w/ async code
4. So, we're going to use a bundles reducer

Cumulative Code Execution

1. Allow all subsuquent cells to reference any previous code cell
   • const color = 'red';
     • Cell 1
   • console.log(color);
     • Cell 2
2. 'Derived' State
   • Inside 'CodeCell', add selector
   • Get code from current cell, and all prior cells
3. We're adding a 'show' function so that we don't have to reference the index of to show anything

   show(console.log('hello'));

   • console.log will actually output to the preview
   • Issue is that subsequent previews will show conent from previous cells
   • We are going to redefine the show function so that subsequent cells get an empty show
     • Current cell gets the real show function

CLI, Launching App, and Local Storage

1. We want to save and load a user's workbook
   • Single file to user's hardrive
   • Single file so user can share
   • File works independently from application
2. How will we launch the app?

   npx jbook serve

   • Open browser and navigate to specific port
3. Node API will be an express server

1. We could go an easy route and use React scripts to host all the different pieces
2. We are instead going to make these separate apps
   • Learn more, and future proof app
3. We are going to create npm packages for each layer in the above stack
   • jbook
   • @jbook/local-api
   • @jbook/public-api
   • @jbook/local-client
4. This allows us to use one part in a different piece
   • react app will use express api
   • express api will use cli
5. Allows us to version pieces and extract common logic into packages
6. We are not building the Public Express API
7. So, we will have multiple packages in our project...how to manage?

Lerna

Documentation

1. Tool for managing a multi-package project
2. Makes it easy to consume updates b/t our different modules on our local machine
   • Think about making an update to a package then consuming that update
   • Usually you push out the updated package
   • Conusmer then updates which version it's pointed to
3. Lerna alternatives
   • Yarn workspaces
   • NPM workspaces
   • Bolt
   • Luigi
4. Installation

   npm install -g --save-exact lerna@3.22.1

5. Lerna Project Directory
6. When using Lerna, we do not manually NPM install modules

   lerna add express --scope=cli

   • Be careful, base lerna add will add package to each local module if you don't include scope

CLI

1. Using Commander

   import { Command } from 'commander';
   
   export const serveCommand = new Command()
   	.command('serve [filename]')
   	.description('Open a file for editing')
   	.option('-p, --port <number>', 'port to run server on', '4005')
   	.action(() => {
   		console.log('Getting ready to serve a file');
   	});

   • [] denotes optional parameters
   • <> denotes if they provide the port option, they must provide a n umber

API

1. Are we actively developing our app on the local machine?

   • Use proxy to local react app dev server

   app.use(
   	createProxyMiddleware({
   		target: 'http://localhost:3000',
   		ws: true,
   		logLevel: 'silent',
   	})
   );

2. Are we running our app on a user's machine?

   • Serve up built files from build dir

   const packagePath = require.resolve('local-client/build/index.html');
   
   app.use(express.static(path.dirname(packagePath)));

Data Persistence

Fetching Cells

1. We are saving our cells locally as a json document

   [
   	{ "content": "# Test", "type": "text", "id": "vqj0q" },
   	{
   		"content": "const a = 'Test';\r\n\r\nshow(a);",
   		"type": "code",
   		"id": "lfwns"
   	}
   ]

2. Through cli args we can open a file in a specific location, otherwise (since we launch the cli manually)
   • It will be in the cli/dist folder

NPM Publishing

1. Make sure package name is unique (self explanatory)
2. Specify which files should be sent to NPM

   "files": [
   	"dist"
   ],

3. Split dependencies
   • Just make sure you don't include dependencies in the "dependencies" section you don't need

   "dependencies": {
   	"@types/express": "^4.17.11",
   	"express": "^4.17.1",
   	"typescript": "^4.2.4"
   }

   • vs

   "dependencies": {
   	"express": "^4.17.1"
   },
   "devDependencies": {
   	"@types/express": "^4.17.11",
   	"typescript": "^4.2.4"
   }

4. Set package to be publicly available
   • It's public by default, but:

   "publishConfig": {
   	"access": "public"
   },

5. If building a cli/configure file to run

   "bin": "dist/index.js",

   • Add #!/usr/bin/env node to index
6. npm publish

   npm login
   npm publish

   • Make sure you are in the root dir of the project
7. Publishing the application

   lerna publish

8. If you build lerna, but new packages aren't published

   lerna publish from-package

TODO

1. If a cell has an error, all subsequent errors show error
   • Limit just to problem cell
2. Make a small change and republish thru lerna
3. Code editor height is fixed
   • Height should adjust based on code in it
4. Better getting started notes in README
5. Context menu for saving?
   • We save automatically now
6. Can we change the command line to launch jbook?

   ghake-jsnote serve
   

7. Dark theme on render side?