RFC: Import Map Support (draft)

[matthewmueller]

Background

In v0.3, we'd like to introduce import maps and remove the Node.js requirement.

While this simplifies the internals, the primary reason is to simplify deploys. Right now Bud applications don't fit within a single buildpack on Fly/Heroku/Vercel, they're a blend of Go and Node. In order to deploy a Bud app, you'll need to write your own Docker file. After v0.3, Bud should look and feel like a stock Go application. You'll still probably need a Procfile, but that's about it.

This was primarily inspired by the Rails 7 demo, where I first learned about import maps and realized you don't need Node anymore.

Goals

1. Manage imports maps either through configuration or the CLI
2. Import map configuration is treated the same as other Bud configuration
3. Configurable CDN
4. Maybe: Long-term caching using Import Maps

API

Configuration

The source of truth for import maps is in the configuration. Import maps are configured in Go files. Here's an idea of how this might work below:

package env

import "github.com/livebud/bud/package/esm"

func Import() esm.Map {
  return esm.Map{
    "lodash": "https://cdn.jsdelivr.net/npm/lodash",
  }
}

(TODO: this section highly depends on the not-yet-written RFC: Env Support)

Open Questions

• Should configuration be done in a configuration language like JSON, HCL or Cue? I like the type-safety, composability of using Go, but if we go with Go, we won't be able to provide configuration to the pre-generators since we'll need to evaluate the environment during that time. This is also a question for RFC: Env Support.

CLI

There should be some way to manage import maps from the CLI. This probably would come as a follow-up to the initial configuration-based release.

Something like:

bud get lodash
bud install lodash
bud import lodash
bud js import lodash
bud im install lodash

Open Questions

• Should we unify go get and import map installation under bud get ? Would there be conflicts? Are they separate concerns?
• Should import map management live under a namespace? You could imagine other commands like uninstall, upgrade, etc.

Usage

bud run

In general, bud run should behave similar to how the browser behaves, where a browser refresh will give you the latest content ever time.

I think this means that our bundler will need to resolve remote imports per request and will use the import map to transform:

• From: import { uniqBy } from 'lodash/uniqBy'
• To: import { uniqBy } from 'https://cdn.jsdelivr.net/npm/lodash'

bud build

bud build should pre-resolve, tree-shake, bundle and minify dependencies. This means it will resolve all dependencies during the build step. It should also prepare the assets for long-term caching (TBD).

Long-Term Caching using Import Maps (maybe)

There's a way to rewrite file paths using import maps. This allows us not to require the user to pass around dynamic paths, but instead use import maps to rewrite file paths. See this clip for more details.

While surely simpler, it doesn't seem as flexible as something like Rail's asset_path because it doesn't support other assets like images.

Resources

• https://www.youtube.com/watch?v=veWpot6zfUQ
• https://deno.land/manual/linking_to_external_code/import_maps
• https://www.youtube.com/watch?v=yOcgGSCrn-c
• https://github.com/trygve-lie/esbuild-plugin-import-map#readme

[syke99]

Should configuration be done in a configuration language like JSON, HCL or Cue? I like the type-safety, composability of using Go, but if we go with Go, we won't be able to provide configuration to the pre-generators since we'll need to evaluate the environment during that time. This is also a question for RFC: Env Support.

I think you'd get the most mileage out of JSON. All web-devs are going to be most familiar with it, and Go has really good built-in interfacing with JSON objects through marshalling/unmarshalling and encoding/decoding when needed. So you wont be scaring anyone off by making them have to learn some other specific language that isn't JSON, and you'll be a lot less likely to run into any headaches that haven't been encountered by another gopher. Plenty of benefit, both for users and maintainers, imo 😋

• Should we unify go get and import map installation under bud get ? Would there be conflicts? Are they separate concerns?
• Should import map management live under a namespace? You could imagine other commands like uninstall, upgrade, etc.

I'm not sure about there being conflicts (though I'd err towards expecting there to be, at least to begin with), but I would suggest the naming convention follow Ruby's with something akin to:

// add es module to import map
bud pin <es module>

// remove es module from import map
bud unpin <es module>

// upgrade es module in import map,
// -u, --upgrade - flag denotes upgrading
// -v, --version - optional flag for specifying
//                 specificversion to upgrade to. 
//                 (default's to '@latest' if not 
//                 specified)
bud pin -u <es module> -v <version number>

This also seems to follow the naming convention for es module URLs across different CDNs. Speaking of, you mentioned being able to configure which CDN to use.

Goals

1. Manage imports maps either through configuration or the CLI
2. Import map configuration is treated the same as other Bud configuration
3. Configurable CDN
4. Maybe: Long-term caching using Import Maps

At least in the CLI, you could have various other flags as well, such as:

-p, --provider - provider, which es module provider you want to pin your module 
                 from (defaults to <whichever CDN you, @matthewmueller , want to default to> 
                 but you can use this flag to choose between <list of CDNs we would want to 
                 support>, (JSPM's available options: 'jspm' | 'jspm.system' | 'skypack' | 
                 'jsdelivr' | 'unpkg'))

-e, --env - env, specify which environment you want the es module's import map 
            to be built for (JSPM's available options: 'browser', 'development', 'module'))

-f, --flatten - flattenScope, specify whether you want the es module's import map's
                scope to be flatten, if possible (default's to false)

-g, --graph - provides a traced analysis graph (may not be provided by all es module CDNs, but is by JSPM)

[matthewmueller]

Thanks for sharing your thoughts Quinn!!

I think you'd get the most mileage out of JSON. All web-devs are going to be most familiar with it, and Go has really good built-in interfacing with JSON objects through marshalling/unmarshalling and encoding/decoding when needed.

I agree the familiarity with JSON is a big one and that it's the safe choice. If we went with this I'd probably go with hujson or "json with comments and commas".

I was slightly leaning towards HCL because you can have functions and avoid excessive strings:

database postgres {
  url = env("POSTGRES_URL") 
}

Admittedly, this isn't the safe or familiar choice.

One of the design requirements is going to be having a configuration schema. Both JSON (via JSONSchema) and HCL support validating against a schema.

A nice to have that you get out of the box with using Go as the configuration language is IDE support for autocomplete and type-checking. I've opened an issue against Schema Store, which is what VSCode uses to provide IDE-like experience for JSON files

but I would suggest the naming convention follow Ruby's with something akin to

While I like the ergonomics of Rail's API, I'm leaning towards something familiar to people that use NPM because NPM usage spans across ecosystems.

I may even be okay with:

bud npm install lodash
bud npm uninstall lodash

Though probably not because that would assume we're shelling out to NPM and expect all the NPM features. Also "Node Package Manager" 😅

Configurable CDN

Ah after your message, I just realized there are two contexts of CDNs.

I was originally talking about being able to configure where you serve assets from your production CDN, not which CDN you download assets from.

I don't expect bud production apps to have import lodash from "jsdelivr.net/npm/lodash", instead it'll all be bundled together where your assets are served from some CDN like Cloudflare with the bud web server as your origin server.

I guess we'll also need some configuration to allow overriding what the default CDN is. Rails 7 mentioned this as well, that they default to jsdelivr, but allow you to configure it. I think we should mostly follow their lead on this one, just keeping in mind that our intended audience is NPM users rather than Rails users.