Assets folder

[matthewp]

Problem

Astro currently has 2 categories of files in its file structure:

1. Things in src/ get built by astro build and placed in dist/ as they are used. However, with the static build (and more importantly, SSR), we can only build things that are statically linked; ie things that are imported somehow.
2. Things in public/ are copied as-is and never touched by the build pipeline.

However there's a 3rd category of files that we don't have a good solution for. These are files that need to be transformed at runtime by components things. Use cases include:

• astro-icon processes icons at runtime to transform them and to inline SVGs.
• Image component implementations might need to dynamically build images at runtime passed on props.
• Processing large datasets such as markdown files that cannot be efficiently built in JavaScript like is currently done with Astro.fetchContent.

Proposal

I propose we have an assets/ top-level folder that is a sibling of src/ and public/. It is like public/ but with these differences:

• Unlike public/ you cannot directly load a file from assets/ in the browser (in dev or prod). It is not exposed to the web server.
• Assets are copied to a dist/server/ (bikeshed) folder to be used at runtime in SSR and in the static build.

---

Components will have access to this folder through these APIs:

Build-time transforms

if you only need to build the file once, such as SSG a markdown file, use load which loads a file from the assets/ folder.

---
import { load } from 'astro/assets';

const { name } = Astro.props;

// name === 'about.md';

let bytes = await load(name);
let html = await transformMarkdown(bytes.toString());
---

{html}

Runtime transforms

---
import { transform } from 'astro/assets';

const { size, format, name } = Astro.props;

// name === 'penguin.png'

// Use a key for determining if this file already has been processed
let cacheKey = size + '-' + 'format';

let publicPath = await transform(name, cacheKey, async bytes => {
  // This is called when there isn't a file with that cacheKey.
  return {
    data: await doExpensiveFileTransform(bytes),
    mimeType: getMime(format)
  };
});
---

<img src={publicPath} />

• transform - Loads a file in the assets folder. In dev this is assets/, in production it is dist/assets/ (bikeshed). A cacheKey is provided so that it can look to see if the file is already transformed in the .cache/ folder. If not the 3rd argument, transformer function is called.

During the build the .cache/ folder is copied to dist/.cache/ if using SSG and dist/client/.cache/ if using SSR.

Filenames will be hashed to allow forever cache headers.

[jonathantneal]

Could you expand on how 'things' are 'imported somehow'? This proposal seems to hinge on this, and so I would like to understand it better. I expect the answer will either strengthen the proposal or negate it!

[FredKSchott]

I think it may help to look at Fonts, Images, and Icons at the same time to answer this question, since each has a different set of requirements. For example:

• Icons should be inlined into SVG strings in the final JS build, to render to SVG in the final HTML
• Fonts must exist as separate files in the final build
• Images must exist as separate files in the final build, to be processed at runtime

I spent some time last night digging into this more, and wrote up some thoughts. This is where I landed: https://gist.github.com/FredKSchott/77e03f1427d28b1d00d830d926a1c18f

Main takeaway was that the real question to answer first is how we want to reference these assets in our code. This proposal assumes that dynamic references are correct, but I believe there might be some real flaws with the approach. Every other SSR-powered build tool that I know of uses ESM imports for asset references (or absolute URLs when a dynamic fallback is needed). I think we need to better understand why:

• dynamic reference: <Icon name="penguin" /> or <Icon name={result.icon} /> or <Image name="penguin" />
• static reference: import penguinSvg from '../assets/penguin.svg'; <Icon svg={penguinSvg} />

pros & cons of static vs. dynamic:

• Pros: will fail build if asset doesn't exist
• Pros: asset import works across all components (React, Svelte, etc), not just .astro
• Cons: More verbose, boilerplate, unfamiliar to some non-JS developers

There's also the <Icon pack="fontawesome" name="penguin" /> use-case to consider. Because this is dynamic, we need to load this asset dynamically at runtime from some source, currently a hosted API. That adds a moving piece to your site, and if api.astro.build/icons/... goes down, then your site degrades.

If we moved to a static reference like import {penguin} from 'astro/icons/fontawesome', then this would be fetched, loaded, and optimized at build-time. We could even remove the API, and use an npm package full of SVGs.

[FredKSchott]

Update: I did some more prototyping on the static approach, and realized fonts may be different enough from icons & images to be handled differently. I'm exploring a config-based system for fonts: https://gist.github.com/FredKSchott/77e03f1427d28b1d00d830d926a1c18f#fonts-added-via-config

I was also able to get a simple prototype working:

[matthewp]

I like a lot of this, this part seems fuzzy to me though:

Astro registers /_astro/images/* to proxy, process & cache the ?src source image

I'm not sure I understand this part. Can you explain more?