Generated Files Manifest

[kevincox]

💬 RFC

Parcel should support generating a manifest file that contains all generated files and some information about them.

I have seen this touched on in a number of places so I want to create a place to centralize the discussion.

Use Cases

Offline Support

If a website wants to be usable offline it will need to prefetch all assets in a service worker. It would be nice to support the following use cases.

• Service Worker precaches all files.
• Service Worker precaches the entire new version before switching over.
• Service Worker precaches some assets. For example some content may not be required for the offline experince.

Deployment Aid.

For deploying it is helpful to know things about a file.

• Content-Type
• Is it immutable? (I think immutable == !entry but would like to see this confirmed).
• A possible extension is header generation such as Content-Security-Policy, Link and anything else.

[kevincox]

Also related to #301 because of the offline case.

[wbinnssmith]

I'm likely going to work on this in the next week or so. The use case is wanting to make entry bundles immutable (have content digests in the filename so a Cache-Control: immutable header and similar can be set). This requires something like generating a manifest file with a stable name which can translate entry files (src/index.js) to bundles (dist/index.[hash].js). These manifests will likely be generated per target.

For this use case, all that's needed is a mapping of entry files to the bundle for which it's an entry asset:

{
  "src/index.js": "dist/index.[hash].js"
}

What else should we include, and what should the manifest look like, @kevincox? Maybe these are really two different manifest files?

For the service worker case, it'll likely need a representation of the entire graph of bundles. For things like headers, I'd also rather we generate data in a generic format (e.g. json) that can be used to generate things like headers. For each bundle, we could include sha{256,384,512} checksums for subresource integrity attributes.

If we combine these into a single manifest, it could look something like this:

  "entries": {
    "src/index.js": "dist/index.[hash].js",
    // ...
  },
  "bundles": {
    "dist/index.[hash].js": {
      "id": "1234abcd",
      "mime": "application/javascript",
      "sha512": "...",
      "children": [
        "dist/page1.[hash].js", 
        "dist/page2.[hash].js"
      ]
    },
    "dist/page1[hash].js": {
      // ...
    }
  }
}

Related: Entry bundle hashing — #3307

[wbinnssmith]

Is it immutable? (I think immutable == !entry but would like to see this confirmed).

This is currently true, but I think they should be separated. Coupling them leads to odd asymmetries, like how service worker bundles are entries (to avoid having their name include a content digest), and web workers are not.

[wbinnssmith]

cc @devongovett

[wbinnssmith]

Related: #4203

[kevincox]

For this use case, all that's needed is a mapping of entry files to the bundle for which it's an entry asset:

I don't follow. Isn't an entry asset its own bundle?

What else should we include

My first thought is that we should include all generated files in the manifest. I was thinking something like:

{
	"outputs": {
		"dist/index.html": {
			"mime": "text/html",
			"is_entry": true,
			"children": [
				"dist/index.[hash].css",
				"dist/index.[hash].js",
			],
			"sources": [
				"src/index.html",
			],
		},
		"dist/index.[hash].css": {
			"mime": "text/css",
			"is_entry": false,
			"children": [
				"dist/background.[hash].png",
			],
			"sources": [
				"src/index.sass",
			],
		},
		"dist/index.[hash].js": {
			"mime": "application/javascript",
			"is_entry": false,
			"children": [],
			"sources": [
				"src/index.js",
				"node_modules/somepackage/index.js",
			],
		},
		"dist/background.[hash].png": {
			"mime": "image/png",
			"is_entry": false,
			"children": [],
			"sources": [
				"src/background.png",
			],
		},
	},
}

Overall I think that more information is better. However if we plan on feeding this back into the application (for example providing information to the service worker on what files it should download) we will have to have a well defined ways to run processes that take this manifest and produce an optimized version with only the needed information.

This is currently true, but I think they should be separated. Coupling them leads to odd asymmetries, like how service worker bundles are entries (to avoid having their name include a content digest), and web workers are not.

This doesn't seem like an asymmetry to me. A service worked needs to have a stable URL and therefore the URL will be mutated over time. A web worker doesn't as it will be loaded immediately and a dynamic URL is good for version consistency.

[kevincox]

Another thing we would probably want to include is the "main asset" for a bundle. For example if I ask parcel to bundle index.pug I would like to be able to identify what file that got turned into.

[kevincox]

Maybe these are really two different manifest files?

I see your point. I still consider your request "non-entry" assets. However you need a way to specify which assets you would like parcel to bundle. Right now parcel only supports bundling some number of entries and their dependencies. However for your use case you would like to bundle these assets without having them traded like entries (particularly avoid giving them a dynamic URL). One way to do this would be to have a separate command line flag.

parcel build --additional-bundle=src/index.js

{
	"outputs": {
		"dist/index.[hash].js": {
			"mime": "application/javascript",
			"is_entry": false,
			"children": [],
			"main_source": "src/index.js",
			"sources": [
				"src/index.js",
				"node_modules/somepackage/index.js",
			],
		},
	},
}

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs.