Handling static assets #292

wants to merge 1 commit into


None yet

4 participants

mde commented Jan 8, 2013

No description provided.

@mde mde referenced this pull request Jan 8, 2013

Stylus or Less Support #290

mde commented Jan 8, 2013

A few requirements:

  1. Some sort of config file that states that source files get baked into what production files
  2. Dev-mode does this on the fly for you (JS will be concatenated but not minified)
  3. Can we use file-extensions on the source files to know what sort of pre-processing to use?

Here's some of the ideas I've been having about it:

Geddy support:

  • add app/assets/{js,css} to the app generator
  • an asset_manifest_link helper which will link to a specific manifest, the given path would be relative to app/assets. This will load the specified manifests compiled contents.
  • allow the staticFilePath option to be an array, and make the default include public and app/assets so files can also be loaded from the assets path, such as images(I'm not entirely sure about this though just a quick thought, I'm not sure it's needed).
  • add an init script for assets so it loads the manifests in app/assets/{js,css} and compiles them on server start.

Example manifest JSON(paths relative to the parent dir):

   "inherits": ["ui"], // Other manifest files to inherit, may also be a single String instead of an Array
   "files": [
      "*", // Include all _files_ in this dir(ignore dirs/manifests)
      "application/base.coffee", // Include the file application/base, ignore manifests though
                   // (Maybe we should exclude file names? It would be easier if they are there, 
                   // don't have to do any weird stuff to get the file path to use)
      "home/*" // Include all _files_ in the dir home(ignoring it's dirs)

Asset manager ideas:

  • Compile manifest contents into mem(Also manage concatenation/minification?), probably something like this[1]
  • Watch files gathered from manifests, then recompile and update it's cache(unless on production of course)
  • Stop watching files on delete/move FS events(but keep in mem for use because the app would still have it?, fs.watch may already stop watching for us IDK)
  • Handle the asset compilation via extension(probably the easiest way, that's what we're doing with the templates)
  • API:
    • addDir("dir/with/manifests") Adds a dir with manifests to a list of manifest dirs
    • compile({/* options */}) Forces a compile of manifests with the given options
    • watch({/* options */}) Watches files included in manifests for FS events, managing them accordingly(should it do an initial compilation? If not it would require a call to compile({}) first)
    • getContents("js/manifest/path") Get the contents of a manifest file, including it's inherited manifests(Here we could manage concatenation/minification, might be easier to update caches if we only minify/concat when getting the actual content)
  • Options:
    • dirs Dirs to include(this could be an alternative to addDir, might be useless though)
    • cache Should we update the cache? Possibly only useful for watch()
    • helpers Object with various helpers, I'm not sure helpers are needed since it's transpiling. BUT, I think we do need a way to have geddy.viewHelpers available to assets just like they are in templates

What do you all think?


cache: {
  "js/main.json": {
    inherits: ["js/ui.json"], // Will include js/ui.json contents as well
    files: {
      "js/base.js": "contents here",
      "js/application/base.coffee": "contents here",
      "js/home/ui.coffee": "contents here"
mde commented Jan 8, 2013

I've been spending some time with this:


So far, the things that occur to me off the top of my head:

  1. I'm not keen on having a 'require' directive that's in JS comments. I think I prefer a plain JSON file that maps paths to packages, instead of having it spread all over the tree of files. I'm interested in arguments for or against this.

  2. I don't think we need any sort of file-watching ability, since in development mode, this stuff gets built on the fly, and in production, this will either be done on demand, or via a Jake task that generates the packaged assets.


What do you mean built on the fly exactly?

mde commented Jan 8, 2013

Just means that there's no need to watch the filesystem if it just does the needed compilation on every request. This is just in development mode. It will not slow things down appreciably, and it makes things way simpler. In fact, in development mode, we don't want the entire build process to run for the JS assets, because we don't want minification.


Ah okay that's fine with me!

mde commented Jan 9, 2013

Had a conversation today with @Techwraith about this, want to do a quick brain-dump:

JSON config

We want some sort of manifest JSON files (hate the word 'manifest,' sounds like XML) that map a set of source-files to a named package. Should probably allow more than one file. Should the lookup locations for these files be configurable?

These files can contain JSON for an object-literal with a "js" or "css" property, which should each be an object-literal where the keys are package-names, and the values are arrays of source-files.

These package names will be what the developer uses when calling the helpers which render tags (something like 'js_package' or 'js_tag' or 'css_package' or 'css_tag,' whatever). In development mode, it will do preprocessing and concatenation, but not minification.


Building will run whatever preprocessing, concat and minify files, and deposit them in the assets directory (should this location be configurable?), using the package-name + some hash based on the most recent mod-time out of all the files in that package. This is preferable to query-string plus timestamp of the build.

The mappings of hashes to package-name will be stored in a generated JSON file that the app can read on startup, and then use for writing out js/css tags.

If the file doesn't exist, you'll get a warning on startup that you haven't generated your assets. There will be an optional command-line switch to generate assets on app-startup.

I can't offhand think of a good, fast way of checking all the source for updated files automatically, so we'll say initially the onus is on the developer to generate assets when something has changed.

The build-process should be able to figure out what preprocessor to use based on the file-extension of the source-files.

What am I missing here, @Techwraith?


Hm I actually already have much of this code written for my project.

My code takes a source directory and generates an assets directory from it. It uses a JSON file to do the concatenation. Files not in the JSON file are just copied over verbatim (i.e. images).

It uses uglify-js to minify javascript, and the clean-css package to minify css.

Would love to talk more about how I can contribute to this.


Would love to see something like this land!


Maybe you could start another PR with your changes in it? Then we'd have actual code to look at and talk about.


@mde sure, i'll clean up my code and start one tonight.

@ben-ng ben-ng referenced this pull request May 17, 2013

Asset management #361

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment