adding plugin search and api docs in ascii doc

Readme.markdown

@@ -78,7 +78,7 @@ assets {
 
 dependencies {
   // Example additional LESS support
-  // assets 'com.bertramlabs.plugins:less-asset-pipeline:2.7.0'
+  // assets 'com.bertramlabs.plugins:less-asset-pipeline:{}'
 }
 
 ```
@@ -166,11 +166,11 @@ dependencies {
 ```
 
 
-Grails Documentation
+Documentation
 -------------
 
-* [API Doc](http://bertramdev.github.io/asset-pipeline)
-* [Grails Doc](http://bertramdev.github.io/grails-asset-pipeline)
+* [API Doc](http://www.asset-pipeline.com/apidoc/index.html)
+* [Doc](http://www.asset-pipeline.com/manual/index.html)
 
 
 For Grails 3 asset-pipeline has to be provided both for Grails and Gradle. An example configuration could be: 

asset-pipeline-core.ipr

@@ -22,6 +22,7 @@
       <module name="asset-pipeline-gradle" target="1.7"/>
       <module name="asset-pipeline-servlet" target="1.7"/>
       <module name="asset-pipeline-spring-boot" target="1.7"/>
+      <module name="asset-pipeline-grails" target="1.7"/>
     </bytecodeTargetLevel>
   </component>
   <component name="CopyrightManager" default="">
@@ -69,6 +70,8 @@
       <module fileurl="file://$PROJECT_DIR$/ratpack-asset-pipeline/ratpack-asset-pipeline.iml" filepath="$PROJECT_DIR$/ratpack-asset-pipeline/ratpack-asset-pipeline.iml"/>
       <module fileurl="file://$PROJECT_DIR$/sass-asset-pipeline/sass-asset-pipeline.iml" filepath="$PROJECT_DIR$/sass-asset-pipeline/sass-asset-pipeline.iml"/>
       <module fileurl="file://$PROJECT_DIR$/asset-pipeline-spring-boot/asset-pipeline-spring-boot.iml" filepath="$PROJECT_DIR$/asset-pipeline-spring-boot/asset-pipeline-spring-boot.iml"/>
+      <module fileurl="file://$PROJECT_DIR$/asset-pipeline-grails/asset-pipeline-grails.iml" filepath="$PROJECT_DIR$/asset-pipeline-grails/asset-pipeline-grails.iml"/>
+      <module fileurl="file://$PROJECT_DIR$/compass-asset-pipeline/compass-asset-pipeline.iml" filepath="$PROJECT_DIR$/compass-asset-pipeline/compass-asset-pipeline.iml"/>
     </modules>
   </component>
   <component name="ProjectRootManager" version="2" languageLevel="JDK_1_8" assert-keyword="true" jdk-15="true" project-jdk-type="JavaSDK" assert-jdk-15="true" project-jdk-name="1.8">

asset-pipeline-docs/build.gradle

@@ -0,0 +1,26 @@
+buildscript {
+    repositories {
+        jcenter()
+    }
+
+    dependencies {
+        classpath 'org.asciidoctor:asciidoctor-gradle-plugin:1.5.0'
+    }
+}
+
+apply plugin: 'org.asciidoctor.gradle.asciidoctor'
+
+
+asciidoctor { 
+    outputDir = new File("$buildDir/docs")
+    options = [
+        doctype: 'book',
+        attributes: [
+            'source-highlighter': 'coderay',
+            toc                 : 'left',
+            'toc-title': 'Table of Contents',
+            idprefix            : '',
+            idseparator         : '-'
+        ]
+    ]
+}

asset-pipeline-docs/src/asciidoc/concepts.adoc

@@ -0,0 +1,121 @@
+[[concepts]]
+== Concepts
+
+Asset-pipeline is a highly extensible asset processing library that is easy to integrate into your jvm based application or to even use standalone with gradle. It works by defining `AssetFile` definitions for different file types in which `Processors` can be chained for processing and conversion to different target content types. But to use it you don't really need to know about any of this (It really does just work).
+
+=== Directives
+
+Similar to other packaging libraries like webpack or grunt, asset-pipeline provides a means to require other files or "modules" into your javascript and/or css. We will call these require patterns `directives`.
+
+NOTE: Asset-pipeline chooses to be dependency syntax agnostic. This makes it very easy to add in extension modules for things like CommonJS.
+
+In asset-pipeline it is a common pattern to list all requirements of your file at the top in a comments section but this is not required. Example:
+
+[source,javascript]
+----
+//This is a JavaScript file with its top level require directives
+//= require jquery
+//= require app/models.js
+//= require_tree views
+//= require_self
+
+console.log("This is my javascript manifest");
+----
+
+Or similarly a CSS file might look like this:
+
+[source,css]
+----
+/*
+This is a comment block in the top of a css file
+*= require bootstrap
+*= require font-awesome
+*= require navigation/header.css
+*= require_tree components
+*/
+
+body {
+  font-size: 12px;
+}
+----
+
+This may look very simple at first but can become really quite powerful. For example, these directives can be nested in other files that you might include. This allows the asset-pipeline to build a dependency-graph, eliminating duplicate requirements, and ensuring everything is loaded in the most optimal order possible.
+
+IMPORTANT: Requirement directives are recursively scanned within each required file, not just the first file.
+
+One other cool aspect of these require directives is the extension is not mandatory. Asset-pipeline will automatically look for any file matching that name (minus the extension) that has a registered `AssetFile` capable of converting it to the intended target file type. i.e. that Bootstrap directive could be including a LESS file or the javascript could easily include a Coffeescript file.
+
+==== Encoding
+
+In some cases it may be necessary to specify an encoding for your assets. An example might be for Japanese characters in a JavaScript file. To do this, two things must be done. First, we simply set the charset attribute when we include are JavaScript file:
+
+[source,html]
+----
+<script src="application.js" charset="utf-8"/>
+----
+
+This should take care of testing in development mode and debugging. However, when we move to production/WAR mode the precompiler has no way to infer the desired encoding for compilation. To accomplish this, we have the `//= encoding` directive. This can be placed at the top of your `application.js` to define the desired compilation encoding.
+
+[source,javascript]
+----
+//=encoding UTF-8
+//=require_self
+//=require_tree .
+----
+
+That's all there is to it.
+
+
+=== Project Structure
+
+Depending on which framework integration is being used (or even if no framework integration is being used) the project of the asset-pipeline files can slightly differ. However, typically this only affects the source folder location while all other aspects of the project structure remain the same.
+
+By default assets live in the `src/assets` directory of the project (except in the case of grails where these live in `src/assets`). However, files should not live at this direct root level and by default asset-pipeline will not detect them if they were. It is a conventional practice that files be organized in one level down subfolders. For example, in a base grails app the following folder structure is used and encouraged:
+
+* `assets/javascripts`
+* `assets/stylesheets`
+* `assets/images`
+* `assets/libs` -- common naming for components that contain both css/javascript/images
+
+Asset-pipeline will automatically add any subfolder in the `assets` directory as a `FileResolver`. Meaning each folder is treated as a root level resolver path. It is also important to note that asset-pipeline does not actually care what file types go where. This is purely for the organizational benefit of the project.
+
+IMPORTANT: When requiring files using require directives do not include these directory names, treat them as root level traversal.
+
+=== Search Paths
+
+When a file is referenced via a require directive or web request, the asset-pipeline checks for the file in several locations.
+
+First it tries to find the file relative to the manifest including it. For example "admin/application.js" looking for "table.js"
+
+[source,javascript]
+----
+// FileName: admin/application.js
+//= require table
+----
+
+The first place we will look is within `src/assets/javascripts/admin/*` We will proceed to do this within all of the asset sub folders across plugins after the main application is searched.
+
+The next place we will look is the root of all src/assets plugin sub folders (e.g. `src/assets/*/table.js`).
+
+Finally all binary plugins are scanned in the `classpath:META-INF/assets` folder, `classpath:META-INF/static` and `classpath:META-INF/resources`.
+
+In all cases, the applications assets folder takes precedence between the search paths, but plugins get scanned as well.
+
+=== Build Structure
+
+When the project is built (in Gradle that's the `assetCompile` task) these folders are flattened and merged. Meaning this first level subdirectory structure disappears and all files are copied into `build/assets`. These files also get md5 digested names for cache busting as well as GZIP versions for compressed file serving (except already compressed images).
+
+In a Java based framework any type of WAR or JAR packaging typically gets detected and assets are automatically moved into `classpath:assets` along with a file `classpath:assets/manifest.properties`. This manifest is a list of every file that was packaged by the asset-pipeline as the key, and the digested name equivalent as the value. This facilitates easy differential syncing between CDNS (like an s3 bucket for cloudfront) as well as fast and easy generation of ETag headers when serving assets from the application (more on this later).
+
+
+=== Relative Urls
+
+With all this renaming of assets with digested names as well as slight restructuring between source and build, what happens with url references specified in `CSS` as well as `HTML`. This is where some familiar with the Ruby on Rails sprockets based asset-pipeline might remember the need for erb helpers to specify path replacement. The asset-pipeline for the JVM, however, takes a different more automatic approach:
+
+All CSS type files go through a processor called the `CssProcessor`. This processor looks for any `url(../path/to/file.png)` type patterns and automatically resolves the asset from the asset-pipeline. If it finds the matching file, the url is automatically replaced with the correct url pattern including the digest name: `url(path/file-dadvbfgdaf123e.png)`.
+
+This relative url replacement is really handy because any external css library that is included in your project (i.e. bootstrap) can be used as is without any need to sift through its code and replace url patterns to match. This feature is also performend on `HTML` files.
+
+TIP: HTML files automatically get relative url replacement making it easy to generate 100% static websites without any need for a dynamic templating engine.
+
+Currently javascript is not scanned for relative path replacement. It is common practice to create a base path variable if required in the html that includes your javascript if your javascript indeed needs to reference images or documents in the asset-pipeline.

asset-pipeline-docs/src/asciidoc/extending.adoc

@@ -0,0 +1,8 @@
+[[extending]]
+== Extending
+
+The asset-pipeline is extremely extensible and easy to customize to suit one's needs. You might extend the asset-pipeline to handle a new type of asset that may need to be preprocessed before being served to the browser, or you may want to define a new custom directive. This guide will go over the basics of how to perform those tasks with ease.
+
+include::extending/assetfile.adoc[]
+include::extending/processors.adoc[]
+include::extending/postprocessors.adoc[]

asset-pipeline-docs/src/asciidoc/extending/assetfile.adoc

@@ -0,0 +1,113 @@
+=== Asset File Definition
+
+The `AssetFile` definition is where our journey begins. This is the defining file for various file types. Without this definition, the asset-pipeline will treat an unknown file type as a standard passthrough resource. As an example, lets first look at the `CssAssetFile` definition.
+
+[source,groovy]
+----
+class CssAssetFile extends AbstractAssetFile {
+  static final String contentType = 'text/css'
+  static extensions = ['css']
+  static compiledExtension = 'css'
+  static processors = [CssProcessor]
+
+  String directiveForLine(String line) {
+    line.find(/\*=(.*)/) { fullMatch, directive -> return directive }
+  }
+}
+----
+
+This file definition is pretty short but allows us to define some very useful information. First, we look at the static definitions at the top of the class. These static definitions are fairly easy to meta-override with Groovy and add additional processors or adjust with added plugins.
+
+The `contentType` property is used to match a file definition with an incoming file request. When the browser requests a `text/css` content-type file , this file is matched and files matching this definition are scanned. The `extensions` list tells asset-pipeline which file extensions to scan through and match. In this case it is just 'css', but in the case of LESS for example, we may be looking for extensions `less`, or `css.less`.
+
+The `compiledExtension` property tells asset-pipelines precompiler what the final file extension should be.
+
+Finally, the `processors` array determines the list of processors that need be run on the file contents before returning a result. This array is executed in order. In this case, we have the `CssProcessor` (a processor for converting the relative image paths and replacing with their cache digested version).
+
+==== Directive Definition
+
+An `assetFile` can specify a REGEXP pattern for require directives. These directives are used to bundle assets together. Some file types don't utilize these require directives and simply returning a null value will cancel directive processing.
+
+[source,groovy]
+----
+  Pattern directivePattern = ~/(?m)\*=(.*)/
+----
+
+NOTE: Used to there was a `directiveForLine` that matched on each individual line. This was changed to support a multiline regex pattern for faster processing.
+
+The example above shows a match pattern for CSS files. This allows it to match require directives for the following example:
+
+[source,groovy]
+----
+/*
+*= require_self
+*= require_file example_b
+*= require_tree .
+*/
+
+body {
+  margin-top:25px;
+}
+----
+
+==== Processing Data Streams
+
+Processors are used to precompile certain assets, and/or adjust the file path contents. The Processor class itself will get a more in depth explanation in the next section. For now, the part we want to look at is the processedStream function.
+
+[source,groovy]
+----
+String processedStream(Boolean precompiler) {
+  def fileText
+  def skipCache = precompiler ?: (!processors || processors.size() == 0)
+
+  if(baseFile?.encoding || encoding) {
+    fileText = file?.getText(baseFile?.encoding ? baseFile.encoding : encoding)
+  } else {
+    fileText = file?.text
+  }
+
+  def md5 = AssetHelper.getByteDigest(fileText.bytes)
+  if(!skipCache) {
+    def cache = CacheManager.findCache(file.canonicalPath, md5)
+    if(cache) {
+      return cache
+    }
+  }
+  for(processor in processors) {
+    def processInstance = processor.newInstance(precompiler)
+    fileText = processInstance.process(fileText, this)
+  }
+
+  if(!skipCache) {
+    CacheManager.createCache(file.canonicalPath,md5,fileText)
+  }
+
+  return fileText
+}
+----
+
+The example above iterates over all of the processor classes defined in our static `processors` variable. This creates a new instance and informs the processor whether this is a developer mode request or being issued by the precompiler (useful for determining if file replacements need to be cache digested or not). The `processedStream` method is now a part of the `AbstractAssetFile` definition and handles cache management if there are processors.
+
+==== Adding the Asset definiton to the list of AssetFiles
+
+Originally we had to add these classes on startup in both runtime and build phases to the `AssetHelper.assetSpecs` array. Thanks to contributions by Graeme Rocher we have been able to simplify this process.
+Simply adding a list file `META-INF/asset-pipeline/asset.specs` to the classpath will automatically get scanned.
+
+Example:
+
+[source,groovy]
+----
+asset.pipeline.HtmlAssetFile
+asset.pipeline.JsAssetFile
+asset.pipeline.CssAssetFile
+----
+
+Another autoscanning file allows us to tack on Processors to already registered AssetFile specifications. This is called the `processor.specs` file and goes in the same `META-INF/asset-pipeline` folder.
+This is a Properties file with the key being the class path of the Processor and the value being a comma delimited list of `AssetFile` classes you want the processor added to.
+
+Example:
+
+[source,groovy]
+----
+asset.pipeline.CssProcessor=asset.pipeline.CssAssetFile,asset.pipeline.LessAssetFile
+----

asset-pipeline-docs/src/asciidoc/extending/postprocessors.adoc

@@ -0,0 +1,3 @@
+=== Post-Processors
+
+Currently, PostProcessor extensibility is not available. This is currently a feature we are implementing to provide easier dropin for custom minifiers and compressors.

asset-pipeline-docs/src/asciidoc/extending/processors.adoc

@@ -0,0 +1,43 @@
+=== Processors
+
+Processors are where the real power of asset-pipeline comes into play. These are the driving force behind making compileable assets such as LESS, and CoffeeScript first class citizens. Gone is the need to run a compiler on the side, and gone is the delay between making changes in development.
+
+A Processor is an implementation of the `Processor` interface via the `AbstractProcessor` class. It must have a constructor with an `AssetCompiler` argument, and it must have a process method. The rest is up to the developer.
+The reason the AssetCompiler is passed is for giving the processor access to manipulate the precompiler phase. If a null precompiler is passed, than development mode is assumed and the processor can infer that. An example use case
+for this is the SassProcessor in the SASS/SCSS Asset Pipeline Plugin. Image sprite generation causes additional image files to be created that need added to the list of files to process.
+
+[source,groovy]
+----
+class CoffeeScriptProcessor extends AbstractProcessor {
+
+  Scriptable globalScope
+  ClassLoader classLoader
+
+  CoffeeScriptProcessor(AssetCompiler precompiler){
+    super(precompiler)
+  }
+
+  String process(String input, AssetFile assetFile) {
+    try {
+      def cx = Context.enter()
+      def compileScope = cx.newObject(globalScope)
+      compileScope.setParentScope(globalScope)
+      compileScope.put("coffeeScriptSrc", compileScope, input)
+      def result = cx.evaluateString(compileScope, "CoffeeScript.compile(coffeeScriptSrc)", "CoffeeScript compile command", 0, null)
+      return result
+    } catch (Exception e) {
+      throw new Exception("""
+        CoffeeScript Engine compilation of coffeescript to javascript failed.
+        $e
+        """)
+    } finally {
+      Context.exit()
+    }
+  }
+}
+----
+
+Above is an excerpt of the `CoffeeScriptProcessor` plugin. This plugin takes advantage of RhinoJS to use the CoffeeScript compiler and provide the application with direct support for CoffeeScript files. The `process` method takes an input, as well as a reference to the asset file definition, and returns a result. To use your processor simply add it to your 'static processors' array on the `AssetFile` definition.
+
+The `LESSProcessor` was not used in this example as it's more complicated due to supporting the `@import` LESS directive and cache dependencies on the cache manager. However, it is a great example to look at and highly recommended.
+

asset-pipeline-docs/src/asciidoc/gradle.adoc

@@ -0,0 +1,9 @@
+[[gradle]]
+== Gradle
+
+This section of the documentation discusses configuration and setup of the asset-pipeline gradle plugin. Most framework integrations use this plugin for builds so it is important to know how it works.
+
+include::gradle/getting_started.adoc[]
+include::gradle/configuration.adoc[]
+include::gradle/java.adoc[]
+