Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assets build #201

Merged
merged 66 commits into from May 16, 2014
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
d34267c
first version of build step for assets
nsams Apr 27, 2014
190453b
add component settings to build
nsams Apr 29, 2014
4662b92
add component events to build
nsams Apr 29, 2014
d8b8b26
add trl cache to build
nsams Apr 29, 2014
102cb26
don't check for db in build, build should be independent of db
nsams Apr 30, 2014
3602c23
only build packages
nsams Apr 30, 2014
2e09547
ccw for component settings in build
nsams Apr 30, 2014
73d0084
create build folders automatically
nsams Apr 30, 2014
5cf2868
if we don't have a db configured yet skip kwc resources
nsams Apr 30, 2014
a10f5ba
add build folder to gitignore
nsams Apr 30, 2014
9960539
cc: check build for up to date git revision
nsams May 2, 2014
41c6e79
fix ccw for assets
nsams May 5, 2014
6042d2d
allow lazy building, required for tests
nsams May 5, 2014
f0f6e50
improve build version check
nsams May 5, 2014
eabd7ed
move sass imports to correct folders to match their import name
nsams May 5, 2014
62df6f5
use libsass/sassc for compiling scss
nsams May 5, 2014
3128b1a
add susy and compass-mixins as individual libraries
nsams May 5, 2014
007e318
don't use stylesheet-url, doesn't work with libsass
nsams May 5, 2014
19bbd56
remove phpsass
nsams May 5, 2014
94d085e
allow detecting of all possible languages statically
nsams May 6, 2014
9b9f405
GoogleMaps dependency: don't inherit File as it doesn't represent a file
nsams May 6, 2014
8e87294
improve dependency toString, always output class name
nsams May 6, 2014
43489ce
ugly workaround assets trl issue: don't use " in trl
nsams May 6, 2014
bd54b3b
first version of uglifyjs usage with sourcemaps support
nsams May 6, 2014
bdd1fc2
fix error in scss code that causes fatal error if compiling using lib…
nsams May 12, 2014
cb46230
simplify assets path handling: only use absolute file path when acces…
nsams May 12, 2014
5355e63
process compiled scss with keeping source map intact
nsams May 12, 2014
bd4bdaa
use hand written implementation to combine multiple sourcemaps
nsams May 12, 2014
4f65212
append sourcemapping url correctly for css files
nsams May 12, 2014
30d8980
move sass include paths to path to be able to load those included fil…
nsams May 12, 2014
65b8583
build sourcemap with our last extension
nsams May 12, 2014
bbd040d
read uglifyjs binary path from config
nsams May 12, 2014
680eb59
source map aware replacmenets for js
nsams May 12, 2014
6f626f3
perform string replacements correctls for js
nsams May 12, 2014
4a53c0d
add progress bar for assets build
nsams May 12, 2014
790855b
don't add newlines in dynamic assets
nsams May 13, 2014
07050d9
correctly replace sourcemappingurl comment
nsams May 13, 2014
de9265d
fix issues with concatted source map
nsams May 13, 2014
0ed12eb
don't require If-Modified-Since header to equal, it also can be after…
nsams May 13, 2014
27bafb3
let scss cache expire if an included file changed
nsams May 13, 2014
ca862fd
support baseUrl in assets
nsams May 13, 2014
ea7a2c2
add script to build all required dependencies
nsams May 13, 2014
5dbb809
fix output-cache-ids saving to cache
nsams May 13, 2014
81f71c6
only save to build folder while building
nsams May 13, 2014
936fe25
add setting to enable/disable lazy building of assets
nsams May 13, 2014
8bd2c4d
pack all maintenance dependencies into a single package
nsams May 13, 2014
5f2bd7a
build maintenance dependencies if enabled
nsams May 13, 2014
ed36f4e
add tests for sourcemap util
nsams May 13, 2014
41ac75c
use replaced contents also when replacing it
nsams May 13, 2014
c89fff6
add providerlist to cache id and read all providers from cache
nsams May 13, 2014
04e1257
add assetsVersion in build
nsams May 13, 2014
17294b6
fix text styles model asset dispatch
nsams May 13, 2014
aabf489
allow running clear-cache and build without having to define server.b…
nsams May 13, 2014
96fd687
check for build folder before starting maintenance controllers
nsams May 13, 2014
227e13b
improve error handling if not yet set up
nsams May 13, 2014
a936f57
check for correctly configured uglifyjs and sassc before building
nsams May 13, 2014
ae6e80e
save uglifyjs build results in library instead of web
nsams May 14, 2014
f1df231
fix tests
nsams May 14, 2014
c5625be
don't clear assets build cache, fill with 'outdated' content
nsams May 14, 2014
4fd90e4
restrict access to sourcemaps to given ip addresses, default to any
nsams May 14, 2014
167d7af
fix lazy load tests
nsams May 15, 2014
18951c6
don't check for build folder if calling clear-cache in kwf
nsams May 15, 2014
7aac7c6
rename getPossibleLanguages in getAvailableLanguages
nsams May 15, 2014
582d659
improve update script
nsams May 15, 2014
94ec702
fix travis build
nsams May 15, 2014
3c91477
use https git repository urls
nsams May 15, 2014
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 4 additions & 0 deletions .travis.yml
Expand Up @@ -4,6 +4,8 @@ php:
before_script:
- echo "extension = apc.so" >> ~/.phpenv/versions/$(phpenv version-name)/etc/php.ini
- echo "extension = memcache.so" >> ~/.phpenv/versions/$(phpenv version-name)/etc/php.ini
- npm install uglify-js
- git clone https://github.com/nsams/libsass.git; cd libsass; make; git clone https://github.com/hcatlin/sassc.git ; cd sassc; SASS_LIBSASS_PATH=.. make; cd ../..
- printf "\n" | pecl install imagick
- echo "short_open_tag = 1" >> ~/.phpenv/versions/$(phpenv version-name)/etc/php.ini
- git clone git://github.com/vivid-planet/library.git
Expand All @@ -13,6 +15,8 @@ before_script:
- echo "[production]" > tests/config.local.ini
- echo "server.domain = localhost" >> tests/config.local.ini
- echo "libraryPath = `pwd`/library" >> tests/config.local.ini
- echo "server.uglifyjs = `pwd`/node_modules/.bin/uglifyjs" >> tests/config.local.ini
- echo "server.sassc = `pwd`/libsass/sassc/bin/sassc" >> tests/config.local.ini
- chmod a+w tests/cache/*
- chmod a+w tests/temp
services:
Expand Down
2 changes: 1 addition & 1 deletion Kwc/Basic/Text/StylesModel.php
Expand Up @@ -46,7 +46,7 @@ public static function getMasterStyles()
$ret = array();
foreach ($package->getDependency()->getFilteredUniqueDependencies('text/css') as $dep) {
if ($dep instanceof Kwf_Assets_Dependency_File) {
$ret = array_merge($ret, self::parseMasterStyles(file_get_contents($dep->getFileName())));
$ret = array_merge($ret, self::parseMasterStyles(file_get_contents($dep->getAbsoluteFileName())));
}
}
Kwf_Cache_SimpleStatic::add($cacheId, $ret);
Expand Down
2 changes: 1 addition & 1 deletion Kwc/Newsletter/Detail/StartNewsletterPanel.scss
Expand Up @@ -44,7 +44,7 @@
border: 1px solid #5B992B;
border-bottom: 1px solid #538C27;
@include box-shadow-pie(#548c29 0px 0px 8px 4px inset);
},
}

&[disabled=disabled], &[disabled] {
@include opacity(0.6);
Expand Down
7 changes: 7 additions & 0 deletions Kwc/Root/LanguageRoot/Component.php
Expand Up @@ -11,9 +11,16 @@ public static function getSettings()
'en'=>'Kwc_Root_LanguageRoot_Language_Component'
)
);
$ret['flags']['hasAvailableLanguages'] = true;
return $ret;
}

public static function getAvailableLanguages($componentClass)
{
$g = Kwc_Abstract::getSetting($componentClass, 'generators');
return array_keys($g['language']['component']);
}

public function getPageByUrl($path, $acceptLangauge)
{
if ($path == '') {
Expand Down
11 changes: 11 additions & 0 deletions Kwc/Root/TrlRoot/Component.php
Expand Up @@ -17,6 +17,17 @@ public static function getSettings()
'uniqueFilename' => true,
);
$ret['childModel'] = new Kwc_Root_TrlRoot_Model(array('de' => 'Deutsch'));
$ret['flags']['hasAvailableLanguages'] = true;
return $ret;
}

public static function getAvailableLanguages($componentClass)
{
$ret = array();
$rows = Kwf_Model_Abstract::getInstance(Kwc_Abstract::getSetting($componentClass, 'childModel'))->getRows();
foreach ($rows as $row) {
$ret[] = $row->id;
}
return $ret;
}

Expand Down
2 changes: 2 additions & 0 deletions Kwf/Acl.php
Expand Up @@ -291,6 +291,8 @@ public function getRoles()
*/
public function loadKwcResources()
{
if (!Kwf_Registry::get('db')) return; //if we don't have a db configured yet skip kwc resources. required to be able to build assets without db

if ($this->_kwcResourcesLoaded) return;
$this->_kwcResourcesLoaded = true;

Expand Down
71 changes: 71 additions & 0 deletions Kwf/Assets/BuildCache.php
@@ -0,0 +1,71 @@
<?php
class Kwf_Assets_BuildCache
{
static private $_buildDir = "build/assets";
public $building = false;
public static function getInstance()
{
static $cache;
if (!isset($cache)) {
$cache = new self();
}
return $cache;
}

public function load($cacheId)
{
$ret = Kwf_Cache_SimpleStatic::fetch('as-'.$cacheId);
if ($ret === false) {
$fileName = self::$_buildDir.'/'.$cacheId;
if (!file_exists($fileName)) {
return false;
}
$ret = unserialize(file_get_contents($fileName));
if ($ret !== false) {
Kwf_Cache_SimpleStatic::add('as-'.$cacheId, $ret);
}
}
return $ret;
}


public function save($cacheData, $cacheId)
{
Kwf_Cache_SimpleStatic::add('as-'.$cacheId, $cacheData);
$fileName = self::$_buildDir.'/'.$cacheId;
return file_put_contents($fileName, serialize($cacheData));
}
/*
public function test($cacheId)
{
$ret = Kwf_Cache_SimpleStatic::fetch('as-mtime-'.$cacheId);
if ($ret === false) {
$ret = self::_getSlowCache()->test($cacheId);
if ($ret !== false) {
Kwf_Cache_SimpleStatic::add('as-mtime-'.$cacheId, $ret);
}
}
return $ret;
}
*/
public function clean()
{
if (!$this->building) {
throw new Kwf_Exception("Can't clear out of build");
}
Kwf_Cache_SimpleStatic::clear('as-');
foreach (glob(self::$_buildDir.'/*') as $f) {
unlink($f);
}
}

public function remove($cacheId)
{
Kwf_Cache_SimpleStatic::_delete('as-'.$cacheId);
$fileName = self::$_buildDir.'/'.$cacheId;
if (file_exists($fileName)) {
unlink($fileName);
return true;
}
}
}
23 changes: 22 additions & 1 deletion Kwf/Assets/Dependency/Abstract.php
Expand Up @@ -16,6 +16,11 @@ public function getContentsPacked($language)
return $this->getContents($language);
}

public function getContentsPackedSourceMap($language)
{
return false;
}

public function setDependencies($type, $deps)
{
$this->_dependencies[$type] = $deps;
Expand Down Expand Up @@ -54,6 +59,10 @@ public function getMTime()
return null;
}

public function warmupCaches()
{
}

public function __toString()
{
return get_class($this);
Expand All @@ -64,6 +73,19 @@ public function toDebug()
return get_class($this).': '.$this->__toString()."\n";
}

public function getRecursiveDependencies()
{
$it = new RecursiveIteratorIterator(new Kwf_Assets_Dependency_Iterator_UniqueFilter(new Kwf_Assets_Dependency_Iterator_Recursive($this, Kwf_Assets_Dependency_Abstract::DEPENDENCY_TYPE_ALL)), RecursiveIteratorIterator::CHILD_FIRST);
$array = array();
$array[] = $this;
foreach ($it as $i) {
if ($i !== $this) {
$array[] = $i;
}
}
return $array;
}

public function getRecursiveFiles()
{
$it = new RecursiveIteratorIterator(new Kwf_Assets_Dependency_Iterator_UniqueFilter(new Kwf_Assets_Dependency_Iterator_Recursive($this, Kwf_Assets_Dependency_Abstract::DEPENDENCY_TYPE_ALL)), RecursiveIteratorIterator::CHILD_FIRST);
Expand Down Expand Up @@ -143,5 +165,4 @@ private function _getFilteredUniqueDependenciesProcessDep($dep, $mimeType, &$pro
}
return $ret;
}

}
15 changes: 2 additions & 13 deletions Kwf/Assets/Dependency/Dynamic/GoogleMapsApiKeys.php
@@ -1,11 +1,6 @@
<?php
class Kwf_Assets_Dependency_Dynamic_GoogleMapsApiKeys extends Kwf_Assets_Dependency_File
class Kwf_Assets_Dependency_Dynamic_GoogleMapsApiKeys extends Kwf_Assets_Dependency_Abstract
{
public function __construct()
{
parent::__construct(null);
}

public function getMimeType()
{
return 'text/javascript';
Expand All @@ -14,12 +9,6 @@ public function getMimeType()
public function getContents($language)
{
$json = json_encode(Kwf_Config::getValueArray('googleMapsApiKeys'));
return "Ext.namespace('Kwf.GoogleMap');\nKwf.GoogleMap.apiKeys = $json;\n";
return "Ext.namespace('Kwf.GoogleMap');Kwf.GoogleMap.apiKeys = $json;";
}

public function getFileName()
{
return null;
}

}
9 changes: 1 addition & 8 deletions Kwf/Assets/Dependency/Dynamic/LoaderConfig.php
@@ -1,12 +1,11 @@
<?php
class Kwf_Assets_Dependency_Dynamic_LoaderConfig extends Kwf_Assets_Dependency_File
class Kwf_Assets_Dependency_Dynamic_LoaderConfig extends Kwf_Assets_Dependency_Abstract
{
protected $_providerList;

public function __construct(Kwf_Assets_ProviderList_Abstract $providerList)
{
$this->_providerList = $providerList;
parent::__construct(null);
}

public function getMimeType()
Expand All @@ -25,10 +24,4 @@ public function getContents($language)
$ret .= "window.Kwf.Loader.config = ".json_encode($config).";\n";
return $ret;
}

public function getFileName()
{
return null;
}

}
2 changes: 1 addition & 1 deletion Kwf/Assets/Dependency/Dynamic/RavenJsDsn.php
Expand Up @@ -14,6 +14,6 @@ public function getContents($language)
$data = array(
'dsn' => Kwf_Config::getValue('ravenJs.dsn')
);
return "Kwf.RavenJsConfig = ".json_encode($data).";\n";
return "Kwf.RavenJsConfig = ".json_encode($data).";";
}
}
71 changes: 59 additions & 12 deletions Kwf/Assets/Dependency/File.php
Expand Up @@ -5,23 +5,39 @@ class Kwf_Assets_Dependency_File extends Kwf_Assets_Dependency_Abstract
private $_mtimeCache;
private $_fileNameCache;

public function __construct($fileName)
public function __construct($fileNameWithType)
{
$this->_fileName = $fileName;
if (substr($fileNameWithType, 0, 1) == '/') {
throw new Kwf_Exception('Don\'t use absolute file names');
}
if (!$fileNameWithType) {
throw new Kwf_Exception("Invalid filename");
}
$this->_fileName = $fileNameWithType;
}

public function getContents($language)
{
return file_get_contents($this->getFileName());
return file_get_contents($this->getAbsoluteFileName());
}

public function getFileName()
public function getType()
{
return substr($this->_fileName, 0, strpos($this->_fileName, '/'));
}

public function getFileNameWithType()
{
return $this->_fileName;
}

public function getAbsoluteFileName()
{
if (isset($this->_fileNameCache)) return $this->_fileNameCache;
static $paths;
if (!isset($paths)) $paths = Kwf_Config::getValueArray('path');

$pathType = substr($this->_fileName, 0, strpos($this->_fileName, '/'));
$pathType = $this->getType();
$f = substr($this->_fileName, strpos($this->_fileName, '/'));
if (isset($paths[$pathType])) {
$f = $paths[$pathType].$f;
Expand All @@ -37,7 +53,7 @@ public function getFileName()
public function getMTime()
{
if (!isset($this->_mtimeCache)) {
$f = $this->getFileName();
$f = $this->getAbsoluteFileName();
if ($f) {
$this->_mtimeCache = filemtime($f);
}
Expand Down Expand Up @@ -88,16 +104,12 @@ public static function createDependency($fileName, Kwf_Assets_ProviderList_Abstr

public function __toString()
{
$ret = $this->_fileName;
if (!$ret) {
$ret = parent::__toString();
}
return $ret;
return $this->_fileName;
}

protected function _getComponentCssClass()
{
$cssClass = realpath($this->getFileName());
$cssClass = realpath($this->getAbsoluteFileName());

static $paths;
if (!isset($paths)) {
Expand Down Expand Up @@ -141,4 +153,39 @@ protected function _getComponentCssClass()
}
return null;
}

private static function _getAbsolutePath($path)
{
if (substr($path, 0, 1)=='.') $path = getcwd().'/'.$path;
$parts = array_filter(explode(DIRECTORY_SEPARATOR, $path), 'strlen');
$absolutes = array();
foreach ($parts as $part) {
if ('.' == $part) continue;
if ('..' == $part) {
array_pop($absolutes);
} else {
$absolutes[] = $part;
}
}
return DIRECTORY_SEPARATOR.implode(DIRECTORY_SEPARATOR, $absolutes);
}

public static function getPathWithTypeByFileName($fileName)
{
static $paths;
if (!isset($paths)) {
$paths = Kwf_Config::getValueArray('path');
foreach ($paths as $k=>$p) {
$paths[$k] = self::_getAbsolutePath($p);
}
$paths = array_reverse($paths);
}
$fileName = self::_getAbsolutePath($fileName);
foreach ($paths as $k=>$p) {
if (substr($fileName, 0, strlen($p)) == $p) {
return $k.substr($fileName, strlen($p));
}
}
return false;
}
}