New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Outdatedness checker detects changed rules, despite rules not being changed #866
Comments
Hi! Could you share the source code for the rule that applies to |
Hi :-) I think that hash of filter parameters (?) pointed me in the right direction. We are using #!/usr/bin/env ruby
tsc_output = Dir.mktmpdir('tsc-')
compile '/assets/js/**/*.ts' do
minify = Nanoc::CoIt::Deployment::BuildEnvironment.optimization_enabled
filter :tsc, output_dir: tsc_output, minify: minify
end
compile '/assets/js/**/*.ts', rep: :map do
source_map = !Nanoc::CoIt::Deployment::BuildEnvironment.optimization_enabled
filter :tsc, output_dir: tsc_output, source_map: source_map
end
route '/assets/js/**/*.ts' do
"#{item.identifier.without_ext}.js"
end
route '/assets/js/**/*.ts', rep: :map do
"#{item.identifier.without_ext}.js.map"
end
postprocess do
FileUtils.rm_rf(tsc_output)
end |
The class Tsc < Nanoc::Filter
identifier :tsc
type :binary
def run(ts_filename, output_dir: './tmp/tsc-build', source_map: false, minify: false)
FileUtils.mkdir_p(output_dir)
compile(output_dir, minify)
result = js_filename(ts_filename, output_dir, source_map)
FileUtils.cp result, output_filename
end
private
# Compiles TypeScript into the given output directory
# @param [String] path to the target directory
# tells the TypeScript compiler where the result should be saved
def compile(output_dir, minify)
return unless Dir["#{output_dir}/*"].empty?
args = %W(npm run tsc -- --outDir #{output_dir})
system(*args.flatten)
uglify(output_dir) if minify
end
def uglify(output_dir)
Dir["#{output_dir}/**/*.js"].each { |file|
system("npm run uglifyjs -- #{file} --compress warnings=false --mangle --output #{file}")
}
end
# Gets the filename of the transpiled js result
# @param [String] path of the TypeScript file beeing compiled
# @param [String] path to the TypeScript output directory
# @param [Boolean] indicator if the source-map should be fetched
def js_filename(ts_filename, output_dir, source_map)
transpiled_source = ts_filename.sub(Dir.pwd, output_dir)
File.join(File.dirname(transpiled_source),
js_ext(transpiled_source, source_map))
end
# Sets .js or js.map as file extension
# @param [String] path of the TypeScript file beeing compiled
# @param [Boolean] indicator if .map should be added as extension
def js_ext(filename, source_map)
File.basename(filename, '.*') + '.js' + (source_map ? '.map' : '')
end
end I believe this works similar to your custom Sass filesystem, but tsc is not Ruby, so we can't let it know about nanoc items ;-) |
I once worked around a similar issue by aliasing
|
This seems to work: #!/usr/bin/env ruby
tsc_output = './tmp/tsc-build'
# Clean up before running and after running. A temporary directory will break
# nanoc's outdatedness checker because the path (and thereby cached tsc filter
# args) change with every invocation of `nanoc compile`.
preprocess do
FileUtils.rm_rf(tsc_output)
end
postprocess do
FileUtils.rm_rf(tsc_output)
end
compile '/assets/js/**/*.ts' do
minify = Nanoc::CoIt::Deployment::BuildEnvironment.optimization_enabled
filter :tsc, output_dir: tsc_output, minify: minify
end
compile '/assets/js/**/*.ts', rep: :map do
source_map = !Nanoc::CoIt::Deployment::BuildEnvironment.optimization_enabled
filter :tsc, output_dir: tsc_output, source_map: source_map
end
route '/assets/js/**/*.ts' do
"#{item.identifier.without_ext}.js"
end
route '/assets/js/**/*.ts', rep: :map do
"#{item.identifier.without_ext}.js.map"
end |
Next issue :-) We have an item that's completely empty (empty haml file, no metadata). It's causing outdatedness for itself as well a as a bunch of dependent items. This line evaluates |
Aha: # The direct predecessors can include nil, which indicates an item that is
# no longer present in the site. Now this gets really strange. I suspect it's our dynamic parts that cause the issue here. Sorry for spamming, I'm trying to document my findings. |
The A |
The The objects we create are static, so I wonder why they're missing from
|
I was able to reproduce this with a small example, and I’ll try to come up with a fix (though I’m out in the next few days, so it’ll take a while). The problem also occurs with a fresh site modified with these steps:
|
The |
No hurry! Thanks for confirming that this is not our fault :-) |
Hello! As I said in the other ticket I'm currently migrating a 3.8 project to 4.2. This project also suffers from outdatedness. They all seem to be related to sass, despite the file(s) or rules being unchanged. Nanoc reports that the Rules have been changed, but they really haven't. I've dug deep into that nanoc source and found that the stored checksum for the sass filter seems incorrect. Rules: compile '/assets/css/app.sass' do
require 'compass'
Compass.add_project_configuration('compass-config.rb')
filter :sass, Compass.sass_engine_options
end For the first run I deleted
After compilation succeeded and require 'nanoc'
module Nanoc
module Filters
class SassWithCompass < Nanoc::Filter
identifier :sass_with_compass
type :text
requires 'compass'
def run(content, params = {})
Compass.add_project_configuration(params.delete(:config_file))
params = params.merge(Compass.sass_engine_options)
Nanoc::Filters::Sass.new(@assigns).setup_and_run(content, params)
end
end
end
end Rules: compile '/assets/css/app.sass' do
filter :sass_with_compass, config_file: 'compass-config.rb'
end A
And subsequent runs yield this:
See the
|
Found the issue within the Sass wrapper. I guess Is there a specific reason nanoc computes cache digests after a filter is run and checks them against digests before running the next time? (The reason the check needs to run before on the seconds run is obvious :-)) What's also interesting that compilation didn't get faster after fixing the digest by duplicating the params hash. It even got slower than just recompile. Haven't checked because I'm on vacation this week. |
Quick repo:
|
Hmm, mutating params passed to a filter will indeed be a problem. I’d disallow it entirely, if I could, but I’m not sure that’s realistic. Does the problem persist if you stop mutating |
#881 is a PR that freezes filter arguments. |
I found this error because we use Compass with Sass as described here. The I did a quick test:
See how the first
|
Even this will always recompiles the Sass file:
|
In the simple example above (#866 (comment)) the problem no longer occurs. But not so much with Sass + Compass, as stated above. I have a workaround for my Sass compilation by decorating the Sass filter with my own filter that loads the configuration, so nanoc sees static parameters passed to my wrapping filter. |
#884 has a fix for the Sass importer problem. |
I’ve created a separate issue, #885, for the dependency on generated items. |
I’ll close this for now, but do let me know if you still experience problems. |
See: * nanoc/nanoc#866 * and way back here: nanoc/nanoc#310
Today I was investigating why our builds take a rather long time despite the fact that we didn't change bytes on disk.
Our site has become rather complex and also a bit dynamic, but with the caches warmed we should not have to wait 30 secs just to compile the same
output
that already sits on disk :-)Today I added some puts statements to nanoc's
outdatedness_checker.rb
and found that nanoc thinks ourRules
have changed. Our mainRules
fileinclude_rules
a bunch of other files that contain the actual rules.Here's some of my output:
Output of
rule_memory_differs_for
:Output of
outdated?
:In this case
contact-channel-link.ts
is filtered bytsc
(TypeScript Compiler) that generates ajs
and amap
file. I verified that they don't change between compilations content-wise (There's also noupdated
output written by nanoc).I wonder what
Tpbv4FX3L15lr3YoTwET58Ck5tw
andGnrgXISbUEBu5yAMT01LlhSXFvo
in the output above are supposed to mean. Hashed parameters of ...?The text was updated successfully, but these errors were encountered: