Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement postprocessing #726

Merged
merged 18 commits into from Nov 29, 2015
Merged
Changes from 12 commits
Commits
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.
+155 −0
Diff settings

Always

Just for now

@@ -92,6 +92,7 @@ def initialize(site, rules_collection, compiled_content_cache:, checksum_store:,
# 3. Preprocess
# 4. Build item reps
# 5. Compile
# 6. Postprocess

# TODO: move elsewhere
def run_all
@@ -103,6 +104,9 @@ def run_all

# Compile
run

# Postprocess
Nanoc::Int::Postprocessor.new(create_view_context, site: @site, rules_collection: @rules_collection).run
end

def run
@@ -233,8 +237,10 @@ def compile_rep(rep)
if can_reuse_content_for_rep?(rep)
Nanoc::Int::NotificationCenter.post(:cached_content_used, rep)
rep.snapshot_contents = compiled_content_cache[rep]
rep.status = :created

This comment has been minimized.

Copy link
@ddfreyne

ddfreyne Nov 27, 2015

Member

Hmm, this :created is incorrect. Some context:

If #can_reuse_content_for_rep? returns true, then that means that the compiled content (which was cached in the previous compilation run) can be reused without actually recompiling the item. This is used when an outdated item includes content from an item that is not outdated, and whose content can thus be reused without recompiling.

In the ItemRepWriter, Nanoc determines whether the resulting file exists already (is_created), is modified (is_modified), or neither. (See these ItemRepWriter lines.)

It’d be useful to make use of the ItemRepWriter modified/created logic that already exists. Creating a snapshot for which a route is defined will make the item rep writer write out the compiled content. The :last snapshot, containing the compiled content after the entire rule is executed, is trigged by the line executor.snapshot(rep, :last) in the #recalculate_content_for_rep method below.

Perhaps the ItemRepWriter could update the item rep to set item_rep.modified = is_modified, with is_modified being whatever ItemRepWriter already calculates. (I don’t think there’s a need for a similar created attribute.) The advantage of this is that only ItemRepWriter determines whether content is deemed changed. In the following cases, ItemRepWriter will treat the content as not modified:

  • An item that is recompiled (#can_reuse_content_for_rep? returns false), but whose content turns out to be identical after compilation
  • An item whose content is reused (#can_reuse_content_for_rep? returns true)
  • An item that’s not even considered for compilation (not considered outdated)
else
recalculate_content_for_rep(rep)
rep.status = :modified
end

rep.compiled = true
@@ -236,6 +236,20 @@ def include_rules(name)
Nanoc::Int::RulesLoader.new(@config, @rules_collection).parse(filename)
end

# Creates a postprocessor block that will be executed after all data is
# loaded and the site is compiled.
#
# @yield The block that will be executed after site compilation completes
#
# @return [void]
def postprocess(&block)
if @rules_collection.postprocessors[rules_filename]
warn 'WARNING: A postprocess block is already defined. Defining ' \
'another postprocess block overrides the previously one.'
end
@rules_collection.postprocessors[rules_filename] = block
end

# @api private
def create_pattern(arg)
case @config[:string_pattern_type]
@@ -23,6 +23,9 @@ class ItemRep
# @return [Enumerable<Nanoc::Int:SnapshotDef]
attr_accessor :snapshot_defs

# @return [Symbol]
attr_accessor :status

# @param [Nanoc::Int::Item] item
#
# @param [Symbol] name
@@ -19,11 +19,19 @@ class RulesCollection
# be executed after all data is loaded but before the site is compiled
attr_accessor :preprocessors

# The hash containing postprocessor code blocks that will be executed after
# all data is loaded and the site is compiled.
#
# @return [Hash] The hash containing the postprocessor code blocks that will
# be executed after all data is loaded and the site is compiled
attr_accessor :postprocessors

def initialize
@item_compilation_rules = []
@item_routing_rules = []
@layout_filter_mapping = {}
@preprocessors = {}
@postprocessors = {}
end

# Add the given rule to the list of item compilation rules.
@@ -6,6 +6,7 @@
require_relative 'services/item_rep_writer'
require_relative 'services/notification_center'
require_relative 'services/preprocessor'
require_relative 'services/postprocessor'
require_relative 'services/recording_executor'
require_relative 'services/rule_memory_calculator'
require_relative 'services/rules_loader'
@@ -0,0 +1,26 @@
module Nanoc::Int
# @api private
class Postprocessor
def initialize(context, site:, rules_collection:)
@context = context
@site = site
@rules_collection = rules_collection
end

def run
ctx = new_postprocessor_context

@rules_collection.postprocessors.each_value do |postprocessor|
ctx.instance_eval(&postprocessor)
end
end

# @api private
def new_postprocessor_context
Nanoc::Int::Context.new({
config: Nanoc::ConfigView.new(@site.config, @context),
items: Nanoc::PostCompileItemCollectionView.new(@site.items, @context),
})
end
end
end
@@ -19,3 +19,6 @@
require_relative 'views/mutable_layout_view'
require_relative 'views/mutable_layout_collection_view'
require_relative 'views/site_view'

require_relative 'views/post_compile_item_view'
require_relative 'views/post_compile_item_collection_view'
@@ -27,6 +27,11 @@ def name
@item_rep.name
end

# @return [Symbol]
def status
@item_rep.status
end

# Returns the compiled content.
#
# @param [String] snapshot The name of the snapshot from which to
@@ -0,0 +1,8 @@
module Nanoc
class PostCompileItemCollectionView < Nanoc::IdentifiableCollectionView
# @api private
def view_class
Nanoc::PostCompileItemView
end
end
end
@@ -0,0 +1,11 @@
module Nanoc
class PostCompileItemView < Nanoc::ItemView

This comment has been minimized.

Copy link
@ddfreyne

ddfreyne Nov 27, 2015

Member

👍 for having PostCompileItemView. I got a little lazy and didn’t create a PreprocessItemView (and friends, like the collection view), but that’ll change Soon™. (See #571.)

def created?
reps.select { |rep| rep.status == :created }
end

def updated?
reps.select { |rep| rep.status == :modified }

This comment has been minimized.

Copy link
@ddfreyne

ddfreyne Nov 27, 2015

Member

Can you replace :modified with :updated? This way, it reflects the name of the method.

This comment has been minimized.

Copy link
@ddfreyne

ddfreyne Nov 27, 2015

Member

See other (big) comment—these two methods might not be necessary.

end
end
end
@@ -30,6 +30,25 @@ def test_preprocess_twice
assert_match(/WARNING: A preprocess block is already defined./, io[:stderr])
end

def test_postprocess_twice
rules_collection = Nanoc::Int::RulesCollection.new
compiler_dsl = Nanoc::Int::CompilerDSL.new(rules_collection, {})

# first time
io = capturing_stdio do
compiler_dsl.postprocess {}
end
assert_empty io[:stdout]
assert_empty io[:stderr]

# second time
io = capturing_stdio do
compiler_dsl.postprocess {}
end
assert_empty io[:stdout]
assert_match(/WARNING: A postprocess block is already defined./, io[:stderr])
end

def test_per_rules_file_preprocessor
# Create site
Nanoc::CLI.run %w( create_site foo )
@@ -61,6 +80,57 @@ def test_per_rules_file_preprocessor
end
end

def test_per_rules_file_postprocessor
# Create site
Nanoc::CLI.run %w( create_site foo )
FileUtils.cd('foo') do
# Create a bonus rules file
File.write(
'more_rules.rb',
"postprocess {}")

# Adjust normal rules file
File.write(
'Rules',
"include_rules 'more_rules'\n\npostprocess {}\n\n" + File.read('Rules'))

# Create site and compiler
site = Nanoc::Int::SiteLoader.new.new_from_cwd
compiler = Nanoc::Int::CompilerLoader.new.load(site)

# Check that the two postprocess blocks have been added
assert_equal 2, compiler.rules_collection.postprocessors.size
refute_nil compiler.rules_collection.postprocessors.first
refute_nil compiler.rules_collection.postprocessors.to_a.last
end
end

def test_postprocessor_updated_method
with_site do |site|
# Create rules
File.open('Rules', 'w') do |io|
io.write <<EOS
compile '*' do
end
route '*' do
end
postprocess do
puts @items.select(&:updated?).length
end
EOS
end

File.open('content/index.html', 'w') { |io| io.write('o hello') }

io = capturing_stdio do
site = Nanoc::Int::SiteLoader.new.new_from_cwd
site.compile
end

assert_match(/1/, io[:stdout])
end
end

def test_include_rules
# Create site
Nanoc::CLI.run %w( create_site foo )
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.