New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#binary? undefined inside XMLSitemap :rep_select #1082

Closed
sunshineco opened this Issue Feb 8, 2017 · 18 comments

Comments

Projects
None yet
2 participants
@sunshineco
Contributor

sunshineco commented Feb 8, 2017

No longer possible to consult binary? within XMLSitemap :rep_select.

Steps to reproduce

Create sitemap.xml with contents:

<%= xml_sitemap :rep_select => lambda {|i| !i.binary?} %>

Expected behavior

Should create sitemap referencing all non-binary items.

Actual behavior

Crashes, complaining that NilClass has no binary? method.

NoMethodError: undefined method `binary?' for nil:NilClass

Details

The item rep itself inside the :rep_select block is not nil, so the error is rather misleading. For instance, debugging with:

<%= xml_sitemap :rep_select => lambda {|i| puts "[#{i}][#{i.path}]"; !i.binary?} %>

results in:

[#<Nanoc::ItemRepView:0x007faa89d09798>][/image.png]
NoMethodError: undefined method `binary?' for nil:NilClass

This code worked correctly (didn't crash and produced expected results) as recently as nanoc 4.5.2, and broke in 4.5.3.

Possibly related to #1068, though it is unclear to me from that discussion if disappearance of binary? is intentional or not.

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Feb 8, 2017

Member

I believe that this happens because #binary? needs the item to be compiled (because binary-ness of an item = binary-ness of its :last snapshot), and Nanoc doesn’t require the item to be compiled, which can cause the :last snapshot not to be available, hence the nil:NilClass error.

Can you try replacing lib/nanoc/base/views/item_rep_view.rb, line 83, which says

      @context.snapshot_repo.get(unwrap, :last).binary?

with

      @context.snapshot_repo.compiled_content(rep: unwrap, snapshot: snapshot).binary?

and see whether the problem persists? If I’m correct, that should fix the problem.

Member

ddfreyne commented Feb 8, 2017

I believe that this happens because #binary? needs the item to be compiled (because binary-ness of an item = binary-ness of its :last snapshot), and Nanoc doesn’t require the item to be compiled, which can cause the :last snapshot not to be available, hence the nil:NilClass error.

Can you try replacing lib/nanoc/base/views/item_rep_view.rb, line 83, which says

      @context.snapshot_repo.get(unwrap, :last).binary?

with

      @context.snapshot_repo.compiled_content(rep: unwrap, snapshot: snapshot).binary?

and see whether the problem persists? If I’m correct, that should fix the problem.

@sunshineco

This comment has been minimized.

Show comment
Hide comment
@sunshineco

sunshineco Feb 8, 2017

Contributor

Thanks for the quick response.

Unfortunately, that doesn't quite work since there is no snapshot variable in context:

NameError: undefined local variable or method `snapshot' for #<Nanoc::ItemRepView:0x007f89a427b840>
Did you mean?  snapshot?
Contributor

sunshineco commented Feb 8, 2017

Thanks for the quick response.

Unfortunately, that doesn't quite work since there is no snapshot variable in context:

NameError: undefined local variable or method `snapshot' for #<Nanoc::ItemRepView:0x007f89a427b840>
Did you mean?  snapshot?
@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Feb 8, 2017

Member

Ohh, my bad… that should have said snapshot: :last rather than snapshot: snapshot.

Member

ddfreyne commented Feb 8, 2017

Ohh, my bad… that should have said snapshot: :last rather than snapshot: snapshot.

@sunshineco

This comment has been minimized.

Show comment
Hide comment
@sunshineco

sunshineco Feb 8, 2017

Contributor

Using this code:

def binary?
    @context.snapshot_repo.compiled_content(rep: unwrap, snapshot: :last).binary?
end

Result:

Nanoc::Int::Errors::CannotGetCompiledContentOfBinaryItem: You cannot access the compiled content of a binary item representation (but you can access the path). The offending item rep is <Nanoc::Int::ItemRep name="default" raw_path="output/SokoSave48x48.png" item.identifier="/SokoSave48x48.png">.
Contributor

sunshineco commented Feb 8, 2017

Using this code:

def binary?
    @context.snapshot_repo.compiled_content(rep: unwrap, snapshot: :last).binary?
end

Result:

Nanoc::Int::Errors::CannotGetCompiledContentOfBinaryItem: You cannot access the compiled content of a binary item representation (but you can access the path). The offending item rep is <Nanoc::Int::ItemRep name="default" raw_path="output/SokoSave48x48.png" item.identifier="/SokoSave48x48.png">.
@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Feb 8, 2017

Member

Argh, looks like I need to investigate this in a bit more detail. The error you’re getting makes sense, but it’s not quite what I had in mind :)

Member

ddfreyne commented Feb 8, 2017

Argh, looks like I need to investigate this in a bit more detail. The error you’re getting makes sense, but it’s not quite what I had in mind :)

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Feb 8, 2017

Member

I made a fix in #1083. It needs more test coverage and I need to dash, but it should work for you!

Member

ddfreyne commented Feb 8, 2017

I made a fix in #1083. It needs more test coverage and I need to dash, but it should work for you!

@sunshineco

This comment has been minimized.

Show comment
Hide comment
@sunshineco

sunshineco Feb 8, 2017

Contributor

Hmm, perhaps I didn't apply the patches correctly locally, but I get this:

NoMethodError: undefined method `binary?' for #<String:0x007fa739cac370>

Stack trace:

  0. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/repos/snapshot_repo.rb:58:in `compiled_content'
  1. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/views/item_rep_view.rb:47:in `compiled_content'
  2. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/views/mixins/with_reps_view_mixin.rb:16:in `compiled_content'

Travis complained, as well.

Contributor

sunshineco commented Feb 8, 2017

Hmm, perhaps I didn't apply the patches correctly locally, but I get this:

NoMethodError: undefined method `binary?' for #<String:0x007fa739cac370>

Stack trace:

  0. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/repos/snapshot_repo.rb:58:in `compiled_content'
  1. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/views/item_rep_view.rb:47:in `compiled_content'
  2. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/views/mixins/with_reps_view_mixin.rb:16:in `compiled_content'

Travis complained, as well.

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Feb 8, 2017

Member
Member

ddfreyne commented Feb 8, 2017

@sunshineco

This comment has been minimized.

Show comment
Hide comment
@sunshineco

sunshineco Feb 8, 2017

Contributor

Getting closer (I think)...

Nanoc::Int::Errors::RecursiveCompilation: The site cannot be compiled because the following items mutually depend on each other:
<Nanoc::Int::ItemRep name="default" raw_path="output/sitemap.xml" item.identifier="/sitemap.xml">.

===== STACK TRACE:

  0. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/item_rep_selector.rb:33:in `each'
  1. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/stages/compile_reps.rb:13:in `run'
  2. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler.rb:131:in `run_all'
  3. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/entities/site.rb:21:in `compile'
  4. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli/commands/compile.rb:408:in `block in run'
  5. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli/commands/compile.rb:443:in `run_listeners_while'
  6. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli/commands/compile.rb:407:in `run'
  7. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/command_runner.rb:12:in `block in call'
  8. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:64:in `handle_while'
  9. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:22:in `handle_while'
  10. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/command_runner.rb:11:in `call'
  11. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command_dsl.rb:231:in `block in runner'
  12. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command.rb:292:in `run_this'
  13. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command.rb:244:in `run'
  14. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command.rb:257:in `run'
  15. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli.rb:45:in `block in run'
  16. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:64:in `handle_while'
  17. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:22:in `handle_while'
  18. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli.rb:43:in `run'
  19. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/bin/nanoc:9:in `<top (required)>'
  20. /Users/sunshine/.rvm/gems/ruby-2.4.0/bin/nanoc:22:in `load'
  21. /Users/sunshine/.rvm/gems/ruby-2.4.0/bin/nanoc:22:in `<main>'
  22. /Users/sunshine/.rvm/gems/ruby-2.4.0/bin/ruby_executable_hooks:15:in `eval'
Contributor

sunshineco commented Feb 8, 2017

Getting closer (I think)...

Nanoc::Int::Errors::RecursiveCompilation: The site cannot be compiled because the following items mutually depend on each other:
<Nanoc::Int::ItemRep name="default" raw_path="output/sitemap.xml" item.identifier="/sitemap.xml">.

===== STACK TRACE:

  0. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/item_rep_selector.rb:33:in `each'
  1. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/stages/compile_reps.rb:13:in `run'
  2. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler.rb:131:in `run_all'
  3. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/entities/site.rb:21:in `compile'
  4. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli/commands/compile.rb:408:in `block in run'
  5. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli/commands/compile.rb:443:in `run_listeners_while'
  6. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli/commands/compile.rb:407:in `run'
  7. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/command_runner.rb:12:in `block in call'
  8. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:64:in `handle_while'
  9. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:22:in `handle_while'
  10. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/command_runner.rb:11:in `call'
  11. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command_dsl.rb:231:in `block in runner'
  12. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command.rb:292:in `run_this'
  13. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command.rb:244:in `run'
  14. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command.rb:257:in `run'
  15. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli.rb:45:in `block in run'
  16. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:64:in `handle_while'
  17. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:22:in `handle_while'
  18. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli.rb:43:in `run'
  19. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/bin/nanoc:9:in `<top (required)>'
  20. /Users/sunshine/.rvm/gems/ruby-2.4.0/bin/nanoc:22:in `load'
  21. /Users/sunshine/.rvm/gems/ruby-2.4.0/bin/nanoc:22:in `<main>'
  22. /Users/sunshine/.rvm/gems/ruby-2.4.0/bin/ruby_executable_hooks:15:in `eval'
@sunshineco

This comment has been minimized.

Show comment
Hide comment
@sunshineco

sunshineco Feb 8, 2017

Contributor

Hmm, aside from the recursion error, I'm also seeing some pretty odd behavior with the patches from #1083 applied. In particular, if I re-run nanoc after it crashes, I get this error:

Errno::ENOENT: No such file or directory @ rb_file_s_size - /var/folders/9m/6361184x6dlbsg6zb1m33kpc0000gn/T/nanoc20170208-807-383bu9/binary_items/0

===== ITEM BEING COMPILED:

Item identifier: /contributed/skins/paninaro01-preview.png
Item rep name:   :thumbnail

===== STACK TRACE:

  0. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/fileutils.rb:719:in `size'
  1. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/fileutils.rb:719:in `compare_file'
  2. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/item_rep_writer.rb:30:in `write'
  3. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/phases/write.rb:15:in `block in run'
  4. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/phases/write.rb:14:in `each'
  5. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/phases/write.rb:14:in `run'
  6. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/phases/mark_done.rb:12:in `run'
  7. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/stages/compile_reps.rb:30:in `compile_rep'
  8. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/stages/compile_reps.rb:14:in `block (2 levels) in run'
  9. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/stages/compile_reps.rb:24:in `handle_errors_while'
  10. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/stages/compile_reps.rb:14:in `block in run'
  11. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/item_rep_selector.rb:23:in `block in each'
  12. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/item_rep_selector.rb:17:in `loop'
  13. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/item_rep_selector.rb:17:in `each'
  14. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/stages/compile_reps.rb:13:in `run'
  15. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler.rb:131:in `run_all'
  16. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/entities/site.rb:21:in `compile'
  17. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli/commands/compile.rb:408:in `block in run'
  18. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli/commands/compile.rb:443:in `run_listeners_while'
  19. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli/commands/compile.rb:407:in `run'
  20. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/command_runner.rb:12:in `block in call'
  21. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:64:in `handle_while'
  22. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:22:in `handle_while'
  23. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/command_runner.rb:11:in `call'
  24. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command_dsl.rb:231:in `block in runner'
  25. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command.rb:292:in `run_this'
  26. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command.rb:244:in `run'
  27. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command.rb:257:in `run'
  28. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli.rb:45:in `block in run'
  29. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:64:in `handle_while'
  30. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:22:in `handle_while'
  31. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli.rb:43:in `run'
  32. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/bin/nanoc:9:in `<top (required)>'
  33. /Users/sunshine/.rvm/gems/ruby-2.4.0/bin/nanoc:22:in `load'
  34. /Users/sunshine/.rvm/gems/ruby-2.4.0/bin/nanoc:22:in `<main>'
  35. /Users/sunshine/.rvm/gems/ruby-2.4.0/bin/ruby_executable_hooks:15:in `eval'

In fact, this crash occurs even after a successful nanoc compile (achieved by temporarily dropping :rep_select from the xml_sitemap invocation).

This crash does not manifest if I start with a clean slate (no output/, no tmp/).

Contributor

sunshineco commented Feb 8, 2017

Hmm, aside from the recursion error, I'm also seeing some pretty odd behavior with the patches from #1083 applied. In particular, if I re-run nanoc after it crashes, I get this error:

Errno::ENOENT: No such file or directory @ rb_file_s_size - /var/folders/9m/6361184x6dlbsg6zb1m33kpc0000gn/T/nanoc20170208-807-383bu9/binary_items/0

===== ITEM BEING COMPILED:

Item identifier: /contributed/skins/paninaro01-preview.png
Item rep name:   :thumbnail

===== STACK TRACE:

  0. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/fileutils.rb:719:in `size'
  1. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/fileutils.rb:719:in `compare_file'
  2. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/item_rep_writer.rb:30:in `write'
  3. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/phases/write.rb:15:in `block in run'
  4. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/phases/write.rb:14:in `each'
  5. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/phases/write.rb:14:in `run'
  6. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/phases/mark_done.rb:12:in `run'
  7. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/stages/compile_reps.rb:30:in `compile_rep'
  8. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/stages/compile_reps.rb:14:in `block (2 levels) in run'
  9. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/stages/compile_reps.rb:24:in `handle_errors_while'
  10. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/stages/compile_reps.rb:14:in `block in run'
  11. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/item_rep_selector.rb:23:in `block in each'
  12. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/item_rep_selector.rb:17:in `loop'
  13. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/item_rep_selector.rb:17:in `each'
  14. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler/stages/compile_reps.rb:13:in `run'
  15. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/services/compiler.rb:131:in `run_all'
  16. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/base/entities/site.rb:21:in `compile'
  17. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli/commands/compile.rb:408:in `block in run'
  18. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli/commands/compile.rb:443:in `run_listeners_while'
  19. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli/commands/compile.rb:407:in `run'
  20. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/command_runner.rb:12:in `block in call'
  21. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:64:in `handle_while'
  22. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:22:in `handle_while'
  23. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/command_runner.rb:11:in `call'
  24. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command_dsl.rb:231:in `block in runner'
  25. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command.rb:292:in `run_this'
  26. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command.rb:244:in `run'
  27. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/cri-2.7.1/lib/cri/command.rb:257:in `run'
  28. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli.rb:45:in `block in run'
  29. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:64:in `handle_while'
  30. /Users/sunshine/.rvm/rubies/ruby-2.4.0/lib/ruby/gems/2.4.0/gems/nanoc-4.6.1/lib/nanoc/cli/error_handler.rb:22:in `handle_while'
  31. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/lib/nanoc/cli.rb:43:in `run'
  32. /Users/sunshine/.rvm/gems/ruby-2.4.0@global/gems/nanoc-4.6.1/bin/nanoc:9:in `<top (required)>'
  33. /Users/sunshine/.rvm/gems/ruby-2.4.0/bin/nanoc:22:in `load'
  34. /Users/sunshine/.rvm/gems/ruby-2.4.0/bin/nanoc:22:in `<main>'
  35. /Users/sunshine/.rvm/gems/ruby-2.4.0/bin/ruby_executable_hooks:15:in `eval'

In fact, this crash occurs even after a successful nanoc compile (achieved by temporarily dropping :rep_select from the xml_sitemap invocation).

This crash does not manifest if I start with a clean slate (no output/, no tmp/).

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Feb 8, 2017

Member

The recursive compilation error is happens because the sitemap item itself is asking whether its own :last snapshot is binary, and in order to figure out, it needs itself to be compiled… a never-ending dependency.

The solution for this could be to figure out up-front whether or not a certain snapshot will be binary. To find out whether a snapshot of a certain item rep is binary, that item rep does not need to be compiled.

Note that previously, Nanoc’s behavior was wrong (although much simpler): the binary-ness of an item rep was the binary-ness of the item, rather than the last snapshot of the item rep; the binary-ness would thus be incorrect if any text-to-binary or binary-to-text filters are used.

A workaround for this error would be to exclude @item from :rep_select.


The Errno::ENOENT does not ring a bell. I’ve not seen it before and can’t reproduce it right now…

Member

ddfreyne commented Feb 8, 2017

The recursive compilation error is happens because the sitemap item itself is asking whether its own :last snapshot is binary, and in order to figure out, it needs itself to be compiled… a never-ending dependency.

The solution for this could be to figure out up-front whether or not a certain snapshot will be binary. To find out whether a snapshot of a certain item rep is binary, that item rep does not need to be compiled.

Note that previously, Nanoc’s behavior was wrong (although much simpler): the binary-ness of an item rep was the binary-ness of the item, rather than the last snapshot of the item rep; the binary-ness would thus be incorrect if any text-to-binary or binary-to-text filters are used.

A workaround for this error would be to exclude @item from :rep_select.


The Errno::ENOENT does not ring a bell. I’ve not seen it before and can’t reproduce it right now…

@sunshineco

This comment has been minimized.

Show comment
Hide comment
@sunshineco

sunshineco Feb 8, 2017

Contributor

While I like a proper solution, such as figuring out ahead-of-time if a rep is binary, I wonder if it's worth a ton of effort if mine is the only site encountering this issue. The idea of excluding @item from :rep_select feels very slightly hacky but might be a good pragmatic compromise. And, of course, I can always use work-arounds myself if it comes to that. For instance, I could assign :items manually when invoking xml_sitemap. I think the only reason I didn't do that is because I didn't want to duplicate in my own code the :is_hidden filtering xml_sitemap.rb is already doing; not because :rep_select was a better choice for my use case.

As for the Errno::ENOENT issue, that particular item rep is created by a very simple filter which runs ImageMagick's convert command on the item to create the thumbnail representation. I can reproduce it reliably in the full site but haven't managed to come up with a minimal reproduction recipe. (I can, of course, send you the full site if you want a crack at it yourself.)

Contributor

sunshineco commented Feb 8, 2017

While I like a proper solution, such as figuring out ahead-of-time if a rep is binary, I wonder if it's worth a ton of effort if mine is the only site encountering this issue. The idea of excluding @item from :rep_select feels very slightly hacky but might be a good pragmatic compromise. And, of course, I can always use work-arounds myself if it comes to that. For instance, I could assign :items manually when invoking xml_sitemap. I think the only reason I didn't do that is because I didn't want to duplicate in my own code the :is_hidden filtering xml_sitemap.rb is already doing; not because :rep_select was a better choice for my use case.

As for the Errno::ENOENT issue, that particular item rep is created by a very simple filter which runs ImageMagick's convert command on the item to create the thumbnail representation. I can reproduce it reliably in the full site but haven't managed to come up with a minimal reproduction recipe. (I can, of course, send you the full site if you want a crack at it yourself.)

@sunshineco

This comment has been minimized.

Show comment
Hide comment
@sunshineco

sunshineco Feb 8, 2017

Contributor

Okay, I managed to come up with a minimal reproduction recipe for the Errno::ENOENT crash. The following script will create a minimal site from scratch in the directory enoent-crash:

mkdir enoent-crash && (
cd enoent-crash &&
mkdir content lib &&

cat <<\EOF >Rules &&
compile '/resource.bin', :rep => :altered do
  filter :alter
  write '/altered.bin'
end
compile '/sitemap.*' do
  filter :erb
  write item.identifier.to_s
end
compile '/**/*' do
  write item.identifier.to_s
end
EOF

cat <<\EOF >nanoc.yaml &&
base_url: http://example.com/
text_extensions: ['xml']
EOF

cat <<\EOF >lib/default.rb &&
include Nanoc::Helpers::XMLSitemap
class Alter < Nanoc::Filter
  identifiers :alter
  type :binary
  def run(filename, params={})
    File.write(output_filename, '')
  end
end
EOF

>content/resource.bin &&

cat <<\EOF >content/sitemap.xml
<%= xml_sitemap :rep_select => lambda {|i| !i.binary?} %>
EOF
)

The first time nanoc compile is invoked for this site, it aborts with the recursive dependency error:

Nanoc::Int::Errors::RecursiveCompilation: The site cannot be compiled because the following items mutually depend on each other:
<Nanoc::Int::ItemRep name="default" raw_path="output/sitemap.xml" item.identifier="/sitemap.xml">.

Each subsquent nanoc compile aborts with the Errno::ENOENT error:

Errno::ENOENT: No such file or directory @ rb_file_s_size - /var/folders/9m/6361184x6dlbsg6zb1m33kpc0000gn/T/nanoc20170208-3438-yv30jy/binary_items/0

The Errno::ENOENT error only manifest if both resource.bin and sitemap.xml are in the item list passed to :rep_select; remove either and the error goes away.

Contributor

sunshineco commented Feb 8, 2017

Okay, I managed to come up with a minimal reproduction recipe for the Errno::ENOENT crash. The following script will create a minimal site from scratch in the directory enoent-crash:

mkdir enoent-crash && (
cd enoent-crash &&
mkdir content lib &&

cat <<\EOF >Rules &&
compile '/resource.bin', :rep => :altered do
  filter :alter
  write '/altered.bin'
end
compile '/sitemap.*' do
  filter :erb
  write item.identifier.to_s
end
compile '/**/*' do
  write item.identifier.to_s
end
EOF

cat <<\EOF >nanoc.yaml &&
base_url: http://example.com/
text_extensions: ['xml']
EOF

cat <<\EOF >lib/default.rb &&
include Nanoc::Helpers::XMLSitemap
class Alter < Nanoc::Filter
  identifiers :alter
  type :binary
  def run(filename, params={})
    File.write(output_filename, '')
  end
end
EOF

>content/resource.bin &&

cat <<\EOF >content/sitemap.xml
<%= xml_sitemap :rep_select => lambda {|i| !i.binary?} %>
EOF
)

The first time nanoc compile is invoked for this site, it aborts with the recursive dependency error:

Nanoc::Int::Errors::RecursiveCompilation: The site cannot be compiled because the following items mutually depend on each other:
<Nanoc::Int::ItemRep name="default" raw_path="output/sitemap.xml" item.identifier="/sitemap.xml">.

Each subsquent nanoc compile aborts with the Errno::ENOENT error:

Errno::ENOENT: No such file or directory @ rb_file_s_size - /var/folders/9m/6361184x6dlbsg6zb1m33kpc0000gn/T/nanoc20170208-3438-yv30jy/binary_items/0

The Errno::ENOENT error only manifest if both resource.bin and sitemap.xml are in the item list passed to :rep_select; remove either and the error goes away.

@sunshineco

This comment has been minimized.

Show comment
Hide comment
@sunshineco

sunshineco Feb 8, 2017

Contributor

By the way, just to be clear, the :rep_select example(s) posted here are minimal versions which allow the reported error(s) to be reproduced. On the actual site, :rep_select is more complex and does indeed filter out sitemap.xml by checking rep.path, so the sitemap is not actually referencing itself in Real Life.

Contributor

sunshineco commented Feb 8, 2017

By the way, just to be clear, the :rep_select example(s) posted here are minimal versions which allow the reported error(s) to be reproduced. On the actual site, :rep_select is more complex and does indeed filter out sitemap.xml by checking rep.path, so the sitemap is not actually referencing itself in Real Life.

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Feb 11, 2017

Member

Thanks for the test case! That’ll be useful.

I’m a bit pressed for time and won’t be able to get to it in the next few days, but I’ll look at it as soon as I have spare time.

Member

ddfreyne commented Feb 11, 2017

Thanks for the test case! That’ll be useful.

I’m a bit pressed for time and won’t be able to get to it in the next few days, but I’ll look at it as soon as I have spare time.

ddfreyne added a commit that referenced this issue Feb 16, 2017

Merge pull request #1083 from nanoc/gh-1082
Require compilation before calling #binary?

ddfreyne added a commit that referenced this issue Feb 16, 2017

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Feb 16, 2017

Member

A fix is in #1084. Can you verify that this works?

Member

ddfreyne commented Feb 16, 2017

A fix is in #1084. Can you verify that this works?

ddfreyne added a commit that referenced this issue Feb 16, 2017

ddfreyne added a commit that referenced this issue Feb 16, 2017

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Feb 16, 2017

Member

I am unable to reproduce your Errno::ENOENT crash, unfortunately, even with the script. :(

Member

ddfreyne commented Feb 16, 2017

I am unable to reproduce your Errno::ENOENT crash, unfortunately, even with the script. :(

@ddfreyne ddfreyne closed this in #1084 Feb 16, 2017

@sunshineco

This comment has been minimized.

Show comment
Hide comment
@sunshineco

sunshineco Feb 17, 2017

Contributor

I am unable to reproduce your Errno::ENOENT crash, unfortunately, even with the script.

The reproduction recipe solely demonstrates a problem with #1083 itself. The crash reproduces reliably with just #1083 applied (which was the only "fix" available at the time the recipe was created).

The crash does not manifest when #1084 is also applied.

A fix is in #1084. Can you verify that this works?

It seems to resolve the problem. Thanks for working on this!

Contributor

sunshineco commented Feb 17, 2017

I am unable to reproduce your Errno::ENOENT crash, unfortunately, even with the script.

The reproduction recipe solely demonstrates a problem with #1083 itself. The crash reproduces reliably with just #1083 applied (which was the only "fix" available at the time the recipe was created).

The crash does not manifest when #1084 is also applied.

A fix is in #1084. Can you verify that this works?

It seems to resolve the problem. Thanks for working on this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment