Skip to content
This repository

Kindle ebook generation with working section navigation #4526

Closed
wants to merge 5 commits into from

5 participants

Daniel Choi mipearson Carlos Antonio da Silva Steve Klabnik Xavier Noria
Daniel Choi

This commit corrects the Kindle ebook navigation to work on the Kindle Touch.

The commit adapts and incorporates the code from https://github.com/danchoi/docrails_kindle

NOTE: this commit introduces a dependency on nokogiri and kindlerb, but I don't
know the proper place outside this commit (which Gemfile or whatnot) to declare
this.

added some commits January 18, 2012
Daniel Choi Generate Kindle ebook with working section navigation
NOTE: this commit introduces a dependency on nokogiri and kindlerb, but I don't
know the proper place outside this commit (which Gemfile or whatnot) to declare
this.
83165e1
Daniel Choi fix frontmatter generation 39d132e
Daniel Choi clear sections/ before regenerating 73c9861
mipearson

Oh nice. As the guy who did the original Kindle port, I was really hoping that somebody with a newer Kindle would jump on this.

+1

Daniel Choi

Thanks for doing the original port @mipearson

Carlos Antonio da Silva

@danchoi your patch does not apply cleanly anymore, could you please bring it up-to-date with master, so it get more changes to get merged? Thanks.

/cc @fxn

Daniel Choi
danchoi commented May 01, 2012

Sorry, I think I messed up my fork of Rails. But the two commits above contain the whole patch.

Daniel Choi
danchoi commented May 01, 2012

I can try to submit a new patch if that's cleaner.

Steve Klabnik
Collaborator

Nope, certainly clean!

Carlos Antonio da Silva

@fxn could you take a look on this? Let me know if I can help somehow, thanks.

Xavier Noria
Owner
fxn commented August 21, 2012

@carlosantoniodasilva thanks for the ping.

Xavier Noria
Owner
fxn commented August 21, 2012

A few remarks:

  • I think it may be a little surprising that a library calls exec, is it needed or could we shell out with system?

  • The name DocrailsKindle is a little suspicious, docrails is not a project or anything. Could we have a different name for that one? Or could we merge both modules? Makes sense? Where is DocrailsKindle used?

  • Some commented out code remains, could you please clean that?

Xavier Noria
Owner
fxn commented August 31, 2012

@danchoi hey, any news over here?

Daniel Choi

I'll try to make the fixes that you want this week @fxn. The changes you're asking for are certainly no big deal & I am happy to make them.

Xavier Noria
Owner

@danchoi thanks!

Daniel Choi

Hi. I cleaned up the patches and redid them as a new patch.

danchoi@cab0a6e

Steve Klabnik
Collaborator

@danchoi if you add this patch to your master branch, github will update the pull request.

Daniel Choi

I added one more commit to clean up the bullet point formatting danchoi@a93ca4c

I committed both of these to my master branch of my fork, but I'm not seeing them show up here.

I think I may have messed things up. Somehow my original fork of rails got deleted (maybe I did it, but I don't remember), and so I made a new fork today and created the these last two commits.

Steve Klabnik
Collaborator

Hm, how strange. @github must be acting a bit weird.

Steve Klabnik
Collaborator

Hey @fxn, this merges cleanly; can we get it in?

Carlos Antonio da Silva

Is the file docrails_kindle.rb being used?

Xavier Noria
Owner

@steveklabnik cool thanks for the ping. I am out not, but I'll try to have a look later today.

Daniel Choi

I think I removed it.

Xavier Noria
Owner

@danchoi as you say this PR does not seem to be sync'ed with the most recent changes. I think it would be better to review the real patch as is today. Would you mind opening a new PR with the last code?

Xavier Noria
Owner

Moved to #8254, closing here.

Xavier Noria fxn closed this November 17, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 5 unique commits by 1 author.

Jan 18, 2012
Daniel Choi Generate Kindle ebook with working section navigation
NOTE: this commit introduces a dependency on nokogiri and kindlerb, but I don't
know the proper place outside this commit (which Gemfile or whatnot) to declare
this.
83165e1
Daniel Choi fix frontmatter generation 39d132e
Daniel Choi clear sections/ before regenerating 73c9861
May 01, 2012
Daniel Choi Merge remote-tracking branch 'upstream/master' 819c651
Daniel Choi Fix Kindle doc generation patch 9aa84e5
This page is out of date. Refresh to see the latest.
8  guides/rails_guides/generator.rb
@@ -117,12 +117,8 @@ def check_for_kindlegen
117 117
     end
118 118
 
119 119
     def generate_mobi
120  
-      opf = "#{output_dir}/rails_guides.opf"
121  
-      out = "#{output_dir}/kindlegen.out"
122  
-
123  
-      system "kindlegen #{opf} -o #{mobi} > #{out} 2>&1"
124  
-      puts "Guides compiled as Kindle book to #{mobi}"
125  
-      puts "(kindlegen log at #{out})."
  120
+      require 'rails_guides/kindle'
  121
+      Kindle.generate(output_dir, mobi)
126 122
     end
127 123
 
128 124
     def mobi
126  guides/rails_guides/kindle.rb
... ...
@@ -0,0 +1,126 @@
  1
+#!/usr/bin/env ruby
  2
+
  3
+unless `which kindlerb` 
  4
+  abort "Please gem install kindlerb"
  5
+end
  6
+
  7
+require 'nokogiri'
  8
+require 'fileutils'
  9
+require 'yaml'
  10
+require 'date'
  11
+
  12
+module Kindle
  13
+  extend self
  14
+
  15
+  def generate(output_dir, outfile)
  16
+
  17
+    Dir.chdir output_dir do 
  18
+
  19
+      # Get html pages in document order
  20
+      toc = File.read("toc.ncx")
  21
+      #puts toc
  22
+      # <ncx xmlns="http://www.daisy.org/z3986/2005/ncx/" version="2005-1" xml:lang="en-US">
  23
+      doc = Nokogiri::XML(toc).xpath("//ncx:content", 'ncx' => "http://www.daisy.org/z3986/2005/ncx/")
  24
+      html_pages = doc.select {|c| c[:src]}.map {|c| c[:src]}.uniq
  25
+      puts html_pages
  26
+
  27
+      frontmatter = []
  28
+
  29
+      html_pages.delete_if {|x| 
  30
+        # we need to treat frontmatter differently
  31
+        if x =~ /(toc|welcome|credits|copyright).html/
  32
+          frontmatter << x unless x =~ /toc/
  33
+          true
  34
+        end
  35
+
  36
+      }
  37
+
  38
+      puts "=> Making one section for all frontmatter."
  39
+      html = frontmatter.map {|x|
  40
+        Nokogiri::HTML(File.open(x)).at("body").inner_html
  41
+      }.join("\n")
  42
+
  43
+      fdoc = Nokogiri::HTML(html)
  44
+
  45
+      # need to turn author h3's into h4's
  46
+      fdoc.search("h3").each { |h3|
  47
+        h3.name = 'h4'
  48
+      }
  49
+      fdoc.search("h2").each { |h2| 
  50
+        h2.name = 'h3'
  51
+        h2['id'] = h2.inner_text.gsub(/\s/, '-')
  52
+      }
  53
+
  54
+      add_head_section fdoc, "Frontmatter"
  55
+      File.open("frontmatter.html",'w'){|f| f.puts fdoc.to_html}
  56
+      html_pages.unshift "frontmatter.html"
  57
+
  58
+      puts "=> Making one section folder per original HTML file"
  59
+      FileUtils::rm_rf("sections/")
  60
+
  61
+      html_pages.each_with_index { |page, section_idx|
  62
+        FileUtils::mkdir_p("sections/%03d" % section_idx)
  63
+        doc = Nokogiri::HTML(File.open(page))
  64
+        title = doc.at("title").inner_text.gsub("Ruby on Rails Guides: ", '')
  65
+        title = page.capitalize.gsub('.html', '') if title.strip == ''
  66
+        File.open("sections/%03d/_section.txt" % section_idx, 'w') {|f| f.puts title}
  67
+        puts "sections/%03d -> #{title}" % section_idx
  68
+        
  69
+        # Fragment the page file into subsections
  70
+        doc.xpath("//h3[@id]").each_with_index { |h3,item_idx|
  71
+          subsection = h3.inner_text
  72
+          content = h3.xpath("./following-sibling::*").take_while {|x| x.name != "h3"}.map {|x| x.to_html}
  73
+          item = Nokogiri::HTML(h3.to_html + content.join("\n"))
  74
+          item_path = "sections/%03d/%03d.html" % [section_idx, item_idx] 
  75
+
  76
+          add_head_section item, subsection
  77
+
  78
+          # fix all image links
  79
+          item.search("img").each { |img|
  80
+            img['src'] = "#{Dir.pwd}/#{img['src']}"
  81
+          }
  82
+
  83
+
  84
+          File.open(item_path, 'w'){|f| f.puts item.to_html}
  85
+          puts "  #{item_path} -> #{subsection}"
  86
+
  87
+        }
  88
+      }
  89
+
  90
+      puts "=> Generating _document.yml"
  91
+
  92
+
  93
+      x = Nokogiri::XML(File.open("rails_guides.opf")).remove_namespaces!
  94
+
  95
+      document = {
  96
+        'doc_uuid' => x.at("package")['unique-identifier'],
  97
+        'title' => x.at("title").inner_text.gsub(/\(.*$/, " v2"),
  98
+        'publisher' => x.at("publisher").inner_text,
  99
+        'author' => x.at("creator").inner_text,
  100
+        'subject' => x.at("subject").inner_text,
  101
+        'date' => x.at("date").inner_text,
  102
+        'cover' => "#{Dir.pwd}/images/rails_guides_kindle_cover.jpg", 
  103
+        'masthead' => nil,
  104
+        'mobi_outfile' => outfile
  105
+      }
  106
+      puts document.inspect
  107
+      File.open("_document.yml", 'w'){|f| f.puts document.to_yaml}
  108
+
  109
+      exec "kindlerb ."
  110
+    end
  111
+  end
  112
+
  113
+  def add_head_section(doc, title)
  114
+    head = Nokogiri::XML::Node.new "head", doc
  115
+    title_node = Nokogiri::XML::Node.new "title", doc
  116
+    title_node.content = title
  117
+    title_node.parent = head
  118
+    css = Nokogiri::XML::Node.new "link", doc
  119
+    css['rel'] = 'stylesheet'
  120
+    css['type'] = 'text/css'
  121
+    css['href'] = "#{Dir.pwd}/stylesheets/kindle.css"
  122
+    css.parent = head
  123
+    doc.at("body").before head
  124
+  end
  125
+
  126
+end
127  railties/guides/rails_guides/docrails_kindle.rb
... ...
@@ -0,0 +1,127 @@
  1
+#!/usr/bin/env ruby
  2
+
  3
+require 'fileutils'
  4
+require 'yaml'
  5
+require 'date'
  6
+require 'nokogiri'
  7
+%w(kindlerb nokogiri).each do |g|
  8
+  begin 
  9
+    require g
  10
+  rescue Gem::LoadError
  11
+    $stderr.puts "Generating Kindle version of guides requires #{g}."
  12
+    exit 1
  13
+  end
  14
+end
  15
+
  16
+
  17
+module RailsGuides
  18
+  module DocrailsKindle
  19
+    def self.add_head_section(doc, title)
  20
+      head = Nokogiri::XML::Node.new "head", doc
  21
+      title_node = Nokogiri::XML::Node.new "title", doc
  22
+      title_node.content = title
  23
+      title_node.parent = head
  24
+      css = Nokogiri::XML::Node.new "link", doc
  25
+      css['rel'] = 'stylesheet'
  26
+      css['type'] = 'text/css'
  27
+      css['href'] = "#{Dir.pwd}/stylesheets/kindle.css"
  28
+      css.parent = head
  29
+      doc.at("body").before head
  30
+    end
  31
+
  32
+    def self.generate(output_dir, outfile)
  33
+      Dir.chdir(output_dir) {
  34
+
  35
+        # Get html pages in document order
  36
+
  37
+        html_pages = Nokogiri::XML(File.read("toc.ncx")).search("navMap//content[@src]").map {|c| c[:src]}.uniq
  38
+
  39
+        frontmatter = []
  40
+
  41
+        html_pages.delete_if {|x| 
  42
+          # we need to treat frontmatter differently
  43
+          if x =~ /(toc|welcome|credits|copyright).html/
  44
+            frontmatter << x unless x =~ /toc/
  45
+            true
  46
+          end
  47
+
  48
+        }
  49
+
  50
+        puts "=> Making one section for all frontmatter."
  51
+
  52
+        html = frontmatter.map {|x|
  53
+          Nokogiri::HTML(File.open(x)).at("body").inner_html
  54
+        }.join("\n")
  55
+
  56
+        fdoc = Nokogiri::HTML(html)
  57
+
  58
+        # need to turn author h3's into h4's
  59
+        fdoc.search("h3").each { |h3|
  60
+          h3.name = 'h4'
  61
+        }
  62
+        fdoc.search("h2").each { |h2| 
  63
+          h2.name = 'h3'
  64
+          h2['id'] = h2.inner_text.gsub(/\s/, '-')
  65
+        }
  66
+
  67
+        add_head_section fdoc, "Frontmatter"
  68
+        File.open("frontmatter.html",'w'){|f| f.puts fdoc.to_html}
  69
+        html_pages.unshift "frontmatter.html"
  70
+
  71
+        puts "=> Making one section folder per original HTML file"
  72
+        FileUtils::rm_rf("sections/")
  73
+
  74
+        html_pages.each_with_index { |page, section_idx|
  75
+          FileUtils::mkdir_p("sections/%03d" % section_idx)
  76
+          doc = Nokogiri::HTML(File.open(page))
  77
+          title = doc.at("title").inner_text.gsub("Ruby on Rails Guides: ", '')
  78
+          title = page.capitalize.gsub('.html', '') if title.strip == ''
  79
+          File.open("sections/%03d/_section.txt" % section_idx, 'w') {|f| f.puts title}
  80
+          puts "sections/%03d -> #{title}" % section_idx
  81
+          
  82
+          # Fragment the page file into subsections
  83
+          doc.xpath("//h3[@id]").each_with_index { |h3,item_idx|
  84
+            subsection = h3.inner_text
  85
+            content = h3.xpath("./following-sibling::*").take_while {|x| x.name != "h3"}.map {|x| x.to_html}
  86
+            item = Nokogiri::HTML(h3.to_html + content.join("\n"))
  87
+            item_path = "sections/%03d/%03d.html" % [section_idx, item_idx] 
  88
+
  89
+            add_head_section item, subsection
  90
+
  91
+            # fix all image links
  92
+            item.search("img").each { |img|
  93
+              img['src'] = "#{Dir.pwd}/#{img['src']}"
  94
+            }
  95
+
  96
+
  97
+            File.open(item_path, 'w'){|f| f.puts item.to_html}
  98
+            puts "  #{item_path} -> #{subsection}"
  99
+
  100
+          }
  101
+        }
  102
+
  103
+        puts "=> Generating _document.yml"
  104
+        x = Nokogiri::XML(File.open("rails_guides.opf")).remove_namespaces!
  105
+
  106
+        document = {
  107
+          'doc_uuid' => x.at("package")['unique-identifier'],
  108
+          'title' => x.at("title").inner_text,
  109
+          'publisher' => x.at("publisher").inner_text,
  110
+          'author' => x.at("creator").inner_text,
  111
+          'subject' => x.at("subject").inner_text,
  112
+          'date' => x.at("date").inner_text,
  113
+          'cover' => "#{Dir.pwd}/images/rails_guides_kindle_cover.jpg", 
  114
+          'masthead' => nil,
  115
+          'mobi_outfile' => outfile
  116
+        }
  117
+        puts document.inspect
  118
+        File.open("_document.yml", 'w'){|f| f.puts document.to_yaml}
  119
+
  120
+        log = "kindegen.log"
  121
+        system "kindlerb . > #{log} 2>&1 "
  122
+        puts "Guides compiled as Kindle book to #{outfile}\n(kindlegen log at #{log}).'"
  123
+      }
  124
+    end
  125
+  end
  126
+
  127
+end
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.