Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse code

- various small updates

  • Loading branch information...
commit f5b7794c25fe3694945527f4e26d49bbf23fe765 1 parent bab2f88
Adam Salter adamsalter authored
33 README.md
Source Rendered
@@ -42,38 +42,37 @@ Other "difficult" Sitemap issues, solved by this plugin:
42 42 Installation
43 43 =======
44 44
  45 +*As a gem*
  46 +
  47 +1. Add the gem as a dependency in your config/environment.rb
  48 + <code>config.gem 'sitemap_generator', :source => 'http://gemcutter.org'</code>
  49 +
  50 +2. rake gems:install
  51 +
  52 +3. Add the following line to your RAILS_ROOT/Rakefile
  53 + <code>begin require 'sitemap_generator/tasks' rescue LoadError end</code>
  54 +
  55 +4. `rake sitemap:install`
  56 +
45 57 *As a plugin*
46 58
47 59 1. Install plugin as normal
48 60
49 61 <code>./script/plugin install git://github.com/adamsalter/sitemap_generator-plugin.git</code>
50 62
51   -2. Installation should create a 'config/sitemap.rb' file which will contain your logic for generation of the Sitemap files. (If you want to recreate this file manually run `rake sitemap:install`)
52 63
53   -3. Run `rake sitemap:refresh` as needed to create Sitemap files. This will also ping all the ['major'][sitemap_engines] search engines. (if you want to disable all non-essential output run the rake task thusly `rake -s sitemap:refresh SILENT=true`)
  64 +Installation should create a 'config/sitemap.rb' file which will contain your logic for generation of the Sitemap files. (If you want to recreate this file manually run `rake sitemap:install`)
  65 +
  66 +You can run `rake sitemap:refresh` as needed to create Sitemap files. This will also ping all the ['major'][sitemap_engines] search engines. (if you want to disable all non-essential output run the rake task thusly `rake -s sitemap:refresh SILENT=true`)
54 67
55 68 Sitemaps with many urls (100,000+) take quite a long time to generate, so if you need to refresh your Sitemaps regularly you can set the rake task up as a cron job. Most cron agents will only send you an email if there is output from the cron task.
56 69
57   -4. Finally, and optionally, add the following to your robots.txt file.
  70 +Optionally, you can add the following to your robots.txt file, so that robots can find the sitemap file.
58 71
59 72 <code>Sitemap: &lt;hostname>/sitemap_index.xml.gz</code>
60 73
61 74 The robots.txt Sitemap URL should be the complete URL to the Sitemap Index, such as: `http://www.example.org/sitemap_index.xml.gz`
62 75
63   -*As a gem*
64   -
65   -1. Add the gem as a dependency in your config/environment.rb
66   - <code>config.gem 'sitemap_generator', :source => 'http://gemcutter.org'</code>
67   -
68   -2. sudo rake gems:install
69   -
70   -3. Add the following line to your Rails.root/Rakefile
71   - <code>begin require 'sitemap_generator/tasks' rescue LoadError end</code>
72   -
73   -4. <code>rake sitemap:install</code>
74   -
75   -5. Follow steps 2-4 above
76   -
77 76
78 77 Example 'config/sitemap.rb'
79 78 ==========
22 Rakefile
@@ -3,24 +3,24 @@ require 'find'
3 3
4 4 begin
5 5 require 'jeweler'
6   - Jeweler::Tasks.new do |gem|
7   - gem.name = "sitemap_generator"
8   - gem.summary = %Q{This plugin enables 'enterprise-class' Google Sitemaps to be easily generated for a Rails site as a rake task}
9   - gem.description = %Q{This plugin enables 'enterprise-class' Google Sitemaps to be easily generated for a Rails site as a rake task}
10   - gem.email = "adam@salter.net "
11   - gem.homepage = "http://github.com/adamsalter/sitemap_generator-plugin"
12   - gem.authors = ["Adam Salter"]
13   - # gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
  6 + Jeweler::Tasks.new do |s|
  7 + s.name = "sitemap_generator"
  8 + s.summary = %Q{This plugin enables 'enterprise-class' Google Sitemaps to be easily generated for a Rails site as a rake task}
  9 + s.description = %Q{This plugin enables 'enterprise-class' Google Sitemaps to be easily generated for a Rails site as a rake task}
  10 + s.email = "adam.salter@codebright.net "
  11 + s.homepage = "http://github.com/adamsalter/sitemap_generator-plugin"
  12 + s.authors = ["Adam Salter"]
  13 + s.files = FileList["[A-Z]*", "{bin,lib,rails,templates}/**/*"]
  14 + # s is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
14 15 end
15   -
16 16 rescue LoadError
17   - puts "Jeweler (or a dependency) not available. Install it with: sudo gem install jeweler"
  17 + puts "Jeweler (or a dependency) not available. Install it with: gem install jeweler"
18 18 end
19 19
20 20 desc 'Default: run unit tests.'
21 21 task :default => :test
22 22
23   -desc 'Test ActiveScaffold.'
  23 +desc 'Test.'
24 24 Rake::TestTask.new(:test) do |t|
25 25 t.libs << 'lib'
26 26 t.pattern = 'test/**/*_test.rb'
2  init.rb
... ... @@ -1,2 +0,0 @@
1   -# Include hook code here
2   -
75 lib/sitemap_generator/tasks.rb
... ... @@ -1 +1,74 @@
1   -load "#{File.dirname(__FILE__)}/../../tasks/sitemap_generator_tasks.rake"
  1 +require 'zlib'
  2 +
  3 +namespace :sitemap do
  4 +
  5 + desc "Install a default config/sitemap.rb file"
  6 + task :install do
  7 + load File.expand_path(File.join(File.dirname(__FILE__), "../../rails/install.rb"))
  8 + end
  9 +
  10 + desc "Delete all Sitemap files in public/ directory"
  11 + task :clean do
  12 + sitemap_files = Dir[File.join(RAILS_ROOT, 'public/sitemap*.xml.gz')]
  13 + FileUtils.rm sitemap_files
  14 + end
  15 +
  16 + desc "Create Sitemap XML files in public/ directory"
  17 + desc "Create Sitemap XML files in public/ directory (set SILENT=true for no output)"
  18 + task :refresh => ['sitemap:create'] do
  19 + ping_search_engines("sitemap_index.xml.gz")
  20 + end
  21 +
  22 + desc "Create Sitemap XML files (don't ping search engines)"
  23 + task 'refresh:no_ping' => ['sitemap:create'] do
  24 + end
  25 +
  26 + task :create => [:environment] do
  27 + include SitemapGenerator::Helper
  28 + include ActionView::Helpers::NumberHelper
  29 +
  30 + start_time = Time.now
  31 +
  32 + # update links from config/sitemap.rb
  33 + load_sitemap_rb
  34 +
  35 + raise(ArgumentError, "Default hostname not defined") if SitemapGenerator::Sitemap.default_host.blank?
  36 +
  37 + links_grps = SitemapGenerator::Sitemap.links.in_groups_of(50000, false)
  38 + raise(ArgumentError, "TOO MANY LINKS!! I really thought 2,500,000,000 links would be enough for anybody!") if links_grps.length > 50000
  39 +
  40 + Rake::Task['sitemap:clean'].invoke
  41 +
  42 + # render individual sitemaps
  43 + sitemap_files = []
  44 + xml_sitemap_template = File.join(File.dirname(__FILE__), '../../templates/xml_sitemap.builder')
  45 + links_grps.each_with_index do |links, index|
  46 + buffer = ''
  47 + xml = Builder::XmlMarkup.new(:target=>buffer)
  48 + eval(open(xml_sitemap_template).read, binding)
  49 + filename = File.join(RAILS_ROOT, "public/sitemap#{index+1}.xml.gz")
  50 + Zlib::GzipWriter.open(filename) do |gz|
  51 + gz.write buffer
  52 + end
  53 + puts "+ #{filename}" unless ENV['SILENT'].present?
  54 + puts "** Sitemap too big! The uncompressed size exceeds 10Mb" if (buffer.size > 10 * 1024 * 1024) && ENV['SILENT'].blank?
  55 + sitemap_files << filename
  56 + end
  57 +
  58 + # render index
  59 + sitemap_index_template = File.join(File.dirname(__FILE__), '../../templates/sitemap_index.builder')
  60 + buffer = ''
  61 + xml = Builder::XmlMarkup.new(:target=>buffer)
  62 + eval(open(sitemap_index_template).read, binding)
  63 + filename = File.join(RAILS_ROOT, "public/sitemap_index.xml.gz")
  64 + Zlib::GzipWriter.open(filename) do |gz|
  65 + gz.write buffer
  66 + end
  67 + puts "+ #{filename}" unless ENV['SILENT'].present?
  68 + puts "** Sitemap Index too big! The uncompressed size exceeds 10Mb" if (buffer.size > 10 * 1024 * 1024) && ENV['SILENT'].blank?
  69 +
  70 + stop_time = Time.now
  71 + puts "Sitemap stats: #{number_with_delimiter(SitemapGenerator::Sitemap.links.length)} links, " + ("%dm%02ds" % (stop_time - start_time).divmod(60)) unless ENV['SILENT'].present?
  72 +
  73 + end
  74 +end
0  install.rb → rails/install.rb
File renamed without changes
0  uninstall.rb → rails/uninstall.rb
File renamed without changes
49 sitemap_generator.gemspec
... ... @@ -1,6 +1,6 @@
1 1 # Generated by jeweler
2   -# DO NOT EDIT THIS FILE
3   -# Instead, edit Jeweler::Tasks in Rakefile, and run `rake gemspec`
  2 +# DO NOT EDIT THIS FILE DIRECTLY
  3 +# Instead, edit Jeweler::Tasks in Rakefile, and run the gemspec command
4 4 # -*- encoding: utf-8 -*-
5 5
6 6 Gem::Specification.new do |s|
@@ -9,60 +9,28 @@ Gem::Specification.new do |s|
9 9
10 10 s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
11 11 s.authors = ["Adam Salter"]
12   - s.date = %q{2009-11-02}
  12 + s.date = %q{2009-11-04}
13 13 s.description = %q{This plugin enables 'enterprise-class' Google Sitemaps to be easily generated for a Rails site as a rake task}
14   - s.email = %q{adam@salter.net }
  14 + s.email = %q{adam.salter@codebright.net }
15 15 s.extra_rdoc_files = [
16 16 "README.md"
17 17 ]
18 18 s.files = [
19   - ".autotest",
20   - ".gitignore",
21   - "MIT-LICENSE",
  19 + "MIT-LICENSE",
22 20 "README.md",
23 21 "Rakefile",
24 22 "VERSION",
25   - "init.rb",
26   - "install.rb",
27 23 "lib/sitemap_generator.rb",
28 24 "lib/sitemap_generator/helper.rb",
29 25 "lib/sitemap_generator/link.rb",
30 26 "lib/sitemap_generator/link_set.rb",
31 27 "lib/sitemap_generator/mapper.rb",
32 28 "lib/sitemap_generator/tasks.rb",
33   - "sitemap_generator.gemspec",
34   - "tasks/sitemap_generator_tasks.rake",
  29 + "rails/install.rb",
  30 + "rails/uninstall.rb",
35 31 "templates/sitemap.rb",
36 32 "templates/sitemap_index.builder",
37   - "templates/xml_sitemap.builder",
38   - "test/mock_app/.gitignore",
39   - "test/mock_app/README",
40   - "test/mock_app/Rakefile",
41   - "test/mock_app/app/controllers/application_controller.rb",
42   - "test/mock_app/app/controllers/contents_controller.rb",
43   - "test/mock_app/app/models/content.rb",
44   - "test/mock_app/config/boot.rb",
45   - "test/mock_app/config/database.yml",
46   - "test/mock_app/config/environment.rb",
47   - "test/mock_app/config/environments/development.rb",
48   - "test/mock_app/config/environments/production.rb",
49   - "test/mock_app/config/environments/test.rb",
50   - "test/mock_app/config/initializers/backtrace_silencers.rb",
51   - "test/mock_app/config/initializers/inflections.rb",
52   - "test/mock_app/config/initializers/mime_types.rb",
53   - "test/mock_app/config/initializers/new_rails_defaults.rb",
54   - "test/mock_app/config/initializers/session_store.rb",
55   - "test/mock_app/config/locales/en.yml",
56   - "test/mock_app/config/routes.rb",
57   - "test/mock_app/config/sitemap.rb",
58   - "test/mock_app/db/migrate/20090826121911_create_contents.rb",
59   - "test/mock_app/db/schema.rb",
60   - "test/mock_app/db/test.sqlite3",
61   - "test/mock_app/public/index.html",
62   - "test/mock_app/script/console",
63   - "test/sitemap_generator_test.rb",
64   - "test/test_helper.rb",
65   - "uninstall.rb"
  33 + "templates/xml_sitemap.builder"
66 34 ]
67 35 s.homepage = %q{http://github.com/adamsalter/sitemap_generator-plugin}
68 36 s.rdoc_options = ["--charset=UTF-8"]
@@ -101,3 +69,4 @@ Gem::Specification.new do |s|
101 69 else
102 70 end
103 71 end
  72 +
74 tasks/sitemap_generator_tasks.rake
... ... @@ -1,74 +0,0 @@
1   -require 'zlib'
2   -
3   -namespace :sitemap do
4   -
5   - desc "Install a default config/sitemap.rb file"
6   - task :install do
7   - load File.expand_path(File.join(File.dirname(__FILE__), "..", "install.rb"))
8   - end
9   -
10   - desc "Delete all Sitemap files in public/ directory"
11   - task :clean do
12   - sitemap_files = Dir[File.join(RAILS_ROOT, 'public/sitemap*.xml.gz')]
13   - FileUtils.rm sitemap_files
14   - end
15   -
16   - desc "Create Sitemap XML files in public/ directory"
17   - desc "Create Sitemap XML files in public/ directory (set SILENT=true for no output)"
18   - task :refresh => ['sitemap:create'] do
19   - ping_search_engines("sitemap_index.xml.gz")
20   - end
21   -
22   - desc "Create Sitemap XML files (don't ping search engines)"
23   - task 'refresh:no_ping' => ['sitemap:create'] do
24   - end
25   -
26   - task :create => [:environment] do
27   - include SitemapGenerator::Helper
28   - include ActionView::Helpers::NumberHelper
29   -
30   - start_time = Time.now
31   -
32   - # update links from config/sitemap.rb
33   - load_sitemap_rb
34   -
35   - raise(ArgumentError, "Default hostname not defined") if SitemapGenerator::Sitemap.default_host.blank?
36   -
37   - links_grps = SitemapGenerator::Sitemap.links.in_groups_of(50000, false)
38   - raise(ArgumentError, "TOO MANY LINKS!! I really thought 2,500,000,000 links would be enough for anybody!") if links_grps.length > 50000
39   -
40   - Rake::Task['sitemap:clean'].invoke
41   -
42   - # render individual sitemaps
43   - sitemap_files = []
44   - xml_sitemap_template = File.join(File.dirname(__FILE__), '../templates/xml_sitemap.builder')
45   - links_grps.each_with_index do |links, index|
46   - buffer = ''
47   - xml = Builder::XmlMarkup.new(:target=>buffer)
48   - eval(open(xml_sitemap_template).read, binding)
49   - filename = File.join(RAILS_ROOT, "public/sitemap#{index+1}.xml.gz")
50   - Zlib::GzipWriter.open(filename) do |gz|
51   - gz.write buffer
52   - end
53   - puts "+ #{filename}" unless ENV['SILENT'].present?
54   - puts "** Sitemap too big! The uncompressed size exceeds 10Mb" if (buffer.size > 10 * 1024 * 1024) && ENV['SILENT'].blank?
55   - sitemap_files << filename
56   - end
57   -
58   - # render index
59   - sitemap_index_template = File.join(File.dirname(__FILE__), '../templates/sitemap_index.builder')
60   - buffer = ''
61   - xml = Builder::XmlMarkup.new(:target=>buffer)
62   - eval(open(sitemap_index_template).read, binding)
63   - filename = File.join(RAILS_ROOT, "public/sitemap_index.xml.gz")
64   - Zlib::GzipWriter.open(filename) do |gz|
65   - gz.write buffer
66   - end
67   - puts "+ #{filename}" unless ENV['SILENT'].present?
68   - puts "** Sitemap Index too big! The uncompressed size exceeds 10Mb" if (buffer.size > 10 * 1024 * 1024) && ENV['SILENT'].blank?
69   -
70   - stop_time = Time.now
71   - puts "Sitemap stats: #{number_with_delimiter(SitemapGenerator::Sitemap.links.length)} links, " + ("%dm%02ds" % (stop_time - start_time).divmod(60)) unless ENV['SILENT'].present?
72   -
73   - end
74   -end

0 comments on commit f5b7794

Please sign in to comment.
Something went wrong with that request. Please try again.