Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Support for publishing to S3 #455

Closed
wants to merge 3 commits into from

5 participants

@jelder

bundle exec rake s3 duplicates the functionality of bundle exec rake rsync using the s3cmd utility.

@parkr
Collaborator

@jelder Nice. Can you re-submit the changes to the rakefile on the current master?

Bump.

@parkr
Collaborator

@imathis and I voted against supporting S3 for a couple reasons. Brandon will elaborate soon.

@parkr parkr closed this
@imathis
Owner

At this point, I don't want to integrate S3 support because it's more involved to test since there are lots of different ways to use S3, like with CloudFront and recently Glacier. People might as well ask for support for Azure, AppFog, etc. and I don't want to be responsible to maintain different deployment mechanisms. I support Rsync because it's dead simple. I also support deployment to GitHub pages and Heroku since they us Git, it's already integrated into the toolchain and these are really easy to test.

Also, I don't use S3 for hosting so testing deployment mechanisms is kind of a pain for me. I'd much rather direct people to a series of steps to enable S3 deployment for themselves. If you want to blog about it, or post a repository with the necessary scripts and add a link to the wiki.

@cmer

:-1: . S3 support would be awesome since so many people deploy Octopress to S3. I hope you'll reconsider.

@imathis
Owner

I am reconsidering.

@imathis imathis reopened this
@parkr parkr commented on the diff
@@ -8,7 +8,12 @@ ssh_user = "user@domain.com"
ssh_port = "22"
document_root = "~/website.com/"
rsync_delete = true
-deploy_default = "rsync"
+deploy_default = "s3"
@parkr Collaborator
parkr added a note

Please remove this change - we'll want to leave deploy_default as "rsync".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@parkr parkr commented on the diff
@@ -8,7 +8,12 @@ ssh_user = "user@domain.com"
ssh_port = "22"
document_root = "~/website.com/"
rsync_delete = true
-deploy_default = "rsync"
+deploy_default = "s3"
+
+## -- S3 Deploy Config -- ##
+# Requires s3cmd. `brew install s3cmd` or see http://s3tools.org/download
@parkr Collaborator
parkr added a note

Perhaps a link to the INSTALL file would be better?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@parkr parkr commented on the diff
@@ -240,6 +245,16 @@ task :rsync do
ok_failed system("rsync -avze 'ssh -p #{ssh_port}' #{exclude} #{"--delete" unless rsync_delete == false} #{public_dir}/ #{ssh_user}:#{document_root}")
end
+desc "Deploy website to s3"
+task :s3 do
+ exclude = ""
@parkr Collaborator
parkr added a note

Thoughts on

exclude = File.exists?("./s3-exclude") ? "--exclude-from '#{File.expand_path('./s3-exclude')}'" : ""

?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@monfresh

There is an official AWS Ruby SDK now, which allows you to do all kinds of things without needing s3cmd.

While doing some website performance optimization research, I came across this gist via this blog post.

That gist allows you to push to S3 and invalidate changed files on Cloudfront, the latter not currently being supported by s3cmd. It also lets you optionally gzip and minify files, and compress images.

I'm planning on writing a blog post explaining how to implement that gist code in an octopress blog and what changes you'll need to make.

@monfresh

@parkr If you want to keep it simple for now with s3cmd, I would at least add the --no-preserve flag to prevent filesystem attributes (mode, ownership, timestamps) from being added to S3 files as metadata entries. See this issue: s3tools/s3cmd#67

I would also add an option to set the storage type to Reduced Redundancy instead of the default Standard Storage:

1) add reduced_redundancy = false in the S3 Deploy Config section

2) add #{"--reduced-redundancy" unless reduced_redundancy == false} in the s3cmd sync call.

@imathis
Owner

@monfresh Thank you so much for pointing me to this. I'd like to avoid s3cmd if possible and I'll be taking a look at this as a possible solution. Report back when you have your blog post written, I'd love to see what you end up doing.

@parkr
Collaborator

We'll definitely want to use some pure-ruby solution if at all possible.

Anyone know if s3cmd supports Windows? We could always print a message if it's not installed and choose to go ahead with it. I've been using it at work and it's really solid.

@parkr
Collaborator

We prefer the solution introduced in #1105 which uses the AWS SDK.

@parkr parkr closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 1, 2012
  1. @jelder

    Add rudimentary S3 support.

    jelder authored
Commits on May 8, 2012
  1. Grrr old ruby users

    Jacob Elder authored
This page is out of date. Refresh to see the latest.
Showing with 34 additions and 22 deletions.
  1. +1 −1  .rbenv-version
  2. +0 −1  Gemfile
  3. +17 −19 Gemfile.lock
  4. +16 −1 Rakefile
View
2  .rbenv-version
@@ -1 +1 @@
-1.9.2-p290
+1.9.3-p125
View
1  Gemfile
@@ -1,7 +1,6 @@
source "http://rubygems.org"
group :development do
- gem 'rake'
gem 'rack'
gem 'jekyll'
gem 'rdiscount'
View
36 Gemfile.lock
@@ -1,22 +1,22 @@
GEM
remote: http://rubygems.org/
specs:
- RedCloth (4.2.8)
+ RedCloth (4.2.9)
albino (1.3.3)
posix-spawn (>= 0.3.6)
blankslate (2.1.2.4)
- chunky_png (1.2.1)
+ chunky_png (1.2.5)
classifier (1.3.3)
fast-stemmer (>= 1.0.0)
- compass (0.11.5)
+ compass (0.12.1)
chunky_png (~> 1.2)
fssm (>= 0.2.7)
sass (~> 3.1)
- directory_watcher (1.4.0)
- fast-stemmer (1.0.0)
- ffi (1.0.9)
- fssm (0.2.7)
- haml (3.1.2)
+ directory_watcher (1.4.1)
+ fast-stemmer (1.0.1)
+ ffi (1.0.11)
+ fssm (0.2.9)
+ haml (3.1.5)
jekyll (0.11.0)
albino (>= 1.3.2)
classifier (>= 1.3.1)
@@ -24,28 +24,27 @@ GEM
kramdown (>= 0.13.2)
liquid (>= 1.9.0)
maruku (>= 0.5.9)
- kramdown (0.13.3)
+ kramdown (0.13.5)
liquid (2.2.2)
maruku (0.6.0)
syntax (>= 1.0.0)
posix-spawn (0.3.6)
- pygments.rb (0.1.3)
- rubypython (>= 0.5.1)
- rack (1.3.2)
- rake (0.9.2)
- rb-fsevent (0.4.3.1)
+ pygments.rb (0.2.12)
+ rubypython (~> 0.5.3)
+ rack (1.4.1)
+ rb-fsevent (0.9.1)
rdiscount (1.6.8)
rubypants (0.2.0)
- rubypython (0.5.1)
+ rubypython (0.5.3)
blankslate (>= 2.1.2.3)
ffi (~> 1.0.7)
- sass (3.1.5)
+ sass (3.1.17)
sinatra (1.2.6)
rack (~> 1.1)
tilt (>= 1.2.2, < 2.0)
- stringex (1.3.0)
+ stringex (1.4.0)
syntax (1.0.0)
- tilt (1.3.2)
+ tilt (1.3.3)
PLATFORMS
ruby
@@ -58,7 +57,6 @@ DEPENDENCIES
liquid (= 2.2.2)
pygments.rb
rack
- rake
rb-fsevent
rdiscount
rubypants
View
17 Rakefile
@@ -8,7 +8,12 @@ ssh_user = "user@domain.com"
ssh_port = "22"
document_root = "~/website.com/"
rsync_delete = true
-deploy_default = "rsync"
+deploy_default = "s3"
@parkr Collaborator
parkr added a note

Please remove this change - we'll want to leave deploy_default as "rsync".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+
+## -- S3 Deploy Config -- ##
+# Requires s3cmd. `brew install s3cmd` or see http://s3tools.org/download
@parkr Collaborator
parkr added a note

Perhaps a link to the INSTALL file would be better?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+s3_bucket = "website.com"
+s3_delete = false
# This will be configured for you when you run config_deploy
deploy_branch = "gh-pages"
@@ -240,6 +245,16 @@ task :rsync do
ok_failed system("rsync -avze 'ssh -p #{ssh_port}' #{exclude} #{"--delete" unless rsync_delete == false} #{public_dir}/ #{ssh_user}:#{document_root}")
end
+desc "Deploy website to s3"
+task :s3 do
+ exclude = ""
@parkr Collaborator
parkr added a note

Thoughts on

exclude = File.exists?("./s3-exclude") ? "--exclude-from '#{File.expand_path('./s3-exclude')}'" : ""

?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ if File.exists?('./s3-exclude')
+ exclude = "--exclude-from '#{File.expand_path('./s3-exclude')}'"
+ end
+ puts "## Deploying website via s3cmd"
+ ok_failed system("s3cmd sync --guess-mime-type --acl-public #{exclude} #{"--delete-removed" unless s3_delete == false} #{public_dir}/ s3://#{s3_bucket}/")
+end
+
desc "deploy public directory to github pages"
multitask :push do
puts "## Deploying branch to Github Pages "
Something went wrong with that request. Please try again.