closes #2: atomic file save with backup #6

Closed
wants to merge 1 commit into
from

Conversation

Projects
None yet
2 participants

Implemented per request using rename.

As implemented, it also create netrc.### backup files before saving in addition to the tempfile.

Behavior easily disabled by passing , false to atomic_write.

Run options: 

# Running tests:

....................................

Finished tests in 0.271142s, 132.7718 tests/s, 199.1576 assertions/s.

36 tests, 54 assertions, 0 failures, 0 errors, 0 skips
lib/file_utility.rb
@@ -0,0 +1,148 @@
+require 'tempfile'
+
+module FileUtility
@geemus

geemus Mar 12, 2012

Owner

This should probably be namespaced inside Netrc to avoid collisions.

lib/file_utility.rb
+ File.open(dest, 'wb', mode) do |d|
+ begin
+ begin # Zero copy version
+ require 'io/splice' # gem install io_splice # linux only
@geemus

geemus Mar 12, 2012

Owner

I don't know if adding the splice requirement is desirable here.

@steakknife

steakknife Mar 13, 2012

Yeap, that was for a different use case.

lib/file_utility.rb
+ # Pass mode to ensure dest is set to a particular file mode, otherwise it is copied from source.
+ #
+ def self.safe_rename(source, dest, make_backup = true, mode = nil)
+ backup(dest) if make_backup
@geemus

geemus Mar 12, 2012

Owner

Backup might not really be needed. I think, if nothing else, that the added complexity required for it seems like more trouble than perhaps it is really worth.

@steakknife

steakknife Mar 13, 2012

Again, different use case. Don't want to fill up dir with a bunch of backups obviously.

lib/file_utility.rb
+ backup(dest) if make_backup
+ mode ||= File.stat(source).mode
+ File.rename source, dest
+ File.chmod(mode, dest) if mode != File.stat(dest).mode
@geemus

geemus Mar 12, 2012

Owner

chmod should be skipped on windows, as it doesn't really do anything in most cases and will raise errors for non-ascii characters in paths.

@steakknife

steakknife Mar 13, 2012

The reason is that renaming temp files leaving yucky setuid. Granted, the mktemp call should be making a file right there to prevent any cross-filesystem hierarchy weirdness.

chown medical experimentation on various platforms:

Lucid works fine:

Using /home/ballard/.rvm/gems/ree-1.8.7-2012.01
~  ᐅ ( umask 777 ; touch ✪ ; ruby -rfileutils -e "FileUtils.chmod(0444,'✪')" ; ls -l ✪ ; rm -f ✪ )
-r--r--r-- 1 ballard ballard 0 2012-03-13 19:14 ✪
~  ᐅ rvm use 1.9.3
Using /home/ballard/.rvm/gems/ruby-1.9.3-p125
~  ᐅ ( umask 777 ; touch ✪ ; ruby -rfileutils -e "FileUtils.chmod(0444,'✪')" ; ls -l ✪ ; rm -f ✪ )
-r--r--r-- 1 ballard ballard 0 2012-03-13 19:14 ✪

Lion works fine:

~  ᐅ rvm use ree
Using /Volumes/Users/barry/.rvm/gems/ree-1.8.7-2012.02
~  ᐅ ( umask 777 ; touch ✪ ; ruby -rfileutils -e "FileUtils.chmod(0444,'✪')" ; ls -l ✪ ; rm -f ✪ )
-r--r--r--  1 barry  admin  0 Mar 13 12:15 ✪
~  ᐅ rvm use 1.9.3
Using /Volumes/Users/barry/.rvm/gems/ruby-1.9.3-p125
~  ᐅ ( umask 777 ; touch ✪ ; ruby -rfileutils -e "FileUtils.chmod(0444,'✪')" ; ls -l ✪ ; rm -f ✪ )
-r--r--r--  1 barry  admin  0 Mar 13 12:15 ✪
~  ᐅ rvm use jruby
Using /Volumes/Users/barry/.rvm/gems/jruby-1.6.7
~  ᐅ ( umask 777 ; touch ✪ ; ruby -rfileutils -e "FileUtils.chmod(0444,'✪')" ; ls -l ✪ ; rm -f ✪ )
-r--r--r--  1 barry  admin  0 Mar 13 12:15 ✪
~  ᐅ rvm use rbx
zsh: correct 'rbx' to '.rbx' [nyae]? n
Using /Volumes/Users/barry/.rvm/gems/rbx-head
~  ᐅ ( umask 777 ; touch ✪ ; ruby -rfileutils -e "FileUtils.chmod(0444,'✪')" ; ls -l ✪ ; rm -f ✪ )
-r--r--r--  1 barry  admin  0 Mar 13 12:15 ✪
~  ᐅ rvm use system
Now using system ruby.
~  ᐅ ( umask 777 ; touch ✪ ; ruby -rfileutils -e "FileUtils.chmod(0444,'✪')" ; ls -l ✪ ; rm -f ✪ )
-r--r--r--  1 barry  admin  0 Mar 13 12:16 ✪

Windows (tested on Windows 2003):

Cygwin: Was able to chmod and create a unicode character filename under cygwin. Windows could not display it, but it appeared correctly over a share. chmoding change NTFS ACEs mapping to (current user, Everyone and pseudo-real user None) per whatever mode is passed.

RubyInstaller 1.9.3: chown throws Error::EINVAL, even running via a unicode script with the magic header.

Cygwin looks like the best way to support unicode Ruby on Windows.

@geemus

geemus Mar 16, 2012

Owner

My experience on windows is similar to the rubyinstaller one you mentioned.

Owner

geemus commented Mar 12, 2012

@steakknife added some inline comments. Also, could you squash the commits? Thanks!

Owner

geemus commented Mar 13, 2012

@dpiddy Mind taking a look at this as well? I think you had a better sense of what you wanted here than I likely do.

SCISPOPD.

lib/netrc.rb
[new_item_prefix+"machine ", m, "\n login ", l, "\n password ", p, "\n"]
end
def save
- File.open(@path, 'w', 0600) {|file| file.print(unparse)}
+ Netrc.parse(Netrc.lex((unparsed = unparse).split "\n"))
+ FileUtility.atomic_write(@path) {|file| file.print(unparsed) }
@geemus

geemus Mar 13, 2012

Owner

Should probably still namespace this as Netrc::FileUtility, just in case.

Owner

geemus commented Mar 13, 2012

Looks like the new commit is lacking the lib/file_utility.rb file? Could you double check that? Thanks!

Yup. Was in the middle of fixing that. ;)

Owner

geemus commented Mar 16, 2012

@steakknife It doesn't look to me like copy_file, synchronous_compare_file and synchronous_compare_file_handle are ever actually called. Is that the case? Can we remove them? Sorry for all the run around, turns out atomic stuff is complex.

Owner

geemus commented Feb 9, 2016

Closing due to inactivity and due to no longer merging cleanly. Please re-open if you'd like to revisit.

@geemus geemus closed this Feb 9, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment