Fanout script: Document and make executable

[islemaster]

While dealing with a low memory page this morning, @sureshc and I spent some time figuring out what the state of our fanout scripts are and realized there's some organizational memory that hasn't been encoded in the scripts themselves. So here are a couple of changes to help with that.

• Make the scripts executable (644 -> 755)
  This removes the need to make them executable every time you'd like to use them, and also means they'll show up in the shell autocomplete when you're logged in to the server.
• Document expected usage pattern
  I've removed some very old TODOs and added documentation to each file explaining that we expect these scripts to run on production-daemon and describing a previously-undocumented prerequisite that the appropriate ssh config be copied into place before the script is run. (Apparently we wipe this config on every deploy.)

I think these are useful, safe changes to make on their own. I'm also interested in some discussion of other changes we could make here.

• @wjordan has a very old pull request that started work on the TODO I'm removing, adding a fanout method to RakeUtils.
• It seems like fanout-dryrun and fanout-parial ought to be flags on the main fanout script, both because it's DRY, and because right now I don't have confidence that dryrun will actually represent the behavior of fanout without doing a diff of the two scripts.
• Is running the command "for real" on console actually a good dry-run option?

[ewjordan]

LGTM - will address your comments in another comment.

[ewjordan]

I agree that dryrun and partial should be flags rather than separate files; I don't remember how it ended up this way, I suspect we intended to clean everything up after HoC but got distracted and just copied the working files into Git as they were so we didn't lose them. I believe the PR you mentioned was a start of that cleanup, but we didn't ever get around to getting the Chef stuff resolved, so put off finalizing it.

Is running the command "for real" on console actually a good dry-run option?
I'm not exactly sure what you mean by this - what would be the alternative?

[islemaster]

I guess dry-run is a fairly specialized concept depending on the command that you're talking about, but my instinct when I first saw that name was that it would do nothing, except maybe print the list of servers it intended to run the command on. I suppose in our setup running commands on console is more useful as a dry-run though.

[ewjordan]

Oh - I just realized that I hadn't looked at that file (the dryrun one). That does seem like odd behavior, I don't remember why we did that at all. I'm totally fine with changing or removing it.