Update fix_permissions, better sync with desimodel, eliminate special config files

[weaverba137]

This PR:

• Closes Better synchronization between desiutil and desimodel #102
• Closes Consider eliminating configuration file support for desiInstall #103
• Closes Group writeability and fix_permissions.sh  #108

[sbailey]

fix_permissions.sh needs a bit more work:

[cori08 temp] mkdir blat && touch blat/foo
[cori08 temp] ls -ld blat && ls -l blat/foo
drwxrwxr-x 2 sjbailey sjbailey 4096 Apr 10 15:32 blat
-rw-rw-r-- 1 sjbailey sjbailey 0 Apr 10 15:32 blat/foo
[cori08 temp] fix_permissions.sh blat
[cori08 temp] ls -ld blat && ls -l blat/foo
drwxrwx--- 2 sjbailey desi 4096 Apr 10 15:32 blat
-rw-rwx--- 1 sjbailey desi 0 Apr 10 15:32 blat/foo

Bug: group execute was added to blat/foo even though it is a plain file and didn't have x to start with.

More subtle: original files had group=sjbailey with write permission, and got converted to group=desi while leaving the write permission, i.e. expanding who could edit this file. In #108 we originally said "fix_permissions.sh should not change the group-writeability bit", but I think what we should have said was "don't change who can write this file". i.e. if the desi group already had write access, leave it that way. And if the desi group doesn't already have write access (either because group write wasn't set, or because it wasn't in the desi group), then change it to the desi group (to ensure read access) but remove group write so that we don't accidentally expand who can write the file.

I also see that it removed world read permissions. We hadn't specified what to do for world bits, but I think that we should either leave those alone, or at least provide an option to leave those alone.

Another test:

Current master:

[cori08 temp] mkdir blat && ls -ld blat
drwxrwxr-x 2 sjbailey sjbailey 4096 Apr 10 15:51 blat
[cori08 temp] fix_permissions.sh blat && ls -ld blat
drwxrws---+ 2 sjbailey desi 4096 Apr 10 15:51 blat
[cori08 temp] touch blat/foo && ls -l blat/foo
-rw-rw---- 1 sjbailey desi 0 Apr 10 15:51 blat/foo

This branch:

[cori08 temp] mkdir blat && ls -ld blat
drwxrwxr-x 2 sjbailey sjbailey 4096 Apr 10 15:51 blat
[cori08 temp] fix_permissions.sh blat && ls -ld blat
drwxrwx--- 2 sjbailey desi 4096 Apr 10 15:51 blat
[cori08 temp] touch blat/foo && ls -l blat/foo
-rw-rw-r-- 1 sjbailey sjbailey 0 Apr 10 15:52 blat/foo

The current master sets ACLs or something such that after running fix_permissions.sh, any new files created in that directory will also have the correct group=desi with read access (good). This is not the case for this branch.

[weaverba137]

Could you please run all of the tests in verbose mode?

[sbailey]

Test 1:

[cori07 sjbailey] mkdir blat && touch blat/foo
[cori07 sjbailey] ls -ld blat && ls -l blat/foo
drwxrwxr-x 2 sjbailey sjbailey 4096 Apr 10 17:04 blat
-rw-rw-r-- 1 sjbailey sjbailey 0 Apr 10 17:04 blat/foo
[cori07 sjbailey] fix_permissions.sh -v blat
Fixing permissions on blat ...
/usr/bin/find blat -user sjbailey -not -group desi -exec chgrp -c -h desi {} ;
changed group of 'blat' from sjbailey to desi
changed group of 'blat/foo' from sjbailey to desi
/usr/bin/find blat -user sjbailey -type f -not -perm /g+r -exec chmod -c g+r {} ;
/usr/bin/find blat -user sjbailey -type f ( -perm /o+rwx -or -not -perm /g+w ) -exec chmod -c g+w,o-rwx {} ;
mode of 'blat/foo' changed from 0664 (rw-rw-r--) to 0660 (rw-rw----)
/usr/bin/find blat -user sjbailey -type d -not -perm 2770 -exec chmod -c 2770 {} ;
mode of 'blat' changed from 0775 (rwxrwxr-x) to 2770 (rwxrws---)
/usr/bin/find blat -user sjbailey -exec /usr/bin/setfacl --remove-all {} ;
/usr/bin/find blat -user sjbailey -type d -exec /usr/bin/setfacl --default -m u::rwx,g::rwx,o::--- {} ;
[cori07 sjbailey] ls -ld blat && ls -l blat/foo
drwxrws---+ 2 sjbailey desi 4096 Apr 10 17:04 blat
-rw-rw---- 1 sjbailey desi 0 Apr 10 17:04 blat/foo

Test 2 current master:

[cori07 sjbailey] mkdir blat && ls -ld blat
drwxrwxr-x 2 sjbailey sjbailey 4096 Apr 10 17:06 blat
[cori07 sjbailey] fix_permissions.sh -v blat && ls -ld blat
Fixing permissions on blat ...
/usr/bin/find blat -user sjbailey -not -group desi -exec chgrp -c -h desi {} ;
changed group of 'blat' from sjbailey to desi
/usr/bin/find blat -user sjbailey -type f -not -perm /g+r -exec chmod -c g+r {} ;
/usr/bin/find blat -user sjbailey -type f ( -perm /o+rwx -or -not -perm /g+w ) -exec chmod -c g+w,o-rwx {} ;
/usr/bin/find blat -user sjbailey -type d -not -perm 2770 -exec chmod -c 2770 {} ;
mode of 'blat' changed from 0775 (rwxrwxr-x) to 2770 (rwxrws---)
/usr/bin/find blat -user sjbailey -exec /usr/bin/setfacl --remove-all {} ;
/usr/bin/find blat -user sjbailey -type d -exec /usr/bin/setfacl --default -m u::rwx,g::rwx,o::--- {} ;
drwxrws---+ 2 sjbailey desi 4096 Apr 10 17:06 blat
[cori07 sjbailey] touch blat/foo && ls -l blat/foo
-rw-rw---- 1 sjbailey desi 0 Apr 10 17:06 blat/foo

Test 2 this branch:

[cori11 desiutil] rm -rf blat
[cori11 desiutil] mkdir blat && ls -ld blat
drwxrwxr-x 2 sjbailey sjbailey 4096 Apr 17 16:16 blat
[cori11 desiutil] fix_permissions.sh -v blat && ls -ld blat
Fixing permissions on blat ...
/usr/bin/find blat -user sjbailey -not -group desi -exec chgrp -c -h desi {} ;
changed group of 'blat' from sjbailey to desi
/usr/bin/find blat -user sjbailey -type f -not -perm /g+r -exec chmod -c g+r {} ;
/usr/bin/find blat -user sjbailey -type d -not -perm -g+rx -exec chmod -c g+rx {} ;
/usr/bin/find blat -user sjbailey -perm /o+rwx -exec chmod -c o-rwx {} ;
mode of 'blat' changed from 0775 (rwxrwxr-x) to 0770 (rwxrwx---)
drwxrwx--- 2 sjbailey desi 4096 Apr 17 16:16 blat
[cori11 desiutil] touch blat/foo && ls -l blat/foo
-rw-rw-r-- 1 sjbailey sjbailey 0 Apr 17 16:16 blat/foo

To ease cutting and pasting if we need to run this again:

Test 1:
rm -rf blat
mkdir blat && touch blat/foo
ls -ld blat && ls -l blat/foo
fix_permissions.sh -v blat
ls -ld blat && ls -l blat/foo

Test 2:
rm -rf blat
mkdir blat && ls -ld blat
fix_permissions.sh -v blat && ls -ld blat
touch blat/foo && ls -l blat/foo

[weaverba137]

@sbailey, are you really certain that you are running the tests on the correct branch? On this branch, fix_permissions only runs setfacl if adding permissions for the apache user, but the setfacl command appears in "Test 2 this branch".

[weaverba137]

Also note, I found the bug that was causing executable bits to be set on files. But that has nothing to do with whether setfacl is run by fix_permissions.sh.

[sbailey]

Indeed, it appears that I ran the verbose version of "test 2 this branch" with the wrong branch. I edited the comment above to have the the correct branch (permissions-and-desimodel). It now shows the (undesired) behavior of files created after running fix_permissions.sh having the old incorrect group.

[weaverba137]

OK, I was trying to simplify fix_permissions.sh so it was orthogonal in some sense to the automated permissions "wrangling" (to use Lisa's term) that NERSC is running. In other words, let the automation deal with things like setting the desi pseudo-user access and the setgid bit (which controls the group id of new files). However maybe that is not the desired design choice, especially if you don't want to wait for the automated wrangling.

Do we want fix_permissions.sh to be exactly the same as the automated wrangling?

Also,

More subtle: original files had group=sjbailey with write permission, and got converted to group=desi while leaving the write permission, i.e. expanding who could edit this file. In #108 we originally said "fix_permissions.sh should not change the group-writeability bit", but I think what we should have said was "don't change who can write this file". i.e. if the desi group already had write access, leave it that way. And if the desi group doesn't already have write access (either because group write wasn't set, or because it wasn't in the desi group), then change it to the desi group (to ensure read access) but remove group write so that we don't accidentally expand who can write the file.

"desi group doesn't already have write access, ... remove group write". This is actually not that easy to implement, because at minimum it would involve passing the results of one find command to another, and might even require replacing fix_permissions.sh with a Python script.

[sbailey]

Comments on comments/questions:

OK, I was trying to simplify fix_permissions.sh so it was orthogonal in some sense to the automated permissions "wrangling" (to use Lisa's term) that NERSC is running. In other words, let the automation deal with things like setting the desi pseudo-user access and the setgid bit (which controls the group id of new files). However maybe that is not the desired design choice, especially if you don't want to wait for the automated wrangling.

Do we want fix_permissions.sh to be exactly the same as the automated wrangling?

Yes. I think the main purpose of the automated wrangling is to catch cases where people didn't run fix_permissions.sh in the first place. And the motivation for a user to run fix_permissions.sh instead of waiting for the automated update would be "if you don't want to wait for the automated wrangling".

"desi group doesn't already have write access, ... remove group write". This is actually not that easy to implement, because at minimum it would involve passing the results of one find command to another, and might even require replacing fix_permissions.sh with a Python script.

How about making the default be "change to group=desi and remove group write" and if users really do want the desi group to have write access, they can followup with "chmod -R g+w dirname"?

[sbailey]

Update: we should also avoid the situation where the user grants write permission and then a subsequent run of fix_permissions.sh removes it again. i.e. if it is already group=desi and group writeable, leave it that way. I think that logic is possible with find.

[weaverba137]

@sbailey, that is the number one point of this update to fix_permissions.sh! I am trying very hard to satisfy that, and I am reasonably confident that it does. I'm starting to think you're not actually reading the code.

[weaverba137]

I feel like we're starting to make up requirements on the DESI data permissions piecemeal. Let's back up and review https://desi.lbl.gov/trac/wiki/Computing/NerscFileSystem#Requirements. Let's expand that, and explain the reasoning behind every step. Then write a script that enforces those requirements. Here are some specific things to consider:

• I've already solved the group-write bit problem. That bit is not touched.
• If you're concerned about world bits, my view is that no world bit should ever be set except on data releases. And even then, only by hand. To do otherwise is to say:
  • Arbitrary DESI collaborators have the power to make DESI data public.
  • We're not going to even try to fix cases where world bits are set carelessly, for example, by a bad umask, or by a botched data transfer.
• Do you and I even have the authority to impose any of this without consulting, say, the Science Committee?

[sbailey]

My "Update" comment was a clarification that my previous suggestion "change to group=desi and remove group write" was not complete, since we don't want to remove group write if the file was already in the desi group.

However, not touching the group write bit does not completely solve the group-write bit problem since that bit interplays with enforcing the requirement that group=desi.

One of the wiki page requirements is "Group-writeable is optional". I suggest that what we really mean by that is "fix_permissions.sh will not change whether the desi group has write access to a file", which is different from saying "fix_permissions.sh will not change the group write bit". Returning to "Test 1":

[cori06 temp] rm -rf blat
[cori06 temp] mkdir blat && touch blat/foo
[cori06 temp] ls -ld blat && ls -l blat/foo
drwxrwxr-x 2 sjbailey sjbailey 4096 Apr 19 15:24 blat
-rw-rw-r-- 1 sjbailey sjbailey 0 Apr 19 15:24 blat/foo
[cori06 temp] fix_permissions.sh -v blat
Fixing permissions on blat ...
/usr/bin/find blat -user sjbailey -not -group desi -exec chgrp -c -h desi {} ;
changed group of 'blat' from sjbailey to desi
changed group of 'blat/foo' from sjbailey to desi
/usr/bin/find blat -user sjbailey -type f -not -perm /g+r -exec chmod -c g+r {} ;
/usr/bin/find blat -user sjbailey -type d -not -perm -g+rxs -exec chmod -c g+rxs {} ;
mode of 'blat' changed from 0775 (rwxrwxr-x) to 2775 (rwxrwsr-x)
/usr/bin/find blat -user sjbailey -perm /o+rwx -exec chmod -c o-rwx {} ;
mode of 'blat' changed from 2775 (rwxrwsr-x) to 2770 (rwxrws---)
mode of 'blat/foo' changed from 0664 (rw-rw-r--) to 0660 (rw-rw----)
[cori06 temp] ls -ld blat && ls -l blat/foo
drwxrws--- 2 sjbailey desi 4096 Apr 19 15:24 blat
-rw-rw---- 1 sjbailey desi 0 Apr 19 15:24 blat/foo

In this case, blat/foo originally was not writeable by the desi group, and after running fix_permissions.sh it became writeable by the desi group. Indeed, it didn't change the group write bit, but it did effectively change the desi group write access since it (correctly!) changed the group to desi. I believe the correct final state to meet the requirements is:

drwxr-s--- 2 sjbailey desi 4096 Apr 19 15:24 blat
-rw-r----- 1 sjbailey desi 0 Apr 19 15:24 blat/foo

(I'm not sure about the 's'). i.e. it should enforce that group=desi but it shouldn't inadvertently change the write access of the desi group in the process.

[weaverba137]

OK, I'm much more clear on this now, but I think this distinction will not matter to most people since (I maintain) most users set permissions completely carelessly. So again, we're back to the question: are we effectively supporting carelessness?

[sbailey]

Many users do set permissions carelessly, and the purpose of fix_permissions.sh is to give them a one-stop shopping method for fixing those permissions if they copy anything into the desi /project space. Regardless, I still assert that we shouldn't inadvertently add write access to the desi group if the original file/directory was not writeable by the desi group. Inadvertently granting write access would be exacerbating carelessness rather than ameliorating it.

[weaverba137]

As I already noted:

"desi group doesn't already have write access, ... remove group write". This is actually not that easy to implement, because at minimum it would involve passing the results of one find command to another, and might even require replacing fix_permissions.sh with a Python script.

So do you actually want me to try to implement this? When you previously responded, my interpretation of the response was that you didn't think it was necessary. That will add quite a bit more work. Also that gets pretty far away from the concept of "fix_permissions.sh does the exact same thing as the automated permission-wrangling script".

Also could you please respond to the comment beginning:

I feel like we're starting to make up requirements on the DESI data permissions piecemeal.

None of the bullet points in that comment have been addressed.

[sbailey]

I may be missing something, but I think we can implement "change to group=desi but don't grant the desi group write access if it didn't already have it" via

#- For files/dirs that aren't group=desi, remove group write prior to changing group.
#- Note that files/dirs with group=desi are unchanged, thus preserving whatever
#- write permission that the desi group had in the first place
/usr/bin/find blat -user sjbailey -not -group desi -exec chmod g-w {} ;

#- Now change everything to group=desi and ensure group readability
/usr/bin/find blat -user sjbailey -not -group desi -exec chgrp -c -h desi {} ;
chmod -R g+rX blat

Responding to the other bullet points:

I feel like we're starting to make up requirements on the DESI data permissions piecemeal. Let's back up and review https://desi.lbl.gov/trac/wiki/Computing/NerscFileSystem#Requirements. Let's expand that, and explain the reasoning behind every step. Then write a script that enforces those requirements.

Thanks for pointing out that wiki page. I had read those requirements and agreed with them, modulo the interpretation of "Group-writeable is optional". I was trying to expand on that in this thread since wikis are not great for conversation threads before finalizing something.

In particular, I interpret it as "either group write or no write is fine, but fix_permissions shouldn't change whether the desi group has write access (which is different than whether it should change the group write bit, since it may be changing the group).

Here are some specific things to consider:

I've already solved the group-write bit problem. That bit is not touched.

I don't consider "that bit is not touched" to have "solved the group-write bit problem", because of the example I have shown of inadvertently granting write permission to the desi group when it didn't previously have it. That is a form of accidentally enforcing group write instead of leaving it as optional.

If you're concerned about world bits, my view is that no world bit should ever be set except on data releases. And even then, only by hand. To do otherwise is to say:
Arbitrary DESI collaborators have the power to make DESI data public.
We're not going to even try to fix cases where world bits are set carelessly, for example, by a bad umask, or by a botched data transfer.
Do you and I even have the authority to impose any of this without consulting, say, the Science Committee?

I'm inclined to have fix_permissions leave world bits alone, but could also see a case for removing world readability by default. It certainly shouldn't grant world readability by default. There may be good cases for sharing some directory with non-desi colleagues (e.g. pointing a NERSC person at an installation of a public github repo)
and it could be a pain if fix_permissions or the auto wrangler kept breaking that. But I don't fell strongly about it either way (though if it does become a pain either way, I will start to feel strongly about it...)

[weaverba137]

@sbailey, I've updated https://desi.lbl.gov/trac/wiki/Computing/NerscFileSystem with a more rigorous definition of file system permission requirements. Let's converge on that, then I'll apply the consensus requirements to the fix_permissions script.

[sbailey]

@weaverba137 I added one question and an extended comment to that wiki page.

In particular, I added a table of suggested before/after cases for fix_permissions.sh since the requirement text was clear that desi group write is optional, but wasn't clear about what fix_permissions.sh should do about it. Simply leaving the group bit alone is different from not changing whether the desi group has write access, and it is unclear to me from this ticket whether we disagree about what should be done or whether we are just misunderstanding each other. If you disagree with any of those examples, please add a comment about what you think the final state should be and why.

[sbailey]

ACLs: ugh.

desi user: if we have to choose between granting the desi user write access and having "obvious" interpretation of the bits from "ls -l", let's favor the more end-user friendly "ls -l". I added that as a proposed requirement on the wiki. As long as the desi group has read access, we can still do disk inventories and invoke NERSC consulting if we need to request something to be deleted, but if the permissions reported by "ls -l" don't match what can actually be done, a lot of users will waste time being very confused.

apache user: is the core problem of ACLs and non-obvious "ls -l" specific to the fact that "desi" is both a user and a group, or does it apply to the apache user too? i.e. is it possible to grant read-access to the apache user without mucking up what "ls -l" reports?

Note: If the core problem is that "desi" is both a user and a group, we could change the name of that production account user. I fear it goes deeper than that though.

[weaverba137]

I'm pretty sure that the ls -l weirdness comes from the write permission, not from any user-group confusion, but I'd have to test this.

Can we please think about this carefully though before we undo all the work that I've done and that NERSC has already done? Personally I think it is far, far weirder that world-writeability determines group-writeability (in the proposed configuration) than /bin/ls reporting unexpected results.

[sbailey]

Let's put a pause on this and discuss it at the data telecon. github + wiki are not proving efficient for converging on this.

[weaverba137]

Agreed. In the meantime, I have confirmed that it's the write permission for the desi user that causes the "phantom" group-write permission. When I apply (only) the read-only permissions for the apache user, the phantom group-write does not appear.

[sbailey]

Updates from desi data call today:

Having "ls -l" provide meaningful results trumps having ACLs that give the desi pseudo-user write access. If we can't have both, prioritize "ls -l". We currently think that means dropping the desi-user write access ACLs.

Proposed update to what to do with group write bits:

• if the group isn't desi:
  • remove group write, add group read, and change the group to desi
• else (i.e. the group already is desi):
  • add group read if it isn't already set, but leave group write alone

We didn't explicitly discuss world bits today, but I propose the following:

• if the group isn't desi:
  • remove world read/write/execute
• else (i.e. the group already is desi):
  • remove world write (!) but leave the read bit alone assuming that if it was set it was set for a good reason

Put another way:

• if the group isn't desi, set permissions to -rw-r----- and change group to desi (and equivalent user/group execute but not world execute for directories).
• if the group is desi, add group read if needed but otherwise leave the bits alone assuming that group write and world read are purposefully set.

If that last suggestion is controversial, removing world read/write no matter what could also be reasonable and then revisit this if we find cases where we really do want world read and don't want fix_permissions.sh to change that.

[weaverba137]

@sbailey, I've read the latest summary. Will get a new version of fix_permissions out soon.

[weaverba137]

Updating my previous pseudo-code example, comments welcome.

for each file and directory {
    if not in desi group {
        remove group write access
        remove all world access
        change group to desi
    } else {
        remove world write
    }
    set group read/traverse/setgid permissions
    if apache access requested {
        set apache account ACL
    }
}

[sbailey]

I agree with that pseudo-code. Thanks.

[weaverba137]

@sbailey, I'm satisfied with the current version of fix_permissions.sh. You may want to try running test_fix_permissions.sh in a scratch directory to check various combinations for yourself.

[sbailey]

Almost there. fix_permissions.sh works for me as expected for correcting existing files, but the setgid bit doesn't seem to be working properly for permissions of files created after running fix_permissions.sh on a directory:

#- Make a directory and run fix_permissions.sh: looks good
$> mkdir blat
$> ls -ld blat
drwxrwxr-x 2 sjbailey sjbailey 4096 May  9 11:16 blat
$> fix_permissions.sh blat
$> ls -ld blat
drwxr-s--- 2 sjbailey desi 4096 May  9 11:16 blat

#- Create a file in that directory and check its group and permissions:
$> touch blat/foo.txt
$> ls -l blat/foo.txt
-rw-rw-r-- 1 sjbailey desi 0 May  9 11:17 blat/foo.txt

#- Oops: blat/foo.txt shouldn't have group write.

blat/foo.txt has the correct group=desi, but it shouldn't have group write since the parent directory blat/ doesn't have group write.

When I run test_fix_permissions.sh on edison I get some error messages from chgrp, though I haven't seen those when running fix_permissions.sh itself.

Dir0/ permission not set properly!
Dir0/File0 permission not set properly!
Dir1/ permission not set properly!
Dir1/File1 permission not set properly!
chgrp: missing operand after 'Dir5'
Try 'chgrp --help' for more information.
chgrp: missing operand after 'Dir5/File5'
Try 'chgrp --help' for more information.
Dir5/ permission not set properly!
Dir5/File5 permission not set properly!
chgrp: missing operand after 'Dir6'
Try 'chgrp --help' for more information.
chgrp: missing operand after 'Dir6/File6'
Try 'chgrp --help' for more information.
chgrp: missing operand after 'Dir7'
Try 'chgrp --help' for more information.
chgrp: missing operand after 'Dir7/File7'
Try 'chgrp --help' for more information.
chgrp: missing operand after 'Dir8'
Try 'chgrp --help' for more information.
chgrp: missing operand after 'Dir8/File8'
Try 'chgrp --help' for more information.
chgrp: missing operand after 'Dir9'
Try 'chgrp --help' for more information.
chgrp: missing operand after 'Dir9/File9'
Try 'chgrp --help' for more information.

[weaverba137]

The only way to enforce permission bits is to use a default ACL, which we were using until we started down the desi pseudo-user path.

The setgid bit enforces group ownership not group permission bits.

Could you please tell me which directory at NERSC your are running test_fix_permissions.sh in?

[weaverba137]

PS, if you want your files to never be group-writeable, you should set your umask accordingly.

There basically is no solution that will aggressively enforce permission bits on all files created into the indefinite future, except very strict ACLs, and then we get back to the problem of /bin/ls lying.

[sbailey]

I was running this on edison in /scratch2/scratchdirs/sjbailey/temp/blat .

I was hoping that fix_permissions.sh would still be able to set the default ACL entries (like current master does) without messing up 'ls -l' output. But if that is another rabbit hole, ok, we can just recommend that people use umask 22 if they don't want group write for new files and move on.

[weaverba137]

Could you run test_fix_permissions.sh again, in the same directory, but this time make certain that when test_fix_permissions.sh calls fix_permissions.sh, it is calling the one in the same bin/ directory, not the one that might be in your $PATH.

[sbailey]

test_fix_permissions.sh and fix_permissions.sh are both coming from my git clone:

[edison06 blat] which test_fix_permissions.sh
/scratch2/scratchdirs/sjbailey/desi/code/desiutil/bin/test_fix_permissions.sh
[edison06 blat] which fix_permissions.sh
/scratch2/scratchdirs/sjbailey/desi/code/desiutil/bin/fix_permissions.sh

I edited my copy of fix_permissions.sh to also "echo hello" to confirm that it is indeed the one being called, and I still get the "chgrp: missing operand after 'Dir9'
Try 'chgrp --help' for more information" messages.

[weaverba137]

OK, then could you please run test_fix_permissions.sh -v?

[sbailey]

Offlist we sorted out that the test_fix_permissions.sh problem was another tcsh/bash difference, i.e. whether $GROUP is set or not. It works now.

I'm ok to merge this as-is as a step in the right direction, though when I see @weaverba137 in person next I'd like to revisit the default ACL question.