Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POSIX compliance #927

Open
OH-AU opened this issue Feb 26, 2019 · 14 comments
Open

POSIX compliance #927

OH-AU opened this issue Feb 26, 2019 · 14 comments

Comments

@OH-AU
Copy link

@OH-AU OH-AU commented Feb 26, 2019

In _extras/guide.md you suggest "Stay within POSIX-compliant commands" and helpfully give an example, however no assistance/referral is provided to check compliance. Having used various UNIX and UNIX-like operating systems, I couldn't tell you if a command I regularly use is POSIX complaint or not. The closest related issue I could find was #708 which asks for assistance in auditing. Perhaps the guide could be updated with a reference to:

  1. https://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html
    Unfortunately, this doesn't cover bash itself. So also include a reference to:
  2. https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_01

Perhaps something like; You can check for POSIX compliance for the shell here and command line utilities here
Thanks.

@colinmorris
Copy link
Contributor

@colinmorris colinmorris commented Feb 28, 2019

Great point. I agree that, as it currently stands, this piece of advice is not very helpful. Even with the additional links you suggest, I think it would still be a little unclear (i.e. it's not clear the spirit that's motivating this rule, or what specific actions instructors are expected to take in response to it).

This is somewhat related to our ongoing discussion at #798.

I wonder about deleting this advice and replacing it with a section with some more informal, general advice about cross-platform issues and consistency, basically stuff like:

  • Where possible, try to use commands that work (and do the same thing) on all platforms. We've aimed for this in the examples given in the lesson.
  • Try to avoid teaching features that are specific to a particular operating system or shell dialect (unless all learners are using it?). For example, ____
  • You may still encounter some scenarios where a command fails or gives different results for certain learners according to their system's configuration. It's worth explaining to learners that there are some slight differences between Bash shells on different systems, but try not to dwell on them, or get bogged into technical explanations for why these differences exist.
  • [Some practical advice for powering through these hiccups during a workshop? e.g. if you have a minority of learners using system Y, you may want to seat them together so they can help each other out?]

@gdevenyi
Copy link
Contributor

@gdevenyi gdevenyi commented Mar 11, 2019

+1 for enhancing that in the instructor's guide

We still have the issue of what to do with the lessons themselves though...

@OH-AU
Copy link
Author

@OH-AU OH-AU commented Mar 13, 2019

Thanks for the comments - the bash conversation you linked to covers a number of related matters. I'm for @colinmorris 's suggestion of making it more generalised - most systems I use default to bash as the shell and the course is largely based around that presumption anyway (take the tab completion section as an example - that's not going to work as documented in ksh, or csh for that matter). And every system has their own custom alias' for common command so I think best effort, generalised as suggested above is reasonable and hopefully any major issues would get picked up on review when commited?

@drj11
Copy link
Member

@drj11 drj11 commented Mar 22, 2019

I would avoid removing the note about POSIX compliance but would support adding a note about the motivation: "Where possible, try to use commands that work (and do the same thing) on all platforms" (quoting @colinmorris above).

I think both are required: Yes, we are trying to use commands that work on all platforms; and, as a practical guideline, we therefore prefer teaching POSIX compliant commands and examples.

At least we can start from there and add links to the relevant POSIX specifications and guides on how to comply. From a personal perspective I have no way to test if commands work on all platforms, as I only use shades of Unix, but at least I can read the POSIX specification.

@LHurst-UoB
Copy link

@LHurst-UoB LHurst-UoB commented Jun 21, 2019

I've been thinking about this a lot lately, as internally at work we've deviated from POSIX in our coding standards in favour of safer code.

I'm not sure, as the session is about using Bash specifically, POSIX-compliance is a helpful target as some of the safer features of BASH, e.g. the double square bracket for ifs/whiles which are safer than the POSIX-compliant '[' command, are not POSIX-compliant.

@drj11
Copy link
Member

@drj11 drj11 commented Jun 24, 2019

The version of bash in OS X is ancient and they Apple are signalling that they are going to drop it: https://scriptingosx.com/2019/06/moving-to-zsh/

So I'm not sure that moving towards bash is going to encourage cross platform work.

@LHurst-UoB
Copy link

@LHurst-UoB LHurst-UoB commented Jun 25, 2019

zsh is almost entirely bash-compatible, the double bracket "bash-ism" is fully supported for example.

This is a bash course - that's what the introduction says, and I've had that reiterated in response to some of my merge requests.

Personally, it doesn't sit well with me that this is called a "Unix Shell" course but the gatekeepers to merging changes say it's a pure bash course.

Each shell provides a complete programming language in its own right, so what we're talking about is trying to present multiple programming languages in a single course; sh, bash, zsh - csh, ksh if you wanted to extend it (which some of our researchers and/or 3rd party research software are still wedded to).

@drj11
Copy link
Member

@drj11 drj11 commented Jun 25, 2019

to most learners: unix shell = unix = bash (in particular, whether the shell is actually bash or not, it may well get called bash).

it's only nerds like us that care.

@aragilar
Copy link

@aragilar aragilar commented Jun 26, 2019

@drj11 I regularly run into systems (I ssh into one every day) where bash isn't the default (login) shell (not my choice, I'd rather use bash), and /bin/sh isn't bash quite regularly (e.g. on Ubuntu/Debian). We already need to tell learners not to use powershell or cmd, telling them not to use (t)csh isn't going to waste that much time (older macs used tcsh as the login shell, and I've seen people have trouble because of it).

I do think the instructor notes need to be more specific (not less). Rather than saying "your shell may have extensions" (which to me sounds very weaselwordy), state something like "GNU shell utilities have additional options (such as the options which start with -- (use the current examples here)) which are beyond that specified in the POSIX standard, and will not work on non-GNU systems. Look up the POSIX standard (include opengroup links here), or try running the commands with busybox or with dash to understand the difference. Search GNU coreutils and GNU findutils to find out more about these additional features the GNU provide." It'd probably be more useful to pull this out and have a general cross platform section and split it into considerations for instructors, and information to pass onto learners, which would answer both the "what is POSIX, why do I care" from instructors, and the "it doesn't work on my system" from learners (the use of --help vs man would also be covered by this).

Aside (this should probably be pushed off to the mailing list, not discussed in a issue): The big question is what do we see the purpose of this lesson being, and how does it fit into a SC course: are we teaching about how systems work (which is functionally what the overall section states is the purpose of the lession); are we teaching how to use "HPC" (systems which provide more resources, whether it's a departmental server, a cluster or the cloud); or is it a lesson we've kept because we teach git and git really only works inside the shell (the third seems like the biggest reason to me).

@kerchner
Copy link

@kerchner kerchner commented Dec 9, 2019

Key differences between zsh and bash to be aware of, as more and more of our learners who use Macs find themselves in Terminals that default to zsh, include configuration - the .bash* files are ignored in favor of .zsh* configuration files (here's one reference about that: https://scriptingosx.com/2019/06/moving-to-zsh-part-2-configuration-files/ ). While it's true that zsh is mostly bash-compatible, we'll want to consider scrubbing the lesson to flag parts that would be materially different for zsh users. Can we consider that task to be the scope of this ticket?

@LHurst-UoB
Copy link

@LHurst-UoB LHurst-UoB commented Dec 11, 2019

I'm not sure that this 'POSIX compliance' ticket is the right place for zsh/bash compatibility - especially as POSIX says sh is "the POSIX shell" so using either bash or zsh automatically makes this course non-compliant ;)

I think at this stage we'd be better agreeing that strict POSIX compliance is a silly aspiration and removing the reference from the contribution guide, resolving the original reason for this issue.

@fthommen
Copy link

@fthommen fthommen commented Dec 11, 2019

I would avoid removing the note about POSIX compliance but would support adding a note about the motivation: "Where possible, try to use commands that work (and do the same thing) on all platforms" (quoting @colinmorris and above).

This would expect the teachers (us) to know (all) other shell/command implementations on (all) other platforms. This is probably not feasible. We would need a very big rosetta stone to keep the overview :-)

In my experience, POSIX compliance is generally not really relevant. Especially on the beginner's level. At least in all environments I have worked so far, POSIX compliance was even completely irrelevant: Users write scripts for their current environment (which is mostly Linux) and there is no requirement whatsoever, that these scripts must work in different (i.e. non-Linux/POSIX) environments. That is not nice from the perspective of compatibility, standardization and interoperability, but it's the reality.

That said I completely agree with @LHurst-UoB that we should remove the POSIX reference from the lesson.

@drj11
Copy link
Member

@drj11 drj11 commented Dec 11, 2019

I've unsubscribed from this issue. Don't at me.

@LHurst-UoB
Copy link

@LHurst-UoB LHurst-UoB commented Dec 11, 2019

In my experience, POSIX compliance is generally not really relevant.

Not only not relevant, often-times undesirable as I mentioned earlier:

we've deviated from POSIX in our coding standards in favour of safer code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants