New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce a more descriptive parameter alias for Get-Content -Raw #7715

Open
mklement0 opened this Issue Sep 5, 2018 · 19 comments

Comments

Projects
None yet
6 participants
@mklement0
Contributor

mklement0 commented Sep 5, 2018

The filesystem provider's -Raw switch for Get-Content is confusingly named:

  • raw suggests reading raw bytes, which is not what it does.

  • conversely, raw doesn't convey that the file is being read as a whole.

Without breaking backward compatibility, a more sensibly named parameter alias could be defined, such as -Whole.

Environment data

Written as of:

PowerShell Core 6.1.0-preview.4
@iSazonov

This comment has been minimized.

Show comment
Hide comment
@iSazonov

iSazonov Sep 6, 2018

Collaborator

The parameter is not documented well :-)

Collaborator

iSazonov commented Sep 6, 2018

The parameter is not documented well :-)

@MartinSGill

This comment has been minimized.

Show comment
Hide comment
@MartinSGill

MartinSGill Sep 6, 2018

I, personally, don't have a problem with it. I know Get-Content always returns an array of strings (at least for text files), so 'raw' to me means just the raw text, without any splitting.

MartinSGill commented Sep 6, 2018

I, personally, don't have a problem with it. I know Get-Content always returns an array of strings (at least for text files), so 'raw' to me means just the raw text, without any splitting.

@mklement0

This comment has been minimized.

Show comment
Hide comment
@mklement0

mklement0 Sep 6, 2018

Contributor

@iSazonov : Good point, but that's a separate issue:

The parameter description in the context of the filesystem-provider-specific help topic is passable, but can be improved: PowerShell/PowerShell-Docs#2697

In the provider-independent help topic - the one that's discoverable far more easily - the generic description is downright misleading:

This parameter is not supported by any providers that are installed with PowerShell.

However, that, and the difficulty in locating the provider-specific help topics, are fundamental, structural problems not specific to -Raw that will go away (hopefully soon), according to PowerShell/PowerShell-Docs#1101 (comment):

Yeah, the provider-specific help is a work-in-progress. Long story but we are working on it. The goal is to wrap all of that content into the provider-affected cmdlets and get rid of the provider-specific docs.

Contributor

mklement0 commented Sep 6, 2018

@iSazonov : Good point, but that's a separate issue:

The parameter description in the context of the filesystem-provider-specific help topic is passable, but can be improved: PowerShell/PowerShell-Docs#2697

In the provider-independent help topic - the one that's discoverable far more easily - the generic description is downright misleading:

This parameter is not supported by any providers that are installed with PowerShell.

However, that, and the difficulty in locating the provider-specific help topics, are fundamental, structural problems not specific to -Raw that will go away (hopefully soon), according to PowerShell/PowerShell-Docs#1101 (comment):

Yeah, the provider-specific help is a work-in-progress. Long story but we are working on it. The goal is to wrap all of that content into the provider-affected cmdlets and get rid of the provider-specific docs.

@mklement0

This comment has been minimized.

Show comment
Hide comment
@mklement0

mklement0 Sep 6, 2018

Contributor

@MartinSGill:

The typical connotation of raw in the context of files is uninterpreted (undecoded) data ("raw bytes").
(As such, you could argue that -Raw should be an alias of -AsByteStream, which is the closest thing PowerShell has to passing bytes through.)

By contrast, -Raw does perform interpretation / decoding, namely decoding the file's bytes to strings based on the file's assumed or explicitly specified character encoding, just as without -Raw.

The only thing that -Raw changes is the partitioning of the data read, so that is what the switch name should reflect, hence my suggestion -Whole

Contributor

mklement0 commented Sep 6, 2018

@MartinSGill:

The typical connotation of raw in the context of files is uninterpreted (undecoded) data ("raw bytes").
(As such, you could argue that -Raw should be an alias of -AsByteStream, which is the closest thing PowerShell has to passing bytes through.)

By contrast, -Raw does perform interpretation / decoding, namely decoding the file's bytes to strings based on the file's assumed or explicitly specified character encoding, just as without -Raw.

The only thing that -Raw changes is the partitioning of the data read, so that is what the switch name should reflect, hence my suggestion -Whole

@BrucePay

This comment has been minimized.

Show comment
Hide comment
@BrucePay

BrucePay Sep 6, 2018

Member

@mklement0 "Raw" in this context means, "undecorated", "not cooked", etc. This is the parameter that the PowerShell team has chosen for this purpose. We could have chosen a different parameter but we did not. Some people will complain that it doesn't mean the right thing to them. That's fine. No matter what we choose, this will always be the case because different people have different perspectives. In PowerShell we try to choose a single term and apply it consistently so that, even if it does not seem intuitive to a person, they only have to learn it once. Sometimes we choose a sub-optimal term but we live with it because you should only have to learn something once. Very rarely, we try and "fix" things like we did with $_ and $PSItem. This sounded good because new users didn't "get" $_. So we added $PSItem as being easier to understand. The feedback we received from people who teach PowerShell is that rather than helping, having to teach two terms made it worse/more confusing for people trying to learn PowerShell.

Member

BrucePay commented Sep 6, 2018

@mklement0 "Raw" in this context means, "undecorated", "not cooked", etc. This is the parameter that the PowerShell team has chosen for this purpose. We could have chosen a different parameter but we did not. Some people will complain that it doesn't mean the right thing to them. That's fine. No matter what we choose, this will always be the case because different people have different perspectives. In PowerShell we try to choose a single term and apply it consistently so that, even if it does not seem intuitive to a person, they only have to learn it once. Sometimes we choose a sub-optimal term but we live with it because you should only have to learn something once. Very rarely, we try and "fix" things like we did with $_ and $PSItem. This sounded good because new users didn't "get" $_. So we added $PSItem as being easier to understand. The feedback we received from people who teach PowerShell is that rather than helping, having to teach two terms made it worse/more confusing for people trying to learn PowerShell.

@mklement0

This comment has been minimized.

Show comment
Hide comment
@mklement0

mklement0 Sep 6, 2018

Contributor

@BrucePay:

"Raw" in this context means, "undecorated", "not cooked", etc. This is the parameter that the PowerShell team has chosen for this purpose.

That would be fine, except, as previously stated, that is not what -Raw actually does:

  • It still decorates its (one) output string.
  • All it does is change the units of data being read from lines (by default) to the whole file.

In other words:

  • The reality of the switch does not align with the stated purpose and this regrettable precedent prevents meaningful use of -Raw in other contexts in alignment with its true purpose.

  • One way of dealing with a misnomer is to provide a more sensibly named alias and to deprecate / deemphasize the old name.

    • In other words: The intent is for the new name to in effect supersede the old one, so the $_ / $PSItem comparison doesn't apply (given that neither is deprecated).

    • Yes, there will be confusion for a while, but over time the sensible name will (hopefully) prevail.

Contributor

mklement0 commented Sep 6, 2018

@BrucePay:

"Raw" in this context means, "undecorated", "not cooked", etc. This is the parameter that the PowerShell team has chosen for this purpose.

That would be fine, except, as previously stated, that is not what -Raw actually does:

  • It still decorates its (one) output string.
  • All it does is change the units of data being read from lines (by default) to the whole file.

In other words:

  • The reality of the switch does not align with the stated purpose and this regrettable precedent prevents meaningful use of -Raw in other contexts in alignment with its true purpose.

  • One way of dealing with a misnomer is to provide a more sensibly named alias and to deprecate / deemphasize the old name.

    • In other words: The intent is for the new name to in effect supersede the old one, so the $_ / $PSItem comparison doesn't apply (given that neither is deprecated).

    • Yes, there will be confusion for a while, but over time the sensible name will (hopefully) prevail.

@BrucePay

This comment has been minimized.

Show comment
Hide comment
@BrucePay

BrucePay Sep 6, 2018

Member

The reality of the switch does not align with the stated purpose and this regrettable precedent prevents meaningful use of -Raw in other contexts in alignment with its true purpose.

As I said, people will have different opinions on how whatever term we choose should be interpreted. Breaking the text into strings can be viewed as "cooking" and thus -Raw should return a single string.

One way of dealing with a misnomer is to provide a more sensibly named alias and to deprecated / deemphasize the old name.

We've never actually deprecated anything and aren't likely to do so anytime soon.

Member

BrucePay commented Sep 6, 2018

The reality of the switch does not align with the stated purpose and this regrettable precedent prevents meaningful use of -Raw in other contexts in alignment with its true purpose.

As I said, people will have different opinions on how whatever term we choose should be interpreted. Breaking the text into strings can be viewed as "cooking" and thus -Raw should return a single string.

One way of dealing with a misnomer is to provide a more sensibly named alias and to deprecated / deemphasize the old name.

We've never actually deprecated anything and aren't likely to do so anytime soon.

@mklement0

This comment has been minimized.

Show comment
Hide comment
@mklement0

mklement0 Sep 7, 2018

Contributor

We've never actually deprecated anything and aren't likely to do so anytime soon.

That genie is already out of the bottle:

-RootModule
[...]
In Windows PowerShell 2.0, this key was called ModuleToProcess.


As I said, people will have different opinions on how whatever term we choose should be interpreted.

You yourself noticed the problem with -Raw int the context of #7537: #7537 (comment)

We added -Raw a long while back to address the perf issue but it doesn't really do the right thing.

Since this discussion has now broadened in scope, let me reiterate and expand on what I proposed as an aside in #7713:

  • Retire the skunked -Raw, as proposed by this issue.

  • Introduce a general-purpose -Bare switch with the following, cross-cmdlet semantics:

Output "bare" objects that are:

  • either: undecorated (have no ETS properties added to them, as added by Get-Content, for instance (#7537) and as a possible solution to #5797)
  • or: not wrapped in helper instances of another type (such as matching text lines getting wrapped in MatchInfo instances by Select-String (#7713))
Contributor

mklement0 commented Sep 7, 2018

We've never actually deprecated anything and aren't likely to do so anytime soon.

That genie is already out of the bottle:

-RootModule
[...]
In Windows PowerShell 2.0, this key was called ModuleToProcess.


As I said, people will have different opinions on how whatever term we choose should be interpreted.

You yourself noticed the problem with -Raw int the context of #7537: #7537 (comment)

We added -Raw a long while back to address the perf issue but it doesn't really do the right thing.

Since this discussion has now broadened in scope, let me reiterate and expand on what I proposed as an aside in #7713:

  • Retire the skunked -Raw, as proposed by this issue.

  • Introduce a general-purpose -Bare switch with the following, cross-cmdlet semantics:

Output "bare" objects that are:

  • either: undecorated (have no ETS properties added to them, as added by Get-Content, for instance (#7537) and as a possible solution to #5797)
  • or: not wrapped in helper instances of another type (such as matching text lines getting wrapped in MatchInfo instances by Select-String (#7713))
@iSazonov

This comment has been minimized.

Show comment
Hide comment
@iSazonov

iSazonov Sep 7, 2018

Collaborator

We have too many such discussions without a conclusion.
I think PowerShell Committee can consider this - should we replace -Raw with -Whole (as alias) or Won't fix?

Collaborator

iSazonov commented Sep 7, 2018

We have too many such discussions without a conclusion.
I think PowerShell Committee can consider this - should we replace -Raw with -Whole (as alias) or Won't fix?

@mklement0

This comment has been minimized.

Show comment
Hide comment
@mklement0

mklement0 Sep 7, 2018

Contributor

@iSazonov: To be clear: -Bare is not the right (alias) name for the current -Raw switch. As proposed in the OP, something like -Whole would make sense.

Deprecating -Raw in favor of something more descriptive such as -Whole would clear the way for use of -Bare with the semantics described above, and -Bare would then become a new switch in the context of Get-Content, for requesting undecorated lines (strings without ETS properties), as suggested in #7537

Contributor

mklement0 commented Sep 7, 2018

@iSazonov: To be clear: -Bare is not the right (alias) name for the current -Raw switch. As proposed in the OP, something like -Whole would make sense.

Deprecating -Raw in favor of something more descriptive such as -Whole would clear the way for use of -Bare with the semantics described above, and -Bare would then become a new switch in the context of Get-Content, for requesting undecorated lines (strings without ETS properties), as suggested in #7537

@BrucePay

This comment has been minimized.

Show comment
Hide comment
@BrucePay

BrucePay Sep 7, 2018

Member

@mklement0

That genie is already out of the bottle:

-RootModule
[...]
In Windows PowerShell 2.0, this key was called ModuleToProcess.

And yet it's still there in version 6, both as a parameter alias and mentioned in the .psd1 file:

# Modules to import as nested modules of the module specified in RootModule/ModuleToProcess
# NestedModules = @()

Deprecated implies "don't use, will be removed at some future time". We don't do this. What we do is deemphasize things.

In this specific case, "RootModule" is semantically much stronger than "ModuleToProcess". There is a clear semantic advantage in using one term over the other whereas -Base and -Raw - are almost synonyms (at least in this context).

You yourself noticed the problem with -Raw int the

Yes :-) I had intended/expected that -Raw should write a stream of undecorated strings into the pipeline however the developer who actually implemented it chose a different interpretation.

Deprecating -Raw in favor of something more descriptive such as -Whole would clear the way for use of -Bare with the semantics described above, and -Bare

Again, -Raw isn't going away so you'd have a cmdlet with both -Raw and -Bare which is pretty confusing (unless you make one an alias of the other in which case the semantic doesn't change.). Also the current -Raw semantic can be "fixed" by adding -Stream as in Out-String -Stream so you'd do gc -raw -stream foo.txt.

@iSazonov The committee doesn't need to be involved in a parameter change but if we do want to propose a new normative pattern, it should go through the RFC process.

Member

BrucePay commented Sep 7, 2018

@mklement0

That genie is already out of the bottle:

-RootModule
[...]
In Windows PowerShell 2.0, this key was called ModuleToProcess.

And yet it's still there in version 6, both as a parameter alias and mentioned in the .psd1 file:

# Modules to import as nested modules of the module specified in RootModule/ModuleToProcess
# NestedModules = @()

Deprecated implies "don't use, will be removed at some future time". We don't do this. What we do is deemphasize things.

In this specific case, "RootModule" is semantically much stronger than "ModuleToProcess". There is a clear semantic advantage in using one term over the other whereas -Base and -Raw - are almost synonyms (at least in this context).

You yourself noticed the problem with -Raw int the

Yes :-) I had intended/expected that -Raw should write a stream of undecorated strings into the pipeline however the developer who actually implemented it chose a different interpretation.

Deprecating -Raw in favor of something more descriptive such as -Whole would clear the way for use of -Bare with the semantics described above, and -Bare

Again, -Raw isn't going away so you'd have a cmdlet with both -Raw and -Bare which is pretty confusing (unless you make one an alias of the other in which case the semantic doesn't change.). Also the current -Raw semantic can be "fixed" by adding -Stream as in Out-String -Stream so you'd do gc -raw -stream foo.txt.

@iSazonov The committee doesn't need to be involved in a parameter change but if we do want to propose a new normative pattern, it should go through the RFC process.

@vexx32

This comment has been minimized.

Show comment
Hide comment
@vexx32

vexx32 Sep 7, 2018

I would be in favour of leaving -Raw as an alias and changing the name to -Bare

I'd potentially be in favour of using -Bytes as a switch to specify to retrieve the bytes in the file per line, which could operate hand in hand with a -Whole switch to read the bytes of the entire file (or just flat out always read all bytes in the file indiscriminately).

vexx32 commented Sep 7, 2018

I would be in favour of leaving -Raw as an alias and changing the name to -Bare

I'd potentially be in favour of using -Bytes as a switch to specify to retrieve the bytes in the file per line, which could operate hand in hand with a -Whole switch to read the bytes of the entire file (or just flat out always read all bytes in the file indiscriminately).

@iSazonov

This comment has been minimized.

Show comment
Hide comment
@iSazonov

iSazonov Sep 8, 2018

Collaborator

@iSazonov The committee doesn't need to be involved in a parameter change but if we do want to propose a new normative pattern, it should go through the RFC process.

@BrucePay We have dozens of useful discussions without a conclusion. As a result, these discussions remain useless. This conclusion can only be made by experts like you. And they are all on the PowerShell committee :-)

Collaborator

iSazonov commented Sep 8, 2018

@iSazonov The committee doesn't need to be involved in a parameter change but if we do want to propose a new normative pattern, it should go through the RFC process.

@BrucePay We have dozens of useful discussions without a conclusion. As a result, these discussions remain useless. This conclusion can only be made by experts like you. And they are all on the PowerShell committee :-)

@mklement0

This comment has been minimized.

Show comment
Hide comment
@mklement0

mklement0 Sep 9, 2018

Contributor

@BrucePay:

We don't do this. What we do is deemphasize things.

Understood - we were just using different words: Deemphasizing was exactly what I was proposing in this case, which, in addition to documenting the better name prominently, would also require making -Raw the alias and -Bare the new parameter name proper, so that the syntax diagrams show the new name.

-Base and -Raw - are almost synonyms (at least in this context).

The advantage here lies not in finding a more descriptive name (though I do think that that -Bare is preferable), but to allow implementing the originally intended functionality without breaking backward compatibility:

That is, -Raw, and -Bare can now coexist:

  • -Raw solely to preserve backward compatibility.

  • -Bare to now implement what -Raw was initially meant to implement, but didn't:

    • output an array of undecorated lines
    • or, if -Whole (the artist formerly known as -Raw) is also specified, a single undecorated string.

And since -Bare hasn't been used before, we're now free to use it as a general pattern across all cmdlets where undecorated / not-wrapped-in-a-helper-type output is desired.

As for candidate cmdlets:

  • -Bare to effect undecorated output:

    • Get-Command, as discussed here
    • As a potential solution for the ConvertTo-Json problem described in #5797
  • -Bare to omit helper wrapper types:

    • Select-String, as discussed in #7713
    • Compare-Object is another candidate (omitting the [pscustomobject] wrappers (which -PassThru already does and passing the input objects through undecorated.

And I wouldn't be surprised if other cmdlets are candidates too (I haven't performed a systematic analysis).

Contributor

mklement0 commented Sep 9, 2018

@BrucePay:

We don't do this. What we do is deemphasize things.

Understood - we were just using different words: Deemphasizing was exactly what I was proposing in this case, which, in addition to documenting the better name prominently, would also require making -Raw the alias and -Bare the new parameter name proper, so that the syntax diagrams show the new name.

-Base and -Raw - are almost synonyms (at least in this context).

The advantage here lies not in finding a more descriptive name (though I do think that that -Bare is preferable), but to allow implementing the originally intended functionality without breaking backward compatibility:

That is, -Raw, and -Bare can now coexist:

  • -Raw solely to preserve backward compatibility.

  • -Bare to now implement what -Raw was initially meant to implement, but didn't:

    • output an array of undecorated lines
    • or, if -Whole (the artist formerly known as -Raw) is also specified, a single undecorated string.

And since -Bare hasn't been used before, we're now free to use it as a general pattern across all cmdlets where undecorated / not-wrapped-in-a-helper-type output is desired.

As for candidate cmdlets:

  • -Bare to effect undecorated output:

    • Get-Command, as discussed here
    • As a potential solution for the ConvertTo-Json problem described in #5797
  • -Bare to omit helper wrapper types:

    • Select-String, as discussed in #7713
    • Compare-Object is another candidate (omitting the [pscustomobject] wrappers (which -PassThru already does and passing the input objects through undecorated.

And I wouldn't be surprised if other cmdlets are candidates too (I haven't performed a systematic analysis).

@mklement0

This comment has been minimized.

Show comment
Hide comment
@mklement0

mklement0 Sep 9, 2018

Contributor

@vexx32:

We already have -AsByteStream (which used to be -Encoding Byte and still is in Windows PowerShell), and by default it already does read the entire file (-Whole is implied, if you will).

While you can't read bytes line by line, doing so is probably not the typical use case for reading bytes.

You can, however, specify how many bytes to read with -TotalCount and/or partition bytes into sub-arrays with -ReadCount.

Contributor

mklement0 commented Sep 9, 2018

@vexx32:

We already have -AsByteStream (which used to be -Encoding Byte and still is in Windows PowerShell), and by default it already does read the entire file (-Whole is implied, if you will).

While you can't read bytes line by line, doing so is probably not the typical use case for reading bytes.

You can, however, specify how many bytes to read with -TotalCount and/or partition bytes into sub-arrays with -ReadCount.

@SteveL-MSFT

This comment has been minimized.

Show comment
Hide comment
@SteveL-MSFT

SteveL-MSFT Sep 19, 2018

Member

@PowerShell/powershell-committee reviewed this. The current use of -Raw is acceptable and therefore no reason to make the proposed change. We would support a proposal to add a type parameter for streaming line-by-line w/o annotations although we did not come to agreement on the naming. -Bare is not different enough from -Raw to communicate functional differences.

Member

SteveL-MSFT commented Sep 19, 2018

@PowerShell/powershell-committee reviewed this. The current use of -Raw is acceptable and therefore no reason to make the proposed change. We would support a proposal to add a type parameter for streaming line-by-line w/o annotations although we did not come to agreement on the naming. -Bare is not different enough from -Raw to communicate functional differences.

@mklement0

This comment has been minimized.

Show comment
Hide comment
@mklement0

mklement0 Sep 19, 2018

Contributor

@SteveL-MSFT

-Bare is not different enough from -Raw to communicate functional differences.

Agreed - but that's precisely why I suggested introducing -Whole and de-emphasizing -Raw.

-Bare wouldn't just benefit Get-Content, but, as I've proposed above, could become a general pattern for all cmdlets where undecorated / unwrapped output is desirable on an opt-in basis.

Contributor

mklement0 commented Sep 19, 2018

@SteveL-MSFT

-Bare is not different enough from -Raw to communicate functional differences.

Agreed - but that's precisely why I suggested introducing -Whole and de-emphasizing -Raw.

-Bare wouldn't just benefit Get-Content, but, as I've proposed above, could become a general pattern for all cmdlets where undecorated / unwrapped output is desirable on an opt-in basis.

@iSazonov

This comment has been minimized.

Show comment
Hide comment
@iSazonov

iSazonov Sep 20, 2018

Collaborator

@mklement0 It make sense to move the Bare proposal in new issue. It would be a new common parameter.

Collaborator

iSazonov commented Sep 20, 2018

@mklement0 It make sense to move the Bare proposal in new issue. It would be a new common parameter.

@mklement0

This comment has been minimized.

Show comment
Hide comment
@mklement0

mklement0 Sep 24, 2018

Contributor

Good idea, @iSazonov - please see #7855

Contributor

mklement0 commented Sep 24, 2018

Good idea, @iSazonov - please see #7855

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment