Add -EndOfLine character and -DiscardBlankLines parameters to Get-Content

[Liturgist]

Summary of the new feature/enhancement

It would be helpful to have an -EndOfLine character parameter in order to read files which use a character to indicate the start of comments.

I want to read a file containing a list of computernames grouped by department. I want to be able to have comment lines to name the groups as well as omit some servers which are known to be unavailable.

# sales
server1
server2  #owned by Bill Smith
# shipping
#server3  #down for hardware failure
server4

Using this list, the following does not work well.

Test-Connection -Computername (Get-Content -Path './hosts.txt') -Count 1

I realize that I could filter them explicitly.

Test-Connection -Computername (Get-Content -Path '.\hosts.txt' | ForEach-Object { if (($_ -ne '') -and ($_[0] -ne '#')) { $_ }})

I would like to use:

Test-Connection -Computername (Get-Content -Path '.\hosts.txt' -EndOfLine '#' -DiscardBlankLines)

[iSazonov]

Dup #3855

[Liturgist]

The #3855 issue is about "delimiters" between lines. That is not what this is about.

[iSazonov]

EndOfLine assumes "delimiter". If you want "line filter" please correct the PR header and description.

[Liturgist]

EndOfLine does not assume "delimiter." "Delimiter," for the reasons mklement0 has noted in #3855, is problematic. It is probably a terminology choice that is regretted. But, it was made, so there you are.

Using LineFilter would not imply the desired result here.

Perhaps there is a better terminology than EndOfLine, but this matches up and would produce the same result as the eol setting in cmd.exe FOR statements.

[mklement0]

I definitely like the feature, but I agree that "end of line" would cause confusion with "newline" , cmd.exe's for terminology notwithstanding.

Omitting the word comment is not an option, I think, but it's tricky to come up with the right name.

---

Let's look at cmd.exe's for /f behavior first - note that I'm assuming "delims=" to read each line in full, as Get-Content does:

• for /f always, invariably ignores empty lines, but never blank lines (at least 1 char., but all-whitespace)

• If you add eol=<char>:

  • A line is omitted if and only if it immediately starts with <char>

    • Note, however, that without delims= and with either the default behavior or with tokens=..., <char> lines preceded by whitespace only would be ignored.

  • Therefore, any other occurrence of <char> is not special, so that with eol=# server2 #owned by Bill Smith would be read as-is, for instance, as would   # no mas.

In short: eol=<char> in the context of for /f essentially means "ignore lines that start with [comment character] <char>".

---

Given that single-character comment sigils virtually always have consider-everything-through-the-end-of-the-current-line-a-comment semantics, I think we can actually omit that aspect in the naming, and use something like -CommentChar instead.

• To simplify matters, with -CommentChar present, we could invariably skip lines that start with the specified character (optionally preceded by whitespace).

  • That said, separately, a switch for omitting empty and blank lines could be useful, but I'd more simply call it -NoBlankLines (mutually exclusive with -Raw).

• However, the bigger question is whether to strip suffixes from lines that contain the comment characters inside a line (that doesn't start with the comment char.); that is, should a line such as server2 #owned by Bill Smith with -CommentChar '#' be read as just server2?

  • The problematic aspect is that unconditional stripping assumes that a comment char. always, unequivocally represents the start of a comment, which cannot generally assumed, without knowing a given file's specific format; e.g., with a line such as foo = "bar#none" # true comment, the result may be undesired.

[Liturgist]

Excellent analysis @mklement0. I would be good with both -CommentChar and -NoBlankLines.

Could there be a switch -CommentForce that would strip from the CommentChar on? Perhaps there is a better name. Yes, the format of the file must be known and not subject to change. We should not violate the "know your data" maxim.

[Liturgist]

Does this merit further work? What is the process to make such a change?

[iSazonov]

@Liturgist Example in your initial post is not common enough (I mean a standards like csv or tabular data). @mklement0 also pointed out this problem. So I don't see how we could go forward.
I think the best thing we could do is Tabular data
#3692 (comment)
From this point of view, Get-Content is not the most suitable place for parsing and filtering.

[Liturgist]

@iSazonov, perhaps it would be good if this were uncommon. There are many files in the world that are process in this way by cmd.exe FOR loops.

WRT @mklement0's comment about unconditional striping from a -CommentChar, that could be a limitation of the mechanism. I am not sure as to the difficulty of parsing quoted text since I would not call myself a language guy. I would think that it is well-known.

What would doing "tablular data" mean? Are you saying that only tab-delimited files could be used?

[iSazonov]

What would doing "tablular data" mean? Are you saying that only tab-delimited files could be used?

In short mentioned above standard generalizes CSV format so that you can use any delimiters for "fields" and "lines". I believe it solves part of what you ask.

[microsoft-github-policy-service]

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

[microsoft-github-policy-service]

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

[microsoft-github-policy-service]

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

[microsoft-github-policy-service]

📣 Hey @Liturgist, how did we do? We would love to hear your feedback with the link below! 🗣️

🔗 https://forms.office.com/r/P926k48jRJ

Microsoft Forms