-
Notifications
You must be signed in to change notification settings - Fork 7.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make ConvertFrom-StringData
more error tolerant
#23790
Comments
It is interesting that you are using an HTTP response as your example, In HTTP protocol you can have multiple entries for the same header which would be incompatible with putting in a hash-table because you can't have duplicate keys.
The http response effectively treats header values as dictionary of list of strings. |
It is just an example. The point is that there are data that mostly conforms to the |
Pragmatically speaking, you could simply filter out lines that aren't of interest along the lines of: curl --silent --head https://www.example.com |
Select-String -Raw : |
ConvertFrom-StringData -Delimiter : However, note that the above creates a hashtable for each and every line (of interest) output by As a result, the above outputs multiple, single-entry hashtables rather than a single hashtable. To create a single hashtable, you'd have to provide all input lines as a single string, which in the simplest case means piping to curl --silent --head https://www.example.com |
Select-String -Raw : |
Out-String |
ConvertFrom-StringData -Delimiter : However, duplicate keys - which, as @rhubarb-geek-nz points out, are a possibility in HTTP headers - then would present a problem - justifiably. However, you did uncover what to me appears to be a genuine bug (since reported in #23793): With multiple, individual strings as input, 'foo=bar', 'broken', 'bar=baz' | ConvertFrom-StringData Currently, the output is:
That is, overall processing was unexpectedly aborted when the The output should be:
That is, processing should have continued after the incorrectly formatted input string (resulting in two hashtables overall). Also, |
Thanks. Please be understanding that I only used curl and HTTP as an example because it is common. If it is causing confusion, I can replace the example with things like output from ffmpeg/ffprobe. Just let me know. The
Yes, I could. But why encourage the users to come up with case-by-case filtering that not only spend CPU time to take out data we don't need to begin with at all, while the proposed update to |
Strictly enforcing data formats improves data quality throughout a system. If you have a relaxed system that accepts anything but ignores it all then you will have trouble later on finding out why none of your values made it though the system. Stick to the "do one thing well" and have ConvertFrom-StringData do the name value conversions and leave filtering to other components that are better suited and can be customised. |
I agree the default behavior should be strict. I'm talking about adding an option or Another example. |
@CrendKing, note that even if the true bug gets fixed (which I've since reported separately, in #23793), it won't help your use case:
So, to spell it out, what you're asking for would require modifying On a related note, the current behavior of emitting a separate hashtable for each input string is awkward, especially when piping an external program's output directly to Both changes discussed are technically breaking ones, though the former is more likely to fall into bucket 3, whereas I think the latter, if it gets implemented, would require opt-in in the form of a new parameter to avoid breaking existing code. |
Thanks. Like you said, my original request is to make the function return non-empty Hashtable from single string input when some line can't be converted. I even showed a workaround that break the single string input into lines, try-catch convert each line and merge all single-entry Hashtables into one. It is doable but very awkward. Also, if this new behavior is protected by a NEW OPTION, for instance |
Well, it is backward-incompatible in the sense that any bug fix is technically a breaking change. That said, even though it would clearly be a bug fix, it has the potential to break existing code, given that using a
Good point, introducing a new switch is a way to avoid backward-compatibility problems, but perhaps the need for that can be avoided if the suggested change is deemed to be a bucket-3 change - but someone would have to do some digging (e.g. analyzing code here on GitHub) to ascertain that. That said, if fixing #23793 is decided against, a new switch is unavoidable.
Yes, though note that your workaround wouldn't work with To avoid a need for |
Summary of the new feature / enhancement
Currently
ConvertFrom-StringData
immediately throws exception if it encounters an invalid data line or duplicate key. Is it a good idea to add an option so thatConvertFrom-StringData
would ignore and skip the erroneous line and continue? This is different to something like-ErrorAction SilentlyContinue
, which skips the whole command.For example, if we use curl to send a HEAD request, we get headers back, which is in the ": " form. However, it also returns the HTTP response code line at the top:
Piping it to
ConvertFrom-StringData -Delimiter ':'
results "ConvertFrom-StringData: Data line 'HTTP/2 200' is not in 'name=value' format." Of course we canSelect-Object -Skip 1
to workaround in this case, but such workaround is not always possible (e.g. invalid data appear in an undetermined line).Implementation wise, just use the option to continue instead of throw at
PowerShell/src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFrom-StringData.cs
Line 68 in 0f3fd20
PowerShell/src/Microsoft.PowerShell.Commands.Utility/commands/utility/ConvertFrom-StringData.cs
Line 78 in 0f3fd20
Proposed technical implementation details (optional)
One can write a function to simulate the feature:
Output:
The text was updated successfully, but these errors were encountered: