New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add parameterset to New-Object
to accept hashtable and emit PSCustomObject
#15848
Comments
Looks to tricky being used broadly. It seems using PS classes is more simple and readable. |
Why not just store the data in a PSCustomObject in the first place? |
My initial thought was to simply use $date = Get-Date
$lottery = 1..49 | Get-Random -Count 7
[pscustomobject]@{Username = $env:username; Drawing= $date;Result = $lottery} |
jdhitsolutions got the idea. Classes are natural to programmers, not ITPro. PowerShell was not designed to be yet another programming language. It was invented to be a practical scripting language with a low entry point for ITPros. They need a simple way to wrap information into objects. Classes are way out of scope to this audience. That said, I am more than open and curious for other approaches as long as they do the job. Can you (iSazonov) elaborate how you think classes could help here, and how the suggested approach is trickier than using classes? Please consider that we are not in a situation where we own software projects and can design classes and constructors. In the intended target group you have to deal with raw information from a number of sources that are beyond own control, and need a way to wrap them into objects without knowing more than the most basic programming principles. |
PowerShell already support using and creating classes out of the box. |
That's a great comment. Most advanced scripters do. But as code review shows, even advanced programmers frequently resort to inappropriate ways like using Basically, anything can be done in PowerShell via direct .NET and types. The ease of use of PowerShell however comes from generic cmdlets which by design shield .NET from the user. So one could argue that we could get rid of all cmdlets since the same can always be done using underlying .NET ;-). So I guess it's a matter of evaluating whether a given cmdlet is simple and reusable enough to be of value, not whether it is unique in what it does. |
[PSCustomObjects] isn't very hard. |
I guess it is a question of how deep you want users to dive into programming. I like to view PowerShell as a video game with different levels. In level 1 you learn how to discover and run individual commands, and in level 2 you learn how to couple multiple commands using pipeline and variables. These two levels are where most ITPro live that I encounter in the IT departments I visit every week, so to help them, new tools need to be in these levels. Level 3 would involve hashtables and types, so yes you can argue that users should be at least educated to level 3 to make use of structured output. However my experience is that level 3 is a barrier for many because it also marks the transition from batch scripters to programmers. I am not hurt in any way if it turned out I am the only one thinking this way ;-). I do have the feeling though that PowerShell is moving away from its original user base, and I suspect this is because most people actively involved in its development seem to be developers or experienced scripters these days. |
I agree that there should be more options for transforming input into objects. I do worry that this particular design would lead to a lot of enumeration related confusion, especially for the type of folks this would be targeting. Either way since this can be done at the command level it would be good to prove the concept with a gallery module like was done with |
This request has quiet some similarities with $env:username |Join $date |Join (,$Lottery) -Name UserName, Drawing, Result
UserName Drawing Result
-------- ------- ------
user 7/30/2021 8:13:03 PM {2, 30, 4, 5…} Note the unary comma for the $env:username |FullJoin @(1..7) |Join $Lottery -Name UserName, Index, Drawing
UserName Index Drawing
-------- ----- -------
user 1 2
2 30
3 4
4 5
5 26
6 12
7 32 What they all (including the linked cases) have in common, is the search for a syntactic way to transpose data: move rows to columns (objects to properties) and/or vice versa. |
I use Powershell daily, both in complex scripts/automation and in one-line commands.....and everywhere in between. Personally, I think this is amazing. There have been a ton of times where I needed to put together some data that would only be used once or twice (making writing some advanced script take more time than just doing it all by hand), as well as converting objects into a decent output to be consumed by another cmdlet or even just dumped out to a file. Both of the functions in this suggestion would definitely be time-saving for ITPros at all levels. Those of us that taught ourselves how to use classes and complexity in Powershell, as well as those who have no idea how to do more than a single one-liner. |
I really doesn't see the issue.
|
Still more complex than what was suggested. What would be the downside to including it as an enhancement? |
@TobiasPSP I 100% agree, I love the complexity that has been added over the years, because Powershell can now do so much more than when I first touched it way back....but if something like this sort of suggestion is scoffed at... I was actually relieved when I saw the simplicity of this suggestion. Yes, we can create custom objects, or use the code in the function itself if we wanted to. But I've found myself trying to combine different arrays or hashtables into a single object (usually for a report or some such) before, and it always involves spending time coding/testing/retesting/etc. I know many ops people that would love to have this so they can get those reports for the audit guys breathing down their necks. Just my 2 cents. Not worth terribly much, of course. |
Ok, now I see the issue. But in a wider perspective. The example provided isn't very complicated if hitting [pscustomobject] to begin with. To wrap my head around the actual data conversion I tried out a oneliner.
|
I like this idea and I think it would be great for all experience levels. Having a cmdlet for it means it's more discoverable and more natural to use, especially in oneliners. The naming is good from a discoverability perspective as it will be easy to find on your favorite search engine. It does feel "wrong" when looking at the current cmdlets and their naming as it doesn't follow the verb-noun style. How about naming it function ConvertTo-Object
{
param
(
[Object[]]
[Parameter(Mandatory,ValueFromPipeline, ParameterSetName="Array")]
$InputArray,
[String[]]
[Parameter(Mandatory, ParameterSetName="Array")]
$PropertyName,
[Switch]
[Parameter(ParameterSetName="Array")]
$DiscardRemaining,
[Switch]
[Parameter(ParameterSetName="Array")]
$AsHashtable,
[HashTable]
[Parameter(Mandatory,ValueFromPipeline, ParameterSetName="HashTable")]
$InputHashTable,
[System.Collections.Specialized.OrderedDictionary]
[Parameter(Mandatory,ValueFromPipeline, ParameterSetName="OrderedHashTable")]
$InputOrdered,
[String[]]
[Parameter(Mandatory=$false, ParameterSetName="OrderedHashTable")]
$PropertyName,
)
...
} I don't know if that design is technically feasible but it should get the idea across at least. The PropertyName parameter for OrderedDictionary would be to allow renaming properties. I don't think it would be sensible to have that for a HashTable as it would be unpredictable and confusing. A function ConvertFrom-Object
{
param
(
[Object]
[Parameter(Mandatory,ValueFromPipeline)]
$InputObject,
[Switch]
[Parameter(ParameterSetName="HashTable")]
$AsHashtable,
[Switch]
[Parameter(ParameterSetName="Array")]
$AsArray
)
...
} |
Thank you all so much for sharing thoughts. I am trying to sum a number of things up. Let me know if I missed anything. ProposalAdd one or more cmdlets to expose object manipulation on discoverable cmdlet level rather than having to use .NET, type conversion, or classes:
Why is that needed?Occasional scripters work on cmdlet level and typically have no deep knowledge about types, hashtables, let alone classes. Even experienced scripters seem to often use There is currently no intuitive self-learning path to go beyond All other techniques for object creation - however simple they may be - require extra knowledge, i.e. about hashtables or classes, which is an extra barrier for novice users still trying to cope with cmdlets and parameters. Experienced users can benefit from these cmdlets, too, as their feature set spans a very powerful range and allows for advanced object manipulation (i.e. sorting properties, removing empty values, merging objects). Is this functionality really needed?Anything the proposed cmdlets do can be done via direct .NET access (type conversion, classes). So once you know how to create new objects via [PSCustomObject] or via classes, it is not much more complicated than using the proposed cmdlets. Yet this proposal was not about teaching PowerShell new tricks in the first place. It is a proposal to lower the barrier for entry-level or occational scripters, and provide generic solutions for common object manipulation tasks. This approach is in good tradition with existing cmdlet design. Cmdlets often exist just to be discoverable or shield the user from .NET. Compare And still there is consensus that it is a great thing that How expensive is it?Implenting the proposed cmdlets is trivial from an engineering point of view. There are no dependencies nor would the package grow. All proof-of-concept functions in this thread are short functions which essentially just wrap a discoverable user interface around the existing PowerShell capabilities. Next stepsI'd suggest to collect the functionality commonly needed to convert unstructured data to structured data and vice versa. Once we know what the feature set should be, we can start composing a cmdlet family and think about naming. PrerequisitesA prototype module can be developed and hosted on github and the gallery so everyone was invited to add ideas or participate in dev or testing. Prereq would be that we get a general thumbs up from the PowerShell working group that ultimately this module can become part of the shipping modules. The proposed cmdlets make no sense if they stay put in a 3rd party module since the target group isn't familiar with the gallery and shouldn't be discriminated in a way that their scripts would only run with a 3rd party helper module being installed. |
@PowerShell/wg-powershell-cmdlets discussed this and agreed with the scenario of making custom object creation easier for new users particularly from a discovery point of view. We propose a change to |
New-Object
to accept hashtable and emit PSCustomObject
Pardon my ignorance but... doesn't that already exist @SteveL-MSFT?
|
It does also support New-Object psobject -Property @{
PSTypeName = 'CountingInfo'
A = 1
B = 2
C = 3
} Though if you need property order to be retained you do need |
You are absolutely right, the original issue title is a bit misleading, and initially this was about a new cmdlet family for a broad range of data-to-object manipulation (and vice versa). In the course of debate here, a number of low hanging fruit were identified that could easily be integrated into the existing New-Object cmdlet. |
Maybe open new issue with clear proposal? |
Thanks for the clarification @TobiasPSP. I think it makes perfect sense to
|
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you. |
2 similar comments
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you. |
This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you. |
This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes. |
Summary of the new feature / enhancement
PowerShell is a object-oriented shell. Programmers are familiar with this concept but admins and occasional scripters often are not. This audience primarily uses simple data types and arrays vs. objects.
By providing a new cmdlet
Convert-ObjectToArray
, the complexity of generating own objects would be significantly reduced. For example, a user could calculate a number of results, then "wrap" them easily into an object and return structured info from own functions:Result:
Experienced scripters would benefit as well. Often system calls return unstructured array information. With
Convert-ArrayToObject
, it would intuitively be possible to impose a structure on that data.The following example queries the eventlog for the newest five installed updates. The actual update names and other event information is stored inside "Properties" as a simple array.
By using
Convert-ArrayToObject
, it would be trivial to impose a meaningful object structure to this array and then emit the result as object:Result:
Proposed technical implementation details (optional)
Convert-ArrayToObject
is no rocket sience but it uses technology not commonly known by the admin audience:It takes an array and a list of property names and turns the array into an object:
When there are not enough property names, all remaining items go into the last property (similar to how variable assignments work):
The switch
-DiscardRemaining
in contrast omits overflow items:On request, the switch
AsHashtable
returns the composed hashtable without converting it to an object:Note
This cmdlet could obviously be created and maintained in the PowerShell Gallery.
However it is just a generic and useful item like i.e. the new ternary operator. It makes sense only when it is part of the shipping language set and does not add additional module dependencies.
The function does not add dependencies to PowerShell and won't enlarge the package. It simply shields the complexity of object generation and hashtables from an entry-level audience and provides experienced scripters with a convenient way of performing every-day tasks.
During code reviews, it turned out that even experienced users often use
Add-Member
to compose objects.Convert-ArrayToObject
could provide a more appropriate alternative in most of these cases as well.Further Thoughts
Ideally,
Convert-ArrayToObject
would be accompanied by a correspondingConvert-ObjectToHashtable
in an effort to better support the conversion of simple data to objects and vice versa:The use case of the latter function would be a plethora of object manipulation, i.e. the exclusion of empty properties, sorting object properties, and displaying object properties line by line:
For reporting purposes, this could be used to sanitize objects: exclude empty properties, sort property names and work around the issue that CSV and excel do not support arrays by automatically flatten arrays, getting rid of those properties that always contain just type names.
-ReturnAsObject
would then return the sanitized hashtable as object, ready to be sent toExport-Csv
orExport-Excel
:This example creates an excel report and correctly shows dependent services that usually would not show in a readable way. To run this example, you need to install the module ImportExcel or replace
Export-Excel
by a default cmdlet such asExport-Csv
:Caveat: removing empty properties via
-ExcludeEmpty
is safe only when processing a single object which is why the previous example isn't removing empty properties. In the context of a single object, though, automatically removing empty properties can be very useful, i.e. to focus on ActiveDirectory attributes for a particular user that are actually set:If you process multiple input objects, removing empty properties could lead to objects with less hashtable keys than others. While this isn't bad a priori, when outputting these objects, PowerShell would show only the properties present in the first object it encounters (as always).
The text was updated successfully, but these errors were encountered: