Support Experimental Features in PowerShell #114

daxian-dbw · 2018-02-23T19:24:57Z

This RFC is for adding support to Experimental Features in PowerShell.
/cc @dantraMSFT

We went through and discussed all comments in this PR. The discussion has been incorporated into the RFC.

'Get-ExperimentalFeature' returns the available 'ExperimentalFeature' instances. For the experimental features that are enabled, the property 'IsEnabled' should be set to true accordingly.

PaulHigin · 2018-03-06T22:08:31Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+```
+
+When adding a new experimental feature, an instance representing the feature needs to be added to the collection for it to be discoverable to users.
+The `Source` for engine experimental features should be specified as `PSEngine`.


Is this always true? If so then why do we need a settable 'Source' property?

Modules can expose experimental features, and for those, the 'Source' would be the modules names. Therefore, the 'Source' property needs to be settable.

PaulHigin · 2018-03-06T22:10:51Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+
+#### Get-ExperimentalFeature
+
+A new cmdlet `Get-ExperimentalFeature` will be added to return all available experimental features of a PowerShell session.


What do you mean by 'available'? Does this include 'hidden' experimental features? Should there be an option to show 'hidden' experimental features?

Here "available" means all experimental features, enabled or not enabled. The returned ExperimentalFeature type has a IsEnabled property, so you will know which ones are enabled/not enabled.

@PaulHigin - experimental implies that people should experiment with these features hence the need for discoverability. What would be the purpose of a "hidden" experimental feature? (And since it's open source, nothing is really hidden)

I'd expect Enable-ExperimentalFeature and Disable-ExperimentalFeature to change config file.

@BrucePay: There is no concept of a hidden experimental feature. In the referenced statement, available could be removed.

PaulHigin · 2018-03-06T22:12:11Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+For each of them, the `Name` and `Description` are from the metadata. The `Source` will be the module name.
+
+Lastly, the cmdlet can go through not-yet-loaded modules that are in the module path,
+in a similar way as `Get-Command` does,


Would it make more sense to have 'Get-Command -Experimental' ?

When an experimental feature is not enabled, the cmdlets or functions subject to this feature is ignored in the command registration phase, so it doesn't make sense to have Get-Command -Experimental.

Yes, but even if it is not enabled I might be interested to learn what experimental features are possible if I opt in. I assume Get-ExperimentalFeature will show this?

Yes, please see the type definition of ExperimentalFeature at line 357. The type ExperimentalFeature has a property Description that will introduce the experimental feature.

@PaulHigin To be clear, Get-Experimental feature enumerates the experimental features but the return type does not explicitly indicate what cmdlets, parameters, or code is impacted by the feature. In short, we're relying on the feature author to provide an appropriate description.

PaulHigin · 2018-03-06T22:45:24Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+however, PowerShell loads the enabled experimental feature list at very early stage,
+where the engine is not ready for the operations required for resolving dependencies,
+such as module discovery.
+Also, there will be an impact to the startup time if we do this resolution when loading the configuration file.


Since Experimental features are non-GA it seems like a modest start-up impact is not too bad. With the extra module processing it seems likely that command discovery will be slower in general too. I think we would want to ensure the module analysis cache file is updated to include experimental information.

Slowdown startup is not acceptable for GA.

That is what I said. So for non-GA a modest impact should be Ok.

iSazonov · 2018-03-07T04:22:22Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+The Experimental Feature support in PowerShell is to provide a mechanism for experimental features to coexist
+with existing stable features in PowerShell or PowerShell modules.
+An experimental feature would not be enabled
+unless a user opt in by properly configuring the `powershell.config.json` file.


Should we get previously RFC for powershell.config.json and the configuration API?

@iSazonov: Would you rephrase the question? I don't know what you're asking.

I believe @iSazonov is talking about another RFC PowerShell Core configuration.
The implementation of this RFC will inevitably touch PSConfiguration.cs to add APIs for reading the experimental feature settings.

Yes, I believe the RFC depends on PowerShell Core configuration

I wouldn't say it depends on that RFC. The configuration RFC discusses how the configuration settings should be organized, and if approved, the implementation is done separately anyways.

iSazonov · 2018-03-07T04:26:04Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+with existing stable features in PowerShell or PowerShell modules.
+An experimental feature would not be enabled
+unless a user opt in by properly configuring the `powershell.config.json` file.
+Experimental features should not affect any stable features


What if new experimental feature is for replacing current feaure? New file system provider? I see tha sample below so my question is about the phrase only.

The FileSystemProviderV2 would replace the existing FileSystemProvider only if that experimental feature is enabled. When it's not enabled, it should not affect the existing FileSystemProvider.

iSazonov · 2018-03-07T04:31:59Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+During the development,
+the `FileSystemProviderV2` can coexist with the existing one
+and be exposed to the user as an experimental feature,
+so that PowerShell team can get early feedback and issue reports.


I think it would be very useful to see activated experimental features in PSVersionTable so we could see this in Issues.

I'm inclined to not adding this information to PSVersionTable yet. The new Get-ExperimentalFeature cmdlet could have switches -Enabled and -Disabled though.

RFC updated to add -Enabled and -Disabled switches.

Without this we have to ask, for ex., in every file system Issue - Did you enable FileSystemProviderV2?

We can add another environment section in the issue template called Get-ExperimentalFeatures -Enabled.
If there is really a demand, we can add information about experimental features to $PSVertionTable as an enhancement, but I don't think it should be added to the RFC yet.

Rather than modify $PSVersionTable or depend on a this command specific to this feature, I think it may be beneficial to consider a an RFC for a generalized PowerShell support cmdlet that provides common environment details asked for by the PowerShell project and many other open source projects for PowerShell. It would be convenient for us and the community. I envision it as a kind of phpinfo() for PowerShell. Adding the command to the template is a good stop-gap though and if anyone else sees value in such a command for PowerShell I will consider an RFC.

iSazonov · 2018-03-07T04:56:37Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+
+### Experimental Feature Names
+
+Experimental features for PowerShell components and built-in modules should be named as `PSXXX`,


I'd expect EF or EXP or EXPS.

PowerShell team owns the prefix PS, and there have been PowerShell built-in modules whose names follow this pattern, like PSDesiredStateConfiguration, PSDiagnostics, PSScheduledJob and etc. So I think we should use PS here.

I am thinking about easy searching in code sources - what about PSEX or PSEXP?

I'm open to PSEXP.

Had a second thought about the naming. I want to avoid using a string of feature name for existence test in engine code, like if (context.EnabledExpFeatures.Contains("PSEXPBlahFeature")). Directly using a string for checks like this will impact readability.

My original design is to have an internal enum that contains the engine experimental features, and the description of each feature goes in a resource file called ExperimentalFeatureDescription.resx. When initializing the engine, it constructs a ReadOnlyCollection<ExperimentalFeature> out of the enum -- the name of an enum member is the experimental feature's name; read the description from the embedded resource. So when referencing an engine experimental feature in the engine code, you have to use the enum. It would serve the readability very well and also reduce errors like spelling a feature name incorrectly when directly using a string.

But I got feedback that

separating the description to a resource file is heavy given experimental features might not live long and we don't do resource string localization anyways;

having name and description together in the code is more readable.

So I gave up the original design and turn to having an exp-feature author to manually add an instance for the read-only collection when adding a new exp-feature.

Now I again tend to have the enum, or a set of const string variables representing each exp-feature, so that we don't directly use literal strings when referencing a exp-feature in the engine.

iSazonov · 2018-03-07T05:06:58Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+For example:
+
+```none
+ModuleName.ExperimentName


Does ExperimentName contains prefix like PSXXX?

I don't think there's any reason to. The PSXXX are for first-party experiments.

iSazonov · 2018-03-07T09:30:48Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+however, PowerShell loads the enabled experimental feature list at very early stage,
+where the engine is not ready for the operations required for resolving dependencies,
+such as module discovery.
+Also, there will be an impact to the startup time if we do this resolution when loading the configuration file.


Slowdown startup is not acceptable for GA.

iSazonov · 2018-03-07T09:34:38Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+
+A new command-line flag `-settingsFile` was added to `pwsh` to allow
+a user to override the instance configuration file `$PSHOME\powershell.config.json` using another configuration file.
+This flag is very useful in testing the experimental features.


For local testing it would be convenient to have pwsh -Experimental Feature1,Feature2.
For CI testing it would be convenient to have pwsh -Experimental EnableAll.

I don't think EnableAll is viable for a few reasons. The primary reasons being

Not all experimental features are compatible

Some experiments are, by design, mutually exclusive.

The processing of command line parameters doesn't happen early enough (we pointed it out in "Open Issues" section when discussing the problems with -settingsFile). So pwsh -Experimental would have the same problem.

It seems we could fix this.

Refactoring the command-line parsing is not covered in this RFC. It should be a separate issue.

Ready PowerShell/PowerShell#6374

iSazonov · 2018-03-07T09:43:32Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+We need to have test pass for every experimental feature introduced in PowerShell built-in components regularly.
+Code changes that are not part of an experimental feature work may cause regression to the experimental feature.
+If we don't have test pass for every built-in experimental feature,
+the experimental features may be broken without us knowing it.


Experimental feature can be a breaking change - so we'll be not only add new tests but we'll have to skip some regular tests.

Excellent point and one we hadn't considered; disabling a test when an experiment is enabled.

iSazonov · 2018-03-07T09:45:55Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+
+The first proposal will cost way more than the second proposal.
+Since the configuration file overriding functionality is mainly for testing,
+the second proposal may be more efficient cost-wise.


I believe it makes sense to split on early parsing and later appling of command-line parameters.

I tend to agree but it may not be justified here.

iSazonov · 2018-03-07T09:48:52Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+### Whether to have telemetry for experimental feature usage
+
+It may be good to have telemetry for the usage of experimental features.
+This work may fall into the more general telemetry support in PowerShell.


This should contain what features enabled.

With the caveat that this section is no more of a placeholder...
I suspect a usage metric would be more desirable (or both). The fact that an experiment is enabled doesn't mean it's was used.

Authors definitely need some way to get feedback about what features are enabled and how they are being used otherwise what's the point of the experiment. Perhaps we could do extra usage telemetry when experimental features are turned on.

Hmm - maybe we should provide cmdlets to allow the users to provide explicit feedback e.g. Submit-ExperimentalFeatureFeedBack -Feature abc -Vote 5 -Comment "Eh - it's ok".

rjmholt · 2018-03-14T17:32:03Z

2-Draft-Accepted/RFCNNNN-Support-Experimental-Features.md

+2. Validating whether references to an non-existing experimental feature have been removed.
+   At some point, the experiment is completed and either removed from or incorporated into the code base.
+   At this point, a tool to scan script or compiled code would be needed to ensure any references to the experiment have been removed.
+


On the static analysis front, I would be in favor of any design choices (like maybe the enum rather than prefixed strings) that would make it easier to run a tool over code. Of course nothing is future proof, but it would be nice to avoid a "Windows 9" kind of situation.

Add support to experimental features RFC: PowerShell/PowerShell-RFC#114 Goals: Allow experimental features to be declared by PowerShell engine and modules. Allow experimental features to be enabled via powershell.config.json Allow Function, Cmdlet, parameters and parameter sets to be shown to the user or hiden from the user depending on whether the associated experimetnal feature is on or off. Allow discover experimental features using cmdlet Get-ExperimentalFeature

…res.md to 4-Experimental-Accepted/RFC0029-Support-Experimental-Features.md

daxian-dbw and others added 6 commits February 5, 2018 09:50

Finish RFC for supporting experimental features

5059ea2

Minor update

fd8be53

Fix typos

ab3ee15

Incorporate further discussions to the RFC (#1)

1a3d4df

We went through and discussed all comments in this PR. The discussion has been incorporated into the RFC.

Add the property 'IsEnabled' to ExperimentalFeature type (#2)

2a83a4b

'Get-ExperimentalFeature' returns the available 'ExperimentalFeature' instances. For the experimental features that are enabled, the property 'IsEnabled' should be set to true accordingly.

Update the 'Comments Due' date

56ba6a6

SteveL-MSFT added the Review - Open for Comments label Feb 28, 2018

PaulHigin reviewed Mar 6, 2018

View reviewed changes

iSazonov reviewed Mar 7, 2018

View reviewed changes

Address some comments

e7fe22f

rjmholt reviewed Mar 14, 2018

View reviewed changes

daxian-dbw mentioned this pull request Jul 6, 2018

Add support to experimental features PowerShell/PowerShell#7242

Merged

11 tasks

SteveL-MSFT added Review - Committee and removed Review - Open for Comments labels Jul 10, 2018

daxian-dbw and others added 2 commits August 6, 2018 18:20

Update the RFC

6cf3104

Update and rename 2-Draft-Accepted/RFCNNNN-Support-Experimental-Featu…

950be17

…res.md to 4-Experimental-Accepted/RFC0029-Support-Experimental-Features.md

SteveL-MSFT merged commit f99b933 into PowerShell:master Aug 8, 2018

daxian-dbw deleted the experimental branch August 9, 2018 22:51

joeyaiello mentioned this pull request Oct 31, 2018

Docs needed for 'Add support to experimental features' MicrosoftDocs/PowerShell-Docs#3030

Closed

joeyaiello removed the Review - Committee label Mar 4, 2019


		#### Get-ExperimentalFeature

		A new cmdlet `Get-ExperimentalFeature` will be added to return all available experimental features of a PowerShell session.


		### Experimental Feature Names

		Experimental features for PowerShell components and built-in modules should be named as `PSXXX`,

Support Experimental Features in PowerShell #114

Support Experimental Features in PowerShell #114

Conversation

daxian-dbw commented Feb 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daxian-dbw Mar 8, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daxian-dbw Mar 12, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daxian-dbw Mar 12, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dantraMSFT Mar 7, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daxian-dbw Mar 8, 2018 •

edited

daxian-dbw Mar 12, 2018 •

edited

daxian-dbw Mar 12, 2018 •

edited

dantraMSFT Mar 7, 2018 •

edited