-
-
Notifications
You must be signed in to change notification settings - Fork 38
Interesting Techniques
Often the values reported by PRTG need to be transformed in some way during or after the deserialization process, such as an empty string being converted into null or a DateTime/TimeSpan being correctly parsed. As the System.Xml XmlSerializer requires that all target properties be public, this presents a variety of issues, requiring "dummy" properties that accept the raw deserialized output and then are correctly parsed via the getter of the "actual" property. Such "raw" properties make a mess of your object's interface, bloat your intellisense and mess up your PowerShell output.
PrtgAPI works around this by implementing its own custom XmlSerializer. Unlike the built in XmlSerializer which generates a dynamic assembly for highly efficient deserialization, PrtgAPI relies upon reflection to iterate over each object property and bind each XML value based the value of each type/property'sXmlElement, XmlAttribute and XmlEnum attributes. This allows PrtgAPI to bind raw values to protected members that are then parsed by the getters of their public counterparts. The PrtgAPI XmlSerializer also has the sense to eliminate a number of common pain points, such as converting empty strings to null.
To improve the performance of deserializing objects, PrtgAPI implements a reflection cache, storing properties, fields and enums of deserialized types. While deserialization performance is not usually noticeable in normal operation of PrtgAPI, this becomes greatly beneficial when executing unit tests, where tests that attempt to deserialize tens of thousands of objects find their performance improved by over 200%.
PrtgClient exposes two event handlers LogVerbose and RetryRequest that expose informational status messages from PrtgAPI's request engine, such as the URL of the request that is being executed as well as that a failed request is being retried. As PowerShell allows for more free-form, batch oriented programming, it is useful to be able to expose this information from secondary output streams such as the verbose and warning streams.
Unfortunately however, due to the nature of PowerShell's execution model, event handlers cannot be simply "wired up" to a single cmdlet and used throughout the life of a pipeline. When a pipeline is being executed, only a single cmdlet may write to an output stream at any given time.
For example, consider the following cmdlet
Get-Sensor | Get-ChannelWhen this cmdlet executes, the following may occur
-
Get-Sensor, as the first cmdlet in the chain, subscribes to theRetryRequestevent handler -
Get-Sensorretrieves an initial 500 sensors from a PRTG Server.Get-Sensoris currently the top most cmdlet -
Get-Channelretrieves the channels for each of the first 500 sensors.Get-Channelis now the top most cmdlet -
Get-Sensorattempts to retrieve an additional 500 sensors (since it is capable of streaming), however PRTG times out, causing the request engine to invoke itsRetryRequestevent handler -
Get-Sensorattempts toWriteWarningtheRetryRequestmessage - PowerShell throws an exception as
Get-Sensoris not currently in a position to write to the warning stream
Having each cmdlet subscribe to the RetryRequest event handler is even worse, as now multiple cmdlets are trying to write to write the same warning at once. We could make each cmdlet "replace" the previous event handler, however this causes a new issue in that in a long chain of cmdlets, cmdlets will be continually stopped and started. We want PrtgAPI to somehow know who is the active cmdlet, so that they will be made responsible for all events triggered by an event handler.
PrtgAPI solves this problem using an EventStack. When the ProcessRecord method of a cmdlet is executed, the current cmdlet's event handlers are activated. When ProcessRecord completes, the event handlers are completely removed. In this way, whoever is currently processing records is the only person who is both capable and allowed to execute an event handler
protected override void ProcessRecord()
{
RegisterEvents();
try
{
ProcessRecordEx();
}
catch
{
UnregisterEvents(false);
throw;
}
finally
{
if (Stopping)
{
UnregisterEvents(false);
}
}
if (IsTopmostRecord())
{
UnregisterEvents(true);
}
}
//Real work is done here.
protected abstract void ProcessRecordEx();Multiple scenarios exist which can cause a cmdlet to stop running. Even if an exception is thrown from a cmdlet, the cmdlet may be executed again by the previous cmdlet if the ErrorActionPreference is Continue. As such, a boolean flag is used to specify whether to pretend to start from scratch as if the cmdlet had never executed in the first place.
By placing this code in a base class, all derived classes can be forced to implement a ProcessRecordEx method, reducing the likelihood they will accidentally overwrite the ProcessRecord method, thereby breaking this functionality.
You have a type that stores a dictionary of objects, however you wish to represent those objects as properties in PowerShell, while still storing and retrieving their values from the dictionary.
class MyType
{
private Dictionary<string, string> dictionary;
public MyType()
{
dictionary.Add("first", "val1");
dictionary.Add("second", "val2");
}
}C:\> Get-MyType
first : val1
second : val2You effectively need to bind the PSObject properties to the dictionary members. In order to do this need to either redirect the getters and setters of the property, or somehow maintain a reference from within the property back to the original value.
As it happens, a PSObject can contain properties that fit all of these scenarios
PSCodePropertyPSScriptPropertyPSVariableProperty
PSCodeProperty and PSScriptProperty take a MethodInfo or ScriptBlock respectively that specifies the action to perform in each scenario. The issue with this however is that for PSCodeProperty, the MethodInfo must both be public and static, polluting our API and preventing us from accessing this, while with PSScriptProperty there's no clear way to convert a C# delegate to a ScriptBlock.
PSVariableProperty however takes a PSVariable. A PSVariable takes a name and a value. Whenever the PSVariableProperty is updated, the PSVariable will be too. This is exactly what we need. All we need to do is maintain a reference to the original variable. As such, instead of storing our value in our dictionary, we'll store our PSVariable. We can then give MyType an indexer to provide normal access to the underlying value.
class MyType
{
public string SomeProp { get; set; } = "banana";
private Dictionary<string, PSVariable> dictionary;
public MyType()
{
Add("first", "val1");
Add("second", "val2");
}
private void Add(string name, object value)
{
dictionary.Add(name, new PSVariable(name, value));
}
public object this[string key]
{
get { return dictionary[key].Value; }
set { dictionary[key].Value = value; }
}
}To output this to PowerShell, we simply need to construct a PSObject that contains all of the properties
class MyType
{
public string SomeProp { get; set; } = "banana";
private Dictionary<string, PSVariable> dictionary;
private PSObject psObject;
public MyType()
{
psObject = new PSObject(this);
Add("first", "val1");
Add("second", "val2");
}
private void Add(string name, object value)
{
var variable = new PSVariable(name, value);
dictionary.Add(name, variable);
psObject.Properties.Add(new PSVariableProperty(variable));
}
}WriteObject(myType);Wait a second, we just output myType instead of psObject! That's not going to work!
The output however may surprise you
C:\> Get-MyType
SomeProp : banana
first : val1
second : val2How did the properties from the internal PSObject get there? Did PowerShell perform some sort of lookup to find our internal PSObject?
This technique abuses an implementation detail of the PSObject type, wherein PowerShell will cache all of the extended properties that belong to a specific type. Since when we created our internal PSObject we bound it to an instance of MyType
psObject = new PSObject(this);Properties we add on this PSObject will also be appended to any other PSObject that consume a MyType in the future. Since every object sent to WriteObject is transformed into a PSObject as it travels through the pipeline, we can just output our normal object and PowerShell will append our properties to our new object. Since this is an implementation detail, you may want to consider having an internal getter for the PSObject instead
internal PSObject PSObject => psObject;TODO
Normally when you define a cmdlet parameter you will give it some sort of specific type
[Cmdlet(VerbsCommon.Get, "Sensor"]
public class GetSensor : PSCmdlet
{
[Parameter(Mandatory = false, ValueFromPipeline = true)]
public Device Device { get; set; }
}When the value of your argument doesn't exactly match that of the parameter (such as specifying an int for a string) PowerShell is able to perform one of several type coercions to make your value fit with the specified type of the parameter.
While you can always accept any old value by simply defining your parameter as type object, by abusing type coercion you can is possible to define type safe parameters that take one of several specified values. The easiest coercion to manipulate is constructor coercion.
public class Either<TLeft, TRight>
{
public TLeft Left { get; }
public TRight Right { get; }
public bool IsLeft { get; }
public Either(TLeft value)
{
// ...
}
public Either(TRight value)
{
// ...
}
}When the left constructor is called, it assigns its value to Left and sets IsLeft to true. Conversely, right assigns Right and sets Isleft to false. By inspecting the value of IsLeft within your cmdlet, you can determine which of the two values was specified (if the parameter even had a value at all)
To utilize the Either class, simply declare it as your parameter's type along with the generic type arguments of the types you want to accept.
public class GetSensor : PSCmdlet
{
[Parameter(Mandatory = false, ValueFromPipeline = true)
public Either<Device, string> Device { get; set; }
}# Get all sensors under devices named "dc-1"
Get-Sensor -Device dc-1
# Get all sensors under the device with ID 1001
Get-Device -Id 1001 | Get-SensorBefore and after each test begins, a number of common tasks must be performed. For example, in PowerShell we must load the PrtgAPI and PrtgAPI.Tests.* assemblies into the session. We cannot just do this once in one test file and forget about it, as tests are split across a number of files and could be run one at a time via Test Explorer.
.NET tests perform common initialization/cleanup via AssemblyInitialize/AssemblyCleanup/TestInitialize methods defined in common base classes of all tests.
Common startup/shutdown tasks can be defined in Pester via the BeforeAll/AfterAll functions, however PrtgAPI abstracts that a step further by completely impersonating the Describe function. When tests call the Describe function they trigger PrtgAPI's Describe, which in turn triggers the Pester Describe with our BeforeAll/AfterAll blocks pre-populated
. $PSScriptRoot\Common.ps1
function Describe($name, $script) {
Pester\Describe $name {
BeforeAll {
PerformStartupTasks
}
AfterAll {
PerformShutdownTasks
}
& $script
}
}Different Describe overrides can be defined in different files, allowing tests to perform cleanup in different ways based on their functionality (such as Get- only tests not needing to perform cleanup on the integration test server). Methods such as AssemblyInitialize in our .NET test assembly can be triggered via our common startup functions, allowing existing testing functionality to be reused.
Integration tests can take an extremely long time to complete, can run in any order and can even cross contaminate. By intercepting key test methods and sprinkling tests with basic logging code, detailed state information can be written to a log file (%temp%\PrtgAPI.IntegrationTests.log) which can be tailed and monitored during the execution of tests
24/06/2017 11:46:00 AM [22952:58] C# : Pinging ci-prtg-1
24/06/2017 11:46:00 AM [22952:58] C# : Connecting to local server
24/06/2017 11:46:00 AM [22952:58] C# : Retrieving service details
24/06/2017 11:46:00 AM [22952:58] C# : Backing up PRTG Config
24/06/2017 11:46:01 AM [22952:58] C# : Refreshing CI device
24/06/2017 11:46:01 AM [22952:58] C# : Ready for tests
24/06/2017 11:46:01 AM [22952:58] PS : Running unsafe test 'Acknowledge-Sensor_IT'
24/06/2017 11:46:01 AM [22952:58] PS : Running test 'can acknowledge indefinitely'
24/06/2017 11:46:01 AM [22952:58] PS : Acknowledging sensor indefinitely
24/06/2017 11:46:01 AM [22952:58] PS : Refreshing object and sleeping for 30 seconds
24/06/2017 11:46:31 AM [22952:58] PS : Pausing object for 1 minute and sleeping 5 seconds
24/06/2017 11:46:36 AM [22952:58] PS : Resuming object
24/06/2017 11:46:37 AM [22952:58] PS : Refreshing object and sleeping for 30 seconds
24/06/2017 11:47:07 AM [22952:58] PS !!! : Expected: {Down} But was: {PausedUntil}
24/06/2017 11:47:07 AM [22952:58] PS : Running test 'can acknowledge for duration'
24/06/2017 11:47:07 AM [22952:58] PS : Acknowledging sensor for 1 minute
24/06/2017 11:47:07 AM [22952:58] PS : Sleeping for 60 seconds
24/06/2017 11:48:07 AM [22952:58] PS : Refreshing object and sleeping for 30 seconds
24/06/2017 11:48:37 AM [22952:58] PS : Test completed successfully
24/06/2017 11:48:37 AM [22952:58] PS : Running test 'can acknowledge until'
24/06/2017 11:48:38 AM [22952:58] PS : Acknowledging sensor until 24/06/2017 11:49:38 AM
24/06/2017 11:48:38 AM [22952:58] PS : Sleeping for 60 seconds
24/06/2017 11:49:38 AM [22952:58] PS : Refreshing object and sleeping for 30 seconds
24/06/2017 11:50:08 AM [22952:58] PS : Test completed successfully
24/06/2017 11:50:08 AM [22952:58] PS : Performing cleanup tasks
24/06/2017 11:50:08 AM [22952:58] C# : Cleaning up after tests
24/06/2017 11:50:08 AM [22952:58] C# : Connecting to server
24/06/2017 11:50:08 AM [22952:58] C# : Retrieving service details
24/06/2017 11:50:08 AM [22952:58] C# : Stopping service
24/06/2017 11:50:21 AM [22952:58] C# : Restoring config
24/06/2017 11:50:21 AM [22952:58] C# : Starting service
24/06/2017 11:50:24 AM [22952:58] C# : Finished
24/06/2017 11:50:25 AM [22952:63] PS : PRTG service may still be starting up; pausing for 60 seconds
24/06/2017 11:51:31 AM [22952:63] PS : Running safe test 'Get-NotificationAction_IT'
DateTime, PID, TID, execution environment and exception details are all easily visible. Showing three exclamation marks against rows that contain a failure is probably the greatest feature of the entire project.