RFC 001: Path Matching/Resolution Strategy

[Tiberriver256]

The original implementation of Polaris was that the path requested by the client had to equal the path declared by the New-PolarisRoute function.

The current implementation takes a matching approach. If the path declared by New-PolarisRoute matches the beginning of the requested path the ScriptBlock will execute.

The current implementation of a more flexible path matching strategy allows for scenarios like the built-in file browser where the remainder of the URL can be assumed to be a relative path to the local file but it can be argued that it is too flexible as URLs that are technically "wrong" will match and the ScriptBlock may execute, as mentioned in #116.

We would like to ask for input from the community on a proposed solution currently in Pull Request #119. The proposal is to match the path strategy of Express.js.

This means we would support RegularExpressions for paths and named parameters within the path:

Example paths:

1. /hello - Would get converted to the following regular expression: ^/hello$. This would guarantee hello would be the only match and return to the original implementation.
2. /hello.* - Would get converted to the following regular expression: ^/hello.*$. This would match the current implementation and allow for any all requests that begin with hello to trigger the desired scriptlbock
3. /hello/:firstname/:lastname - Would get converted to the following regular expression: ^/hello/(?<firstname>.*)/(?<lastname>.*). The $Matches from the regular expression would then be converted to a PSCustomObject and be made available within the ScriptBlock as something like $Request.Parameters

As this is a breaking change we would love feedback and additional suggestions on the implementation. Thanks @TimCurwick for the feedback on the PR to bring this out for a more open discussion.

*EDIT:
This would introduce the following breaking change. The below request URL would match in the current implementation and would no longer match in the new implementation.

Route path: /flights
Request URL: http://localhost:3000/flights/myspecialflight

[Tiberriver256]

Full set of examples taken from the express.js documentation and converted to my idea of what it would look like for Polaris:

This route path will match requests to the root route, /.

New-PolarisRoute -Path "/" -ScriptBlock {
   $Response.Send('root')
}

This route path will match requests to /about.

New-PolarisRoute -Path "/about" -ScriptBlock {
   $Response.Send('about')
}

This route path will match requests to /random.text.

New-PolarisRoute -Path "/random.text" -ScriptBlock {
   $Response.Send('random.text')
}

Here are some examples of route paths based on string patterns.

This route path will match acd and abcd.

New-PolarisRoute -Path "/ab?cd" -ScriptBlock {
   $Response.Send('ab?cd')
}

This route path will match abcd, abbcd, abbbcd, and so on.

New-PolarisRoute -Path "/ab+cd" -ScriptBlock {
   $Response.Send('/ab+cd')
}

This route path will match abcd, abxcd, abRANDOMcd, ab123cd, and so on.

New-PolarisRoute -Path "/ab*cd" -ScriptBlock {
   $Response.Send('/ab*cd')
}

This route path will match /abe and /abcde.

New-PolarisRoute -Path "/ab(cd)?e" -ScriptBlock {
   $Response.Send('/ab(cd)?e')
}

Examples of route paths based on regular expressions:

This route path will match anything with an “a” in it.

New-PolarisRoute -Path [RegEx]::New("a") -ScriptBlock {
   $Response.Send('a')
}

This route path will match butterfly and dragonfly, but not butterflyman, dragonflyman, and so on.

New-PolarisRoute -Path [RegEx]::New(".*fly$") -ScriptBlock {
   $Response.Send('.*fly$')
}

Route parameters

Route parameters are named URL segments that are used to capture the values specified at their position in the URL. The captured values are populated in the req.params object, with the name of the route parameter specified in the path as their respective keys.

Route path: /users/:userId/books/:bookId
Request URL: http://localhost:3000/users/34/books/8989
$Request.parameters: { "userId": "34", "bookId": "8989" }

To define routes with route parameters, simply specify the route parameters in the path of the route as shown below.

New-PolarisRoute -Path "/users/:userId/books/:bookId" -ScriptBlock {
   $Response.Send($Request.Parameters)
}

The name of route parameters must be made up of “word characters” ([A-Za-z0-9_]).

Since the hyphen (-) and the dot (.) are interpreted literally, they can be used along with route parameters for useful purposes.

Route path: /flights/:from-:to
Request URL: http://localhost:3000/flights/LAX-SFO
$Request.Parameters: { "from": "LAX", "to": "SFO" }

Route path: /plantae/:genus.:species
Request URL: http://localhost:3000/plantae/Prunus.persica
$Request.Parameters: { "genus": "Prunus", "species": "persica" }

To have more control over the exact string that can be matched by a route parameter, you can append a regular expression in parentheses (()):

Route path: /user/:userId(\d+)
Request URL: http://localhost:3000/user/42
$Request.Parameters: {"userId": "42"}

[ChrisLGardner]

This looks like a pretty cool change, I've just started looking at Polaris as a possible alternative to Universal Dashboard for hosting some REST APIs (well as a middle layer to talk to another API that doesn't support CORS).

I'm only doing GETs so I can see some use for this for me, I'm interested to see what other people are doing with Polaris that this might break for them.

[TylerLeonhardt]

@Tiberriver256 you know you have my FULL support. This change looks soooo awesome. I can't wait 😄

@ChrisLGardner I love UD but I'll say Polaris does beat it on its size. Polaris uses HttpListener (for now at least...) which is built in where UD bundles up ASP.NET (for 5.1) and ASP.NET Core (for 6+) which can make it a bit like drinking from a firehose. But UD is soooo powerful.

[ChrisLGardner]

Yeah, I've got UD hooked up to IIS for https so I'm hoping to do something similar with Polaris if possible. I believe UD uses Kestrel which is what makes it possible but I haven't looked at httplistener enough to see if I can achieve the same thing. If it's http only then I still have a few ideas for how I can use it as part of this for now and then when https support of some form appears I'll move over to purely Polaris.

…

________________________________ From: Tyler James Leonhardt <notifications@github.com> Sent: Wednesday, May 9, 2018 12:03:44 AM To: PowerShell/Polaris Cc: Chris Gardner; Mention Subject: Re: [PowerShell/Polaris] RFC 001: Path Matching/Resolution Strategy (#120) @Tiberriver256<https://github.com/Tiberriver256> you know you have my FULL support. This change looks soooo awesome. I can't wait 😄 @ChrisLGardner<https://github.com/ChrisLGardner> I love UD but I'll say Polaris does beat it on its size. Polaris uses HttpListener (for now at least...) which is built in where UD bundles up ASP.NET (for 5.1) and ASP.NET Core (for 6+) which can make it a bit like drinking from a firehose. But UD is soooo powerful. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#120 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AQDqFj9qsiu9jspBLHnl1P0xPRXg4cOZks5twiRQgaJpZM4T10jv>.

[Tiberriver256]

@ChrisLGardner - HttpListener can do https. I think we'll need a few changes possibly and we'll have to document the setup. Tracking that in #107.

I also don't expect us to break any use cases but I think some paths might have to be updated in code if they were expecting it to continue to be just a match. We will leave this out here for the full two weeks before accepting a PR just because it sounds like we need to be sensitive to any use cases out there.

[TylerLeonhardt]

We are also hoping to see if we can swap in Kestrel for Httplistener if we can keep the size low (in other words, not bring in all of ASP.NET if possible) that's tracked in #12.

[TimCurwick]

Here is the functionality I want/need:

New-PolarisRoute -Path '/Snoobits' -Scriptblock {}

Matches http://localhost/Snoobits
Matches http://localhost/Snoobits/Careers
Matches http://localhost/Snoobits/Careers/Application.html
Does not match http://localhost/Snoobits2

If multiple routes are defined in the path, the route closest to the leaf takes precedence.

New-PolarisRoute -Path '/Snoobits' -Scriptblock {} -Exactmatch

Prevents partial matches.

We can, of course, integrate all of these functionalities, but I don't want to clutter it up with unnecessary features that make it more complicated or challenging to use in real life.

I agree with adding handling for route parameters (though not necessarily the approach), but what is the use case for string and regex matching?

[TimCurwick]

I also need simple matching to be case insensitive, and for more efficient implementation, I would argue for there to be no exceptions.

[Tiberriver256]

So in the proposal here you could achieve your goal with this command if you didn't need / want to capture any parameters:

New-PolarisRoute -Path '/Snoobits/.*' -Scriptblock {}

If you wanted to capture the remainder of the path for use in your scriptblock you would write it as

New-PolarisRoute -Path '/Snoobits/:RemainingPath' -Scriptblock {}

Which would produce the following results:

Request URL: http://localhost:3000/Snoobits/Careers/Application.html
$Request.Parameters: { "RemainingPath": "Careers/Application.html" }

Request URL: http://localhost:3000/Snoobits/Careers
$Request.Parameters: { "RemainingPath": "Careers" }

I'm with you on the case sensitive thing. I had not thought of that but PowerShell being case insensitive on a -match is somewhat non-typical. We could default it to use -cmatch and add a parameter for -CaseInsensitiveMatch or something of that sort?

To answer your question about the benefits / use case of going to regular expressions and parameters for path matching. The parameters is really just taking advantage of regular expressions as .net allows named captures. The RegEx though guarantees that the path matching strategy can really be bent to anyone's use case. If you can think of some way you want to match / not-match you can do it in regex with enough Googling, but it's still friendly enough to just drop in a "/hello-world" path and have a match for "http://localhost:3000/hello-world" and not match "http://localhost:3000/hello-world1235" as you might typically expect.

[TimCurwick]

We don't want to build something that can do anything. There's different project for that, named "PowerShell". We need something that is optimized to meet the needs of the people that will be using it.

If we end up with pattern matching functionality as a side effect of how we implement design requirements, that's fine, but I don't think it is itself a design requirement.

If you are using matching as you've proposed it, how would you handle precedence?

Scenario:
Route A: /Reports
Route B: /Reports/SQLServer1/Instance1
Route C: /Reports/SQLServer1/Instance2

/Reports goes to A
/Reports/SQLServer1 goes to A
/Reports/SQLServer1/Instance1 goes to B
/Reports/SQLServer1/Instance1/Log1 goes to B

[TimCurwick]

Here’s what I propose.

Syntax

#  Exact match
New-PolarisRoute -Scriptblock {} -Route /Folder1

#  Match descendant objects
New-PolarisRoute -Scriptblock {} -Route /Folder1 -MatchChildItems

#  Regex match
New-PolarisRoute -Scriptblock {} -Match /Folder1/.*fly$

#  PowerShell string pattern match
New-PolarisRoute -Scriptblock {} -Like /Folder1/SQL*

#  Route parameters
New-PolarisRoute -Scriptblock {} -Route /Folder1 -RouteParameter /:firstname/:lastname 

Objects
Replace (and possibly rename) $Polaris.ScriptRoutes with array of PolarisRoute objects.

class PolarisRoute {
    [String]$Method
    [String]$Route
    [String]$RouteParameter
    [Scriptblock]$Scriptblock
    [Boolean]$MatchChildItems
    [Boolean]$IsRegexMatch
    [Boolean]$IsPatternMatch } 

Add $Polaris property [Hashtable]$Routes (or some better name). As routes are added to array $ScriptRoutes, they are also added to a hashtable for fast, case insensitive exact matching.
$Routes.Add( "$Method:$Route", $_ )`

Matching logic

#  Exact match, case insensitive
If ( $Route["$Method`:$Route"] )
	{ $MatchingRoute = $Route["$Method`:$Route"] }
Else
    {
	#  Child item match, case insensitive
    ForEach ( $RouteCandidate in <# from $Route to "/" by atom#> )
        {
        If ( $Route["$Method`:$RouteCandidate"].MatchChildItems )
            {
            $MatchingRoute = $Route["$Method`:$RouteCandidate"]
            break
            }
        }
    }

If ( -not $MatchingRoute )
    {
    #  Regex and pattern matching routes in order added to Polaris object
    ForEach ( $RouteCandidate in $ScriptRoutes ) 
        {
        If ( $RouteCandidate.IsRegexMatch )
            {
            If ( $Route -match $RouteCandidate.Route )
                {
                $MatchingCandidate = $RouteCandidate
                break
                }
            }
        If ( $RouteCandidate.IsPatternMatch )
            {
            If ( $Route -like $RouteCandidate.Route )
                {
                $MatchingCandidate = $RouteCandidate
                break
                }
            }
        }
    }

If ( $MatchingRoute.RouteParameter )
    {
    <# Process route parameters #>
    } 

[Tiberriver256]

Thanks for the alternate solution proposal, I will have to mull this over a little bit.

I don't think the goal is to re-create PowerShell in path matching (I'm guessing that's Internet sarcasm though :) ) but I do somewhat appreciate the comparison. My goal would be to create something that is simple and easy to use but also scales well to really complex scenarios which I feel like expressJS accomplished really well with their path matching strategy as there have been a lot of tools successfully built on top of it.

To answer your question on precedence. The example you gave wouldn't really have any worries about precedence because they don't have any patterns or regular expressions.

Scenario:
Route A: /Reports
Route B: /Reports/SQLServer1/Instance1
Route C: /Reports/SQLServer1/Instance2

/Reports goes to A
/Reports/SQLServer1 gets 404 as it doesn't exist
/Reports/SQLServer1/Instance1 goes to B
/Reports/SQLServer1/Instance1/Log1 gets 404 as it doesn't exist

This should be the default behavior for basic strings entered in as paths as it is the default behavior in the frameworks that I have tried and is pretty much the expected behavior. It matches what I tell it to match and nothing more. If I ask it to match "/Reports" and it matches "/Reports/SQLServer1" I think that would be considered unexpected behavior by most people. @tylerl0706 feel free to correct me if I'm wrong or jump in on this too.

In a scenario with multiple regular expressions or string patterns with a wildcard at the end my PR is handling the precedence by running the longest (most specific) first, so if you wanted to use all wildcard string patterns like this:

Scenario:
Route A: /Reports*
Route B: /Reports/SQLServer1/Instance1*
Route C: /Reports/SQLServer1/Instance2/logs/*

This would be your experience:

/Reports goes to A
/Reports/SQLServer1 goes to A
/Reports/SQLServer1/Instance1 goes to B
/Reports/SQLServer1/Instance1/myrandomsite goes to B
/Reports/SQLServer1/Instance1/logs/1 goes to C

Question on your proposal there, what is intent of the -MatchChildItems switch? It's a little tough to tell from the code.

[TylerLeonhardt]

Just want to chime in and say that I think @Tiberriver256 is right. Most webservers (thinking about Express, Flask, Nancy (I think)) would not allow this:

Scenario:
Route A: /Abc

/Abc/Foo to match A

For the other web servers this would throw a 404 back.

I do, however, really like the compromise of using the wildcard which is very familiar to PowerShell users.

@Tiberriver256's idea allows for simple route creation, while also giving complexity to users who need it.

That said, I really value @TimCurwick's feedback and thoughts. He's done a ton of awesome things for Polaris. What are your thoughts given this addition information?