Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fixing filtered update performance #3233

Merged
merged 3 commits into from Jun 5, 2018
Merged

fixing filtered update performance #3233

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
151ec58
Merge upstream master/5.169.6 from fsprojects/Paket
viktor-svub May 30, 2018
21abd29
update with filter performance fix
viktor-svub May 31, 2018
e49b28a
ensure the filter regex is created only once
viktor-svub Jun 4, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
15 changes: 5 additions & 10 deletions src/Paket.Core/Common/Domain.fs
View file
Open in desktop
Expand Up @@ -96,31 +96,26 @@ type QualifiedPackageName =
let packageName = PackageName packageName
QualifiedPackageName (groupName, packageName)

type PackageMatch(ex:String) =
member this.Expression = Regex("^" + ex + "$", RegexOptions.CultureInvariant ||| RegexOptions.IgnoreCase)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure this no longer creates regex objects on every access?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pretty much :)
as far I understand F# this should be constructor syntax, filling the Expression member with result of the expression after equals
and I'm seeing the described improvement on my machine, getting filtered update in approximately same time as full one
edit: including Debug build, so it's not just a result of compiler optimization, and it's indeed most likely called only on the filter instance creation

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decompiled into C#:

		[CompilationMapping(SourceConstructFlags.ObjectType)]
		[Serializable]
		public class PackageMatch
		{
			public PackageMatch(string ex) : this()
			{
				this.ex = ex;
			}

			public Regex Expression
			{
				get
				{
					return new Regex("^" + this.ex + "$", RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);
				}
			}

			internal string ex;
		}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know there is a syntax to initialize it only once, I think it is something with val. Tbh, in that regard F# is really confusing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And would it maybe make sense to add ``RegexOptions.Compiled`? https://stackoverflow.com/questions/513412/how-does-regexoptions-compiled-work#7707369

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a sharplab.io example: https://sharplab.io/#v2:DYLgZgzgPg9gDgUwHYAIDKBPCAXBBbAWAChjsNEUBhACgQA8QUcAnASyQHMBKFAXmJQo8+AEYJmKAG4BDYCgCidOMwQQIrGKl4p6KANQoARCOnNDAwYNLkEVAPoAlBAGMYeOAFdctBk2xtOHn4iQWE8MQlsAAtWCAA6RWVVdU0+HTp9IxMzIA===

You can see here that thanks to member val the property value is computed once on construction and then just reused, whereas for C_Recompute the property is recomputed each access.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we don't have access to System.Text.RegularExpressions in sharplab so that's why I didn't repro your scenario exactly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh my... :) Ok, I'll revise it to create the instance only once for sure.
At this moments it seems possible that all the slowdown was caused just by RegexOptions.Compiled while creating instance per comparison in either case.


// Represents a filter of normalized package names
[<System.Diagnostics.DebuggerDisplay("{ToString()}")>]
type PackageFilter =
| PackageName of PackageName
| PackageFilter of string
| PackageFilter of PackageMatch

member this.Match (packageName : PackageName) =
match this with
| PackageName name -> name = packageName
| PackageFilter f ->
let regex =
Regex("^" + f + "$",
RegexOptions.Compiled
||| RegexOptions.CultureInvariant
||| RegexOptions.IgnoreCase)

regex.IsMatch (packageName.CompareString)
| PackageFilter f -> f.Expression.IsMatch(packageName.CompareString)

static member ofName name = PackageFilter.PackageName name

override this.ToString() =
match this with
| PackageName name -> name.ToString()
| PackageFilter filter -> filter
| PackageFilter filter -> filter.Expression.ToString()


type DomainMessage =
Expand Down
2 changes: 1 addition & 1 deletion src/Paket.Core/Installation/UpdateProcess.fs
View file
Open in desktop
Expand Up @@ -272,7 +272,7 @@ let UpdatePackage(dependenciesFileName, groupName, packageName : PackageName, ne
let UpdateFilteredPackages(dependenciesFileName, groupName, packageName : string, newVersion, options : UpdaterOptions) =
let dependenciesFile = DependenciesFile.ReadFromFile(dependenciesFileName)

let filter = PackageFilter.PackageFilter(packageName.ToString())
let filter = PackageFilter.PackageFilter(PackageMatch packageName)

let dependenciesFile =
match newVersion with
Expand Down