Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSOE-353: Add duplicate SQL query detector in Lombiq.UITestingToolbox #216

Open
wants to merge 70 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
f224aff
Adding counter infrastructure
dministro Oct 23, 2022
d87e325
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Oct 23, 2022
34b538b
Using interfaces
dministro Oct 24, 2022
7574e9a
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Oct 24, 2022
beef486
Implementing required exception constructors
dministro Oct 24, 2022
6c3602f
Adjusting tests
dministro Oct 25, 2022
06f8eaa
Fixing typo
dministro Oct 25, 2022
3cd03e3
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Oct 25, 2022
8335d59
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Nov 7, 2022
914c291
Update Lombiq.Tests.UI/Services/CounterDataCollector.cs
dministro Dec 3, 2022
1100237
Update Lombiq.Tests.UI/Services/Counters/CounterProbeBase.cs
dministro Dec 3, 2022
df3c3c2
Update Lombiq.Tests.UI/Services/Counters/ICounterValue.cs
dministro Dec 3, 2022
1a0fd1a
Update Lombiq.Tests.UI/Services/Counters/NavigationProbe.cs
dministro Dec 3, 2022
8e61c22
Update Lombiq.Tests.UI.Samples/Tests/BasicOrchardFeaturesTests.cs
dministro Dec 3, 2022
a20311c
Update Lombiq.Tests.UI/Services/CounterConfiguration.cs
dministro Dec 3, 2022
f033443
Merge branch 'issue/OSOE-353' of https://github.com/Lombiq/UI-Testing…
dministro Dec 3, 2022
cac8255
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Dec 3, 2022
169fa26
Adding SessionProbe
dministro Dec 3, 2022
cc941b0
Adding better exclusion and defaults
dministro Dec 4, 2022
8e854fc
Addressing "Add docs to the properties."
dministro Dec 6, 2022
dff34c3
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Dec 6, 2022
33fd4fe
Fixes after merge
dministro Dec 6, 2022
f0d8fc0
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Dec 15, 2022
0a8189c
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Dec 26, 2022
d0a088f
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Jan 13, 2023
5d32c2f
Fixing analyzer errors
dministro Jan 13, 2023
448a46e
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Jan 25, 2023
0e1cdfe
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Feb 13, 2023
eb8adc9
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Mar 22, 2023
b416eb5
Fixes after merge
dministro Mar 22, 2023
bc91080
Fixing possible thread safety violation
dministro Mar 24, 2023
b3e0600
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Mar 24, 2023
278fdcc
Addressing "S4457: Split this method into two..." analyzer error
dministro Mar 27, 2023
2fd58bd
Adding per url threshold configuration support
dministro Apr 3, 2023
98a596e
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro May 18, 2023
8c05e71
Adding per page configuration implementation
dministro May 19, 2023
097d091
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Jul 17, 2023
030c692
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Aug 27, 2023
406e6ed
Fixing analyzer violations
dministro Aug 27, 2023
6bf2917
Adding ProbedSqliteConnection
dministro Aug 27, 2023
acf440f
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Sep 3, 2023
3342c9d
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Sep 8, 2023
126c97b
Improving log and exception message
dministro Sep 10, 2023
82c9090
Improving logs
dministro Sep 10, 2023
f50c13c
Throwing session probe exception in test context
dministro Sep 23, 2023
8a0b33c
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Oct 1, 2023
de38ccd
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Oct 19, 2023
8561fac
Fixing test
dministro Oct 23, 2023
fb9be54
Adding comment
dministro Oct 23, 2023
4935a11
Fix spelling
dministro Oct 23, 2023
c2906c7
Apply suggestions from code review
dministro Nov 5, 2023
8c58730
Apply suggestions from code review
dministro Nov 12, 2023
c953c8c
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Jan 30, 2024
3343bd8
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Apr 9, 2024
4f334fe
Fixes after merge
dministro Apr 9, 2024
027486d
Fixing analyzer violations
dministro Apr 9, 2024
fa3a95c
Fixing analyzer violations again
dministro Apr 9, 2024
0e4f177
Adding custom type for storing db command parameters in configuration…
dministro May 6, 2024
849553f
Allowing to enable/disable counter subsystem by configuration
dministro May 7, 2024
9203e36
Adding a bit more information to CounterThresholdException message
dministro May 7, 2024
a52cff1
Removing unnecessary Should.NotThrowAsync()
dministro May 7, 2024
30b4997
Adding test to demonstrate the scenario when the counter thresholds a…
dministro May 7, 2024
2d5abae
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro May 7, 2024
1b500d8
Removing unnecessary configuration
dministro May 12, 2024
bca3add
Renaming `PageLoadProbe` -> `RequestProbe` and `PageLoadProbeMiddlewa…
dministro May 12, 2024
a1531d5
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro May 12, 2024
71663b9
Adding more comments
dministro May 13, 2024
1dea988
Merge branch 'dev' of https://github.com/Lombiq/UI-Testing-Toolbox in…
dministro Jul 14, 2024
2c7e741
Spelling
dministro Jul 14, 2024
a6f685c
Fixing analyzer warnings
dministro Jul 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions Lombiq.Tests.UI/Exceptions/CounterThresholdException.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
using Lombiq.Tests.UI.Services.Counters;
using System;
using System.Text;

namespace Lombiq.Tests.UI.Exceptions;

// We need constructors with required informations.
#pragma warning disable CA1032 // Implement standard exception constructors
Piedone marked this conversation as resolved.
Show resolved Hide resolved
public class CounterThresholdException : Exception
#pragma warning restore CA1032 // Implement standard exception constructors
{
public CounterThresholdException(
ICounterProbe probe,
ICounterKey counter,
ICounterValue value)
: this(probe, counter, value, message: null, innerException: null)
{
}

public CounterThresholdException(
ICounterProbe probe,
ICounterKey counter,
ICounterValue value,
string message)
: this(probe, counter, value, message, innerException: null)
{
}

public CounterThresholdException(
ICounterProbe probe,
ICounterKey counter,
ICounterValue value,
string message,
Exception innerException)
: base(FormatMessage(probe, counter, value, message), innerException)
{
}

private static string FormatMessage(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generate exception messages that are easier to understand than:

DbExecuteCounterKey
SELECT [Document].* FROM [Document] WHERE [Document].[Type] = @type LIMIT 1
[0]Type = OrchardCore.Settings.SiteSettings, OrchardCore.Settings

IntegerCounterValue value: 26
Counter value is greater then DbCommandExecutionRepetitionThreshold, threshold: 22

I as an ordinary developer who didn't set up these counter thresholds but just wanted to change something unrelated but accidentally implemented a SELECT N+1 issue, should be able to understand what I did wrong.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While the current message is better, it's still cryptic when you first see it.

SessionProbe, [GET]https://localhost:9341/Admin
Database reader read counter
Query: SELECT [Document].* FROM [Document] WHERE [Document].[Type] = @type LIMIT 1
Parameters: [0]Type = OrchardCore.AdminMenu.Models.AdminMenuList, OrchardCore.AdminMenu
Count: 2
Counter value is greater then SessionThreshold.DbReaderReadThreshold, threshold: 0.

Start the exception with a message that explains, in plain English, what happened.

ICounterProbe probe,
ICounterKey counter,
ICounterValue value,
string message) =>
new StringBuilder()
.AppendLine(probe.DumpHeadline())
.AppendLine(counter.Dump())
.AppendLine(value.Dump())
.AppendLine(message ?? string.Empty)
.ToString();
}
60 changes: 60 additions & 0 deletions Lombiq.Tests.UI/Extensions/CounterDataCollectorExtensions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
using Lombiq.Tests.UI.Services.Counters;
using Lombiq.Tests.UI.Services.Counters.Data;
using System.Data;
using System.Data.Common;
using System.Threading;
using System.Threading.Tasks;

namespace Lombiq.Tests.UI.Extensions;

public static class CounterDataCollectorExtensions
{
public static int DbCommandExecuteNonQuery(this ICounterDataCollector collector, DbCommand dbCommand)
{
collector.Increment(DbExecuteCounterKey.CreateFrom(dbCommand));
return dbCommand.ExecuteNonQuery();
}

public static Task<int> DbCommandExecuteNonQueryAsync(
this ICounterDataCollector collector,
DbCommand dbCommand,
CancellationToken cancellationToken)
{
collector.Increment(DbExecuteCounterKey.CreateFrom(dbCommand));
return dbCommand.ExecuteNonQueryAsync(cancellationToken);
}

public static object DbCommandExecuteScalar(this ICounterDataCollector collector, DbCommand dbCommand)
{
collector.Increment(DbExecuteCounterKey.CreateFrom(dbCommand));
return dbCommand.ExecuteScalar();
}

public static Task<object> DbCommandExecuteScalarAsync(
this ICounterDataCollector collector,
DbCommand dbCommand,
CancellationToken cancellationToken)
{
collector.Increment(DbExecuteCounterKey.CreateFrom(dbCommand));
return dbCommand.ExecuteScalarAsync(cancellationToken);
}

public static DbDataReader DbCommandExecuteDbDatareader(
this ICounterDataCollector collector,
DbCommand dbCommand,
CommandBehavior behavior)
{
collector.Increment(DbExecuteCounterKey.CreateFrom(dbCommand));
return dbCommand.ExecuteReader(behavior);
}

public static Task<DbDataReader> DbCommandExecuteDbDatareaderAsync(
this ICounterDataCollector collector,
DbCommand dbCommand,
CommandBehavior behavior,
CancellationToken cancellationToken)
{
collector.Increment(DbExecuteCounterKey.CreateFrom(dbCommand));
return dbCommand.ExecuteReaderAsync(behavior, cancellationToken);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
using Lombiq.Tests.UI.Constants;
using Lombiq.Tests.UI.Pages;
using Lombiq.Tests.UI.Services;
using Lombiq.Tests.UI.Services.Counters;
using OpenQA.Selenium;
using OpenQA.Selenium.Support.UI;
using System;
Expand Down Expand Up @@ -39,7 +40,10 @@ public static Task GoToAbsoluteUrlAsync(this UITestContext context, Uri absolute
await context.Configuration.Events.BeforeNavigation
.InvokeAsync<NavigationEventHandler>(eventHandler => eventHandler(context, absoluteUri));

context.Driver.Navigate().GoToUrl(absoluteUri);
using (new NavigationProbe(context.CounterDataCollector, absoluteUri))
{
context.Driver.Navigate().GoToUrl(absoluteUri);
}
Piedone marked this conversation as resolved.
Show resolved Hide resolved

await context.Configuration.Events.AfterNavigation
.InvokeAsync<NavigationEventHandler>(eventHandler => eventHandler(context, absoluteUri));
Expand Down
62 changes: 62 additions & 0 deletions Lombiq.Tests.UI/Services/CounterConfiguration.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
using Lombiq.Tests.UI.Exceptions;
using Lombiq.Tests.UI.Services.Counters;
using Lombiq.Tests.UI.Services.Counters.Data;
using Lombiq.Tests.UI.Services.Counters.Value;
using System;
using System.Collections.Generic;
using System.Linq;

namespace Lombiq.Tests.UI.Services;

public class CounterConfiguration
{
/// <summary>
/// Gets the counter configuration used in the setup phase.
/// </summary>
public PhaseCounterConfiguration Setup { get; } = new();

/// <summary>
/// Gets the counter configuration used in the running phase.
/// </summary>
public PhaseCounterConfiguration Running { get; } = new();

public static Action<ICounterProbe> DefaultAssertCounterData(PhaseCounterConfiguration configuration) =>
probe =>
{
if (probe is NavigationProbe or CounterDataCollector)
{
var executeThreshold = probe is NavigationProbe
? configuration.DbCommandExecutionRepetitionPerNavigationThreshold
: configuration.DbCommandExecutionRepetitionThreshold;
var executeThresholdName = probe is NavigationProbe
? nameof(configuration.DbCommandExecutionRepetitionPerNavigationThreshold)
: nameof(configuration.DbCommandExecutionRepetitionThreshold);
var readThreshold = probe is NavigationProbe
? configuration.DbReaderReadPerNavigationThreshold
: configuration.DbReaderReadThreshold;
var readThresholdName = probe is NavigationProbe
? nameof(configuration.DbReaderReadPerNavigationThreshold)
: nameof(configuration.DbReaderReadThreshold);

AssertIntegerCounterValue<DbExecuteCounterKey>(probe, executeThresholdName, executeThreshold);
AssertIntegerCounterValue<DbReadCounterKey>(probe, readThresholdName, readThreshold);
}
};

public static void AssertIntegerCounterValue<TKey>(ICounterProbe probe, string thresholdName, int threshold)
where TKey : ICounterKey =>
probe.Counters.Keys
.OfType<TKey>()
.ForEach(key =>
{
if (probe.Counters[key] is IntegerCounterValue counterValue
&& counterValue.Value > threshold)
{
throw new CounterThresholdException(
probe,
key,
counterValue,
$"Counter value is greater then {thresholdName}, threshold: {threshold.ToTechnicalString()}");
dministro marked this conversation as resolved.
Show resolved Hide resolved
}
});
}
34 changes: 34 additions & 0 deletions Lombiq.Tests.UI/Services/CounterDataCollector.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
using Lombiq.Tests.UI.Services.Counters;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;

namespace Lombiq.Tests.UI.Services;

public sealed class CounterDataCollector : CounterProbeBase, ICounterDataCollector
{
private readonly ConcurrentBag<ICounterProbe> _probes = new();
public override bool IsRunning => true;
public Action<ICounterProbe> AssertCounterData { get; set; }

public void AttachProbe(ICounterProbe probe) => _probes.Add(probe);

public void Reset()
{
_probes.Clear();
Clear();
}

public override void Increment(ICounterKey counter)
{
_probes.SelectWhere(probe => probe, probe => probe.IsRunning)
.ForEach(probe => probe.Increment(counter));

base.Increment(counter);
Piedone marked this conversation as resolved.
Show resolved Hide resolved
}

public override string DumpHeadline() => nameof(CounterDataCollector);
public override string Dump() => DumpHeadline();
public void AssertCounter(ICounterProbe probe) => AssertCounterData?.Invoke(probe);
public void AssertCounter() => AssertCounter(this);
}
13 changes: 13 additions & 0 deletions Lombiq.Tests.UI/Services/Counters/CounterKey.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
namespace Lombiq.Tests.UI.Services.Counters;

// The Equals must be implemented in consumer classes.
#pragma warning disable S4035 // Classes implementing "IEquatable<T>" should be sealed
public abstract class CounterKey : ICounterKey
#pragma warning restore S4035 // Classes implementing "IEquatable<T>" should be sealed
{
public abstract bool Equals(ICounterKey other);
protected abstract int HashCode();
public override bool Equals(object obj) => Equals(obj as ICounterKey);
public override int GetHashCode() => HashCode();
public abstract string Dump();
}
64 changes: 64 additions & 0 deletions Lombiq.Tests.UI/Services/Counters/CounterProbe.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
using System;
using System.Text;

namespace Lombiq.Tests.UI.Services.Counters;

public abstract class CounterProbe : CounterProbeBase, IDisposable
Piedone marked this conversation as resolved.
Show resolved Hide resolved
{
private bool _disposed;

public override bool IsRunning => !_disposed;
public ICounterDataCollector CounterDataCollector { get; init; }

public override string Dump()
{
var builder = new StringBuilder();

builder.AppendLine(DumpHeadline());

foreach (var entry in Counters)
{
builder.AppendLine(entry.Key.Dump())
.AppendLine(entry.Value.Dump());
}

return builder.ToString();
}

protected CounterProbe(ICounterDataCollector counterDataCollector)
{
CounterDataCollector = counterDataCollector;
CounterDataCollector.AttachProbe(this);
}

protected virtual void OnAssertData() =>
CounterDataCollector.AssertCounter(this);

protected virtual void OnDispose()
{
}

protected virtual void Dispose(bool disposing)
{
if (!_disposed)
{
try { OnAssertData(); }
catch { throw; }
finally
{
if (disposing)
{
OnDispose();
}

_disposed = true;
}
}
}

public void Dispose()
{
Dispose(disposing: true);
GC.SuppressFinalize(this);
}
}
39 changes: 39 additions & 0 deletions Lombiq.Tests.UI/Services/Counters/CounterProbeBase.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
using Lombiq.Tests.UI.Services.Counters.Value;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;

namespace Lombiq.Tests.UI.Services.Counters;

public abstract class CounterProbeBase : ICounterProbe
{
private readonly ConcurrentDictionary<ICounterKey, ICounterValue> _counters = new();

public abstract bool IsRunning { get; }
public IDictionary<ICounterKey, ICounterValue> Counters => _counters;

protected void Clear() => _counters.Clear();

public virtual void Increment(ICounterKey counter) =>
_counters.AddOrUpdate(
counter,
new IntegerCounterValue { Value = 1 },
(_, current) => TryConvertAndUpdate<IntegerCounterValue>(current, current => current.Value++));

public abstract string DumpHeadline();

public abstract string Dump();

private static TCounter TryConvertAndUpdate<TCounter>(ICounterValue counter, Action<TCounter> update)
{
if (counter is not TCounter value)
{
throw new ArgumentException(
$"The type of ${nameof(counter)} is not compatible with ${typeof(TCounter).Name}");
dministro marked this conversation as resolved.
Show resolved Hide resolved
}

update.Invoke(value);

return value;
}
}
50 changes: 50 additions & 0 deletions Lombiq.Tests.UI/Services/Counters/Data/DbCommandCounterKey.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Text;

namespace Lombiq.Tests.UI.Services.Counters.Data;

public abstract class DbCommandCounterKey : CounterKey
{
private readonly List<KeyValuePair<string, object>> _parameters = new();
public string CommandText { get; private set; }
public IEnumerable<KeyValuePair<string, object>> Parameters => _parameters;

protected DbCommandCounterKey(string commandText, IEnumerable<KeyValuePair<string, object>> parameters)
{
_parameters.AddRange(parameters);
CommandText = commandText;
}

public override bool Equals(ICounterKey other)
{
if (ReferenceEquals(this, other)) return true;

return other is DbExecuteCounterKey otherKey
&& GetType() == otherKey.GetType()
&& CommandText == otherKey.CommandText
&& Parameters
.Select(param => (param.Key, param.Value))
.SequenceEqual(otherKey.Parameters.Select(param => (param.Key, param.Value)));
}

public override string Dump()
{
var builder = new StringBuilder();

builder.AppendLine(GetType().Name)
.AppendLine(CultureInfo.InvariantCulture, $"\t{CommandText}");
var commandParams = Parameters.Select((parameter, index) =>
FormattableString.Invariant(
$"[{index.ToTechnicalString()}]{parameter.Key ?? string.Empty} = {parameter.Value?.ToString() ?? "(null)"}"))
.Join(", ");
builder.AppendLine(CultureInfo.InvariantCulture, $"\t\t{commandParams}");

return builder.ToString();
}

protected override int HashCode() => StringComparer.Ordinal.GetHashCode(CommandText);
public override string ToString() => $"[{GetType().Name}] {CommandText}";
}
Loading