Tests | Performance Tests Initial base

[cheenamalhotra]

Configure runnerconfig.json file with connection string and preferred settings to run Benchmark Jobs.
Run all enabled tests with below command:

cd src\Microsoft.Data.SqlClient\tests\PerformanceTests
dotnet run -c Release -f netcoreapp3.1|net5.0

Only "Release Configuration" applies to Performance Tests

[Wraith2]

How are these going to work given that we're not allowed to publish or compare any sql server benchmark numbers due to licensing?

[cheenamalhotra]

As of this test project, it's intended to be for development/review ease, we're not planning to publish benchmarks yet. We'll definitely go for legal approvals before doing that. Perf Project would simply live here for benchmarking aid.

/cc @David-Engel

[Wraith2]

Don't expect any answer from legal. They've been "working on it" for almost 3 years now with respect to TE benchmarking
TechEmpower/FrameworkBenchmarks#3786

[cheenamalhotra]

FYI as per our queries with legal team, Microsoft products and teams are pre-approved for performance benchmarking and publishing. Community and customers however would need to seek approval to do so, and licensing restrictions apply to anyone non-Microsoft.

We will update docs accordingly and can proceed with this PR.

[Wraith2]

Is there a way for community members to get approved?

[cheenamalhotra]

That is a tough area, SQL Server owner team can only provide official approval. It seems Microsoft has been involved in TechEmpower/FrameworkBenchmarks#3786 to seek approval, but is still in progress. They are the first links IMO.

[Wraith2]

If you scroll to the bottom of the thread you linked you'll find the last post is by me, and it commemorates the third anniversary of the thread being opened and there is still no official response. Which is why I asked, I'm wondering if there's another way to contact someone because that thread has disappeared into internal bureaucracy and has become a delocalized question that no-one owns and everyone is waiting for someone else on.

If the perf of this library isn't adequate to put onto the TE charts then that can be worked on. As I know you're well aware we've made some excellent performance enhancements and we know there's more we can do with time and hard work.
If there are further improvements needed then we can work on them if we know what they are, refusing to participate at all just makes it look as if the results are known to be bad and means people just won't be interested in a product that seems not to have the confidence of it's owner behind it.

[cheenamalhotra]

We shall try to change this perception to best of our capacity @Wraith2 .
I can't speak for the past, but our team's present goals are including perf benchmarking and we do value driver performance and would like to target it hard for better user experience.

It will take a bit of work and time to come up with a proper process, but we're beginning it soon!
So let's give it a fresh start :)

Maybe @saurabh500 or @David-Engel have some leads to resurrect the TM discussion, so I will leave it to them.

[DavoudEshtehari]

Nice job 🎉