-
Notifications
You must be signed in to change notification settings - Fork 63
Add benchmark project for performance tests #294
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
673cb37
add benchmark project
lucyzhang929 e04aed2
fix logger
lucyzhang929 330eaa5
fix logger
lucyzhang929 152ac4b
copy functions to benchmark output folder
lucyzhang929 55b67bd
cleanup
lucyzhang929 0527079
move setup methods to base
lucyzhang929 45649cb
add output binding perf
lucyzhang929 58b1b04
clean up
lucyzhang929 81a78ad
add README
lucyzhang929 b917abc
fix readme
lucyzhang929 3282f0e
add link to integration tests
lucyzhang929 8e4130b
typo
lucyzhang929 7839c00
cleanup + comments
lucyzhang929 cf141c5
address pr comments
lucyzhang929 1e1e059
fix readme
lucyzhang929 fba8bb3
Merge branch 'main' into luczhan/benchmark
lucyzhang929 9dbbeaf
Merge branch 'main' into luczhan/benchmark
lucyzhang929 2c23040
fix integration test base
lucyzhang929 40b41f4
pr comments
lucyzhang929 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
36 changes: 36 additions & 0 deletions
36
performance/Microsoft.Azure.WebJobs.Extensions.Sql.Performance.csproj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
|
|
||
| <PropertyGroup> | ||
| <OutputType>Exe</OutputType> | ||
| <TargetFramework>netcoreapp3.1</TargetFramework> | ||
| <!-- Need to set root namespace to empty for IDE0130 to work properly - otherwise it errors out on top-level namespaces for some reason --> | ||
| <RootNamespace></RootNamespace> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <PackageReference Include="BenchmarkDotNet" /> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\test\Microsoft.Azure.WebJobs.Extensions.Sql.Tests.csproj" /> | ||
| <ProjectReference Include="..\samples\samples-csharp\Microsoft.Azure.WebJobs.Extensions.Sql.Samples.csproj" /> | ||
| </ItemGroup> | ||
|
|
||
| <Target Name="CopySamples" AfterTargets="Build"> | ||
| <ItemGroup> | ||
| <_CSharpCopyItems Include="..\samples\samples-csharp\bin\$(Configuration)\$(TargetFramework)\**\*.*" /> | ||
| </ItemGroup> | ||
| <Copy SourceFiles="@(_CSharpCopyItems)" DestinationFolder="$(OutDir)\SqlExtensionSamples\CSharp\%(RecursiveDir)" /> | ||
| <Message Text="Copied C# Samples output to $(OutDir)\SqlExtensionSamples\CSharp" Importance="high" /> | ||
| </Target> | ||
|
|
||
| <!-- Copy C# Samples output to BenchmarkDotNet project bin (https://github.com/dotnet/BenchmarkDotNet/issues/946) --> | ||
| <ItemGroup> | ||
| <None Include="$(OutDir)\**\*"> | ||
| <CopyToOutputDirectory>Always</CopyToOutputDirectory> | ||
| <Link>%(RecursiveDir)\%(Filename)%(Extension)</Link> | ||
| <Visible>True</Visible> | ||
| </None> | ||
| </ItemGroup> | ||
|
|
||
| </Project> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| # Running Performance Tests | ||
|
|
||
| ## Pre-requisites | ||
| The performance tests are based on the IntegrationTestBase class. Follow the instructions to set up the pre-requisites for integration tests [here](../test/README.md#running-integration-tests). | ||
|
|
||
| ## Run | ||
| The performance tests use BenchmarkDotNet to benchmark performance for input and output bindings. | ||
|
|
||
| Run the tests from the terminal. | ||
| ``` | ||
| cd performance | ||
| dotnet run -c Release | ||
| ``` | ||
|
|
||
| ## Results | ||
| The test results will be generated in the BenchmarkDotNet.Artifacts folder. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| // Copyright (c) Microsoft Corporation. All rights reserved. | ||
| // Licensed under the MIT License. See License.txt in the project root for license information. | ||
|
|
||
| using BenchmarkDotNet.Running; | ||
|
|
||
| namespace Microsoft.Azure.WebJobs.Extensions.Sql.Performance | ||
| { | ||
| public class SqlBindingPerformance | ||
| { | ||
| public static void Main() | ||
| { | ||
| BenchmarkRunner.Run<SqlInputBindingPerformance>(); | ||
| BenchmarkRunner.Run<SqlOutputBindingPerformance>(); | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| // Copyright (c) Microsoft Corporation. All rights reserved. | ||
| // Licensed under the MIT License. See License.txt in the project root for license information. | ||
|
|
||
| using System.Net.Http; | ||
| using System.Threading.Tasks; | ||
| using Microsoft.Azure.WebJobs.Extensions.Sql.Samples.Common; | ||
| using Microsoft.Azure.WebJobs.Extensions.Sql.Samples.InputBindingSamples; | ||
| using Microsoft.Azure.WebJobs.Extensions.Sql.Tests.Common; | ||
| using Microsoft.Azure.WebJobs.Extensions.Sql.Tests.Integration; | ||
| using BenchmarkDotNet.Attributes; | ||
|
|
||
| namespace Microsoft.Azure.WebJobs.Extensions.Sql.Performance | ||
| { | ||
| public class SqlInputBindingPerformance : IntegrationTestBase | ||
| { | ||
| [GlobalSetup] | ||
| public void GlobalSetup() | ||
| { | ||
| this.StartFunctionHost(nameof(GetProductsTopN), SupportedLanguages.CSharp); | ||
| Product[] products = GetProductsWithSameCost(10000, 100); | ||
| this.InsertProducts(products); | ||
| } | ||
|
|
||
| [Benchmark] | ||
| [Arguments("1")] | ||
| [Arguments("10")] | ||
| [Arguments("100")] | ||
| [Arguments("1000")] | ||
| [Arguments("10000")] | ||
| public async Task<HttpResponseMessage> GetProductsTest(string count) | ||
| { | ||
| return await this.SendInputRequest("getproductstopn", count); | ||
| } | ||
|
|
||
| [GlobalCleanup] | ||
| public void GlobalCleanup() | ||
| { | ||
| this.Dispose(); | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| // Copyright (c) Microsoft Corporation. All rights reserved. | ||
| // Licensed under the MIT License. See License.txt in the project root for license information. | ||
|
|
||
| using System.Net.Http; | ||
| using System.Threading.Tasks; | ||
| using Microsoft.Azure.WebJobs.Extensions.Sql.Tests.Integration; | ||
| using Microsoft.Azure.WebJobs.Extensions.Sql.Tests.Common; | ||
| using Microsoft.Azure.WebJobs.Extensions.Sql.Samples.Common; | ||
| using Microsoft.Azure.WebJobs.Extensions.Sql.Samples.OutputBindingSamples; | ||
| using Newtonsoft.Json; | ||
| using BenchmarkDotNet.Attributes; | ||
|
|
||
|
|
||
| namespace Microsoft.Azure.WebJobs.Extensions.Sql.Performance | ||
| { | ||
| public class SqlOutputBindingPerformance : IntegrationTestBase | ||
| { | ||
| [GlobalSetup] | ||
| public void StartAddProductsArrayFunction() | ||
| { | ||
| this.StartFunctionHost(nameof(AddProductsArray), SupportedLanguages.CSharp); | ||
| } | ||
|
|
||
| [Benchmark] | ||
| [Arguments(1)] | ||
| [Arguments(10)] | ||
| [Arguments(100)] | ||
| [Arguments(1000)] | ||
| public async Task<HttpResponseMessage> AddProductsArrayTest(int count) | ||
| { | ||
| Product[] productsToAdd = GetProductsWithSameCost(count, 100); | ||
| return await this.SendOutputPostRequest("addproducts-array", JsonConvert.SerializeObject(productsToAdd)); | ||
| } | ||
|
|
||
| [IterationCleanup] | ||
| public void IterationCleanup() | ||
| { | ||
| // Delete all rows in Products table after each iteration | ||
| this.ExecuteNonQuery("TRUNCATE TABLE Products"); | ||
| } | ||
|
|
||
| [GlobalCleanup] | ||
| public void GlobalCleanup() | ||
| { | ||
| // Delete the database | ||
| this.Dispose(); | ||
| } | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.