Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ReadStreamToArraySegment use BufferListStream directly when available #19823

Merged
merged 1 commit into from
Mar 25, 2021

Conversation

danielmarbach
Copy link
Contributor

@danielmarbach danielmarbach commented Mar 25, 2021

Additional split of #19683

All SDK Contribution checklist:

This checklist is used to make sure that common guidelines for a pull request are followed.

  • Please open PR in Draft mode if it is:
    • Work in progress or not intended to be merged.
    • Encountering multiple pipeline failures and working on fixes.
  • If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
  • I have read the contribution guidelines.
  • The pull request does not introduce breaking changes.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

SDK Generation Guidelines

  • The generate.cmd file for the SDK has been updated with the version of AutoRest, as well as the commitid of your swagger spec or link to the swagger spec, used to generate the code. (Track 2 only)
  • The *.csproj and AssemblyInfo.cs files have been updated with the new version of the SDK. Please double check nuget.org current release version.

Additional management plane SDK specific contribution checklist:

Note: Only applies to Microsoft.Azure.Management.[RP] or Azure.ResourceManager.[RP]

  • Include updated management metadata.
  • Update AzureRP.props to add/remove version info to maintain up to date API versions.

Management plane SDK Troubleshooting

  • If this is very first SDK for a services and you are adding new service folders directly under /SDK, please add new service label and/or contact assigned reviewer.
  • If the check fails at the Verify Code Generation step, please ensure:
    • Do not modify any code in generated folders.
    • Do not selectively include/remove generated files in the PR.
    • Do use generate.ps1/cmd to generate this PR instead of calling autorest directly.
      Please pay attention to the @microsoft.csharp version output after running generate.ps1. If it is lower than current released version (2.3.82), please run it again as it should pull down the latest version,

Old outstanding PR cleanup

Please note:
If PRs (including draft) has been out for more than 60 days and there are no responses from our query or followups, they will be closed to maintain a concise list for our reviewers.

@ghost ghost added Service Bus customer-reported Issues that are reported by GitHub users external to the Azure organization. labels Mar 25, 2021
@ghost
Copy link

ghost commented Mar 25, 2021

Thank you for your contribution @danielmarbach! We will review the pull request and get back to you soon.

@ghost ghost added the Community Contribution Community members are working on the issue label Mar 25, 2021
stream.CopyTo(memStream, StreamBufferSizeInBytes);

return new ArraySegment<byte>(memStream.ToArray());
switch (stream)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19042
AMD Ryzen 9 3950X, 1 CPU, 32 logical and 16 physical cores
.NET Core SDK=5.0.201
  [Host]     : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
  Job-AYYPIA : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT

InvocationCount=800000  
Method Mean Error StdDev Ratio RatioSD Gen 0 Gen 1 Gen 2 Allocated
ConvertViaMemoryStream 306.6 ns 6.13 ns 15.94 ns 1.00 0.00 0.1100 - - 928 B
PatternMatch 132.8 ns 1.13 ns 0.88 ns 0.48 0.04 0.0325 - - 280 B

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

    using System;
    using System.IO;
    using System.Text;
    using BenchmarkDotNet.Attributes;
    using BenchmarkDotNet.Configs;
    using BenchmarkDotNet.Diagnosers;
    using BenchmarkDotNet.Exporters;
    using BenchmarkDotNet.Jobs;
    using Microsoft.Azure.Amqp;

    [Config(typeof(Config))]
    public class BufferListStreamDirectly
    {
        private BufferListStream stream;
        private AmqpMessage amqpMessage;

        private class Config : ManualConfig
        {
            public Config()
            {
                AddExporter(MarkdownExporter.GitHub);
                AddDiagnoser(MemoryDiagnoser.Default);
                AddJob(Job.Default.WithInvocationCount(800000));
            }
        }

        [IterationSetup]
        public void Setup()
        {
            stream = new BufferListStream(new[] {new ArraySegment<byte>(Encoding.UTF8.GetBytes("Hello"))});
            amqpMessage = AmqpMessage.Create(stream, false);
        }

        [IterationCleanup]
        public void Cleanup()
        {
            amqpMessage.Dispose();
        }

        private const int StreamBufferSizeInBytes = 512;

        [Benchmark(Baseline = true)]
        public ArraySegment<byte> ConvertViaMemoryStream()
        {
            using var messageStream = amqpMessage.ToStream();
            using var memStream = new MemoryStream(StreamBufferSizeInBytes);
            messageStream.CopyTo(memStream, StreamBufferSizeInBytes);
            return new ArraySegment<byte>(memStream.ToArray());
        }

        [Benchmark]
        public ArraySegment<byte> PatternMatch()
        {
            using var messageStream = amqpMessage.ToStream();
            return messageStream switch
            {
                BufferListStream bufferListStream => bufferListStream.ReadBytes((int) stream.Length),
                _ => throw new InvalidOperationException()
            };
        }
    }

Copy link
Member

@jsquire jsquire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @danielmarbach! I'll defer to Josh for approval and merge, but assuming that we're getting a BufferListStream back from the AMQP library in most cases, this looks like it could be a nice efficiency with little risk.

@JoshLove-msft
Copy link
Member

Thanks, @danielmarbach! I'll defer to Josh for approval and merge, but assuming that we're getting a BufferListStream back from the AMQP library in most cases, this looks like it could be a nice efficiency with little risk.

As discussed in the other PR, we do always get back a BufferListStream, but looking in master it has been changed to a MemoryStream, so this still makes me a bit nervous. I don't think there are imminent plans to ship out of master though, and if others think this is a good calculated risk I'm happy to go with it. Still, it would be awesome if we could add coverage for the default case via a unit test, but we can always file an issue for that.

@danielmarbach
Copy link
Contributor Author

sigh even the virtual was removed on that method. Basically, a complete breaking change also for all inheritors of that abstract class 🤷

@JoshLove-msft
Copy link
Member

/azp run net - servicebus - tests

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@JoshLove-msft JoshLove-msft merged commit db10ffc into Azure:master Mar 25, 2021
@danielmarbach danielmarbach deleted the sender-buffered-stream branch March 30, 2021 04:47
azure-sdk pushed a commit to azure-sdk/azure-sdk-for-net that referenced this pull request Jul 26, 2022
Task 15001443: Remove identity field from the ActionGroup swagger as it had wrong type, and it is unused (Azure#19823)

* Task 15001443: Bug 14937463: Should throw a bad request with bad resourceId

* Revert "Task 15001443: Bug 14937463: Should throw a bad request with bad resourceId"

This reverts commit 4cfc48993c5ac8d7bf562c9156ba039db80644c9.

* Remove unused property

* Revert

* Revert "Revert"

This reverts commit b5ef90dca82008fc09cf07c5fa484b1dd880ee4f.

* Remove outdated api

* Also remove kind

* Remove kind

Co-authored-by: Thomas Pham <thompham@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Community Contribution Community members are working on the issue customer-reported Issues that are reported by GitHub users external to the Azure organization. Service Bus
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants