Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider adding Text overloads to support UTF8 text #41919

Closed
davidfowl opened this issue May 30, 2022 · 27 comments · Fixed by #42595
Closed

Consider adding Text overloads to support UTF8 text #41919

davidfowl opened this issue May 30, 2022 · 27 comments · Fixed by #42595
Assignees
Labels
api-approved API was approved in API review, it can be implemented area-minimal Includes minimal APIs, endpoint filters, parameter binding, request delegate generator etc area-web-frameworks feature-minimal-actions Controller-like actions for endpoint routing Perf
Milestone

Comments

@davidfowl
Copy link
Member

davidfowl commented May 30, 2022

Background and Motivation

We've added the utf8 text literals to C# 11 and it would be nice to add some convenience helpers for producing results with utf8 string content.

Proposed API

namespace Microsoft.AspNetCore.Http;

public static class Results
{
+    public static IResult Text(ReadOnlySpan<byte> utf8Content, string? contentType = null, Encoding? contentEncoding = null, int? statusCode = null) 
}

public static class TypedResults
{
+    public static ContentHttpResult Text(ReadOnlySpan<byte> utf8Content, string? contentType = null, Encoding? contentEncoding = null, int? statusCode = null);
}

Usage Examples

// UTF8 literals
app.MapGet("/utf8-0", () => Results.Text("Hello World"u8));
app.MapGet("/utf8-1", () => TypedResults.Text("Hello World"u8));

// UTF8 Memory<byte>, not the most efficient by yah
var utf8FileBuffer = File.ReadAllBytes("/utf8Files/utf8file1.txt");

// Response with the first 30 bytes
app.MapGet("/u8-5", () => Results.Text(utf8FileBuffer[..30]));

Risks

None

@davidfowl davidfowl added api-suggestion Early API idea and discussion, it is NOT ready for implementation api-ready-for-review API is ready for formal API review - https://github.com/dotnet/apireviews and removed api-suggestion Early API idea and discussion, it is NOT ready for implementation labels May 30, 2022
@ghost
Copy link

ghost commented May 30, 2022

Thank you for submitting this for API review. This will be reviewed by @dotnet/aspnet-api-review at the next meeting of the ASP.NET Core API Review group. Please ensure you take a look at the API review process documentation and ensure that:

  • The PR contains changes to the reference-assembly that describe the API change. Or, you have included a snippet of reference-assembly-style code that illustrates the API change.
  • The PR describes the impact to users, both positive (useful new APIs) and negative (breaking changes).
  • Someone is assigned to "champion" this change in the meeting, and they understand the impact and design of the change.

@gfoidl
Copy link
Member

gfoidl commented May 30, 2022

Does

ReadOnlyMemory<byte> foo = "foo"u8;

work without intermediate byte[]-allocation?
For ROS it works as expected, but I'm not aware that the same works for ROM (which doesn't mean that latest discussion around that topic includes support for ROM too, of course).

@davidfowl
Copy link
Member Author

I believe that allocates but @stephentoub would know.

@stephentoub
Copy link
Member

That won't compile as of the latest language design and compiler version. You'd need to write:

ReadOnlyMemory<byte> foo = "foo"u8.ToArray();

which does of course allocate. The code you wrote with the existing compiler allocates as well.

@davidfowl
Copy link
Member Author

Hmm I need to get an updated compiler..

@rafikiassumani-msft rafikiassumani-msft added this to the 7.0-preview7 milestone Jun 2, 2022
@halter73
Copy link
Member

halter73 commented Jun 2, 2022

Do we expect people to use pooled ReadOnlyMemory<byte> with this API? Do we need to copy? Unfortunately, since we're returning the IResult, we cannot immediately write to the response body.

@halter73 halter73 added api-suggestion Early API idea and discussion, it is NOT ready for implementation and removed api-ready-for-review API is ready for formal API review - https://github.com/dotnet/apireviews labels Jun 6, 2022
@halter73
Copy link
Member

halter73 commented Jun 6, 2022

API review notes:

  • @stephentoub Is there type for UTF8 literals like "foo"u8 other than ReadOnlyMemory<byte> that would allow us to directly take it as a parameter without .ToArray()?
  • Let's wait on approval until we can test with new language version.

@stephentoub
Copy link
Member

Is there type for UTF8 literals like "foo"u8 other than ReadOnlyMemory that would allow us to directly take it as a parameter with .ToArray()?

The natural type of "..."u8 is ReadOnlySpan<byte>. There are no other types it implicitly or explicitly target types.

@halter73 halter73 added api-needs-work API needs work before it is approved, it is NOT ready for implementation and removed api-suggestion Early API idea and discussion, it is NOT ready for implementation labels Jun 6, 2022
@halter73
Copy link
Member

halter73 commented Jun 6, 2022

If we're going to have to copy either way which it seems like we will because there's no way for the developer returning Results.Text(...) to know when they can reclaim pooled memory, maybe we should just take ReadOnlySpan<byte>.

@davidfowl
Copy link
Member Author

@halter73 Sure.

@halter73 halter73 added api-ready-for-review API is ready for formal API review - https://github.com/dotnet/apireviews and removed api-needs-work API needs work before it is approved, it is NOT ready for implementation labels Jun 6, 2022
@ghost
Copy link

ghost commented Jun 6, 2022

Thank you for submitting this for API review. This will be reviewed by @dotnet/aspnet-api-review at the next meeting of the ASP.NET Core API Review group. Please ensure you take a look at the API review process documentation and ensure that:

  • The PR contains changes to the reference-assembly that describe the API change. Or, you have included a snippet of reference-assembly-style code that illustrates the API change.
  • The PR describes the impact to users, both positive (useful new APIs) and negative (breaking changes).
  • Someone is assigned to "champion" this change in the meeting, and they understand the impact and design of the change.

@halter73
Copy link
Member

halter73 commented Jun 13, 2022

API Review:

  • Do we need Memory overloads too?
    • Not for now. Span is easy enough and we want to copy anyway.
  • Do we want to add an overload to Results.Stream() supporting leaveOpen:?
    • Maybe. Not yet though.

Api approved!

namespace Microsoft.AspNetCore.Http;

public static class Results
{
+    public static IResult Text(ReadOnlySpan<byte> utf8Content, string? contentType = null, int? statusCode = null) 
}

public static class TypedResults
{
+    public static ContentHttpResult Text(ReadOnlySpan<byte> utf8Content, string? contentType = null, int? statusCode = null);
}

@davidfowl davidfowl self-assigned this Jun 13, 2022
@halter73 halter73 added api-approved API was approved in API review, it can be implemented and removed api-ready-for-review API is ready for formal API review - https://github.com/dotnet/apireviews labels Jun 13, 2022
@Tarun047
Copy link

Hi Guys,
I'd like to take this one up, if it's available,
As this my first time around here, I'm not sure if it's okay to create a PR directly.
I'm thinking this is the change to be done to
aspnetcore/src/Http/Http.Results/src/TypedResults.cs:
Screenshot 2022-06-18 at 11 23 38 AM

And for aspnetcore/src/Http/Http.Results/src/Results.cs:

public static IResult Text(ReadOnlySpan<byte> utf8Content, string? contentType = null, Encoding? contentEncoding = null, int? statusCode = null) => TypedResults.Text(utf8Content, contentType, contentEncoding, statusCode);

@davidfowl
Copy link
Member Author

Almost. We don't want to convert to a String here.

@halter73 We missed an API (my bad) there should also be a new Utf8ContentHttpResult that we create as part of this:

public sealed partial class Utf8ContentHttpResult : IResult
{
    public Memory<byte> ResponseContent { get; internal init; }

    public string? ContentType { get; internal init; }

    public int? StatusCode { get; internal init; }

    public Task ExecuteAsync(HttpContext httpContext);
}

@davidfowl
Copy link
Member Author

davidfowl commented Jul 6, 2022

Made a few changes to the API:

  • Removed the content encoding, these are UTF8 APIs
  • Added Utf8ContentHttpResult
namespace Microsoft.AspNetCore.Http;

public static class Results
{
+    public static IResult Text(ReadOnlySpan<byte> utf8Content, string? contentType = null, int? statusCode = null) 
}

public static class TypedResults
{
+    public static Utf8ContentHttpResult Text(ReadOnlySpan<byte> utf8Content, string? contentType = null, int? statusCode = null);
}

+public sealed partial class Utf8ContentHttpResult : IResult, IStatusCodeHttpResult, IContentTypeHttpResult
+{
+   public ReadOnlyMemory<byte> ResponseContent { get; internal init; }
+
+    public string? ContentType { get; internal init; }
+
+    public int? StatusCode { get; internal init; }
+    
+    int IStatusCodeHttpResult.StatusCode => StatusCode ?? 200;
+
+    public Task ExecuteAsync(HttpContext httpContext);
+}

@davidfowl
Copy link
Member Author

@Tarun047 my bad, I forgot you wanted to do this one 😞

@Tarun047
Copy link

Tarun047 commented Jul 6, 2022

No worries @davidfowl 😀

@halter73 halter73 added api-ready-for-review API is ready for formal API review - https://github.com/dotnet/apireviews and removed api-approved API was approved in API review, it can be implemented labels Jul 6, 2022
@ghost
Copy link

ghost commented Jul 6, 2022

Thank you for submitting this for API review. This will be reviewed by @dotnet/aspnet-api-review at the next meeting of the ASP.NET Core API Review group. Please ensure you take a look at the API review process documentation and ensure that:

  • The PR contains changes to the reference-assembly that describe the API change. Or, you have included a snippet of reference-assembly-style code that illustrates the API change.
  • The PR describes the impact to users, both positive (useful new APIs) and negative (breaking changes).
  • Someone is assigned to "champion" this change in the meeting, and they understand the impact and design of the change.

@BrennanConroy
Copy link
Member

Utf8ContentHttpResult should also implement IContentHttpResult

@davidfowl
Copy link
Member Author

It could also implement IValueResult (both generic and non generic) but I don’t have a strong opinion here

@davidfowl
Copy link
Member Author

OK now it matches what ContentHttpResult would have.

@rafikiassumani-msft rafikiassumani-msft added the feature-minimal-actions Controller-like actions for endpoint routing label Jul 7, 2022
@halter73
Copy link
Member

halter73 commented Jul 7, 2022

Is there a particular reason that the IStatusCodeHttpResult implementation is explicit?

@halter73
Copy link
Member

halter73 commented Jul 7, 2022

I think I see why. Should we make IStatusCodeHttpResult.StatusCode nullable instead?

@brunolins16
Copy link
Member

I think I see why. Should we make IStatusCodeHttpResult.StatusCode nullable instead?

In my opinion, I don't think we should because the only reason for IStatusCodeHttpResult is to describe a HttpResult that contains a StatusCode.

I did not want to change it in my PR but, in my opinion, ContentHttpResult, JsonHttpResult and Utf8ContentHttpResult should be changed to non-nullable StatusCode and explicitly set a default value (HTTP 200 OK) avoiding relying on the underlying behavior (similar issue happened in MVC #27165).

@halter73
Copy link
Member

halter73 commented Jul 7, 2022

in my opinion, ContentHttpResult, JsonHttpResult and Utf8ContentHttpResult should be changed to non-nullable StatusCode and explicitly set a default value (HTTP 200 OK) avoiding relying on the underlying behavior

I think there's a big difference between a result that always sets a 200 OK result vs a result that just skips setting the status code. Certain result types like Utf8ContentHttpResult can do either based on its state. I think that's why it's important that IStatusCodeHttpResult.StatusCode can take distinct values (200 and null respectively in this case) for when a status code is set vs when it's skipped.

@davidfowl
Copy link
Member Author

Did we close on this? Should it be nullable or not?

@halter73
Copy link
Member

API Review Notes:

  • Should IStatusCodeHttpResult.StatusCode be nullable?
    • Yes.
    • This means that any result type with non-nullable status codes should now implement the interface explicitly.
namespace Microsoft.AspNetCore.Http;

public static class Results
{
+    public static IResult Text(ReadOnlySpan<byte> utf8Content, string? contentType = null, int? statusCode = null) 
}

public static class TypedResults
{
+    public static Utf8ContentHttpResult Text(ReadOnlySpan<byte> utf8Content, string? contentType = null, int? statusCode = null);
}

+public sealed partial class Utf8ContentHttpResult : IResult, IStatusCodeHttpResult, IContentTypeHttpResult
+{
+   public ReadOnlyMemory<byte> ResponseContent { get; }
+
+    public string? ContentType { get; }
+
+    public int? StatusCode { get;  }
+
+    public Task ExecuteAsync(HttpContext httpContext);
+}

public interface IStatusCodeHttpResult
{
-    public int StatusCode { get; }
+    public int? StatusCode { get; }
}

// Plus a lot of changes to other types implementing IStatusCodeHttpResult

Approved!

@mkArtakMSFT mkArtakMSFT added api-approved API was approved in API review, it can be implemented and removed api-ready-for-review API is ready for formal API review - https://github.com/dotnet/apireviews labels Jul 17, 2022
@dotnet dotnet locked as resolved and limited conversation to collaborators Aug 16, 2022
@davidfowl davidfowl added the Perf label Oct 1, 2022
@amcasey amcasey added the area-minimal Includes minimal APIs, endpoint filters, parameter binding, request delegate generator etc label Jun 2, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
api-approved API was approved in API review, it can be implemented area-minimal Includes minimal APIs, endpoint filters, parameter binding, request delegate generator etc area-web-frameworks feature-minimal-actions Controller-like actions for endpoint routing Perf
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants