API Proposal: File.ReadLinesAsync

[khellang]

Now that IAsyncEnumerable<T> has landed, I'd like to propose the following new methods to the File type:

namespace System.IO {
    public static partial class File {
        public static System.Collections.Generic.IEnumerable<string> ReadLines(string path) { throw null; }
        public static System.Collections.Generic.IEnumerable<string> ReadLines(string path, System.Text.Encoding encoding) { throw null; }
+       public static System.Collections.Generic.IAsyncEnumerable<string> ReadLinesAsync(string path, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
+       public static System.Collections.Generic.IAsyncEnumerable<string> ReadLinesAsync(string path, System.Text.Encoding encoding, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }
    }
}

The methods would basically be async counterparts to File.ReadLines, so they would mirror what File.ReadAllLinesAsync does, i.e. use an asynchronous, sequential StreamReader, but read line by line lazily, without allocating a list or array.

These would fill the remaining gap for async reading done by the File type, as mentioned in https://github.com/dotnet/corefx/issues/11220.

[GrabYourPitchforks]

Is the CancellationToken parameter at the end correct? I thought the pattern was that the IAsyncEnumerable<T>-returning component didn't take a CancellationToken and that a token was instead passed via an extension method.

[khellang]

Is the CancellationToken parameter at the end correct? I thought the pattern was that the IAsyncEnumerable<T>-returning component didn't take a CancellationToken and that a token was instead passed via an extension method.

I think so, yes. I'm assuming the CancellationToken parameter would be annotated with an EnumeratorCancellationAttribute and used inside the actual implementation, like this:

runtime/src/libraries/System.Threading.Channels/src/System/Threading/Channels/ChannelReader.netcoreapp.cs

Lines 19 to 28 in 4f9ae42

	public virtual async IAsyncEnumerable<T> ReadAllAsync([EnumeratorCancellation] CancellationToken cancellationToken = default)
	{
	while (await WaitToReadAsync(cancellationToken).ConfigureAwait(false))
	{
	while (TryRead(out T item))
	{
	yield return item;
	}
	}
	}

This was the only similar API I could find in code base:

runtime/src/libraries/System.Threading.Channels/ref/System.Threading.Channels.netcoreapp.cs

Line 16 in 4f9ae42

public virtual System.Collections.Generic.IAsyncEnumerable<T> ReadAllAsync(System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }

[khellang]

Any hint on when this will be up for review? 👼

[GrabYourPitchforks]

We usually get to APIs in chronological order (by date opened) unless they're marked as blocking.

[terrajobst]

Video

• Looks good as proposed
• The [EnumeratorCancellation] attribute might be needed in the implementation if we use yield return, but it doesn't have to be part of the ref.

namespace System.IO
{
    public static partial class File
    {
        public static IAsyncEnumerable<string> ReadLinesAsync(string path, CancellationToken cancellationToken = default);
        public static IAsyncEnumerable<string> ReadLinesAsync(string path, System.Text.Encoding encoding, CancellationToken cancellationToken = default);
    }
}

[carlossanlop]

@khellang will you get a chance to work on this in the next week or two? Otherwise, I should move it to Future.

[khellang]

@carlossanlop I've already done most of the work, but I won't be able to polish it up and submit a PR in the coming weeks as I'm on vacation 😢

[carlossanlop]

No worries. Let's move it to Future then.

[eiriktsarpalis]

Would really love to see this included in 7.0.0

[vcsjones]

@eiriktsarpalis There was a PR open for this that didn't get merged, I think this API is blocked on #20824 if I understood the PR's comments correctly.

[huoyaoyuan]

@khellang Now #20824 is unblocked. Do you want to work on this again?

[khellang]

Thanks for the heads up, @huoyaoyuan! I'm pretty swamped with Other Stuff right now, so I can't commit to anything. I can try to breathe life back into #43108 when I have time, but if someone beats me to it, I won't mind 😅

[AraHaan]

Wouldn't it be worth making ReadAllLinesAsync then call into this new API then to optimize them a bit as well?

[lateapexearlyspeed]

Hi, I would try to learn and fix it, not sure if can assign to me, here is my draft PR which will be marked as ready, thanks.

[lateapexearlyspeed]

Hi, this PR is marked as ready for review now, thanks.