Aff and the correct way to do async/await pattern functionally.

[EivindAntonsen]

Hi!

I'm looking to learn more about this lib and functional programming in general. Either/Option etc has been great so far, but now my next project has mostly async code which I tried for a bit with regular Eithers but it didn't look quite correct. Then I stumbled upon Aff in the wiki, but needless to say I just got more confused... I don't quite get what they do other than the vague notion that they're for asynchronous effects.

Say you have a simple api integration with a third party, you have a SpotifyHttpClient.cs for example.
And let's assume the contract for one method looks like this: async Task<Playlist[]> GetUserPlaylistsAsync(string userId)

How would Aff change how you solve this? Because the method is Async, it has to return a Task as far as I understand, but a Task doesn't fit into the same method chains that an Either does.. so I'm a bit confused. Any help appreciated. :D

[louthy]

Because I don't know the library you're using, I've created this mock of it:

namespace SpotifyAPI
{
    public record Track(string Artist, string Title);
    public record Playlist(string Name, Track[] Tracks);
    
    public static class SpotifyHttpClient
    {
        public static Task<Playlist[]> GetUserPlaylistsAsync(string userId) => 
            default;
    }
}

For the simplest version of Aff (the non-runtime version), you can do this:

namespace SpotifyAff
{
    public static class Spotify
    {
        public static Aff<Seq<Playlist>> getUserPlaylist(string userId) =>
            Aff(async () => 
                Seq(await SpotifyAPI.SpotifyHttpClient
                                    .GetUserPlaylistsAsync(userId))
                                    .Map(p => new Playlist(p.Name, Seq(p.Tracks))));
    }

    public record Playlist(string Name, Seq<SpotifyAPI.Track> Tracks);
}

Essentially we're wrapping up the Task based functionality into Aff. This does a little extra work to convert the arrays into language-ext Seq types, because they're easier to work with.

Once you have wrapped up (lifted) your functionality into the Aff space, you can use them in Aff expressions, where the async/await stuff is handled automatically.

So, you could filter:

// Finds playlists containing songs by Led Zeppelin
Aff<Seq<Playlist>> r = Spotify.getUserPlaylist("123")
                              .Filter(lists =>
                                  lists.Exists(list => 
                                      list.Tracks.Exists(track => 
                                          track.Artist == "Led Zeppelin"))); 

You could reduce and just get a sequence of numbered tracks:

// Numbers each track and formats to a string
Aff<Seq<string>> r = Spotify.getUserPlaylist("123")
                            .Map(lists => lists.Bind(list => list.Tracks))
                            .Map(tracks => Naturals.Zip(tracks)
                                                   .Map(track => $"{track.Item1}. {track.Item2.Title}")
                                                   .ToSeq());

But, critically, as with all monadic types, they work with LINQ. And LINQ is how the monadic binding works (think chaining of operations).

Aff<Seq<Playlist>> both = from p1 in Spotify.getUserPlaylist("123")
                          from p2 in Spotify.getUserPlaylist("456")
                          select p1 + p2;

Now, because Aff is lazy, just doing the operations above will have no effect. So you need to call Run() on an Aff for it to execute. So, to run the last example above, you do this:

Fin<Seq<Playlist>> result = await both.Run();

Which returns a concrete result. Fin is like Either<Error, A> and it supports Match like Either does - which is how you find out if the operation succeed or failed.

Another thing to note is that you can call both.Run() multiple times. So, it's possible to pre-build Aff expressions and run them multiple times.

Again, I don't know the Spotify client, but I suspect that SpotifyHttpClient isn't static as I've shown above. If not, then you could consider using the runtime version of Aff. This allows a 'context' to be passed through the Aff expression without you having to do it manually.

If I change the SpotifyHttpClient to be an instance class, not static:

namespace SpotifyAPI
{
    public class SpotifyHttpClient
    {
        public Task<Playlist[]> GetUserPlaylistsAsync(string userId) => 
            default;
    }
}

Then create a 'trait' interface for all things spotify:

    public interface HasSpotify
    {
        SpotifyAPI.SpotifyHttpClient SpotifyClient { get; }
    }

Then I can create a runtime that is the environmental context that gets passed through the Aff:

public readonly struct Runtime : 
    HasSpotify,
    HasCancel<Runtime>
{
    public Runtime(
        SpotifyHttpClient spotifyClient,
        CancellationTokenSource source,
        CancellationToken token
    )
    {
        SpotifyClient = spotifyClient;
        CancellationTokenSource = source;
        CancellationToken = token;
    }
    
    public SpotifyHttpClient SpotifyClient { get; }
    
    public Runtime LocalCancel
    {
        get
        {
            var src = new CancellationTokenSource();
            return new(SpotifyClient, src, src.Token);
        }
    }

    public CancellationToken CancellationToken { get; }
    public CancellationTokenSource CancellationTokenSource { get; }
}

It also needs to implement HasCancel<RT> so that the cancellation token can be passed through automatically too.

Now we can update the Spotify wrapper to use a generic runtime:

    public static class Spotify<RT>
        where RT : struct, HasSpotify, HasCancel<RT>
    {
        public static Aff<RT, Seq<Playlist>> getUserPlaylist(string userId) =>
            Aff<RT, Seq<Playlist>>(async rt => 
                Seq(await rt.SpotifyClient
                            .GetUserPlaylistsAsync(userId))
                            .Map(p => new Playlist(p.Name, Seq(p.Tracks))));
    }

Notice how RT has three constraints:

• struct - which is essential for knowing it's not null
• HasSpotify - so we know we can get to the SpotifyClient member
• HasCancel<RT> - which is a requirement for the runtime Aff system, but also allows access to the cancellation-token if necessary

Then we can write generic functions that takes a runtime of any flavour, as long as it has specific constraints:

    static Aff<RT, Seq<Playlist>> concatPlaylists<RT>(string user1, string user2) 
        where RT : struct, HasSpotify, HasCancel<RT> =>
            from p1 in Spotify<RT>.getUserPlaylist("123")
            from p2 in Spotify<RT>.getUserPlaylist("456")
            select p1 + p2;

This allows for building of different runtimes that support unit-testing (for example). Or, you can just bake in the known Runtime that we've already built:

    static Aff<Runtime, Seq<Playlist>> concatPlaylists(string user1, string user2)  =>
            from p1 in Spotify<Runtime>.getUserPlaylist("123")
            from p2 in Spotify<Runtime>.getUserPlaylist("456")
            select p1 + p2;

This isn't unit-testable, but removes the needs for the constraints.

Then we can create a runtime, with an instance of the SpotifyHttpClient and it will automatically get passed through to the wrapped functions without you having to manually pass it around:

        var runtime = new Runtime(new SpotifyAPI.SpotifyHttpClient(), default, default);
        var aff = concatPlaylists<Runtime>("user123", "user456");
        var result = await aff.Run(runtime);

concatPlaylists doesn't care about what's going on behind it, it just knows that it needs a runtime and that it must support HasSpotify. This can allow for the HasSpotify capability to grow without changing the code that depends on it.

The idea here is to create a runtime that captures all of your IO, not just the Spotify API, but file-access, configuration, etc. into one place (the runtime), and then define declarative functions (that return Eff and Aff) that declare their requirements (HasSpotify, HasCancel, etc.). You inject the runtime into the Aff and Eff (when calling Run) at the edges of your application. The edges are:

• Main - this entry point should lead to the creation of a runtime, and then everything else runs inside the Aff
• web-requests - this entry point could use a pre-built runtime (created in Main) and then call .LocalCancel to create a local context for the web-request
• etc.

There are two 'built-in' runtimes for live and test environments. Which will give you an idea of how these are built. They can't be extended, so you need to copy/paste them as a starting point, then add the capabilities that your application needs (like spotify API access).

But off the bat, they have the following traits:

        HasCancel<Runtime>,
        HasConsole<Runtime>,
        HasFile<Runtime>,
        HasEncoding<Runtime>,
        HasTextRead<Runtime>,
        HasTime<Runtime>,
        HasEnvironment<Runtime>,
        HasDirectory<Runtime>

Which maps to a lot of the System.IO capabilities.

This may seem like a lot of effort for not much gain, but what you get is:

• Abstraction over the IO of your application
  • Means it can be swapped out, improved, without the core logic changing
• Declarative dependency injection (the runtime)
• Robust error handling
  • The ability to work with exceptional and non-exceptional errors
  • Pattern matching for catching errors (@catch)
• Guards
• Retry/Repeat/Fold with scheduling
• Timeout handling
• Trivial parallel processing via fork and SequenceParallel

And in the next release:

• Lifting of observables into Aff and Eff
• Lifting of enumerables into Aff and Eff
• Automatic resource management

There's lots of examples in the EffectsExamples project. await is called exactly once in Main

[EivindAntonsen]

Thanks so much for the in-depth response!
This clarifies a lot about the mysterious (well, to me anyway) Aff/Eff and their runtimes. :) Naming standards in functional programming (Lst, Aff, Ef, Seq etc) has confused me a bit, so it's nice to see their intent and usage explained in full and with examples. Really makes it much more accessible and easy to understand that way.
I'll give this a read through and by the looks of it I think it'll be nice to use as well.

quick question: I notice that the method signature is no longer marked with async. What are the consequences for this with regards to returning idle threads back into the threadpool?

[louthy]

I notice that the method signature is no longer marked with async

async is only there to enable the await keyword in the body of an expression. C# doesn't need that to use Task or ValueTask, back in the day we used to use task.ContinueWith(x => ...), like old javascript promises. Behind the scenes everything is still using C# tasks and leverages async/await where appropriate, so the return type isn't important here, what it wraps is still using Task and ValueTask, and is therefore still running on the built-in thread-pool.

[EivindAntonsen]

Thanks a lot, that information about async and thread-pool makes sense. :)

After some trial and mostly user error on the asynchronous side I am not able to get the code you posted above to compile, I only get the following compiler error in Jetbrains Rider:

Further more it only seems to want to return Aff and not what I'd expect from this method (different code example).

It seems to resolve to an overload that returns just a ValueTask, and not ValueTask<HttpRequestMessage>.

If I change it to call .Effect then it does work (by at least compiling and returning Unit), but I feel like I am misunderstanding something here since you are not calling that in your code example.

Eff<int>.Effect(() => 1).Run().Match(i => i.ToString(), error => error.Message);  // works
Eff<int>(() => 1).Run().Match(i => i.ToString(), error => error.Message); // compiler error stating struct name is not valid at this point

Sorry for this incoherent rambling, but I am abit confused here. :)

[evermanwa]

I did something like this, and it appears to work:

In your screenshot, you have an Aff that wraps a HttpRequestMessage but in your .Map you are returning string content. I'm nowhere near an authority on the library or FP, just throwing my .02 in there. Let me know if I am completely off, and if Paul over-rides anything definitely go with what he says.

Edit: I only explicitly put the "string" on the aff to make it more obvious.

[EivindAntonsen]

I figured it out. I did not do using static LanguageExt.Prelude. This all works like a charm now.