How to compose multiple IO effects with Eff<RT> and a custom Runtime?

[IlyaVassyutovich]

Hello!

Dearest @louthy (*), I can't express how grateful I am for your work. This is the most incredible project in terms of both fun and growth for me as a software engineer.

* And other contributors, of course. For your questions, suggestions, donations. I love you!

I have a question about Eff/IO. Please, advise :)

Setup

LanguageExt v5.

I have two effectful operations:

// 1. Load all device IDs
Task<Seq<DeviceId>> LoadDevices();

// 2. Load activity for a single device
Task<DateTimeOffset> GetActivity(DeviceId deviceId);

And some pure business logic that works on the collected data — for example, reducing each DateTimeOffset to DateOnly:

// Pure — no IO
static Seq<DateOnly> ToDateOnly(Seq<DateTimeOffset> activities) =>
    activities.Map(a => DateOnly.FromDateTime(a.DateTime));

Goal

I want to follow the "impureum sandwich" pattern:

1. Impure shell — load all devices, then load all their activities into memory
2. Pure core — run business logic over the collected data
3. Return the result

Something along these lines (pseudo-code):

static Eff<RT, Seq<DateOnly>> GetAllActivities() =>
    from devices    in LoadDevices<RT>()
    from activities in devices.Traverse(d => GetActivity<RT>(d))
    select ToDateOnly(activities);   // pure step at the end

Questions

1. Trait design — How should the traits (interfaces) on RT be defined so the runtime is injectable, similar to the FileIO / HasFile<RT> pattern in the docs?

2. Composing the pipeline — How to properly Traverse / Map over the device sequence inside Eff while keeping the IO layer flat (all reads upfront, then pure transforms)?

3. Different runtimes — What if LoadDevices and GetActivity require different runtime capabilities (e.g. different Has<RT, …> constraints)? How do you compose effects that depend on different parts of the runtime without creating a god-runtime that implements everything?

[louthy]

Hi Ilya, firstly thank you for sponsoring language-ext, it's very much appreciated and goes a long way to motivating me with the project.

I'm still away at the moment, trying to squeeze the last drops out of the ski season! I'll be back home next Thursday (26th), so I'll be able to reply to this in-depth then.

Hopefully some of the other experts here can help in the meantime 👍

[IlyaVassyutovich]

Wishing you a great end to the ski season — good snow on the slopes and safe riding with no incidents! ☃️

[aloslider]

Hello, clarify me if I misunderstood something.

How should the traits (interfaces) on RT be defined so the runtime is injectable, similar to the FileIO / HasFile<RT> pattern in the docs?

My guess is that you want to wrap these two methods like this:

record DeviceId;

interface DeviceIOTrait
{
	Task<Seq<DeviceId>> LoadDevicesAsync();
	Task<DateTimeOffset> GetActivityAsync(DeviceId deviceId);
}

record DeviceIOImpl : DeviceIOTrait
{
	public static readonly DeviceIOTrait Default = new DeviceIOImpl();

    public Task<DateTimeOffset> GetActivityAsync(DeviceId deviceId) =>
        throw new NotImplementedException();

    public Task<Seq<DeviceId>> LoadDevicesAsync() =>
		throw new NotImplementedException();
}

record MyRuntime : Has<Eff<MyRuntime>, DeviceIOTrait>
{
    static K<Eff<MyRuntime>, DeviceIOTrait> Has<Eff<MyRuntime>, DeviceIOTrait>.Ask =>
		pure<Eff<MyRuntime>, DeviceIOTrait>(DeviceIOImpl.Default);
}

How to properly Traverse / Map over the device sequence inside Eff while keeping the IO layer flat (all reads upfront, then pure transforms)

I think you've done it already in your example,

static class DeviceModule<RT> where RT : Has<Eff<RT>, DeviceIOTrait>
{
	static Eff<RT, DeviceIOTrait> trait =>
		+Has<Eff<RT>, RT, DeviceIOTrait>.ask; // + is equivalent to .As()

	public static Eff<RT, Seq<DateOnly>> getAllActivities() =>
		from dev in trait
		from devices in liftIO(() => dev.LoadDevicesAsync())
		from dates in devices.Traverse(getActivity) * BusinessLogic.toDatesOnly // * is equivalent to .Map()
		select dates;

	static Eff<RT, DateTimeOffset> getActivity(DeviceId deviceId) =>
		from dev in trait
		from activity in liftIO(() => dev.GetActivityAsync(deviceId))
		select activity;
}

static class BusinessLogic
{
	public static Seq<DateOnly> toDatesOnly(Seq<DateTimeOffset> activities) =>
		activities.Map(a => DateOnly.FromDateTime(a.DateTime));
}

What if LoadDevices and GetActivity require different runtime capabilities (e.g. different Has<RT, …> constraints)?

You specify the constaints on RT as usual:

static class DeviceModule<RT>
	where RT :
	    Has<Eff<RT>, DeviceIOTrait>,
	    Has<Eff<RT>, DirectoryIO> // for example

How do you compose effects that depend on different parts of the runtime without creating a god-runtime that implements everything?

I haven't yet dealt with so called god-runtimes, but currently I don't see any way of splitting them and composing in one computation, as well as I don't see any problems with god-runtime: make it partial and split the code across multiple files per capability, or just use #region.

[IlyaVassyutovich]

Thank you very much for your response. TBH, I have no idea (yet) what these things are — Has, .ask, Ask, in general I'm quite new to FP and some concepts are completely new for me.

I'll take some time to understand your response, read some docs.

[micmarsh]

ask is in general related to reader monads and "reader monad-like" types, that's most important to nail down as the general concept.

Paul's article is of course great https://paullouth.com/higher-kinds-in-csharp-with-language-ext-part-10-readert-monad-transformer/, but any kind of tutorial or introduction in any language depending on your background may be relevant, especially if you don't want to worry about monad-transformers yet.

[micmarsh]

To chime in on 3. specifically: while there indeed no way to compose concrete runtimes as @aloslider said (which is really just how C# objects in general are), are able to get a lot of decomposition and separation of concerns based on the constraints on your generic RT type.

In your example with only two traits, it may not be a big deal, but if you find, for example, that you're implementing a lot of file-system based utilities that only care about DirectoryIO in DeviceModule, you can easily split those off into a separate module that only depends on DirectoryIO, composing methods from the two classes within other methods that depend on both traits as needed.

That way, you do still have to define the "god-runtime" up front, but once you drill down to the actual application logic you find that nothing even knows about the concrete runtime type, and only cares about the minimal set of traits that it needs.