Suggestions for handling asynchronous data

[karljj1]

Hi,

We have some data types we use for formatting, but sometimes they're not quite ready yet. To handle this, we use the persistent variables system to trigger an update event and reformat, which usually works fine. However, we hit an issue when we've got further selectors in the mix.

Take this example: "Hello {myAsyncData.name}". Here, if "myAsyncData" isn't ready yet, we get an error because it can't find "name".

What's a smart way to stop formatting for a specific selector when we know that any further selectors are gonna fail because we're not ready yet?

[karljj1]

PR #375

[axunonb]

Currently I'm on vacation and thus a bit limited by available tools. So sorry for the delay.

You're looking for a smart solution for scenarios,

• where an ISource cannot return the requested data, because it's not available yet, and
• where the formatting step must therefore be skipped, as it could cause exceptions without the expected data

The implementation in the PR would be a solution indeed.
I have a small concern with using the exception message as the text to output. A separate property would be more open for any extensions. The class could be internal?
What I suggest here is to give the exception a name containing the cause, instead of the action to be taken, e.g. instead of AbortFormattingException maybe DataNotAvailableException or DataNotReadyException.

Another way could be the following:

• Extend the ISource with a property bool CancelFormatting which can be set for whatever reason (default is false of course). If true, the formatters will not be evaluated.
• All ISources are invoked with an FormattingInfo object, although the signature is ISelectorInfo.

• SmartFormat/src/SmartFormat/SmartFormatter.cs

  Line 588 in 245d3b0

  var handled = sourceExtension.TryEvaluateSelector(formattingInfo);

  
  That means that the ISource is able to formattingInfo.Write("Any Text") directly.

Unfortunately I could not evaluate this alternative, but maybe you'd like to consider it. What do you think?

[karljj1]

Thanks for the feedback, the name DataNotReadyException does indeed sound better.

The suggestion to extend ISource is an interesting one although it does break design guidelines to avoid changing interfaces that have shipped, it tends to break peoples existing code.
Perhaps the introduction of a new interface would work instead? ICancellableSource?

I'm currently using the PersistentVariables source to process a custom IVariable which is where the async code is, so that would need to communicate with the source to cancel the processing.

Id like to move my IVariable into its own ISource in the future so ill revisit this then and use an internal exception in our fork for now.
Thanks!

[axunonb]

Probably this idea is a bit late now, but still - if anybody else comes across this question:

• The selectorInfo in TryEvaluateSelector(ISelectorInfo selectorInfo) can safely be casted to a FormatterInfo.
• This gives access to FormattingInfo.Write(...). Use this to write your "not available" message.
• Then set the FormattingInfo.Result = null and leave TryEvaluateSelector with return true; for "success".
• The formatters will still be invoked, but output nothing because of the null value.

Code snippet:

In the YourCustomSourceExtension : ISource:

var fi = (FormattingInfo) selectorInfo;
fi.Write("waiting...");
fi.Result = null;
return true;

[Test]
public void Source_Cannot_Deliver_Value()
{
    var smart = Smart.CreateDefaultSmartFormat();
    smart.SourceExtensions.Add(new YourCustomSourceExtension());
    var result = smart.Format("{0:d:0.00}", 1);
    Assert.That(result, Is.EqualTo("waiting..."));
}

[axunonb]

@karljj1 Coming back to the idea "Extend the ISelectorInfo with a property bool CancelFormatting which could be set for whatever reason (default is false of course). If true, the formatters will not be evaluated." If we did this considering the default interface implementations available since C# v8, this would not be breaking change, would it?

A reason why I'm coming back to this questions is: I'm currently reconsidering the LogiCalcFormatter. NCalc v4 significantly improved performance and GC.
Because all the source and formatter extensions are handled inside the SmartFormatter, it could be helpful to move these method to an ExtensionInvoker class, which could provide more fine-grained handling for dealing with extensions. Example: Get the value for a single Placeholder and format a single Placeholder.
If this is not too vague: Any thoughts from your wish list?

[karljj1]

Hey.
I need to catch up on the latest .Net features :)
We are currently still using 2.x, one of the reasons is we can't use a lot of the newer .Net features, however, we have plans to move a lot of our code into core in a newer version which does support the newer .Net features. So we would be happy with these changes as we plan to upgrade to 3.x when we do this move and treat it as a clean break between the old system. So breaking changes are ok. We will still need to fork and make some changes (the nullable stuff doesn't work with our serializer) so if it does cause problems we can work around them in our fork.

[axunonb]

Hi - I see, so welcome to the future :)
Btw. there is a more up-to-date description about breaking changes compared to IMO outdated design guidelines.

Re extracting kind of ExtensionInvoker from SmartFormatter, I'll give it a try, but will certainly appreciate your review.

[axunonb]

Started to have a closer look at default interface implementations for Smart.Format yesterday.
Sadly, they don't work on NetStandard2.0 and .NET Framework - and never will 🙄
We pay the bill for supporting older target frameworks with more breaking changes or less innovation.

[axunonb]

@mikasoukhov As I noticed in your fork of SmartFormat, you have added the ISourceAsync interface, which I think is a nice feature. I rather thought about sources, that use e.g. a web api.
You have implemented ISourceAsync to ReflectionSource. Where do you see the advantage of async over the caching that we have implemented some time ago (this is not part of your fork)?
@karljj1 What's your opinion about ISourceAsync?

[karljj1]

Can you link me to ISourceAsync?

[axunonb]

Sure: https://github.com/StockSharp/Ecng/blob/6235d628fe516cee4466780a595d5aecb172c658/SmartFormat/Extensions/ReflectionSource.cs#L15
This implementation requires async all the way up to FormatAsync(...).

[mikasoukhov]

Yes, we implemented the ISourceAsync interface as we use it for text generation based on SmartFormat. This was specifically done for the web part of our application, where we have additional extensions that query the database during text generation. The primary reason for implementing this interface was to properly handle the ability to stop text generation when needed.

For desktop solutions, I see this as an excessive measure, and honestly, I don't see any connection with caching.

[axunonb]

Thanks for explaining how you're using the interface. Might be an interesting extension to SmartFormat.
Indeed, in this context there is no connection to caching.