Draft version of FPBench 1.1 standards.

[billzorn]

Changes:

• Support for multiple and mixed-precision computations
• Semantics of numerical operations clarified as rounded real operations
• Precision can be specified per-operation using the rounding context
• New constant formats, including rational numbers and hexadecimal floating-point
• New metadata properties

A staging website with the draft versions of the 1.1 standards can be found here.

[pavpanchekha]

After looking over it carefully, it looks like the specified behavior of annotations is a little odd. If I have multiple parameters that affect rounding, maybe like this:

(FPCore (x)
 :precision binary32
 :round toZero
 (sin (! :precision binary64 (+ x (! :round toNearest (* 2 y))))))

As written the standard demands the rounding context be thrown away and replaced by the annotations. As a result, we have the following operations:

• sin in 32 bits rounding to zero
• + in 64 bits rounding as default
• * in default bits rounding to nearest

The bolded parts don't seem to make sense to me—it seems like they should inherit from enclosing annotations, but that doesn't happen. What's the reasoning here? Are there situations where inheriting is undesirable?

[billzorn]

Pavel is correct: the 1.1 standard, as originally written, always throws away the old context at each annotation and replaces it with only the specific properties provided by the annotation. This has the effect of making the standard very clear and easy to implement, but imposes what I now believe is too high a cost in terms of usability.

Pavel's example illustrates this perfectly: one would probably expect a nested (! :precision binary64) annotation to leave the current rounding mode untouched, rather than magically reverting it to whatever the default is. It also makes sense for nested annotations to build on each other: we should be able to write something like

(! :precision binary64 (- (! :round toZero (+ a b)) (! :round :toPositive (+ a b))))

without the inner rounding mode annotations blowing away other parts of the context.

I plan to rework the 1.1 standard slightly so that contexts inherit properties as expected, and annotations only change the properties provided, without clearing the rest of the context.

Allowing this behavior does make the standard slightly more complex. It also raises the question: is it possible to "undefine" a property in the context, to set it back to the default? Perhaps providing a property with no value, such as (! :precision :round nearestEven) to changing the rounding mode to nearestEven but revert the precision to whatever the default is.

[pavpanchekha]

One thing to think about for resetting to a default is that we generally do not standardize a default value, and allow implementations to choose their own. Is resetting to the default something you want to do? I suppose so, but is it too harmful to just force you to specify the default you expect?

[billzorn]

This is a good point: what is the default value supposed to be? As the standard is written, the default is always some magical thing defined by the tool. Another way of doing things would be to say that within a benchmark, the default is whatever is specified at the top level, in the overall properties of the FPCore (and if that is unspecified then it becomes the magical default supplied by the tool).

This seems like it adds a new capability, but I don't actually think it does. The typical application would be a case where you want to share a specific rounding context across many (but maybe not all) operations in a benchmark, and quickly change that context to create multiple related benchmarks. One way of accomplishing this would be to set this shared context at the top level, and frequently revert properties to default inside the benchmark expression.

However, the same effect can be achieved by moving the other operations, that you don't want to use the shared context, out into let bindings where they do not affect inheritance of the shared context. This also has the advantage that it can be done in a modular way: any annotation can be widely shared with appropriate use of let bindings under it, not just one given at top level.

I think the best solution is to allow properties to be inherited, to update only the properties supplied in an annotation, and to not provide any mechanism to revert a property to default or remove it from the context. To me, this seems like the simplest and most intuitive approach. I can't think of any cases where a useful behavior cannot be implemented on top of it with clever use of let bindings.

[pavpanchekha]

Yes, I agree. The inheritance mechanism already provides something like a default, and if you're going to go about resetting to default that seems sufficient.

[billzorn]

The latest update allows properties to be inherited as discussed. I have also added some extended examples.

There is not an extended example of using let to control context inheritance, just the small example in the Rounding section. This might be worth making a more detailed example for - thoughts?

[pavpanchekha]

I've made some revisions; they were major enough that I figured I'd push it to a new branch. The changes are largely reorganization, but I also wrote new text to describe rounding of function applications.

[pavpanchekha]

Ok, I think we have had plenty of time for discussions and multiple passes. I'm merging this as a draft standard, with the aim of making it a full standard once we've updated the FPBench tooling.