fn:scan-left and fn:scan-right - produce accumulation of results

[dnovatchev]

fn:scan-left and fn:scan-right - produce accumulation of results

In XPath 4.0 so far we still don't have a convenient way to express the functionality of producing a series of accumulated (accrued) results when applying a folding function over a collection (sequence, array, ...) of items. The general use-case for this is the task to produce a sequence of running totals when applying an operation over a sequence of data points: produce the partial sums of loan payments over fixed periods, produce the compounded amounts of a deposit with fixed interest rate over years, ..., etc.

Two functions (shamelessly borrowed from Haskell):

• fn:scan-left
• fn:scan-right

---

fn:scan-left

This function has a similar signature to that of fn:fold-left and produces the same final result, however it produces the complete (ordered) sequence of all partial results from every new value the accumulator gets during the evaluation of fn:fold-left.

Signature

fn:scan-left($input	as item()*,	
             $zero	as item()*,	
             $action	as function(item()*, item()) as item()*	
) as array(*)*

Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

The function is equivalent to the following implementation in XPath(return clause added for completeness):

let $scan-left-inner := function($seq as item()*, 
                                 $zero as item(), 
                                 $fun as function(item()*, item()) as item()*,
                                 $self as function(*)
                               ) as array(*)*                               
{
  let $result := [$zero]
   return
     if(empty($seq)) then $result
       else
         (
           $result, $self(tail($seq), $fun($zero, head($seq)), $fun, $self)  
         )
},

$scan-left := function($seq as item()*, 
                       $zero as item(), 
                       $fun as function(item()*, item()) as item()*
                     ) as array(*)*  
{
  $scan-left-inner($seq, $zero, $fun, $scan-left-inner)
}  

  return
    $scan-left(1 to 10, 0, op('+'))

Examples:

 $scan-left(1 to 10, 0, op('+'))

produces:

[0]
[1]
[3]
[6]
[10]
[15]
[21]
[28]
[36]
[45]
[55]

fn:scan-right

This function has a similar signature to that of fn:fold-right and produces the same final result, however it produces the complete (ordered) sequence of all partial results from every new value the accumulator gets during the evaluation of fn:fold-right.

Signature

fn:scan-right($input	as item()*,	
              $zero	as item()*,	
              $action	as function(item()*, item()) as item()*	
) as array(*)*

Properties

This function is ·deterministic·, ·context-independent·, and ·focus-independent·.

Rules

The function is equivalent to the following implementation in XPath(return clause added for completeness):

let  $scan-right-inner := function($seq as item()*,
                                   $zero as item()*,
                                   $f as function(item(), item()*) as item()*,                          
                                   $self as function(*)
                                  ) as array(*)*
{
    if(empty($seq)) then [$zero]
      else
        let $rightResult := $self(tail($seq), $zero, $f, $self)
         return
            ([$f(head($seq), head($rightResult))], $rightResult)
},

$scan-right := function($seq as item()*,
                        $zero as item()*,
                        $f as function(item(), item()*) as item()*
                       ) as array(*)*
{
  $scan-right-inner($seq, $zero, $f, $scan-right-inner)
}   

return
  $scan-right(1 to 10, 0, op('+'))  

Examples:

 $scan-right(1 to 10, 0, op('+'))

produces:

[55]
[54]
[52]
[49]
[45]
[40]
[34]
[27]
[19]
[10]
[0]

[ChristianGruen]

Related: https://docs.basex.org/wiki/Higher-Order_Functions_Module#hof:scan-left

[liamquin]

Looks like these could be defined in terms of the existing fold functions?

[ChristianGruen]

Looks like these could be defined in terms of the existing fold functions?

With a scan, the intermediate results are returned as well. While it’s possible to generate a sequence with the intermediate results with fold, it’s more convenient to have a custom function for it.

I know that Dimitre will disagree, but as hardly anyone uses fold-right in practice, a single scan function should suffice (e.g., as Kotlin has done it).

[liamquin]

Thanks for replying. Agree about fold-right! And in the XSLT 3 course i run, some people have said they'd struggled for ages with fold-left() before having an example and working through it. But i am not sure that they would find scan() any easier, and having two functions to learn instead of one is a burden sometimes. I’d vote for less-is-more on this one.

[ChristianGruen]

Thanks for replying. Agree about fold-right! And in the XSLT 3 course i run, some people have said they'd struggled for ages with fold-left() before having an example and working through it. But i am not sure that they would find scan() any easier, and having two functions to learn instead of one is a burden sometimes. I’d vote for less-is-more on this one.

Yes, learning folds may take a while. Until today, I don’t understand how our spec tries to explain how they work (and I wish we would revise the text; related: #864 (comment)). Instead, I learned folding by looking at other languages (or books ;).

My feeling is that once you’ve understood folds, scans are easy. Maybe we can find a better name, though.

[michaelhkay]

I have certainly found the need for something equivalent to scan-left. I agree that it's very difficult to explain these functions to users (and that's especially true for fold-right, which I still struggle with myself) and we should certainly try to do a better job on explanation (especially by example), but I don't think that's an absolute reason for leaving them out.

[dnovatchev]

Here is more about the use-cases for the scan functions: https://www.reddit.com/r/haskell/comments/s737lq/scan/

And especially here: https://www.cs.cmu.edu/~guyb/papers/Ble93.pdf

[dnovatchev]

As there seem to be no more comments, I will proceed to creating a PR.

Any objections?

[ChristianGruen]

PRs are always helpful.

[dnovatchev]

Submitted the PR, can be viewed here now: #957