Proposal: Variadic Extra Arguments for map, fold, and filter Clarity Functions

[dreadful-dev]

Variadic Arguments for filter, fold, and map

Preamble

SIP Number: XXX

Title: Variadic Extra Arguments for map, fold, and filter Clarity Functions

Authors:

• Jake Krogman https://github.com/dreadful-dev
• Aakanksha Mahajan https://github.com/amahajan87

Consideration: Technical, Governance

Type: Consensus

Status: Draft

Created: 6/16/23

License: BSD-2-Clause

Sign-off:

Layer: Consensus (hard fork)

Discussions-To: https://github.com/stacksgov/sips

Abstract

This SIP proposes a modification to the map, fold, and filter functions in the Clarity language to accept variadic extra arguments. This change would enhance the usability of these built-in functions, enabling cheaper operations such as filtering a value from a list.

License and Copyright

This SIP is made available under the terms of the BSD-2-Clause license. This SIP's copyright is held by the Stacks Open Internet Foundation.

Introduction

The map, filter, and fold functions in the Clarity language are key building blocks for many smart contract interactions. Currently, these functions only take a limited number of arguments. This proposal intends to extend these functions to accept variadic extra arguments, enhancing their functionality and providing developers with more flexibility.

Specification

map

The map function will be extended to accept variadic extra arguments:

(map func sequence_A sequence_B ... sequence_N ...args)

The map function applies func to each corresponding element of the input sequences and outputs a list of the same type containing the outputs from those function applications. The extra arguments allow developers to pass additional information into the function without altering the sequence inputs.

Examples

Old syntax

(define-private (multiply-by-two (x int)) (* x 2))
(define-private (example-map) (map multiply-by-two (list 1 2 3 4 5)))

New Syntax with variadic arguments

(define-private (multiply-by-n (x int) (n int)) (* x n))
(define-private (example-map) (map multiply-by-n (list 1 2 3 4 5) 2))

filter

The filter function will be extended to accept variadic extra arguments:

(filter func sequence ...args)

The filter function applies func to each element of the input sequence, returns the same sequence with any elements removed for which func returned false. The extra arguments allow developers to pass additional information into the function for more complex filtering operations.

Examples

Old syntax

(define-private (is-even (x int)) (is-eq (mod x 2) 0))
(define-private (example-filter) (filter is-even (list 1 2 3 4 5)))

New Syntax with variadic arguments

(define-private (greater-than-n (x int) (n int)) (> x n))
(define-private (example-filter) (filter greater-than-n (list 1 2 3 4 5) 2))

The next example has showcases the highest impact of all examples as the newly proposed functionality reduces the amount of code needed and reduce the contract state mutations, thus reducing cost of implementing this type of functionality.

Old syntax

(define-private (remove-principal-from-list (l (list 10 principal)) (p principal))
  (begin 
    (asserts! (is-none (var-get working-principal-storage)) (err "INVALID CONTRACT STATE"))
    (var-set working-principal-storage (some p))

    (let ((new-list (filter filter-principal-from-list l)))
      (var-set working-principal-storage none)
      (ok new-list)
    )
  )
)

(define-private (filter-principal-from-list (p principal))
  (not (is-eq (var-get working-principal-storage) (some p)))
)

New Syntax with variadic arguments

(define-private (remove-principal-from-list (l (list 10 principal)) (p principal))
  (filter filter-principal-from-list l p)
)

(define-private (filter-principal-from-list (current principal) (to-remove principal))
  (not (is-eq current to-remove))
)

fold

The fold function will be extended to accept variadic extra arguments:

(fold func sequence_A initial_B ...args)

The fold function condenses sequence_A into a value of type B by recursively applying func to each element of the input sequence and the output of a previous application of func. The extra arguments allow developers to pass additional information into the function for more complex folding operations.

Examples

Old syntax

(define-private (sum (x int) (acc int)) (+ x acc))
(define-private (example-fold) (fold sum (list 1 2 3 4 5) 0))

New Syntax with variadic arguments

(define-private (weighted-sum (x int) (acc int) (w int)) (+ (* x w) acc))
(define-private (example-fold) (fold weighted-sum (list 1 2 3 4 5) 0 2))

Backwards Compatibility

This change is backwards compatible as it adds functionality to existing Clarity functions. Existing smart contracts that do not use these functions with extra arguments will remain unaffected. However, smart contracts utilizing the extended functions will be incompatible with nodes not implementing this SIP.

Activation

This SIP will require a hard-fork and should be bundled with other changes that also require a hard-fork to minimize network disruption.

Reference Implementations

[To be added once there is a pull request or commit on Github]

Related Work

#27

References

1. Clarity Language Reference: https://docs.stacks.co/references/language-functions

[obycode]

Thanks for kicking off the new process with an excellent writeup!

This is definitely something I've wished for often when writing contracts. I would definitely support this addition.

[MarvinJanssen]

This change is backwards compatible as it adds functionality to existing Clarity functions. Existing smart contracts that do not use these functions with extra arguments will remain unaffected. However, smart contracts utilizing the extended functions will be incompatible with nodes not implementing this SIP.

The map function is actually variadic. But since we have epoch switches it would be ok, albeit confusing.

So in terms of your map example: what you want is already possible, you just have to repeat your input a bunch of times.

(define-private (multiply-by-n (x int) (n int)) (* x n))
(define-private (example-map) (map multiply-by-n (list 1 2 3 4 5) (list 2 2 2 2 2)))

Imagine if we had lambdas. 😁

(define-private (example-map) (map (lambda ((x int)) (* x 2)) (list 1 2 3 4 5)))

[hugocaillard]

The map function is actually variadic. But since we have epoch switches it would be ok, albeit confusing.

It would be super confusing for map indeed. To know which arguments are from the sequences or the fixed ones. So this would need to be covered.

I really like the lambda idea as well. There is a proposal for lambda / local functions here: #56

[dreadful-dev]

The other option I was toying with was a single optional tuple that could effectively allow for any number of extra arguments within a single data type. The motivation for this proposal was primarily for developer ease of use, so I certainly don't want to make anything more confusing.

My other motivation was reducing costs (both deployment and runtime) associated with high level operations within clarity, such as filtering an item from a list. One of my assumptions was that deployment costs would be increased if one used a loop unrolling strategy to repeat an item in a list (in the case of the map function), and that runtime costs would be increased by using a tuple because of how values are accessed within tuples.

Is there a gas model for clarity available for me to evaluate some of my assumptions against?