Moving away from the scoreboard

[aksiome]

With newer versions of minecraft the scoreboard is becoming more and more obsolete. This thread is here so we can discuss about new possibilities while being really careful about performance penalty, limitations, and ease of use.

Here is my first proposal:

• Each feature has a bs:out and bs:in storage for example bs:out math.sin. The advantage is that results are not overwritten like with the bs.out.0 score, and that we are not limited to integer. We can have vector, floating point numbers, ...
• The bs:in storage is just here for ease of use meaning you can call a function using macro arguments and not provide the bs:in, this way people can copy data directly into it and have other default arguments like scale already setup
• Each function that only output a single value should also return it. Since we can only return integer when returning floats we should always consider having a macro argument named scale

Here is an example of what this could look like when using the library

# This are default inputs defined at __load__
# we also could have a global key that defines the default scale level for all functions that uses it
data modify storage bs:in math.sin set value {value:0d,scale:1000}
data modify storage bs:in math.cos set value {value:0d,scale:1000}

# When using the function with macro arguments and storing the result right away
execute store result entity @s Pos[0] double 0.1 run function #bs.math:sin {value:85.54d,scale:10}

# When chaining multiple operations and using the bs.in storage (ik this example is really dumb)
data modify storage bs:in math.cos.value set from entity @s Pos[0]
execute store result storage bs:in math.sin.value double 0.001 run function #bs.math:cos with storage bs:in math.cos
execute store result entity @s Pos[0] double 0.001 run function #bs.math:sin with storage bs:in math.sin

# When using vector output (this one is also quite stupid)
function #bs.vector:get_from_orientation
data modify entity @s Pos set from storage bs:out vector

Pros:

• It's a bit more easier to use and read. And in a lot of case it reduces the number of code the user has to write
• We are not limited to integer when it comes to inputs and outputs
• It reduces the scoreboard usage, and the number of objectives

Cons:

• Performance penalty of using macros
• With a single output we can't really do this anymore:
  execute as @e[type=creeper] run function bs.vector:get_from_orientation
  But I don't think it's really a problem. Nowadays @s is used a lot more than global context. I also think that the ease of use argument is not valid as it does not encourage best practices as you can see below.

# bad_example.mcfunction
execute as @e[type=creeper] run function bs.vector:get_from_orientation
execute as @e[type=creeper] run function bs.vector:do_whatever
execute as @e[type=creeper] run function bs.vector:do_whatever

# better_example.mcfunction
# also more performant since we don't have to filter through the entity list everytime
execute as @e[type=creeper] run function myawesome_whatever

# myawesome_whatever.mcfunction
function bs.vector:get_from_orientation
function bs.vector:do_whatever
execute at @s run function bs.vector:do_whatever

[Leirof]

I'm worried about performance issues, but also and especially, with accessibility. NBT are advanced datapacking notions.

I'm maybe biased because I'm still not confortable with NBT, but I think that if I'm in this case, I'm probably not the only one. Even scoreboard that are a lot more basic than NBT, I still remember that it was clearly not trivial at the begining...

[Leirof]

However, exploiting thes new arguments remain a good idea, but I think it would be better to use it as an alternative way of programing, such as bs.math:sin/macro

Afterward, if the user want to use the sin/macro function as the default one, he can overwrite the #bs.math:sin tag

[Ethanout]

I can't agree more

[aksiome]

Yeah i think it's a nice idea. When we'll feel comfortable about it we can switch the tags and deprecate the old system.

[theogiraudet]

Hey!
So, I'm also worried about performance, my first opinion about that is that we need to use macro only if it's really simplify the development of a function.
But the main problem about that is we lose consistency by having several way to input a value.

So here is my opinion about the current state:

• For me scoreboard are less usable than NBT as function parameters:

  • We use @s bs.in.[0-9] to pass data to a function. This solution lack of clarity because we don't know by reading the code what is the meaning of the parameter.

  • I find it hard to follow the flow of my program when I have several functions because all variables are global. Where my variable is initialized? At this specific line, which functions have modified this variable and after several function call, how can I be sure that I have the righ value in it? We need to be confident about other functions/users respect the convention.

  • The user needs to move output to input, input to data or even output to data in order to do operations which is not really obvious, at the risk of having their value overwrited if they don't move the values.

  • It's not always easy to see wich inputs are for wich function call.

  • We can only manage integer, nor arrays, string or floats.

  • Working only with score as input make the state management really hard, especially in the case of multiple recursive calls.
    Let's take an example, I want to compute in Minecraft the depth of a binary tree. We define the depth as the length of the longest path.
    We define a path as a path from the root to the a leaf.
    Here the function compute_depth in Python:

    def compute_depth(tree: Tree) -> int:
       if !tree:
         return 0
       depth_left = compute_depth(tree.left)
       depth_right = compute_depth(tree.right)
       return 1 + (depth_left if depth_left > depth_right else depth_right)
        

    Here the function bs.test:compute_depth in Minecraft, I suppose the tree is encoded as a NBT structure:

     $data modify storage bs:test tree set $(tree)
     scoreboard players set #result bs.out.0 0
     # I use tree.data to check if the current tree isn't empty
     execute unless data storage bs:test tree.data run return 0
     
     scoreboard players set #left_depth bs.data 0
     scoreboard players set #rigth_depth bs.data 0
     
     function bs.test:compute_depth with storage bs:test left.right
     scoreboard players operation #left_depth bs.data = #result bs.out.0
    
     function bs.test:compute_depth with storage bs:test tree.right
     scoreboard players operation #rigth_depth bs.data = #result bs.out.0
    
    scoreboard players operation #result bs.out.0 = #left_depth bs.data
    scoreboard players operation #result bs.out.0 > #right_depth bs.data
    scoreboard players add #result bs.out.0 1

    But this program... Doesn't work. Why? Because Python like other programming languages has a "context" where all variables of current scope are stored. When we enter in a new scope, a new context is added to the stack and when we go out a scope we remove the first context of the stack. In Minecraft, we don't have that with scoreboard. Indeed, after the second recursive call, #left_depth doesn't have the same value than before, that is a really big problem. And I didn't really see a dynamic way to resolve this issue with scoreboard only, except by simulating a stack and creating a new entity each time we enter in a new call. And for me, this is the bigger issue when we work with scoreboard as input: we don't have local context and this can be really difficult for many programmers. With macro variable, we have this "context" because each macro variable is specific to the current function call. If you check the code carefully, we also have the same issue with the storage. But in this case, it's easier to resolve the issue by creating an array and use this array as a stack. or by directly using the macro instead of placing the tree into a storage at the begining.

So, the scoreboard aren't a perfect solution. Can we do better with macro and storages?

• The main issue is about performance, we need to evaluate the loss of performance before choosing an approach. For me, it's the main issue and if the impact is really significant, I think we have to drop this idea and use macros only when it's not possible to deal only with scoreboards/storages or when it's much simple to use macros.
• While we are in the "macro world", we have the local context.
• We aren't restricted by the type of the input/output.
• We have parameters in the same line as the function call, that is easier to read than the old way.
• At this moment, the return command can only return integer, so to return a NBT compound we need to use a storage and fallback with several identical problems than scoreboards (readable issues, overlaping...).
  • It's easier to pass a NBT compound in a storage to a function.
  • But it's less conveniant to chain with arithmetic operations

I'm a little afraid of a mixed approach, as the library would lose its coherence.
I'm not a big fan of an approach like bs.math:sin/macro, for the same reason. Especially when some function cannot really have another approach than the macro approach (logs for instance).

[Leirof]

I'm not a big fan of an approach like bs.math:sin/macro, for the same reason. Especially when some function cannot really have another approach than the macro approach (logs for instance).

This has obviously to be explicited in the doc

[aksiome]

I was thinking that we could just check for arguments and fallback to the legacy / current behavior for now when the macro command cannot be executed. Then when we feel comfortable deprecate and remove the old syntax?

[theogiraudet]

I forgot one issue about macro:

• We don't have type checking, if a user give a string where we want a number, the function will not be executed.

[Ethanout]

Once marco io or scoreboard io is used, there may be some unnecessary conversion process if the user needs to compute the values themselves, which is my biggest concern

[Leirof]

This is why proposing two variants can be interesting... but also more boring to maintain ^^'

[Leirof]

Oooor, actually, the I used an argument that can be reversed again myself 😅

I said on Discord that the lib is mainly focused on accessibility. Performance is secondary. I also said that we can create function variation that use macro to have practical but not optimized functions... so we can reverse the logic and use the macro as the "classical" way, and create feature variation that use scoreboard to make a more optimised one like bs.math:sin/optimized

For the inner part of a feature, there is no real problem, we can optimize it as we can, it's just a problem of user interface

[theogiraudet]

I think we still need to do some benchmarks to see the difference about performance. But after that, if we consider this difference negligible so yeah I agree with that.

[aksiome]

Building on the first proposal, here is a revisited version where we keep the best of both worlds without loosing performance and introducing too much complexity. I think that if we follow strict rules, ease of use and global coherence should not really be an issue 😉

Interface

Storage:

Input: bs:in module.feature.{inputs}
Output: bs:out module.feature

Scoreboard:

Input: bs.in $module.feature.{inputs}
Output: bs.out $module.feature | $module.feature.{outputs}

Considerations:

• The scoreboard should mirror exactly the storage in order to keep things simple.
• Macros arguments should also mirror the storage. This way we can use storage input as macro arguments.
• Since we can return a single value but we can't have single value input with macros, we should use module.feature.value for single input in storage.
• That's also why the scoreboard output have multiple definitions, it should mirror the storage, for example:
  $module.feature when returning a single number value or $module.feature.0, $module.feature.1 when returning a vec2

Implementation

Feature structure:

Names here are only a proposition / placeholder. I hate the python convention and I can't find a way to name these better 😂

• feature/__macro__.mcfunction
• feature/__storage__.mcfunction
• feature/__score__.mcfunction (optional)

About complexity (how to not make a mess):

All methods should return a result if it is a single value.
We should keep the rule of conservation, inputs are never overwriten.
Foating point operations should have a common input parameter (accuracy?) to define either the scaling of returned integer or the accuracy in storage. When there is no such parameter we should try to always use the same scaling/accuracy, i was thinking 3 digits precision?
The __macro__ function always output in storage and can always take the storage input as the argument list. There are 2 ways to implement them:

• update the storage then return the __storage__ function (most simple but not most performant)

  $data modify storage bs:in module.feature.value set value $(value)
  return run function __storage__

• update the input score, run the __score__ method then return the result in the storage output (which add some complexity, and a little bit of code duplication, we write to storage output from the __macro__ and __storage__ functions)

  $scoreboard players set $module.feature.value bs.in $(value)
  # A little more complex when working with multiple outputs but still not much of an issue
  return run execute store result storage bs:out module.feature int 1 run function __score__

When we don't need the scoreboard to compute the result or it adds too much complexity we just omit the __score__ function.
If there is a __score__ function then __storage__ could be implemented this way:

execute store result score $module.feature.value bs.in run data get storage bs:in module.feature.value 1
# A little more complex when working with multiple outputs but still not much of an issue
return run execute store result storage bs:out module.feature int 1 run function __score__

By default all function tags are using the __storage__ method when we think that macro could replace them we just have to update the tags.

Pros:

• We are macro ready
• We can more easily chain operations
• We can return decimal which I think is better than returning scaled integer
• In some cases storage is inevitable which is not true for the scoreboard
• We still have access to the scoreboard methods when performance matter

Cons:

• Using the storage means a negligeable loss in performance
• Only advanced users or us internally are gonna use the scoreboard methods
• Maybe a bit harder to maintain but I think that it should not be a problem if we follow these rules

[Leirof]

The main problem I see with storage is that every single storage content is global, not proper to a specific entity. It implies that every function should be called globally, and then force the user to create he's own interface function if he want to use the function on several entities at the same time

Macros doesn't suffer of thes problems because we can directly inject scores in the JSON compound

Moreover, most of operation in a map are made on scores, so using storage as an interface require to often migrate data from score to storage. In my opinion, the global consistency doesn't justify to ask the user to do such efforts. Storage should then be used only when it is required

Finally, I'm a bit inconfortable with the idea of having several ways to call a function. It is nice because it let the user chose which method he prefere, but the counterpart is that the documentation will be a lot more complexe.... which is not good on the accessibility point of view