Modular movements

[TheDudeFromCI]

I believe it might be easier to implement a modular "plugin-like" system to movements to better configure movements, write cleaner API, and add more flexibility for new movement paths and styles.

For example, if the bot can mine, add the dig-based movements to the movements container. If it can't dig, don't load that module. This will actually result in fewer nodes overall by trimming out actions that can't be preformed.

Some example movement modules:

• Basic Movement
• Swimming
• Digging
• Placing
• Large Jumps
• Ladders/Vines
• Doorways
• Elytra (Way, way down the road)
• Custom Movements defined by the user such as:
  • Opening Chests to get more scaffolding blocks
  • Crafting a new pickaxe if one breaks

Each movement module will be in charge of actually executing the movement as well, which will keep the codebase clean and easy to read. All digging based behavior is kept in the digging module and nowhere else.

In addition, each module contains it's own config for stuff like dig cost as well as the overall config on the actual movements container specifying range and timeout like normal.

[TheDudeFromCI]

I can make a PR for this. To make a smoother transition, I'll start by having all current movements in a singular "core" module. These can be subdivided later down the road.

[Karang]

The idea I had for this was to make the movement generator completly data-driven (using .json to describe the possible movements). This approach works very well for protodef that we use for the protocol. We can first implement an interpreter that read the json and generate the moves, then if optimisation is needed, "compile" the generator like we did for protodef.

We could also take inspiration from https://github.com/CheezBarger/mineflayer-pathfinder that I discovered recently (In particular: https://github.com/CheezBarger/mineflayer-pathfinder/tree/master/DefaultConditions).

The really big advantage of having a data description of the movements is that we can write dumb rules and optimize them automatically (remove redundant tests, make a decision tree, etc..)

Of course what you said still apply: we can have different json files for each movements categories and enable/disable them.

For the movement execution: there is not that much special actions that can be performed: (move, jump, place, break, click, etc..) but we can combine them in different ways. I think it would be better to keep the actions concept separate from the move concept (a move is a combination of actions).

[TheDudeFromCI]

Oh, okay. I see what you're aiming for. Basically the config files contain sets of checks for specific block configurations to register available moves by compiling sets of actions. Actually, that's a pretty sweet idea. I like it.

If someone wanted to make configs detailed enough, you could use subpixel movements and complex parkour jumps and all that. I like the idea.

Make a configurable action which checks if the block in front of the bot is a door and then register an active action on the block. Or check if the bot is inside of a ladder block and allow vertical movements.

Is there and current implementation in the works, yet? I imagine the first step would be to make some simple Json files for basic movements, and then work on parsing those.

Can you create a new branch for developing this system that I can push to? Might take quite a few revisions to get working and fully functional, lol.

[Karang]

I made an example of what the json could look like some time ago:

const moves = [
  {
    // forward
    prerequists: [
      { delta: [1, 0, 0], condition: 'safe', canBreak: true },
      { delta: [1, 1, 0], condition: 'safe', canBreak: true },
      { delta: [1, -1, 0], condition: 'physical', canPlace: true }
    ],
    cost: 1,
    delta: [1, 0, 0],
    symetry: 'radial'
  },

  {
    // 1 gap jump
    prerequists: [
      { delta: [1, -1, 0], condition: 'safe' },
      { delta: [1, 0, 0], condition: 'safe' },
      { delta: [1, 1, 0], condition: 'safe', canBreak: true },
      { delta: [1, 2, 0], condition: 'safe', canBreak: true },
      { delta: [0, 2, 0], condition: 'safe', canBreak: true },
      { delta: [2, 0, 0], condition: 'safe', canBreak: true },
      { delta: [2, 1, 0], condition: 'safe', canBreak: true },
      { delta: [2, -1, 0], condition: 'physical', canPlace: true }
    ],
    cost: 2,
    delta: [2, 0, 0],
    symetry: 'radial'
  },

  {
    // forward jump up
    prerequists: [
      { delta: [1, 1, 0], condition: 'safe', canBreak: true },
      { delta: [1, 2, 0], condition: 'safe', canBreak: true },
      { delta: [0, 2, 0], condition: 'safe', canBreak: true },
      { delta: [1, 0, 0], condition: 'physical', canPlace: true, prerequists: [{ delta: [1, -1, 0], condition: 'physical', canPlace: true }] }
    ],
    cost: 2,
    delta: [1, 1, 0],
    symetry: 'radial'
  },

  {
    // forward drop down
    prerequists: [
      { delta: [1, -1, 0], condition: 'safe', canBreak: true },
      { delta: [1, 0, 0], condition: 'safe', canBreak: true },
      { delta: [1, 1, 0], condition: 'safe', canBreak: true },
      { delta: [1, 'landing', 0], condition: 'safe' }
    ],
    cost: 1,
    delta: [1, 'landing', 0],
    symetry: 'radial'
  },

  {
    // diagonal
    prerequists: [
      { delta: [1, -1, 1], condition: 'physical' },
      { delta: [1, 0, 0], condition: 'safe', canBreak: true },
      { delta: [0, 0, 1], condition: 'safe', canBreak: true },
      { delta: [1, 0, 1], condition: 'safe', canBreak: true },
      { delta: [1, 1, 0], condition: 'safe', canBreak: true },
      { delta: [0, 1, 1], condition: 'safe', canBreak: true },
      { delta: [1, 1, 1], condition: 'safe', canBreak: true }
    ],
    cost: Math.SQRT2,
    delta: [1, 0, 1],
    symetry: 'radial'
  },

  {
    // up
    prerequists: [
      { delta: [0, 2, 0], condition: 'safe', canBreak: true },
      { delta: [0, 0, 0], condition: 'physical', canPlace: true, jump: true }
    ],
    cost: 1,
    delta: [0, 1, 0],
    symetry: 'none'
  },

  {
    // down
    prerequists: [
      { delta: [0, 'landing', 0], condition: 'safe' },
      { delta: [0, -1, 0], condition: 'safe', canBreak: true }
    ],
    cost: 1,
    delta: [0, 'landing', 0],
    symetry: 'none'
  }
]

I was trying to get a feel about what tool would be needed to describe as much movements as possible. I got a bit stuck on how to specify parametric moves (like the fall move that have a while loop to get the landing block).

Indeed the first step is to get a very basic json and start experimenting. I'm ok with having both implementations next to each other in the master branch, as long as we have the possibility to choose the implementation (which is the case with the setMovements() function). This will simplify developments and comparison with the current system.

[TheDudeFromCI]

As for repeatable movements, like falling, this would be handled in the type. Since each action type would need to be coded in order to execute, (I.e, Move 1 block forward, Interact with Block, Place Block, etc), you can add a type setting to each action. This would work with your delta option to express how to perform it. For example,

{
	//...
	
	delta: [1, 1, 0], // Jump up onto block
	type: 'jump'	
	
	//...
},
{
	//...
	
	delta: [1, 0.5, 0], // Walk up halftile
	type: 'walk',
	halftile: true,
	
	//...
},
{
	//...
	
	delta: [1, 1, 0], // Walk up stair
	type: 'walk',
	stair: true,
	
	//...
},
{
	//...
	
	delta: [1, 0, 0], // Sneak forward one block
	type: 'sneak'	
	
	//...
},
{
	//...
	
	delta: [1, 0, 0], // Walk forward one block
	type: 'walk'	
	
	//...
},
{
	//...
	
	delta: [1, 0, 0], // Sprint forward one block
	type: 'sprint'	
	
	//...
},

This is a rough idea and would need a lot of tweaking, of course.

[Karang]

That's the global idea.
I'm not sure that sprint should be an action, because it doesn't make sense to sprint only on small distances, I see it more like a post-processing (see the free movement experiment), but that's a detail. (And walk up half a tile, is just move forward because the moves are adjusted by the physic, no need to make a special case)
One thing to note is that a move should always have a destination (because that's what is used to advance the astar algorithm). It can have as many actions as we want to reach this destination.

[TheDudeFromCI]

Yeah, I agree. I was just giving an example. Actions like sprint and sneak would be post-processing, yeah. I was just making an example as to how modifiers can be used to interpret delta in a specific way.

Also, isn't the delta the destination? The bot's position plus the delta (modified in running as needed for actions like falling) is the next node position. Right?

[Karang]

yes delta can represent a destination, but also a block position (to break or place)

[TheDudeFromCI]

That's all in the action. A move is a set of actions, so the move would have a delta and each action could have a delta.

{
	// Mine forward
	
	delta: [1, 0, 0]
	actions: [
		{
			type: 'mine',
			delta: [1, 0, 0]
		},
		{
			type: 'mine',
			delta: [1, 1, 0]
		},
		{
			type: 'walk',
			delta: [1, 0, 0]
		},
	]
}

However, the overall delta might be redundant, as it's litterally just the sum of all walking/jumping actions.

[Karang]

Yes, we can deduce the move delta from the actions. Do you want to start something in a PR ? Would be easier to talk with concrete examples and start testing.

[TheDudeFromCI]

Sure. I'll start a basic config and we'll go from there.

[TheDudeFromCI]

I submitted a PR for the first config proposal.

I just have one question: Why are we combining actions together for a single movement? Thinking about it now, it would mean that we would need to make quite a few different movements available for each possible combination of actions. And even if this was procedural, this means that each node generates quite a few neighbor nodes while pathfinding.

By splitting each action into its own movement, you'd get fewer nodes overall being generated. Plus, the nodes would be actually become more flexible and be able to perform significantly more complex behaviors naturally. Such as only mining blocks required to see a block to mine it without moving towards it. Or making a 1x1 tunnel and using a trapdoor to crawl through it. Or even replicate Baritone build schematics in only a few lines of code. Honestly, I could write down a dozen more.

What are the reasons for combining them?

[Karang]

The reason is to not confuse the astar algorithm. It search to minimise the cost function f=g+h with the idea that each move increase g by the cost of the move but decrease h (the heuristic) as it bring us closer to the goal. If a move only increase g but doesn't decrease h (such as a node containing only the action of breaking or placing a block) it will get assigned a low priority reseluting in being processed last. This basically defeat the whole purpose of astar and end up being too slow. In a perfect world we would also include the dig and place cost in the heuristic but that would result in too many computation.

This is why a move should always have a destination: because it is needed to change the heuristic (a move where you don't actually move, would simply be ignored by the algorithm until it runs out of options)

[TheDudeFromCI]

Ah, I see. Yes, when separating it, you would need to add changes to the world state to the hash of each block as it's technically a completely separate location. (Just in a higher dimension) It would work correctly with heuristics in that case but does make more dimensions to walk through which only are unless in specific situations.

Useful for making complex, intricate paths, but it would indeed complicate things a bit too much for standard pathfinding. It's something I would probably do with an injection approach from a separate plugin rather than including it in the base plugin.