@botmation/core v2 - architecture, features, goals

[mrWh1te]

Goals

• stronger typing Upgrade pipeRunner typing #85
  • i.e. remove as SomeTypeOrInterface
  • have the Pipe value type carry (and change) through an assembly line, function by function
• Upgrade the injects system
  • replace spread array of injects for some kind of 1:1 map
  • related injectMap()() #58
• Decouple Puppeteer page from BotAction request() & pagelessPipe()() #54
  • therefore you'll need like puppeteerPage(page)(...actions) BotAction to support Puppeteer based bots by injecting page
  • opens door for other fits (i.e. API Router, general async functional programming library, etc)

chain(
   browser( // uses puppeteer to inject a browser instance
       page( // uses browser inject to inject a new page instance
           // ... actions that rely on `page` inject
       )
   )
)

Considerations
These ideas are under consideration (not committed):

• Support sync BotAction's Support sync BotAction's #41
  • improves performance by removing Promises when they are not necessary
• Improve discoverability (learning)
  • add a graph of relate-able data like tags/terms for a search component in the Documentation's sidebar to narrow down links by relevant content

Open to suggestions
Please share thoughts, ideas, concerns for @botmation/core v2

[mrWh1te]

A bot is a system

look for ways to build systems that can accomplish the same goals, with the same functionality, but composed differently i.e. different injects

Consider using puppeteer page vs a website's api, using the same anon functions, just composed differently, with different rudimentary botaction's (foundational level)

[mrWh1te]

If the pipe object were moved to the start of the BotAction params, before the spread injects (instead of after), as a new standard, the chain() BotAction could be deprecated, while enabling the pipe value to be injected directly, instead of wrapping it in a branded pipe object

downside is that without chain(), there is no assembly line that can run more effectively when not using pipe values in BotAction's assembled

What if chain(), by default, passed in an undefined initial param? That way BotAction params can have a consistent order, regardless of being ran inside a chain() or pipe()

leave the pipe value branded (wrapped) in a pipe object, than set a convention of using a helper function for "unboxing" the ...injects array along with the pipe object safely, by returning an undefined value if no pipe object exists in the injects. Similar to what's going on now.

what if BotAction injects was not a spread array, but just an array to destructure? Therefore, the type of the pipe value doesn't have to be lost when "appended" to a "spread" array of injects i.e. BotAction(page, injects?, pipeValue?)

[mrWh1te]

if injects is no longer spread array, it could be a Map / object that supports both looking up injects by key or by index

In that case, both options of array (number index) and Map (string keys) could be used to store and retrieve injects.

However, that could grow into an anti-pattern, therefore should probably stick with one or the other, and based on prevailing DI systems, a key based Map should scale better in terms of handling more complex behaviors

[mrWh1te]

if injects (not spread array) is a key->value dictionary, one can de-structure the object by key without an additional function

injects()() BotAction still works (with scope in layers) as long as injects is treated as immutable

[mrWh1te]

It's possible to consolidate injects and pipe value when it goes into a BotAction via the function's params.

In this way, a chain could be given a pipe value to then give to the 1st BotAction. Without adjusting the injects, this would consequently work like switchPipe()() too, because it would inject the same pipe value for each BotAction. It would consolidate the two BotAction's. Currently, switchPipe is expressive for this type of functionality, while chain is easy/clean runner of plain BotAction's without some kind of shared state via pipe value.

So in the end it could be a pure curry flow with one param, an unique injects map which interfaces uniquely with these higher order assembling functions (i.e. piping new values, adding deeper scoped injects, etc)

food for thought

[mrWh1te]

in this case, there could be a reserved key in the injects map like __value__ or something more expressive like pipe, depending on how the standard approach for unboxing/destructuring the injects object in a BotAction becomes

probably with a helper function

[ideation]

[mrWh1te]

Maybe what is ultimately injected is a plain JSON object to be destructured, but if need be, the assembly line running the actions can use a Map or something else, if more advantageous

i inner exchange the words assembler and assembly line from time to time. In the context of Botmation, it's referring to Assembly Line BotAction's, the main functions used to curry other botaction functions together

[mrWh1te]

With the new packages release, v4 plans for the original botmation package are slated for v2 of the new @botmation/core package

same upgrade plans, but different package name

[mrWh1te]

switching the injects system from a spread array to a key based Map has been moved from "consideration" to "included" in the v2 update

[mrWh1te]

what if pipe value was simply in the injects and you didnt have to return a value by a BotAction, to change it. Instead, its immutable, you simply next a new value so BotActions done in parallel can still work

then a BotAction can return something when it wants to signal something to the higher scope ie AbortLineSignal, etc

[mrWh1te]

could even use the injects to contain signals

therefore, BotActions no longer return anything - there is no need

[mrWh1te]

could include #39 into this

as state is a map of key->values that can be observed/reduced

[mrWh1te]

these ideas will effect what ultimately gets included in the Core package

puppeteer specific stuff can be moved into a new auxiliary package

[mrWh1te]

express API auxiliary package

[mrWh1te]

an end goal: bots communicating with each other, collaborating, work together to orchestrate their actions

[mrWh1te]

get rid of other types of BotActions like ConditionalBotAction through the use of stronger typing & generics

[mrWh1te]

injects being a Record-like type that appends key->value types as injects are added

[mrWh1te]

what about renaming BotAction to Action ?

[mrWh1te]

rename injects -> props ?

similar to React's FC syntax

[mrWh1te]

every branching BotAction (loops, conditionals, etc) return a value if last assembled BotAction returns

then chain()() can ignore it

[mrWh1te]

Branch v2-core contains working, without test coverage, alpha code for the described re-architecture of this discussion.

It focuses on two things

1. De-couple Puppeteer
2. Replace Injects with a de-structurable JSON object

Now devs can build other programs, ie API's, with Botmation. Puppeteer bots are opt-in. At its Core, Botmation no longer includes any Actions that use a Puppeteer Page or Browser. That has been separated into a new, to be published package @botmation/puppeteer.

Which leads to a new package naming issue. Existing auxiliary packages rely on Puppeteer. The shift away from pure Puppeteer integration makes it possible to build bots with some of these websites' public API's such as Reddit. Therefore, is it important to rename existing auxiliary packages with a puppeteer- prefix ie @botmation/puppeteer-reddit to differentiate from @botmation/reddit as a more core package that focuses on integrating with their API's?

[mrWh1te]

Once tests cover new code, a new @botmation/express package will be explored for building API's with Express using Botmation. Create Express based API's that activate Puppeteer bots via authenticated requests in a single chain - all readable, reusable, and reviewable (testable).

Note:
BotAction has been renamed to Action