Task Queueing

[modora]

The current implementation runs tasks in a predefined sequence. While this is functional, the goal is that the end user does not have to meddle with the task sequencing. The proposed alternative method is to implement a event-driven task sequencer. This requires an event handler for the plugin dispatcher.

Using an event system means that specific events need to be defined. Here are the options for an event based system:

1. Bag of events
2. Central event handler

The problem with the first implementation is that the events are seemingly random and there is no logic to prevent a plugin from firing if need be. For instance, a checksum plugin has no event dependencies so it will fire immediately. However, another plugin might fire and modify the file. We need some way to control when tasks are to be executed.

The issue with the second method is that events need to be predefined. I don't want to restrict the developers to working with a finite set of events.

A proposed compromise is to implement the handler but have the event set be extendable with another plugin system. If a developer feels that no event matches their needs, create their own and add it to the pool.

This approach has its own issues. One of the immediate issues I can see is if your plugin has a custom dependent event, how will that dependency be met. Other devs would have to support that event which is not going to happen... More likely, there would have to be many built-in events that get extended over time. This breaks the project philosophy of "simple extensions." It should be easy for someone to come in and make their own plugins.

[modora]

Some issues with adding include the fact that your plugin is not very extendable. Other plugins would have to support your events. Additionally, if I were a plugin dev, I would hate not knowing which events were available. Therefore, fixed events are necessary. To still allow for flexibility, how about create a fixed set of events for a given dispatcher and make dispatchers themselves a driver-typed plugin

[modora]

New idea. Rather than figuring out how to schedule tasks. Just let all tasks run in parallel until they reach a dependency clause. Most likely, the dependency is going to be some variable in the variable pool that has not yet been initialized. We can just add checkpoints throughout the task that saves the local variable state and if some dependency has not been met yet, then pause the execution and run another task. Once all tasks have been iterated through, come back to the paused tasks and try again. At this point, the dependent task has executed and it is possible that the dependent variable has been added (otherwise, that task is waiting for some other dependency). This process continues until all tasks have been run or until an infinite loop has been detected. We can check for an infinite loop by looking at whether the checkpoints in the all remaining, paused tasks are the same between iterations.

Here are some implementation requirements for this system:

1. Locks are needed on shared variables and the metafile
2. A checkpoint system that is able to continue execution of code at some arbitrary line