Progress Logs: The model, thoughts and brainstorm, issues (incoherent)

[Hamza-ye]

• The recursive form element structure is powerful, it allows efficient retrieval and manipulation of elements using their paths. This pattern fits well because we can resolve dependencies and update the UI dynamically and selectively much easier.

Dependencies/References:

any other elements, libraries, or tools that this depends on or interacts with.

• Rule Engine, Filtering, Parsing, Expressions Evaluation
• Abstract Form elements model
• logging
• Resolving dependencies | bind evaluation context | Action behaviors
• Scoped Dependency lookup
• Status Management and Minimizing UI’s rebuild

[Hamza-ye]

The current dependency look up: look for parent after reaching the element root parent it doesn't go down to look into siblings (not in the same path of the element), what i want to do regarding the look-up is implement something like Scope-based Dependency Lookup: Dependencies should follow a scoped lookup model, similar to variable scopes in programming languages, it starts by looking for the dependency in the local section, moves up to the parent, and finally checks global scope (siblings and root level).

Currently, I construct the elements in a tree like abstracted from model, all form elements having a common element with different specialized concretes all can be consolidated into a single root, a unified processing, unified enhancement. structuring with all elements starting from a single root node and traversing the tree both upwards and downwards from any node.

• organize the hierarchy in a way that supports navigating through different levels of scope while maintaining efficient access to sibling elements. A common root parent for all elements,

• A form element state (value, visibility, disable/enable, validity ...etc) can depend on other form elements state or the outside. (dependencies, and dependents || observers/listeners and notifiers.

• tree nested complexity while maintaining the state of each element.

[Hamza-ye]

quick Notes

• tree model: many issues in the previous model were resolved them without even realizing.
• dependencies resolving, tracking: looking up a form element’s dependencies based on its rules and other configuration and track these dependencies changing properties.
• resolving dependencies:
  • Resolve during instantiation (but some are not created yet).
  • Delay resolving until after all elements have been constructed, but some elements might
  • Lazy resolving: decide when an element wants to decide based on its dependencies that

Improvements/Next Steps, Potential Issues and Ideas:

• Form elements lifecycle management: form elements in the current model have a similarities with flutter widgets, especially the lifecycle.

• when a form element gets notified it reevaluate its status, sometimes it might be hidden, sometimes it needs to notify only if its state changed from the previous one. Sections, Repeat Sections, and Repeat Items are also form elements that group other form elements, they might have their rules, action, or dependency needing different strategy of binding, listening, or notifying their dependencies 👇🏻:

  • shouldNotify()
  • didUpdateDependency()
  • didElementUpdate()
  • …

• form Element have and onChangeDependency method that called when a change in dependncy.

void onDependencyChanged(String notifierName, dynamic value,
      {bool notify = true}) {
  ///.. implementation
  }

• onDependencyChanged much like didUpdateDependency in a Widget's state lifeCycle but didUpdateDependency have no arguments.

Dependencies/References:

any other elements, libraries, or tools that this depends on or interacts with.

• Rule Engine, Filtering, Parsing, Expressions Evaluation
• Abstract Form elements model
• logging
• Resolving dependencies | bind evaluation context | Action behaviors
• Scoped Dependency lookup
• Status Management and Minimizing UI’s rebuild

[Hamza-ye]

Key Challenges:

• Circular Dependency: Form elements often rely on each other's states for creation.
• State-Based Creation: Deciding whether to create a form control depends on its initial state.
• Conditional Visibility: Form elements may become hidden or visible based on various conditions.
• Widget Selection: Choosing the appropriate widget type and input actions.
• Cleanup and Disposal: Managing form controls when elements become hidden.
• Parent/Grandparent Section Rebuilds: Handling rebuilds properly due to state changes.

Specific Issues:

• An Element initial State based on the loaded data
• the order of form element creation.
• Constructing form controls before evaluating dependencies.
• Avoiding unnecessary widget creation.
• Fixing the keyboard actions dynamically.
• Cleaning up UI and model when elements become hidden.
• the behavior of form builders during rebuilds.

strategies to optimize form element creation and rendering.

[Hamza-ye]

New things I learned:

• I wAs confused about how the controller works. When I tried to make the state separate from the controller, This made the stream reactivity mess up the state changes I wanted. Like, if I set an element to be hidden and disabled, then made it not hidden anymore, it stayed disabled only.
• Making form controls before checking dependencies wasn't a good idea. I think we should make controls when we need them, but I gotta test this more to be sure.
• Right now, I put too many properties in the form element and ended up getting them from other places. I think be better if the model just had the properties that show up in the UI and affects the UI directly, it should be connected to that part. This way, we can make the form element model-state smaller.

Ways to make form element creation and rendering better:

• Accessors make it easier for form controls to work with the UI, especially for custom widgets or inputs. This helps separate complex UI stuff from handling form values.
• A special accessor could handle visibility and other properties based on the form control's state. This might make the code simpler.

Dependencies/References:

any other elements, libraries, or tools that this task depends on or interacts with.

• current commit
• Rule Engine, Filtering, Parsing, Expressions Evaluation
• Abstract Form elements model
• logging
• Resolving dependencies | bind evaluation context | Action behaviors
• Scoped Dependency lookup
• Status Management and Minimizing UI’s rebuild

[Hamza-ye]

I need to centralize event registration, deregistration, and cleanup for my form elements while traversing the tree and managing dependencies. The challenge is ensuring that when an element depends on another by name, it registers and listens to the closest one within its scope—especially when there are multiple elements with the same name, taking Scoped Dependency lookup into account.

I’m thinking about two approaches, but I’m not sure which is best:

1. Path-based resolution: Elements in my form come with a path, but expressions only reference names (not paths). I could resolve the path for each dependency upfront, during serialization or storage, ensuring that each element registers based on a unique path rather than just a name. This way, notifications would be sent specifically to listeners associated with that path.
2. Hierarchical dispatching: Another idea is to have the dispatcher send notifications hierarchically based on the path. Elements would register dependencies by name and provide their current path, and the dispatcher would resolve the closest dependency during the notification process.

I’m looking for fast-to-implement-and-expand-later ideas on how to best implement this while decoupling these mechanisms. Example:

{
    "form": "inventoryForm",
    "fields": {
        "name": "transactionInfo",
        "type": "Section",
        "fields": [
            {
                "name": "transaction",
                "type": "SelectOne",
                "listName": "transactionTypes"
            },
            {
                "name": "supplier",
                "type": "Text",
                "rules": [
                    {
                        "expression": "#{transaction} == 'supply'",
                        "action": "Show"
                    }
                ]
            }
        ]
    }
}

My current approach involves retrieving the dependencies at runtime using an extension on the FieldTemplate and the Rule:

extension FieldTemplateDependencies on FieldTemplate {
  List<String> get dependencies {
    List<String> dependencySet = [];
    for (final rule in rules) {
      final ruleDependencies = rule.dependencies;
      dependencySet.addAll(ruleDependencies);
    }
    return dependencySet.toSet().toList();
  }

  /// from the choiceFilter expression
  List<String> get filterDependencies {
    List<String> dependencyList = [];
    final fieldPattern = RegExp(r'#\{(.*?)\}');

    if (type.isSelectType) {
      if (choiceFilter != null) {
        final filterDependencies = fieldPattern
            .allMatches(choiceFilter!)
            .map((match) => match.group(1)!)
            .toList();
        dependencyList.addAll(filterDependencies);
      }

      return dependencyList.toSet().toList();
    }

    return [];
  }

  String? get evalChoiceFilterExpression =>
      choiceFilter?.replaceAll("#{", "").replaceAll("}", "");
}

extension RuleDependencies on Rule {
  List<String> get dependencies {
    if (expression != null) {
      final fieldPattern = RegExp(r'#\{(.*?)\}');

      return fieldPattern
          .allMatches(expression!)
          .map((match) => match.group(1)!)
          .toSet()
          .toList();
    }
    return [];
  }

  String? get evalExpression =>
      expression.replaceAll("#{", "").replaceAll("}", "");
}

• Maybe the easier and quicker to implement way would be to expand this extension extract and lookup the path or centerlize it in its own module, so instead of returning the supplier field dependency as [”transaction”] it returns [”transactionInfo.transaction”] ← nearest one in supplier path in case another one in another part with the same name exist, it will look recursively and get the nearest and then the field will register as listener for the path and dependency will notify with their paths instead of just the name.
• The field would then register as a listener for the path, and dependencies would notify with their paths rather than just their names.

[Hamza-ye]

notion’s AI pot Ideas about this 😃😋:

Your approach of expanding the extension to include path lookup is a good starting point and can be relatively quick to implement. However, there are some established patterns in software design that might provide more robust and scalable solutions for your centralized dependency management problem. Here are a few suggestions:

• Observer Pattern with Hierarchical Subject: This is similar to your hierarchical dispatching idea. Implement a hierarchical subject (the form) where each element is both an observer and a subject. When a change occurs, it notifies its parent, which then notifies its own observers. This naturally handles scoping and closest-match scenarios.
• Pub/Sub with Namespaces: Use a publish-subscribe system where topics are namespaced according to the element's path. Subscribers can listen to specific namespaces or use wildcards for broader subscriptions. This approach is flexible and can easily handle complex dependency scenarios.
• Dependency Injection Container: Create a DI container that manages dependencies based on both name and path. This centralizes dependency resolution and can be easily extended to handle complex scoping rules.

Your path-based resolution ideais actually quite close to these established patterns, particularly the Pub/Sub with Namespaces approach. It's a good direction that can be expanded upon.

For a quick implementation that can be expanded later, I would suggest starting with your path-based resolution approach. It's straightforward to implement and aligns well with your current structure. You can then evolve this into a more robust Pub/Sub or Observer system as your needs grow.

To implement this:

1. Modify your dependency extraction to include the full path of each dependency.
2. Create a central registry that maps full paths to elements.
3. When registering listeners, use the full path instead of just the name.
4. When notifying, traverse up the path to find the closest matching dependency.

This approach allows you to start simple and gradually introduce more complex patterns as needed, while addressing your immediate concerns about scoped dependency lookup.

[Hamza-ye]

Materialized Paths, for abstracting some of the complexities, if not all:

🤔

Due to the size of the project, sometimes I decide a goal and think of the details of achieving it, but while working towards that goal I get lost in the details and for a moment I forget the goal’s details I need to implement and end up working in a path that finishes far away from the goal

Modeling the tree structure of elements differently might simplify things:

**We can Model Tree Structures with**

• Parent References
• with Child References.
• with an Array of Ancestors
• with Materialized Paths

maintaining everything as tree with Child References as in the current model seems ideal, it naturally algins with the nested UI structures, makes each form element understands its hierarchy and dependencies clearly, and rule evaluation where parent-child relationships are involved is easier to implement.

However, as I delved deeper into the code I realized that managing deeply nested trees can get complex, especially with rule evaluations that don’t involve child-parent relationship.

I need balance…

• Sections and repeated sections primarily serve to organize and group fields, with limited additional functionality. They share only a few common behaviors with fields, such as visibility and properties that mainly affect the UI and the final structure of submitted data.
• Fields, on the other hand, are more complex—they depend on other fields and hold values. Attempting to conceptualize sections alongside fields introduced unnecessary complexity.

Storing fully re-constructable relationships using Model Tree Structures with Materialized Paths, seems to suite my case the most, but I don’t know the complexities that may arise down the road, the goal is partly clear in my mind and I partly thought about the details I need to implement to reach it.

Using materialized paths the previous example would become like:

{
  "form": "inventoryForm",
  "name": "inventoryForm",
  "fields": [
    {
      "name": "transactionInfo",
      **"type": "Section",
      "path": "transactionInfo",**
    },
    {
      "name": "transaction",
      "listName": "transactionTypes",
      **"type": "SelectOne",
      "path": "transactionInfo.transaction",**
    },
    {
      "name": "supplier",     
      "rules": [
        {
          "expression": "#{transaction} == 'supply'",
          "action": "Show"
        }
      ],
      **"type": "Text",**
      **"path": "transactionInfo.supplier"**    
    }
  ]
}

With a few tree-walking methods, this could abstract away many of the complexities i faced.. instead of tracking nested elements and parents as in the current model, it would flatten the tree's depth into fields and with the help of Path and ValueType for indicating the nature of each element.

more about it later....

---

Dependencies/References:

• Things to consider/Explore
• Quick notes about the form elements and their dependencies