CMB2 and Gutenberg, JSON, Nested Data

[rogerlos]

This is a proposal, not a bug, as such.

So here's a wee idea for CMB2 (3?) which would allow it to be useful in the theoretical post-metabox WordPress 5.0 world. As a bonus (?) it would make CMB2 a go-to solution for anyone dealing with JSON configuration data in WordPress.

Any reference to "group" or "box" below should be seen as interchangeable.

Background

I was working on a CMB2 plugin which handles nested groups (and string indices on groups and fields), but realized the level of "hacks" needed was ridiculous when CMB2 really isn't too far away from being able to do this already. The following is what I think would be needed for CMB2 to be able to deal with this type of data structure:

"core": {
    "stuff": {
        "repeating1": {
            "do": "this",
            "if": [
                "whatever",
                "oops"
            ],
            "hm": "yep"
        },
        "repeating2": {
            "do": "another",
            "if": [
                "whatever",
                "oops"
            ]
        },
    },
    "version": "1"
},
"actions": [
    {
        "id": "myid",
        "got": "example"
    },
    {
        "id": "myid2",
        "got": "example2"
    }
]

A lot of the time, JSON of this nature can be quite deeply nested, and (for me) encountering JSON which is only a couple of nesting-levels deep is pretty rare.

Premises

• WordPress is moving away from metaboxes (WordPress 5.0 and Gutenberg seems to be using JSON-type configuration in addition to ditching metaboxes).
• Complicated and deeply nested JSON (or similar) is an increasingly common and expected way to encounter data.
• CMB2 would still a valuable library for managing user admin input in a post-metabox world due to its flexible field types and other great features.

CMB2 Now

• PHP: CMB2 is hard-wired for a very specific data structure, essentially:
  $my_box[ $my_group ][ $group_iterator ][ $my_field ][ $field_iterator ]
• PHP/JS: CMB2 cannot use string indices for groups or fields, which is essential for handling JSON
• JS: CMB2 spends a fair amount of effort at what amounts to avoiding iterators on group items.

So, We Can Say...

• CMB2 at its core is a great field-rendering engine.
• Fields shouldn't care about their parent container.
• "Groups" and "metaboxes" are basically the same, the main difference is their markup (assuming a post-metabox world).
• Numeric iterators are an internal convenience (and sometimes match "real-world" data structures), but limit how data can be saved; they also complicate CMB2's Javascript.
• Current CMB2 box configuration is conceptually similar to JSON Schema

Goals

• CMB2 fields (and their great flexibility, established behavior and codebase) remain largely intact.
• CMB2 should not really care much about the incoming or outgoing data format, as long as it can identify fields and render/save them (ie, provide an interface and then a specific data model implementation, such as JSON or YAML)
• CMB2, both when saving and when reading, separates the data (with various unpredictable indices) from the data's schema, which allows CMB2 to not care about the indices.
• CMB2 data can be create, import or export data as (for example) JSON, complete with schema.
• No special mechanism is needed for repeating boxes/groups/blocks/etc
• CMB2 admin forms use simpler jQuery methods for group/repeating

Suggested Modifications

This seems intimidating, but a lot of the pieces are already in place within CMB2.

Abstract the Display of CMB2 Objects

I believe that this might simplify CMB2, as each box only needs to carry information about itself and not care about where its being presented or where its data is being saved as long as a parent is carrying that information.

• Use an abstract class or interface to control the output of boxes.
• Route individual boxes to the most appropriate display model, which returns markup: Is this presented as a metabox? As a group? As a Gutenberg block? As a page? As a Gutenberg container? As a custom something or other? Who cares! Get back an vprintf ready string (or similar) and plugin in the box data.

CMB2 Objects Can Have Parents

• Introduce the concept of "parent" to the CMB2 model.
• Groups become CMB2 instances, stored in CMB2_Boxes, potentially with other CMB2 objects as parents.
• CMB2 Objects inherit their parents characteristics, allowing re-use of boxes in various contexts. The most distant ancestor can dictate where the data is ultimately saved, for example, without requiring each box to declare that as well.

CMB2 No Longer Cares About Indices

Well, it cares, but the actual indices themselves are stored on a per-CMB2 instance basis and can simply be retrieved for saving/rendering purposes.

• Because the schema is separate from the data itself, CMB2 doesn't need to rely on the indices to try and parse what it is.
• An internal method and property can keep track of the specific index for a specific CMB2 instance.
• When rendering or saving a field, calling something like $field->get_indices(); would trigger a step up through any/all parent CMB2 boxes and return something like [root_example][hello][][another][howdy]

CMB2 JavaScript Definitely Doesn't Care About Indices

I have working JS which allows the manipulation of nested elements in CMB2, though it's meaningless without matching PHP code. It does this by exchanging "greedy" parent and child selectors for non-greedy versions. You can have repeating groups inside repeating groups inside repeating groups (ad naseum) and it will properly move/add/delete them and their children without affecting parents or siblings.

If we take our new CMB2 data model, our JS can do the following.

• Because parent and child selectors are/can be limited to the same nesting level, swapping rows is a doddle, and can easily accomodate drag and drop or a very quick and simple jQuery call
• Adding a new repeating item is an ajax call which asks CMB2 to return an empty child object, built from the schema.
• Duplicating is a clone() call. If the item has a string index, ask for a new one when the clone button is pushed, or do a WP "-1" thing with the existing index.
• All numeric indices are represented by empty brackets in name attributes. They can also be hot-swapped since there is an easy this reference to the particular row
• Text indexes are shown in interface, clicking allows editing.
• On submit, as form is serialized, empty brackets are populated with incrementing numeric indicators. Because the serialization of the form preserves the order, CMB2 doesn't have to try and keep track as the form is manipulated by the user.
• This also allows easy sending of the CMB2 form data as a JSON object for very complex forms.

As a bonus, this eliminates the current error situation where swapping rows with repeating elements can see data loss (because of how CMB2 tries to avoid fiddling with iterators).

Saving Data

When a form submission is handed to PHP, CMB2 should not care about the various indices, allowing the user/JS to set them. All CMB2 cares about is where the data is in the schema, and what the original "root" (ie, lowest level sent to form) indices were.

• Step 1: Remove original root indices from WP.
• Step 2: Save form contents at proper schema point.

Example: we've presented one of the "repeating" objects from my bogus JSON at the top of this post in an admin form. Processing that submission would be something like (ignoring index collision testing, normal sanitizing, etc):

• Step 1:
  unset( $myoption['core']['stuff'][ $post_original_key ]);
• Step 2:
  $myoption['core']['stuff'][ $post_key ] = $post_array;
update_option( 'myoption', $myoption )

Existing Resources

I am fairly confident a PHP/JSON library could be leveraged to give a leg-up on the data abstraction bit. My recent pull request probably contains some code which could be used as a basis for a "group/box" display class.

Conclusion

This is a fair amount of work. But: It would make CMB2 a lot more flexible and allow it to stay relevant in a Gutenberg world. Deeply nested data is reality for some of us (and may be for all of us post WP-5.0). Having CMB2 capable of handling that data would be incredible.

(I'd be willing to put a fair amount of work into this, as I would use it constantly. Most of my use of CMB2 is in configuring complicated one-off plugins. For example, when configuring a bespoke "SSO" plugin which allowed WP to log in users via a legacy internal API (with the data sent via URL payload), I ended up with eight option pages, with three to nine tabs each, to give them the flexibility needed to work across their WP installs.

I would be interested on feedback from @sc0ttkclark who I think is using CMB2 [in the pods project] to handle complex data structures.)

[scottsawyer]

I think this would be a fantastic step toward modernizing CMB2 and using it for managing flexible structured data. One of the recurring issues ( #565 , #543 , #1065 , and probably others ) is the topic of nested groups / repeaters.

I think an interesting addition that the changes you are proposing would be adding a selectable group type of control. In the Drupal space, the Paragraphs project ( https://www.drupal.org/project/paragraphs ), you can set up "container" paragraphs ( something like Groups / Box concept ) which contain fields. If multiple paragraphs are configured, the editor can select between any number of paragraphs.

Let's say we have 2 groups:
Group 1 ( 2 Columns, each with 2 fields ):

• Group 1.1
  • Field 1 ( Title )
  • Field 2 ( Text )
• Group 1.2
  • Field 1 ( Image )
  • Field 2 ( Text )

Column 1	Column 2
Title	Image
Text	Text

Group 2 ( 3 Columns, each with 2 fields ):

• Group 2.1
  • Field 1 ( Title )
  • Field 2 ( Text )
• Group 2.2
  • Field 1 ( Image )
  • Field 2 ( Text )
• Group 2.3
  • Field 1 ( Image )
  • Field 2 ( Text )

Column 1	Column 2	Column 3
Title	Image	Image
Text	Text	Text

( I realize I am displaying more of an implementation detail, but using CMB2 for layout control is a realistic use-case )

Then an Add Group button would allow you to select one of the group "types" configured for that post type, and you can add as many as you want of that type. It would actually be a metabox field of Groups, a reference to a Group configuration.

This would allow for creating complex data structures in a manageable way. I am kind of sad to see this issue hasn't received much attention in the last year.