Persistence API

[gabizou]

Persistence API introduces the common data structure to represent any possible data produced by the SpongeAPI.

This adds interfaces for DataContainer, DataView, and DataOptions.

Using a DataContainer is very simple:

    @Subscribe(order = Order.FIRST, ignoreCancelled = false)
    public void onPlayerInteract(PlayerInteractBlockEvent event) {
        Optional<ItemStack> itemOption = event.getPlayer().getItemInHand();
        if (itemOption.isPresent()) {
            DataContainer itemContainer = itemOption.toContainer();
            MyCustomDataManager.logItem(event.getPlayer(), itemContainer);
            Optional<List<String>> loreOptions = itemContainer.getList("lore");
            if (loreOptions.isPresent()) {
                for (String string : loreOptions.get()) {
                    System.out.println(string);
                }
            } 
        }
    }

Being that these data structures are map based, serialization to various data handlers, such as NBT, flat file, SQL, etc. is possible.

As well, adding annotations for SerializableAs and DataPath to mark methods and fields with keys that can easily be referenced for automated serialization (example: Gson).
Example usage:

@SerializableAs("Banner")
public interface BannerData extends DataSerializable {

    @DataPath("id") String getId();

    @DataPath(collapse = true) Vec3i getPosition();

    @DataPath("base") DyeColor getBaseColor();

    @DataPath(key = "patterns") List<BannerPattern> getPatterns();


    @SerializableAs(key = "pattern", compoundable = true)
    interface BannerPattern extends DataSerializable {

        // Since banners use dye to create their colors
        @DataPath("color") DyeColor getColor();

        @DataPath("pattern") List<String> getPatterns();

    }


  void setPosition(Vec3i pos);
  void setColor(DyeColor color);
  void setPatterns(List<BannerPattern> patterns);
  void setPattern(BannerPattern pattern);

}

@SerializableAs(key = "position", compoundable = true)
public class Vec3i implements DataSerializable {

    @DataPath("x") int x;
    @DataPath("y") int y;
    @DataPath("z") int z;

}

Finally, displaying the importance of the viability for BannerData as a DataContainer, the following would exist:

"Banner" // Name of the DataContainer
    |- "id" -> String
    |- "x" -> Integer
    |- "y" -> Integer
    |- "z" -> Integer
    |- "base" -> DyeColor
    |- "patterns" -> List<BannerPattern>
         |- "BannerPattern" -> BannerPattern
         |    |- "pattern" -> String
         |    |- "color" -> DyeColor
         |- "BannerPattern" -> BannerPattern
         |    |- "pattern" -> String
         |    |- "color" -> DyeColor
         |- "BannerPattern" -> BannerPattern
              |- "pattern" -> String
              |- "color" -> DyeColor

As visible in the example, the map based data structure, DataContainer#getString("id") would return the actual string id provided from the BannerData and likewise,
DyeColor color = BannerData.serialize().getList("patterns").get(0).getSerializable("color", DyeColor.class);

In essence, we can now safely use these maps to not only serialize and deserialize SpongeAPI objects, but internally, we can use various implementations to not just translate, but manipulate similar data structures in NBT (like in this case, banner data).

Likewise, all of the DataContainer system is designed to allow serializing EntitySnapshots for various reasons.

ToDo:

• Abstract data structure to represent and store data
• Serialization interfaces for Systems that can persist said abstract data structure
• Offline Chunk Iterators

[gabizou]

With the changes, I used @gratimax's BannerData example for a possible comparison to #253 in regards to creating an NBT library based on this Persistence API which would primarily reside in the Sponge implementation.

[octylFractal]

👍

[ZephireNZ]

Nice, I really like this. Bukkit had serialization, but oddly it separated it from NBT, making converting from one to the other difficult.

[ST-DDT]

This look very nice.
Two things for discussion:

1. a getList(path,class) : List method which can be used to shorten the Banner example.
2. make DataPath key optional to allow usage of the fieldname as key.

[gabizou]

2. make DataPath key optional to allow usage of the fieldname as key.

Why? That defeats the purpose of telling developers what keys to use for getting certain data.

1. a getList(path,class) : List method which can be used to shorten the Banner example.

I don't understand what this would do.

[ST-DDT]

1. This method will convert the entire list of data to the specified list. Combining getSerialized and getList.
   This way you could use getDarkest(banner.getSerializedList('patterns', BannerPattern.class)) : BannerPattern without having to implement either the conversion yourself or the method a second time for the dataview as args.

2. In most cases the DataPath key will be the same as the fieldname. So there is no real benefit in stating it twice. But no issue either.

[maxov]

getSerialiable -> getSerializable

[ST-DDT]

Will not work if path does not contain any separators.

[gabizou]

Updated the main comment with a quasi todo list.

[riking]

As this is a snapshot, all serialized data is thread-safe and may be stored.

[gabizou]

@SpongePowered/developers would like some more input on this PR to get it underway.

Currently, we're pondering options on removing the annotations to reduce the "magic voodoo" of annotations. Essentially, we'd be leaving the entire decision making of the tags to the implementation and just state that the data objects translate into DataViews/Containers.

[Lunaphied]

Hmm, I liked the Annotations, they seemed much nicer and conveyed more information about what's going on. @gabizou can you please show the Banner example without tags for some context?

[me4502]

I don't think giving the implementation most of the control is a good idea

[octylFractal]

I'd love to keep the annotations around, they're not that magic.

[gabizou]

@modwizcode the BannerData would look exactly the same without annotations:

public interface BannerData extends DataSerializable {

    String getId();

    Vec3i getPosition();

    DyeColor getBaseColor();

    void setBaseColor(DyeColor color);

    List<BannerPattern> getPatterns();

    void addPattern(BannerPattern pattern);

    void setPatterns(List<BannerPatterns> patterns);

    interface BannerPattern extends DataSerializable {

        DyeColor getColor();

        List<String> getPatterns();

    }
}

[gabizou]

I'd love to keep the annotations around, they're not that magic.

They're somewhat magic when it comes to debugging why something is breaking in the serialization process because the annotation doesn't exist. Plus it makes writing future parts of the API a bit convoluted and dependent on the Minecraft implementation.

[progwml6]

the annotations do have some uses when the serialized fields need to have different names that can't be java method names

[octylFractal]

@progwml6 I think the idea is that the implementation would define the names, not the methods. This itself is a bad idea, because now we can't have custom classes.

[progwml6]

the annotations need to at a minimum alert the implementation that a field/method requires a specific name in order to get some information from a forge mod, etc.

[sk89q]

While I may sound like a broken record, I like how Jackson does it:

http://wiki.fasterxml.com/JacksonAnnotations

• Jackson lets you configure it to use fields, getters, setters, or explicit annotations only
• @JsonAutoDetect to override the above configuration on a per-class basis
• @JsonProperty to override the name of a property
• @JsonTypeInfo to support polymorphic type handling
• @JsonSerialize to specify a specific serialization routine for a given class
• @JsonDeserialize to specify a specific deserialization routine for a given class

Jackson is a pretty powerful and big package, but those are the features most relevant to us in my opinion. The polymorphic type handling would be nice, but whether it's worth the effort in our specific case is another question (although it is pretty handy when working with real use cases).

In addition, Jackson lets you configure many of those things without needing to actually annotate the class.

[gabizou]

Just my opinion:

Pros of using Jackson annotations

• Readily available library for automatic serialization
• Serialization is automatic (no need for serialization methods)
• Safe serialization to JSON/NBT

Cons:

• Possible incompatibilities with other data structures that do not follow the NBT structure
• Leaks implementation
• Debugging is still made difficult due to Jackson 'magic'