Let mods intercept XNB content load

[Pathoschild]

Let mods override data loaded from the game's *.xnb data files. This could be used to edit data (e.g. add a new crop or change an item price), patch custom sprites into a spritesheet, and do interesting things like seasonal or contextual textures.

(Split from #173.)

Rationale

Mods which change images or content currently overwrite the game's *.xnb data files. These mods have major disadvantages:

• In SDV 1.2+, many of the XNBs have a different version for each language. That means XNB mods will only work when the language is set to English, unless they create a separate XNB replacement file for each language even for language-agnostic changes (like prices).
• XNB mods are hard to manage (since you need to keep track of which files were changed by which mod) and uninstall (since you need to recover the original game files).
• XNB mods conflict if they change the same file (which is common).
• XNB mods can't patch a file (only fully replace it), which exacerbates the previous point.
• XNB mods have no standard install process.

Use cases

These are the possible use cases for mods which override content:

use case	support in
add new entries to data content (new crops, NPCs, items, etc)	✓ 2.0
edit or replace entries in data content (edit crop seasons, item prices, etc)	✓ 2.0
patch image data (edit existing sprites, add new sprites)	✓ 2.0
add tilesheets to use in custom maps	✓ 2.0
edit map data (edit layout, add new logic, create new areas, etc)	✘ edit maps after they're loaded

[Pathoschild]

Proposed implementation

Based on discussion on the SDV Discord and in #173, SMAPI would...

1. replace the game's LocalizedContentManager with its own instance (using a change introduced for SMAPI in SDV 1.2.13+);
2. override the content manager's Load method;
3. raise a ContentEvents.AfterAssetLoaded event for mods to intercept;
4. write the modified data to the underlying content manager's cache.

The ContentEvents.AssetLoading event would pass an IContentEventHelper argument to event listeners:

/// <summary>Raised when an XNB file is being read into the cache. Mods can change the data here before it's cached.</summary>
/// <param name="sender">The event sender.</param>
/// <param name="e">The helper which encapsulates access and changes to the content being read.</param>
void ReceiveAssetLoading(object sender, IContentEventHelper content);

This IContentEventHelper would simplify changes to the content being loaded by providing methods to add/edit dictionary entries, patch images, etc.

Per discussion with @ClxS, overriding the content manager like this shouldn't cause any trouble with Farmhand compatibility. Farmhand doesn't have an equivalent API yet, but creating one like this is a possibility to be determined later.

Example usage

This code adds data for a crop (see #256 about unique ID handling):

public void OnContentAfterLoad(object sender, IContentEventHelper content)
{
   if (content.IsPath(@"Data\Crops")) // normalises the path before comparison
   {
      content
        .AsDictionary<int, string>()
        .Set(1000, "...crop data string...");
   }
}

This code makes all crops grow in any season:

public void OnContentAfterLoad(object sender, IContentEventHelper content)
{
   if (content.IsPath(@"Data\Crops")) // normalises the path before comparison
   {
      content
        .AsDictionary<int, string>()
        .Set((key, value) =>
        {
            string[] fields = value.Split('/');
            var newFields = new[] { fields[0], "spring summer fall winter" }.Concat(fields.Skip(2)).ToArray();
            return string.Join("/", newFields);
        });
      }
   }
}

This code inverts the colours of all textures:

private void OnContentAfterLoad(object sender, IContentEventHelper content)
{
    if (content.Data is Texture2D)
    {
        // get texture pixels
        Texture2D texture = content.AsImage().Data;
        Color[] pixels = new Color[texture.Width * texture.Height];
        texture.GetData(pixels);

        // invert pixels
        for (int i = 0; i < pixels.Length; i++)
        {
            Color color = pixels[i];
            if (color.A != 0) // not transparent
                pixels[i] = new Color(byte.MaxValue - color.R, byte.MaxValue - color.G, byte.MaxValue - color.B, color.A);
        }

        // set data
        texture.SetData(pixels);
    }
}

Pros & cons

Pros:

• ✓ Fairly easy to implement.
• ✓ Can recreate most XNB mods in code.
• ✓ Mod frameworks can add extension methods like content.AddCustomCrop(…).
• ✓ Mods can implement exotic features like seasonal sprites, location/time-aware dialogue, etc.
• ✓ Reduced conflicts compared to XNB mods, since (a) mods can change specific things instead of the entire XNB file, and (b) mods can generate unique IDs to avoid key conflicts.
• ✓ Simpler maintenance than XNB mods, since they won't need to be updated for every change to the XNB file.
• ✓ SMAPI mods can be added/removed easily; XNB mods can be difficult to uninstall unless you kept track of which files were edited and kept backups (or use an XNB mod manager).

Cons:

• ✘ Letting mods override content using a low-level API increases the risk of unique ID conflicts; see Manage unique IDs for custom content #256.

[AllTheGoodNamesAreTakenAlready]

Is ContentEvents.AfterEveryLoad mentioned in #173 still a possibility? At the moment I can force a specific asset reload using reflection, which has allowed me to do some interesting things (catch a fish, throw it in the farm pond, reload Data\Locations.xmb to change the fish data, now I have a pond full of Carp) but this feels hackish and I'm afraid it may have consequences that I'm not aware of yet (new to C#, know nothing about XNA)

[Pathoschild]

@AllTheGoodNamesAreTakenAlready It's still a possibility, but the initial version will probably focus on the easier on-first-load case.

[Pathoschild]

Proposed implementation B

Implementation

This is roughly the same implementation as proposal A, but uses interface instances registered through the content API instead of a content event. This approach is inspired by Entoarox Framework's content injectors and loaders.

Example usage

Explicit implementation

The mod would create an implementation of IAssetEditor to make all crops grow in any season:

/// <summary>Edit crop data so all crops grow in any season.</summary>
public class CropSeasonEditor : IAssetEditor
{
   /*********
   ** Public methods
   *********/
   /// <summary>Get whether this instance can edit the given asset.</summary>
   /// <param name="asset">Basic metadata about the asset being loaded.</summary>
   public bool CanEdit<T>(IContentMetadata asset)
   {
      return asset.IsPath("Data\Crops");
   }

   /// <summary>Edit a matched asset.</summary>
   /// <param name="asset">A helper which encapsulates metadata about an asset and enables changes to it.</param>
   public void Edit<T>(IContentHelper asset)
   {
      content
         .AsDictionary<int, string>()
         .Set((key, value) =>
         {
            string[] fields = value.Split('/');
            var newFields = new[] { fields[0], "spring summer fall winter" }.Concat(fields.Skip(2)).ToArray();
            return string.Join("/", newFields);
         });
   }
}

The mod would then add it through the content API:

helper.Content.Editors.Add(new CropSeasonEditor());

The mod could optionally remove it later using regular list methods:

helper.Content.Editors.RemoveAll(p => p is CropSeasonEditor);
helper.Content.Editors.Clear();

Lambda implementation

When a mod doesn't need the full flexibility of a content editor, it could use simplified methods to add editors. Here's the same example as above with a simplified method:

// make all crops grow in any season
helper.Content.Editors.Add(@"Data\Crops", asset =>
{
   content
      .AsDictionary<int, string>()
      .Set((key, value) =>
      {
         string[] fields = value.Split('/');
         var newFields = new[] { fields[0], "spring summer fall winter" }.Concat(fields.Skip(2)).ToArray();
         return string.Join("/", newFields);
      });
});

Extensions

We could optionally add extensions to further encapsulate common changes, like this:

// make all crops grow in any season
helper.Content.Editors.AddDataEditor<int, string>(@"Data\Crops", (key, value) =>
{
   string[] fields = value.Split('/');
   var newFields = new[] { fields[0], "spring summer fall winter" }.Concat(fields.Skip(2)).ToArray();
   return string.Join("/", newFields);
});

Pros & cons

This has the same pros & cons as approach A:

Pros:

• ✓ Fairly easy to implement.
• ✓ Can recreate most XNB mods in code.
• ✓ Mod frameworks can add extension methods like content.AddCustomCrop(…).
• ✓ Mods can implement exotic features like seasonal sprites, location/time-aware dialogue, etc.
• ✓ Reduced conflicts compared to XNB mods, since (a) mods can change specific things instead of the entire XNB file, and (b) mods can generate unique IDs to avoid key conflicts.
• ✓ Simpler maintenance than XNB mods, since they won't need to be updated for every change to the XNB file.
• ✓ SMAPI mods can be added/removed easily; XNB mods can be difficult to uninstall unless you kept track of which files were edited and kept backups (or use an XNB mod manager).

Cons:

• ✘ Letting mods override content using a low-level API increases the risk of unique ID conflicts; see Manage unique IDs for custom content #256.

Pros compared to proposal A:

• ✓ All content logic is available through the same API.
• ✓ Design supports asset loaders (which provide missing assets) in a future version. Since only one mod can load an asset, the multicast events in proposal A aren't a good match.

Cons compared to proposal A:

• ✘ More verbose implementation. This is addressed by the proposed lambda implementations and extensions, if they're implemented too.
• ✘ This would be the only part of SMAPI that uses interface/lambda injection, instead of API operations and events. (That isn't necessarily a blocker, but new APIs should be consistent and use the same paradigms as the rest of SMAPI where possible.)

[Pathoschild]

Tentatively scheduled for SMAPI 1.15 (internal implementation only). This will let Entoarox Framework work without overriding the content manager, which breaks SMAPI's content features.

[Entoarox]

@Pathoschild #BlamePathos for implementing a CM without even considering that EF did it first! :P