1.0.0 Specification

[jamierocks]

Preamble

Lorenz has been in development now for over 2 years - in which time it has gone through a number of iterations and been used in countless projects. It has been refined to such an art that the readers in Lorenz feel galaxies apart from their counterparts in Nocturne. Despite the many leaps that Lorenz has made - no 1.x.x release has been made. I am making it my mission this year to address the lack of a Lorenz 1.x, and am working on a number of pre-1.0.0 interim releases.

Timeline

Please note that this is only a rough estimate of what the timeline will be like - I have a very busy year and may well not stick to this timeline particularly well. Having said that I have been generous on the later releases to attempt to compensate for that

• 0.1.0 - February 2018
• 0.2.0 - March 2018
• 0.3.0 - April/May 2018
• 0.4.0 - July/August 2018
• 1.0.0 - Late 2018

Required Features

There are a number of features that are required for a 1.0.0 release, though the eventually feature list may well be larger than this.

• Support for both reading and writing in the SRG format (including the two variants: TSRG and CSRG)
• Support for both reading and writing in the JAM format
• Support for both reading and writing in the Enigma format

Format Specifics

There are a number of things that need to exist in order to support the formats, and further complexities to support multiple, differing, formats side-by-side.

SRG

The SRG and variant formats support the following mapping types, with the given data:

• Class Mappings -> obfuscated name, de-obfuscated name
• Field Mappings -> obfuscated name, de-obfuscated name
• Method Mappings -> obfuscated name, obfuscated signature, de-obfuscated name

JAM + Enigma

Whereas both the JAM and Enigma formats support the following mapping types, with the given data (note the extra inclusions in bold):

• Class Mappings -> obfuscated name, de-obfuscated name
• Field Mappings -> obfuscated name, obfuscated signature, de-obfuscated name
• Method Mappings -> obfuscated name, obfuscated signature, de-obfuscated name
• Method Parameter Mappings -> obfuscated name, de-obfuscated name, index

These differences make it difficult to support the two mapping formats - especially in cases where it is desired to convert an SRG mappings file to an Enigma mappings file, for example. There are a few solutions that could be used:

• Introduce FieldTypeProviders that would find the type of fields without an associated type
• Introduce bytecode analysis to Lorenz

I won’t get into the arguments for and against the above solutions here, other than mention that I am swaying towards the former.

Incorporating into Nocturne

One key element of the Lorenz 1.0.0 release is making sure it is in such a state that it could be incorporated into Nocturne to replace the ageing mappings model that exists. I won’t reiterate any specific formats that need be supported here - however it is worth mentioning that Nocturne has a number of hooks within the mappings model. There are a few solutions that could be used:

• Convert the mappings model to be composed of interfaces, providing a default implementation
• Introduce an event bus throughout the model to allow for tools such as Nocturne to listen on specific events

Again, I will not comment on arguments for or against here - but I am swaying towards the former option.

Closing Note

There is no set-in-stone feature list for each pre-1.0.0 release currently, as I will select a number of features I wish to address in that development-cycle at the start of the cycle alongside a GitHub issue to track the progress made.

This specification exists to inform myself and others of what needs to be done for Lorenz 1.0.0, however the pre-1.0.0 releases will better inform me of specifics that I have not yet had to consider. Keep an eye out for my further, more-technical specifications for those releases.

[caseif]

Regarding support for information not contained by the SRG format, the simplest solution (IMO) would be to allow Lorenz to discover the types of fields dynamically from a provided binary. This would be relatively straightforward and introduce very few if any edge cases - I'm not aware of any project that makes use of fields with identical names. As for method parameters, this feature can simply be ignored, since it's not required by any present formats.

As far as integration into Nocturne goes, we should enter a period of preparation such that Nocturne's mapping model is isolated from any other components (something which should be done anyway for that matter), so that Lorenz can effectively serve as a drop-in replacement.

[jamierocks]

Regarding support for information not contained by the SRG format, the simplest solution (IMO) would be to allow Lorenz to discover the types of fields dynamically from a provided binary.

I had initially been against this, however given every use of Lorenz I had (and can think of) already does some form of bytecode analysis - I'm fine with it. Though Lorenz should not be opinionated with what representation of the bytecode is used (e.g. ASM or FF AST).

IMO the best solution is to go down the provider route, and require them when constructing the MappingSet.

As for method parameters, this feature can simply be ignored, since it's not required by any present formats.

Method Parameter Mappings are supported by both JAM and Enigma.

[jamierocks]

Just to provide a brief update, I'm still working towards a goal of releasing Lorenz 1.0.0 before the end of the year.

Lorenz has had a large amount of testing over the past few months, with extensive testing being done with Survey which have shown that Lorenz is reliable. When Lorenz releases 1.0.0, I will be able to do the same for Survey and replace the current system in Nocturne. I will also be able to introduce jar remapping into Nocturne very nicely though Survey.

Lorenz's API is fairly stable now, and there probably won't be any major differences between now and release - though that's not definitive. There will be additional features however.

[jamierocks]

Another update.

I'm on track to release 0.4.0 sometime this month, and though I'm planning for an additional 0.5.0 release prior to 1.0.0 - things are progressing nicely.

Lorenz now supports storing field types, and though I want to write some unit tests to test things out - implementing Enigma+JAM isnt going to be a hard task. I want to make further changes to the readers/writers before proceeding however to make for a nicer UX.

I have already done the ground work to support binary formats such as kin, though additional format support (kin, enigma, jam) will be for 0.5.0 - there will be modularisation :D

I'm still undecided on a few minor details, though Lorenz 1.0.0 could well be released by the end of August / early September - leading the way for its use in Nocturne 1.3.0 and Survey 1.0.0.

[jamierocks]

Everything here is done

When the API is stable, and me and minecrell are happy with it - 1.0.0 will be release 🎉