Gherkin - Token matching refactor

[ciaranmcnulty]

Summary

This refactors how Tokens are matched to reduce the amount of null checks needed

Details

Currently, Token has a bunch of match* fields that get initialised as null and then should be set by the TokenMatcher. By the time the gherkin document is being built we know they won't be null, but that isn't really enforced in the type system.

In the PHP implementation we run static analysis which flags up all these potential nulls, so we had to handle them explicitly. It seems the Java implementation suffers slightly less because of differences in how null is handled.

This change reifies the idea of matching as a new TokenMatch object. Rather than each field being individually nullable, a token has a match which is either null or a TokenMatch object. This means downstream during document building, we can deal only with TokenMatch, which has non-null fields.

A better approach may have been to model UnmatchedToken vs MatchedToken, but that would require more extensive changes to the Parser, which currently is mutating a token that is passed by reference. I'd be willing to have a stab at that instead.

This is only changed in PHP but I could port to Java (and look at any relevant other implementations)

How Has This Been Tested?

Existing unit tests are modified to reflect the new behaviour

Types of changes

• Bug fix (non-breaking change which fixes an issue).
• New feature (non-breaking change which adds functionality).
• Breaking change (fix or feature that would cause existing functionality to not work as expected).

Checklist:

• The change has been ported to Java.
• The change has been ported to Ruby.
• The change has been ported to JavaScript.
• The change has been ported to Go.
• The change has been ported to .NET.
• I've added tests for my code.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have updated the CHANGELOG accordingly.

[aurelien-reeves]

It looks really interesting 👍

I guess other implementation would benefit from such refactoring. Also, porting it to a well known implementation like ruby, js or java could also help us making sure it remains stable until the chain end

[aslakhellesoy]

I think this is a great improvement! Agree with @aurelien-reeves it would be desireable to make this consistent across implementations.

[ciaranmcnulty]

I'm happy to have a stab at Java in this branch... dotnet if I get brave enough