Support back compat for ngram hash

[Ivanidzo4ka]

fixes #1982

[TomFinley]

foundColumnFeature [](start = 20, length = 18)

Were you intending to assert this somewhere? Also, could you please not use the ISchema derived functions, but the newer Schema ones? We're going to be getting rid of the ISchema ones sooner or later. In this case, preferred would be GetColumnOrNull.

machinelearning/src/Microsoft.ML.Core/Data/Schema.cs

Line 64 in 7e21afa

public Column? GetColumnOrNull(string name)

[TomFinley]

Thank you @Ivanidzo4ka !

[justinormont]

Thanks for the new test! Should we be testing the specific output values too? (or is this somewhere I'm not seeing it?)

[justinormont]

Not to nickle-and-dime, but is 273KB needed to add to the repo? Existing repo zips to 18.3MB. Seems a simple ngram-hash model should be tiny.

Looks like space in your model is from an ngram dictionary & normalizer. Perhaps set bits=10 & max=100 to minify.

Command: (now 18KB)
maml.exe Train loader=TextLoader{col=Sentiment:R4:0 col=SentimentText:TX:1 header=+} data=D:\src\fork-machinelearning\test\data\wikipedia-detox-250-line-data.tsv out=d:\tlc\1.model.zip xf=CopyColumns{col=Label:Sentiment} xf=WordBagTransform{col=Column:SentimentText tok=WordTokenizeTransform weighting=TfIdf max=100} xf=CategoricalTransform{col=Column kind=Key} xf=NgramHashTransform{col=Features1:Column all=- rehash=+ bits=10} xf=NgramTransform{col=Features2:Column max=100} xf=Concat{col=Features:Features1,Features2}

[sfilipi]

"wikipedia-detox-250-line-data.tsv" [](start = 51, length = 35)

should we stick to TestDatasets.Sentiment.trainFilename

[sfilipi]

VersionTransformer [](start = 26, length = 18)

it would be more intuitite, i think, if called: CurrentVersion.

[TomFinley]

The trouble with naming it current version is that it is not a descriptive name, and it could change in the future. it is better to describe what changed then. The reason is, if we think about how these are used, they are used in the conditional checks during loading to do this or that. So, if we had some model format change in such a way that we added some information "Foo," then we should name that VersionFoo or something, so that the inevitable conditional test, if (header.Version >= VersionFoo) or what have you, makes sense. That would not if it was named CurrentVersion. It would also lead to bugs, since people might say, "hey, the current version changed, I'll change this." But it is absolutely essential that they do not, since we are using that field for a very specific test. So it is absolutely essential that it not be named something like CurrentVersion.

[sfilipi]