New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is custom preprocessing available? #2492

Open
sob3kx opened this Issue Feb 10, 2019 · 2 comments

Comments

2 participants
@sob3kx
Copy link

sob3kx commented Feb 10, 2019

Hi,

I am working on productivization of Keras's VGG16 model in C#. The VGG16's weights are adapted from Caffe and demands turning image into BGR and substracting mean of channels over dataset from the channels of pixel. If my explaination is not understandable, you can see the code here in the first else branch of _preprocess_numpy_input.

Is there an option to do such thing in ML.NET? If not, is there an option to skip the graph definition of preprocessing steps and defining own preprocessing? If not, are there any plans for such feature?

Thanks

@Ivanidzo4ka

This comment has been minimized.

Copy link
Member

Ivanidzo4ka commented Feb 10, 2019

https://github.com/dotnet/machinelearning/blob/master/docs/code/MlNetCookBook.md#how-can-i-define-my-own-transformation-of-data
It's not optimal but doable.
Regaridng BGR order, our pixel extractor right now doing ARGB order, and it's possible what till 1.0 release it will remain the same, but as workaround you can always create three transforms which would extract Blue, Green and Red channel and then combine them together.

Also if your preprocessing step is just to offset by certain value, you can use offset parameter in transform (we also have scale if you need to scale your values)

      var pipeline = mlContext.Transforms.LoadImages(imagesFolder, ("ImageObject", "ImagePath"))
                          .Append(mlContext.Transforms.Resize("ImageObject", imageWidth: 100, imageHeight: 100))
                          .Append(mlContext.Transforms.ExtractPixels(new ImageAnalytics.ImagePixelExtractingEstimator.ColumnInfo("Green", "ImageObject",
                          colors: ImageAnalytics.ImagePixelExtractingEstimator.ColorBits.Blue, offset: 103.939f))
                          .Append(mlContext.Transforms.ExtractPixels(new ImageAnalytics.ImagePixelExtractingEstimator.ColumnInfo("Red", "ImageObject",
                          colors: ImageAnalytics.ImagePixelExtractingEstimator.ColorBits.Blue, offset: 116.779f))
                          .Append(mlContext.Transforms.ExtractPixels(new ImageAnalytics.ImagePixelExtractingEstimator.ColumnInfo("Blue", "ImageObject",
                          colors: ImageAnalytics.ImagePixelExtractingEstimator.ColorBits.Blue, offset: 123.68f)))
                          .Append(mlContext.Transforms.Concatenate("GRB", "Green", "Red", "Blue"));
@sob3kx

This comment has been minimized.

Copy link
Author

sob3kx commented Feb 11, 2019

Thanks,
the channels processing part seems to be doing good work, although I would probably need more "combine" kind of function, than concatenate. Concatenate will take 3 columns and concatenate them in [a, b, c] order, while for image classification you need a[0], b[0], c[0], a[1], b[1], c[1] ...

I am going to try to find out how to combine columns in such way tommorow and report back.

@Ivanidzo4ka Ivanidzo4ka referenced a pull request that will close this issue Feb 18, 2019

Open

[WIP] Introduce order for pixel extraction #2602

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment