Create FeatureVector from numpy array of RGB or HSV data

[whitews]

Hi,

I'd like to use the Python API to get feature metrics from color image regions in memory. I've been able to do this using grayscale data using the as_ndarray() method in PyImageMatrix. However, there doesn't seem to be a similar mechanism to expose _clr_plane data from ImageMatrix in the Python API?

Any chance this could be implemented?

[colettace]

This is a great enhancement request. The temporary work around is to write the RGB to temp file (hopefully memory mapped for speed) and load that. I'll put it on the todo list, but prob won't be able to get to it for a while. Also we <3 PRs if you wanna take a stab at it!

[colettace]

All but one of WND-CHARM's algorithms work on grayscale pixel planes (The exception being the Color Histogram). Unless you specifically ask for the Color Feature Set, WND-CHARM will take in an RGB TIFF and flatten it to grayscale according to the formula:

static inline double RGB2GRAY(const RGBcolor rgb) {
return((0.2989rgb.r+0.5870rgb.g+0.1140*rgb.b));
}

If you don't need the Color-specific features, but don't want to lose information in the flattening operation, you can treat the R, G and B channels as their own grayscale and run the battery of algorithms 3 times and end up with 3 times the features, and you can combine them into one big feature space on the back end. The literature explains many use cases like this.

However some people may want the Color Feature Set. The feature families that comprise the standard WND-CHARM Color Feature Set are as follows:

• Color Histogram on the raw RGB pixels
• Take 4 color-related image transforms...
  1. Color Transform
  2. Hue Transform (RGB2HSV, then take the H channel)
  3. Fourier of the Hue Transform
  4. Chebyshev of the Hue Transform
• ...and each of the following feature algorithms done on each of the image transforms listed above:
  1. Chebyshev-Fourier Coefficients
  2. Chebyshev Coefficients
  3. Zernike Coefficients
  4. Comb Moments
  5. Haralick Textures
  6. Multiscale Histograms
  7. Tamura Textures
  8. Radon Coefficients
  9. Fractal Features
  10. Pixel Intensity Statistics
  11. Gini Coefficient

For those of you who are working in Python and need to get RGB pixel plane information into WND-CHARM, I can recommend a work-around until this enhancement is implemented. In your own program do an RGB->HSV colorspace conversion, take the H channel, and pass it to WND-CHARM as a grayscale image in the standard way you would in Python. That will get you all the algorithms in the third bulletpoint above (and more) calculated on transforms 2, 3 and 4 in the second bulletpoint - almost all the way there.

A tip: when you combine the features from grayscale and HSV into one Feature Space make sure you change the name of the features coming out of that operation by placing the string "Hue ()" inside the innermost parentheses of each feature name. For example the feature name Fractal Features (Chebyshev ()) [19] would become Fractal Features (Chebyshev (Hue ())) [19]. Now all the features can exist in one matrix, be ranked and sorted, etc.

Just know that if you go this route, you would essentially be making your own Feature Set, which may not be compatible with the values you would have gotten had you wrote a temp file to TIFF and calculated color features that way. I recommend doing a statistical analysis to make sure the features are the same if you ever have an application that would use features calculated both ways.