Add UIImage to RGB converter #8

Closed
mfentross opened this Issue Jan 8, 2016 · 6 comments

Comments

Projects
None yet
4 participants
@mfentross

Quoting the contribute page:

DeepLearningKit is missing basic conversion from e.g. UIImage to RGB (the example network supports 32x32x3 CIFAR RGB Image Format, but has no conversion from UIImage to it). Check out e.g. Drawing Images From Pixel Data – In Swift and Image Processing in iOS Part 1: Raw Bitmap Modification for inspiration.

@atveit

This comment has been minimized.

Show comment
Hide comment
@atveit

atveit Jan 21, 2016

Member

Hi @torb - do you have time for this?

Member

atveit commented Jan 21, 2016

Hi @torb - do you have time for this?

@several27

This comment has been minimized.

Show comment
Hide comment
@several27

several27 Feb 1, 2016

I guess something like that could be used.
I haven't created pull request, because I didn't know in which file put it.
Inspired by: https://github.com/scottsievert/swix/blob/master/swix/swix/swix/matrix/m-image.swift#L81

UIImage to four (R, G, B, A) float matrices converter:

func imageToMatrix(image: UIImage) -> ([Float], [Float], [Float], [Float])
{
    let imageRef = image.CGImage
    let width = CGImageGetWidth(imageRef)
    let height = CGImageGetHeight(imageRef)
    let colorSpace = CGColorSpaceCreateDeviceRGB()
    let bytesPerPixel = 4
    let bytesPerRow:UInt = UInt(bytesPerPixel) * UInt(width)
    let bitsPerComponent:UInt = 8
    let pix = Int(width) * Int(height)
    let count:Int = 4 * Int(pix)

    // Pulling the color out of the image
    let rawData = UnsafeMutablePointer<UInt8>.alloc(4 * width * height)
    let temp = CGImageAlphaInfo.PremultipliedLast.rawValue
    let context = CGBitmapContextCreate(rawData, Int(width), Int(height), Int(bitsPerComponent), Int(bytesPerRow), colorSpace, temp)
    CGContextDrawImage(context, CGRectMake(0, 0, CGFloat(width), CGFloat(height)), imageRef)

    // Unsigned char to double conversion
    var rawDataArray: [Float] = Array(count: count, repeatedValue: 0.0)
    vDSP_vfltu8(rawData, vDSP_Stride(1), &rawDataArray, 1, vDSP_Length(count))

    // Indices matrix
    var i: [Float] = Array(count: pix, repeatedValue: 0.0)
    var min: Float = 0.0
    var step: Float = 4.0
    vDSP_vramp(&min, &step, &i, vDSP_Stride(1), vDSP_Length(i.count))

    func increaseMatrix(var matrix: [Float]) -> [Float]
    {
        var increaser: Float = 1.0
        vDSP_vsadd(&matrix, vDSP_Stride(1), &increaser, &matrix, vDSP_Stride(1), vDSP_Length(i.count))

        return matrix
    }

    // Red matrix
    var r: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &r, vDSP_Stride(1), vDSP_Length(r.count))

    increaseMatrix(i)
    // Green matrix
    var g: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &g, vDSP_Stride(1), vDSP_Length(g.count))

    increaseMatrix(i)
    // Blue matrix
    var b: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &b, vDSP_Stride(1), vDSP_Length(b.count))

    increaseMatrix(i)
    // Alpha matrix
    var a: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &a, vDSP_Stride(1), vDSP_Length(a.count))

    return (r, g, b, a)
}

I guess something like that could be used.
I haven't created pull request, because I didn't know in which file put it.
Inspired by: https://github.com/scottsievert/swix/blob/master/swix/swix/swix/matrix/m-image.swift#L81

UIImage to four (R, G, B, A) float matrices converter:

func imageToMatrix(image: UIImage) -> ([Float], [Float], [Float], [Float])
{
    let imageRef = image.CGImage
    let width = CGImageGetWidth(imageRef)
    let height = CGImageGetHeight(imageRef)
    let colorSpace = CGColorSpaceCreateDeviceRGB()
    let bytesPerPixel = 4
    let bytesPerRow:UInt = UInt(bytesPerPixel) * UInt(width)
    let bitsPerComponent:UInt = 8
    let pix = Int(width) * Int(height)
    let count:Int = 4 * Int(pix)

    // Pulling the color out of the image
    let rawData = UnsafeMutablePointer<UInt8>.alloc(4 * width * height)
    let temp = CGImageAlphaInfo.PremultipliedLast.rawValue
    let context = CGBitmapContextCreate(rawData, Int(width), Int(height), Int(bitsPerComponent), Int(bytesPerRow), colorSpace, temp)
    CGContextDrawImage(context, CGRectMake(0, 0, CGFloat(width), CGFloat(height)), imageRef)

    // Unsigned char to double conversion
    var rawDataArray: [Float] = Array(count: count, repeatedValue: 0.0)
    vDSP_vfltu8(rawData, vDSP_Stride(1), &rawDataArray, 1, vDSP_Length(count))

    // Indices matrix
    var i: [Float] = Array(count: pix, repeatedValue: 0.0)
    var min: Float = 0.0
    var step: Float = 4.0
    vDSP_vramp(&min, &step, &i, vDSP_Stride(1), vDSP_Length(i.count))

    func increaseMatrix(var matrix: [Float]) -> [Float]
    {
        var increaser: Float = 1.0
        vDSP_vsadd(&matrix, vDSP_Stride(1), &increaser, &matrix, vDSP_Stride(1), vDSP_Length(i.count))

        return matrix
    }

    // Red matrix
    var r: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &r, vDSP_Stride(1), vDSP_Length(r.count))

    increaseMatrix(i)
    // Green matrix
    var g: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &g, vDSP_Stride(1), vDSP_Length(g.count))

    increaseMatrix(i)
    // Blue matrix
    var b: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &b, vDSP_Stride(1), vDSP_Length(b.count))

    increaseMatrix(i)
    // Alpha matrix
    var a: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &a, vDSP_Stride(1), vDSP_Length(a.count))

    return (r, g, b, a)
}
@ashmanov

This comment has been minimized.

Show comment
Hide comment
@ashmanov

ashmanov Feb 25, 2016

I've tried this code, but it wrote same values to each of RGBA channels. Here's the fixed code:

func imageToMatrix(image: UIImage) -> ([Float], [Float], [Float], [Float])
{
    let imageRef = image.CGImage
    let width = CGImageGetWidth(imageRef)
    let height = CGImageGetHeight(imageRef)
    let colorSpace = CGColorSpaceCreateDeviceRGB()
    let bytesPerPixel = 4
    let bytesPerRow:UInt = UInt(bytesPerPixel) * UInt(width)
    let bitsPerComponent:UInt = 8
    let pix = Int(width) * Int(height)
    let count:Int = 4 * Int(pix)

    // Pulling the color out of the image
    let rawData = UnsafeMutablePointer<UInt8>.alloc(4 * width * height)
    let temp = CGImageAlphaInfo.PremultipliedLast.rawValue
    let context = CGBitmapContextCreate(rawData, Int(width), Int(height), Int(bitsPerComponent), Int(bytesPerRow), colorSpace, temp)
    CGContextDrawImage(context, CGRectMake(0, 0, CGFloat(width), CGFloat(height)), imageRef)

    // Unsigned char to double conversion
    var rawDataArray: [Float] = Array(count: count, repeatedValue: 0.0)
    vDSP_vfltu8(rawData, vDSP_Stride(1), &rawDataArray, 1, vDSP_Length(count))

    // Indices matrix
    var i: [Float] = Array(count: pix, repeatedValue: 0.0)
    var min: Float = 0.0
    var step: Float = 4.0
    vDSP_vramp(&min, &step, &i, vDSP_Stride(1), vDSP_Length(i.count))

    func increaseMatrix(var matrix: [Float]) -> [Float]
    {
        var increaser: Float = 1.0
        vDSP_vsadd(&matrix, vDSP_Stride(1), &increaser, &matrix, vDSP_Stride(1), vDSP_Length(i.count))

        return matrix
    }

    // Red matrix
    var r: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &r, vDSP_Stride(1), vDSP_Length(r.count))

    increaseMatrix(i)
    min = 1.0
    vDSP_vramp(&min, &step, &i, vDSP_Stride(1), vDSP_Length(i.count))
    // Green matrix
    var g: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &g, vDSP_Stride(1), vDSP_Length(g.count))

    increaseMatrix(i)
    min = 2.0
    vDSP_vramp(&min, &step, &i, vDSP_Stride(1), vDSP_Length(i.count))
    // Blue matrix
    var b: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &b, vDSP_Stride(1), vDSP_Length(b.count))

    increaseMatrix(i)
    min = 3.0
    vDSP_vramp(&min, &step, &i, vDSP_Stride(1), vDSP_Length(i.count))
    // Alpha matrix
    var a: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &a, vDSP_Stride(1), vDSP_Length(a.count))

    return (r, g, b, a)
}

I've tried this code, but it wrote same values to each of RGBA channels. Here's the fixed code:

func imageToMatrix(image: UIImage) -> ([Float], [Float], [Float], [Float])
{
    let imageRef = image.CGImage
    let width = CGImageGetWidth(imageRef)
    let height = CGImageGetHeight(imageRef)
    let colorSpace = CGColorSpaceCreateDeviceRGB()
    let bytesPerPixel = 4
    let bytesPerRow:UInt = UInt(bytesPerPixel) * UInt(width)
    let bitsPerComponent:UInt = 8
    let pix = Int(width) * Int(height)
    let count:Int = 4 * Int(pix)

    // Pulling the color out of the image
    let rawData = UnsafeMutablePointer<UInt8>.alloc(4 * width * height)
    let temp = CGImageAlphaInfo.PremultipliedLast.rawValue
    let context = CGBitmapContextCreate(rawData, Int(width), Int(height), Int(bitsPerComponent), Int(bytesPerRow), colorSpace, temp)
    CGContextDrawImage(context, CGRectMake(0, 0, CGFloat(width), CGFloat(height)), imageRef)

    // Unsigned char to double conversion
    var rawDataArray: [Float] = Array(count: count, repeatedValue: 0.0)
    vDSP_vfltu8(rawData, vDSP_Stride(1), &rawDataArray, 1, vDSP_Length(count))

    // Indices matrix
    var i: [Float] = Array(count: pix, repeatedValue: 0.0)
    var min: Float = 0.0
    var step: Float = 4.0
    vDSP_vramp(&min, &step, &i, vDSP_Stride(1), vDSP_Length(i.count))

    func increaseMatrix(var matrix: [Float]) -> [Float]
    {
        var increaser: Float = 1.0
        vDSP_vsadd(&matrix, vDSP_Stride(1), &increaser, &matrix, vDSP_Stride(1), vDSP_Length(i.count))

        return matrix
    }

    // Red matrix
    var r: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &r, vDSP_Stride(1), vDSP_Length(r.count))

    increaseMatrix(i)
    min = 1.0
    vDSP_vramp(&min, &step, &i, vDSP_Stride(1), vDSP_Length(i.count))
    // Green matrix
    var g: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &g, vDSP_Stride(1), vDSP_Length(g.count))

    increaseMatrix(i)
    min = 2.0
    vDSP_vramp(&min, &step, &i, vDSP_Stride(1), vDSP_Length(i.count))
    // Blue matrix
    var b: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &b, vDSP_Stride(1), vDSP_Length(b.count))

    increaseMatrix(i)
    min = 3.0
    vDSP_vramp(&min, &step, &i, vDSP_Stride(1), vDSP_Length(i.count))
    // Alpha matrix
    var a: [Float] = Array(count: pix, repeatedValue: 0.0)
    vDSP_vindex(&rawDataArray, &i, vDSP_Stride(1), &a, vDSP_Stride(1), vDSP_Length(a.count))

    return (r, g, b, a)
}
@atveit

This comment has been minimized.

Show comment
Hide comment
@atveit

atveit Feb 28, 2016

Member

Thanks for solutions, please create pull requests next time :)

I've added support for this myself, using https://github.com/kNeerajPro/ImagePixelFun - have a look at b4c47d2

Will keep this ticket open until I've added a blog post about it.

Amund

Member

atveit commented Feb 28, 2016

Thanks for solutions, please create pull requests next time :)

I've added support for this myself, using https://github.com/kNeerajPro/ImagePixelFun - have a look at b4c47d2

Will keep this ticket open until I've added a blog post about it.

Amund

@atveit atveit closed this Feb 28, 2016

@atveit

This comment has been minimized.

Show comment
Hide comment
@atveit

atveit Feb 28, 2016

Member

Hi @several27 and @ashmanov - have added your code for UIImage conversion now, will update the recent blog post with this :-) Thanks! 👍

See 88faf01

Member

atveit commented Feb 28, 2016

Hi @several27 and @ashmanov - have added your code for UIImage conversion now, will update the recent blog post with this :-) Thanks! 👍

See 88faf01

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment