Efficient pixel conversions

[dCubelic]

Hi,

I am new to halide, and am wondering what would be the best scheduling to achieve most efficient pixel conversion from any of RGB/BGR/RGBA/BGRA to any of RGB/BGR/RGBA/BGRA

Currently I have the algorithm split in two functions, the one where destination pixel type is 3 bytes long, and the one where it's 4 (with alpha channel).

swap 3:

        input.dim( 0 ).set_stride( inputBytes );
        input.dim( 1 ).set_stride( inputBytes * input.width() );
        input.dim( 2 ).set_stride( 1 );
        input.dim( 2 ).set_bounds( 0, inputBytes );

        swapped( x, y, c ) = input( x, y, c );
        swapped( x, y, 0 ) = input( x, y, 2 );
        swapped( x, y, 2 ) = input( x, y, 0 );

        swapped.dim( 0 ).set_stride( 3 );
        swapped.dim( 1 ).set_stride( 3 * input.width() );
        swapped.dim( 2 ).set_stride( 1 );
        swapped.dim( 2 ).set_bounds( 0, 3 );

        swapped.reorder( c, x, y ).vectorize( x, 3 ).unroll( c ).parallel( y );

swap 4:

        input.dim( 0 ).set_stride( inputBytes );
        input.dim( 1 ).set_stride( inputBytes * input.width() );
        input.dim( 2 ).set_stride( 1 );
        input.dim( 2 ).set_bounds( 0, inputBytes );

        swapped( x, y, c ) = Halide::cast< std::uint8_t >( 255 );
        swapped( x, y, 0 ) = input( x, y, 2 );
        swapped( x, y, 1 ) = input( x, y, 1 );
        swapped( x, y, 2 ) = input( x, y, 0 );

        swapped.dim( 0 ).set_stride( 4 );
        swapped.dim( 1 ).set_stride( 4 * input.width() );
        swapped.dim( 2 ).set_stride( 1 );
        swapped.dim( 2 ).set_bounds( 0, 4 );

        swapped.reorder( c, x, y ).vectorize( x, 4 ).unroll( c ).parallel( y );

The image in memory is interleaved (R1 G1 B1 R2 G2 B2 R3...), and I am loading it to halide buffer like this:

auto imageBuffer{ Halide::Runtime::Buffer< std::uint8_t, 3 >::make_interleaved( reinterpret_cast< std::uint8_t * >( source.data() ), source.width(), source.height(), sizeof( SrcPixel ) ) };

I am benchmarking against OpenCV implementations of the same things, and currently achieving around 5-10 times worse results. Do you have any suggestions on improving this?

[dCubelic]

I also tried running Adams2019 auto scheduler and got something like this:

swap 3:

        swapped.update(1)
            .split(y, y, yi, 4, TailStrategy::GuardWithIf)
            .reorder({x, yi, y})
            .parallel(y);
        swapped.update(0)
            .split(y, y, yi, 4, TailStrategy::GuardWithIf)
            .reorder({x, yi, y})
            .parallel(y);
        swapped
            .split(y, y, yi, 4, TailStrategy::ShiftInwards)
            .split(c, c, ci, 32, TailStrategy::GuardWithIf)
            .vectorize(ci)
            .compute_root()
            .reorder({ci, c, x, yi, y})
            .parallel(y)
            .reorder_storage(c, x, y);

swap 4:

        swapped.update(2)
            .split(x, x, xi, 120, TailStrategy::GuardWithIf)
            .split(y, y, yi, 60, TailStrategy::GuardWithIf)
            .reorder({xi, yi, x, y})
            .fuse(x, y, x)
            .parallel(x);
        swapped.update(1)
            .split(x, x, xi, 120, TailStrategy::GuardWithIf)
            .split(y, y, yi, 60, TailStrategy::GuardWithIf)
            .reorder({xi, yi, x, y})
            .fuse(x, y, x)
            .parallel(x);
        swapped.update(0)
            .split(x, x, xi, 120, TailStrategy::GuardWithIf)
            .split(y, y, yi, 60, TailStrategy::GuardWithIf)
            .reorder({xi, yi, x, y})
            .fuse(x, y, x)
            .parallel(x);
        swapped
            .split(x, x, xi, 120, TailStrategy::ShiftInwards)
            .split(y, y, yi, 60, TailStrategy::ShiftInwards)
            .split(c, c, ci, 32, TailStrategy::GuardWithIf)
            .vectorize(ci)
            .compute_root()
            .reorder({ci, c, xi, yi, x, y})
            .fuse(x, y, x)
            .parallel(x)
            .reorder_storage(c, x, y);

That is much faster than what I did initially, but still almost 2 times worse than OpenCV.

[abadams]

I think you want something like this (this is for RGB to BGR, but the approach generalizes to other conversions):

    swapped(x, y, c) = mux(c, {input(x, y, 2), input(x, y, 1), input(x, y, 0)});

    swapped.vectorize(x, 16, TailStrategy::GuardWithIf)
        .reorder(c, x, y)
        .unroll(c)
        .parallel(y, 16, TailStrategy::GuardWithIf);

    input.dim(0).set_stride(3);
    input.dim(2).set_bounds(0, 3).set_stride(1);
    swapped.output_buffer().dim(0).set_stride(3);
    swapped.output_buffer().dim(2).set_bounds(0, 3).set_stride(1);

Notes:

• Each f(...) = ... implies a separate loop nest, with a separate schedule. You've only provided a schedule for the first loop nest, so for most of the work your code is unscheduled. It's simpler to just use a single definition which muxes on the color channel, then reorder it innermost and unroll it.
• Vectorize within a factor of two of the native width for the type in question. My machine is avx2, which has 32-byte vectors natively. I vectorized 16-bytes wide, because it was a little faster (because x86 is bad at doing vector shuffles that cross 16-byte boundaries). 3/4 is way too narrow.
• The parallel factor to use is image size-dependent and you should test lots of options, but 8 or 16 is often good.