SLP vectorizer fails to vectorize unrolled loop

[davidbolvansky]

Bugzilla Link	47178
Version	trunk
OS	Linux
CC	@anton-afanasyev,@fhahn,@RKSimon,@rotateright

Extended Description

#include <stdint.h>
#include <cstddef>

#define STRIPE_WIDTH 32
#define STRIPE_MASK   (STRIPE_WIDTH - 1)

static int16_t dither_line[2 * STRIPE_WIDTH] = {
#if STRIPE_WIDTH > 8
     8, 40,  8, 40,  8, 40,  8, 40,  8, 40,  8, 40,  8, 40,  8, 40,
    56, 24, 56, 24, 56, 24, 56, 24, 56, 24, 56, 24, 56, 24, 56, 24,
#else
     8, 40,  8, 40,  8, 40,  8, 40,
    56, 24, 56, 24, 56, 24, 56, 24,
#endif
};


void ass_stripe_pack_c(uint8_t *dst, ptrdiff_t dst_stride, const int16_t *src,
                       uintptr_t width, uintptr_t height)
{
    for (uintptr_t x = 0; x < width; x += STRIPE_WIDTH) {
        uint8_t *ptr = dst;
        for (uintptr_t y = 0; y < height; ++y) {
            const int16_t *dither = dither_line + (y & 1) * STRIPE_WIDTH;
            for (int k = 0; k < STRIPE_WIDTH; ++k)
                ptr[k] = (uint16_t) (src[k] - (src[k] >> 8) + dither[k]) >> 6;
                //ptr[k] = (255 * src[k] + 0x1FFF) / 0x4000;
            ptr += dst_stride;
            src += STRIPE_WIDTH;
        }
        dst += STRIPE_WIDTH;
    }
    uintptr_t left = dst_stride - ((width + STRIPE_MASK) & ~STRIPE_MASK);
    for (uintptr_t y = 0; y < height; ++y) {
        for (uintptr_t x = 0; x < left; ++x)
            dst[x] = 0;
        dst += dst_stride;
    }
}

GCC and ICC can vectorize it.

https://godbolt.org/z/nXCxW8

[anton-afanasyev]

This is not the job for SLP Vectorizer, but for the Loop Vectorizer. GCC and ICC can vectorize this by comparing ptr and src and taking the difference which is rather big for source and destination not to alias.

I've reduced it to:

#include <stdint.h>
#include

//#define STRIPE_WIDTH 32
#define STRIPE_WIDTH 8

void foo(uint8_t *ptr, const int16_t *src)
{
for (int k = 0; k < STRIPE_WIDTH; ++k)
ptr[k] = src[k];
}

(https://godbolt.org/z/Ph1hfc)

[davidbolvansky]

Loop is unrolled before Loop Vectorizer.

[anton-afanasyev]

Yes, but SLP can't vectorize it anyway. Vectorization here is conditional, SLP can't provide it -- it can check the pointers difference for the constant pointers only. Loop Vectorizer could detect unrolling here and make a vectorization. Or some metadata change among passes. This could be an architecture issue to solve this.

[davidbolvansky]

mentioned in issue llvm/llvm-bugzilla-archive#50256

[alexey-bataev]

The reduced form

void foo(uint8_t * __restrict__ ptr, const int16_t *src)
{
    for (int k = 0; k < STRIPE_WIDTH; ++k)
        ptr[k] = src[k];
}

is vectorized by SLP