运用NEON指令集加速RGB与YUV相互转换 #5

Denislyl · 2018-11-08T03:09:37Z

最近做H264编解码的工作，使用了Android平台MediaCodec，进行硬编解码，以减少CPU的压力。

然后发现适用性最强的格式是YUV420P和YUV420SP，绝大部分Android设备都可以支持这两种格式的H264输入。

（如何查看Android设备所支持格式，见我的上一篇文章《Android平台MediaCodec避坑指北》)

但是Android API level 23. 开始不被推荐，推荐COLOR_FormatYUV420Flexible.

那我们采集到视频帧数据如果是RGB888或者UYUV422，那就需要进行格式转换。

如果刚好用的NV21，NV12，或者YV12，I420有可能不用转换，也有可能需要转换。见《Android平台MediaCodec避坑指北》坑点3

先说一下开发环境，使用的是，Windows10 + AndroidStudio3.0 + CMake

NEON相关的学习资料 ，可以看这个
https://www.jianshu.com/p/16d60ac56249
https://blog.csdn.net/chshplp_liaoping/article/details/12752749
简单小例子：
http://hilbert-space.de/?p=22
翻译：
https://zhuanlan.zhihu.com/p/24702989

Neon支持检测
https://developer.android.com/ndk/guides/cpu-arm-neon.html

0、Neon指令

Neon指令可以在arm官网下到文档NEON Programmer’s Guide Version: 1.0
里面有许多特色长宽指令，8_8 变为 16_8，收窄指令 16_8 变为 8_8 ，饱和指令等等，
要多看看文档才能了然于胸。

1、使用Intrinsic_C还是汇编

Intrinsic_C用起来简单，还能跨armv7，armv8，每一个API接口都对应着相应的一条指令。由于充满了sign/unsign，int、short、char，8x8,16x8 ，加减乘，长宽，收窄，饱和，位移等等，在arm_neon.h里有几百个Api，但是殊途同归，大同小异，每了解1个汇编指令，就能通吃10~20个Api。
这里有份帮助文档：https://blog.csdn.net/fengbingchun/article/details/38085781
汇编代码就不能跨armv7，armv8了，但是真正懂得每个汇编指令的意思，才能优化到极致。
懂了汇编，反过来写Intrinsic_C也大有帮助。

2、开启Neon之旅

在你的CMakeLists.txt配置文件里，加入这两句，使用mk文件也一样。
（如果不需要的话可以不加）

add_definitions("-D__ARM_NEON__=1")
set_property(SOURCE ${SRC_FILE}  APPEND_STRING PROPERTY COMPILE_FLAGS " -mfpu=neon")

也可以加在build.gradle里，AndroidStudio3.0就是方便。

        externalNativeBuild {
            cmake {
                cFlags "-mfpu=neon"
                cppFlags "-std=c++11 -frtti -fexceptions"
                arguments "-DANDROID_ARM_NEON=TRUE"
            }
        }

3、CMake开启ASM

enable_language(ASM)
这句不加，你就不能使用.S汇编文件。

4、使用IntrinsicC库

非常简单，只需#include <arm_neon.h>

5、Show me the code

相关转换函数，例子是rgb转yuv420sp nv12的，反过来也是一样道理。

先上C++版本：

void FrameFormatConvert::RGBtoNV12(byte* nv12, byte* rgb, int width, int height)
{
	if(CheckIsSupportNeon())
	{
#ifndef WIN32
		LOGD("start neon nv12");
		//rgb888_2_nv12_neon(nv12, rgb, width, height, width >> 4);
		rgb888_2_nv12_intrinsic(nv12, rgb, width, height);
		return;
#endif
	}
	int frameSize = width * height;

	int yIndex = 0;
	int uvIndex = frameSize;

	int R, G, B, Y, U, V;
	int index = 0;
	for (int j = 0; j < height; j++)
	{
		for (int i = 0; i < width; i++)
		{

			//a = (argb[index] & 0xff000000) >> 24; // a is not used obviously
			//R = (argb[index] & 0xff0000) >> 16;
			//G = (argb[index] & 0xff00) >> 8;
			//B = (argb[index] & 0xff) >> 0;
			R = rgb[index];
			index++;
			G = rgb[index];
			index++;
			B = rgb[index];
			index++;

			// well known RGB to YUV algorithm
			Y = ((66 * R + 129 * G + 25 * B + 128) >> 8) + 16;
			U = ((-38 * R - 74 * G + 112 * B + 128) >> 8) + 128;
			V = ((112 * R - 94 * G - 18 * B + 128) >> 8) + 128;

			// NV21 has a plane of Y and interleaved planes of VU each sampled by a factor of 2
			//    meaning for every 4 Y pixels there are 1 V and 1 U.  Note the sampling is every other
			//    pixel AND every other scanline.
			// NV12  YYYYYYYY UVUV 
			// NV21  YYYYYYYY VUVU
			nv12[yIndex++] = (byte)((Y < 0) ? 0 : ((Y > 255) ? 255 : Y));
			if (j % 2 == 0 && index % 2 == 0)
			{
				nv12[uvIndex++] = (byte)((U < 0) ? 0 : ((U > 255) ? 255 : U));
				nv12[uvIndex++] = (byte)((V < 0) ? 0 : ((V > 255) ? 255 : V));
			}
		}
	}
}

然后是Intrinsic_C：

void FrameFormatConvert::rgb888_2_nv12_intrinsic(byte * nv12, byte * rgb, int width, int height)
{
#ifndef WIN32
	const uint8x8_t u8_zero = vdup_n_u8(0);
	const uint8x8_t u8_16 = vdup_n_u8(16);
	const uint16x8_t u16_rounding = vdupq_n_u16(128);

	const int16x8_t s16_zero = vdupq_n_s16(0);
	const int8x8_t s8_rounding = vdup_n_s8(-128);
	const int16x8_t s16_rounding = vdupq_n_s16(128);

	byte* UVPtr = nv12 + width * height;
	int pitch = width >> 4;

	for (int j = 0; j < height; ++j)
	{
		for (int i = 0; i < pitch; ++i)
		{
			// Load rgb 16 pixel
			uint8x16x3_t pixel_rgb = vld3q_u8(rgb);

			uint8x8_t high_r = vget_high_u8(pixel_rgb.val[0]);
			uint8x8_t low_r = vget_low_u8(pixel_rgb.val[0]);
			uint8x8_t high_g = vget_high_u8(pixel_rgb.val[1]);
			uint8x8_t low_g = vget_low_u8(pixel_rgb.val[1]);
			uint8x8_t high_b = vget_high_u8(pixel_rgb.val[2]);
			uint8x8_t low_b = vget_low_u8(pixel_rgb.val[2]);

			// NOTE:
			// declaration may not appear after executable statement in block
			uint16x8_t high_y;
			uint16x8_t low_y;			

			// 1. Multiply transform matrix (Y′: unsigned, U/V: signed)
			// 2. Scale down (">>8") to 8-bit values with rounding ("+128") (Y′: unsigned, U/V: signed)
			// 3. Add an offset to the values to eliminate any negative values (all results are 8-bit unsigned)
			uint8x8_t scalar = vdup_n_u8(66);
			high_y = vmull_u8(high_r, scalar);
			low_y = vmull_u8(low_r, scalar);

			scalar = vdup_n_u8(129);
			high_y = vmlal_u8(high_y, high_g, scalar);
			low_y = vmlal_u8(low_y, low_g, scalar);

			scalar = vdup_n_u8(25);
			high_y = vmlal_u8(high_y, high_b, scalar);
			low_y = vmlal_u8(low_y, low_b, scalar);

			high_y = vaddq_u16(high_y, u16_rounding);
			low_y = vaddq_u16(low_y, u16_rounding);

			uint8x8_t u8_low_y = vshrn_n_u16(low_y, 8);
			uint8x8_t u8_high_y = vshrn_n_u16(high_y, 8);

			low_y = vaddl_u8(u8_low_y, u8_16);
			high_y = vaddl_u8(u8_high_y, u8_16);

			uint8x16_t pixel_y = vcombine_u8(vqmovn_u16(low_y), vqmovn_u16(high_y));

			// Store
			vst1q_u8(nv12, pixel_y);

			if (j % 2 == 0)
			{
				uint8x8x2_t mix_r = vuzp_u8(low_r, high_r);
				uint8x8x2_t mix_g = vuzp_u8(low_g, high_g);
				uint8x8x2_t mix_b = vuzp_u8(low_b, high_b);

				int16x8_t signed_r = vreinterpretq_s16_u16(vaddl_u8(mix_r.val[0], u8_zero));
				int16x8_t signed_g = vreinterpretq_s16_u16(vaddl_u8(mix_g.val[0], u8_zero));
				int16x8_t signed_b = vreinterpretq_s16_u16(vaddl_u8(mix_b.val[0], u8_zero));

				int16x8_t signed_u;
				int16x8_t signed_v;

				int16x8_t signed_scalar = vdupq_n_s16(-38);
				signed_u = vmulq_s16(signed_r, signed_scalar);

				signed_scalar = vdupq_n_s16(112);
				signed_v = vmulq_s16(signed_r, signed_scalar);

				signed_scalar = vdupq_n_s16(-74);
				signed_u = vmlaq_s16(signed_u, signed_g, signed_scalar);

				signed_scalar = vdupq_n_s16(-94);
				signed_v = vmlaq_s16(signed_v, signed_g, signed_scalar);

				signed_scalar = vdupq_n_s16(112);
				signed_u = vmlaq_s16(signed_u, signed_b, signed_scalar);

				signed_scalar = vdupq_n_s16(-18);
				signed_v = vmlaq_s16(signed_v, signed_b, signed_scalar);

				signed_u = vaddq_s16(signed_u, s16_rounding);
				signed_v = vaddq_s16(signed_v, s16_rounding);

				int8x8_t s8_u = vshrn_n_s16(signed_u, 8);
				int8x8_t s8_v = vshrn_n_s16(signed_v, 8);

				signed_u = vsubl_s8(s8_u, s8_rounding);
				signed_v = vsubl_s8(s8_v, s8_rounding);

				signed_u = vmaxq_s16(signed_u, s16_zero);
				signed_v = vmaxq_s16(signed_v, s16_zero);

				uint16x8_t unsigned_u = vreinterpretq_u16_s16(signed_u);
				uint16x8_t unsigned_v = vreinterpretq_u16_s16(signed_v);

				uint8x8x2_t result;
				result.val[0] = vqmovn_u16(unsigned_u);
				result.val[1] = vqmovn_u16(unsigned_v);

				vst2_u8(UVPtr, result);
				UVPtr += 16;
			}

			rgb += 3 * 16;
			nv12 += 16;
		}
	}
#endif
}

最后是汇编版：

 /*
 	RGB_2_NV21.Neon.S
 */
    .text
    .align
    .global rgb888_2_nv12_neon
    .type   rgb888_2_nv12_neon, %function

/*the param of the function*/
#define Dst_NV12_Ptr	r0
#define Src_RGB_Ptr		r1
#define width			r2
#define height			r3
#define pitch			r4 /*width>>4*/

/*the param using in function*/
#define XCount			r5
#define YCount			r6

#define UV_Ptr			r7

#define	CData0			q10
#define	CData16			d22
#define CData66			d23
#define CData129		d24
#define CData25 		d25	
#define	CData128		d26
#define CData38 		d27
#define CData74 		d28
#define CData112 		d29
#define CData94 		d30
#define CData18 		d31

rgb888_2_nv12_neon:
		PUSH            {r4-r12, lr}					/* 10 words */
        VPUSH           {q0-q7}							/* 8Q -> 32 words */
		VPUSH           {q8-q15}						/* 8Q -> 32 words */
		
		/* 向量加载常数 */
		VMOV.s16    CData0	,	#0
		VMOV.u8		CData16	,	#16
		VMOV.u8		CData66	,	#66
		VMOV.u8		CData129,	#129
		VMOV.u8		CData25 ,	#25
		VMOV.u8		CData128,	#128
		VMOV.u8		CData38 ,   #38
		VMOV.u8		CData74 ,	#74
		VMOV.u8		CData112,	#112
		VMOV.u8		CData94 ,	#94
		VMOV.u8		CData18 ,	#18

		/* char* UVPtr = Dst_NV12_Ptr + width*height; */ 
		MUL			UV_Ptr, width, height									
		ADD			UV_Ptr, Dst_NV12_Ptr		

		MOV YCount, height								/*	int YCount = height;   */
		CMP YCount, #0									/*	if (YCount == 0) return;*/
		BEQ endColNormal
	beginColNormal:										/*	do
															{*/
		MOV 	XCount, pitch							/*	int XCount = pitch;*/
		BEQ     endRowNormal							/*	if (XCount == 0) continue;*/
	beginRowNormal:										/*	do
															{*/
		/* d0 d1 d2 as RGB, d3 d4 d5 as RGB */
		VLD3.8	{d0-d2}, [Src_RGB_Ptr]! 
		VLD3.8	{d3-d5}, [Src_RGB_Ptr]! 

		#------------------------------------------
		/* 计算Y值 Y = ((66 * R + 129 * G + 25 * B + 128) >> 8) + 16; */

		VMULL.u8 q3, d0, CData66
		VMULL.u8 q4, d3, CData66
		VMLAL.u8 q3, d1, CData129
		VMLAL.u8 q4, d4, CData129
		VMLAL.u8 q3, d2, CData25
		VMLAL.u8 q4, d5, CData25

		VADDW.u8 q3 , q3, CData128
		VADDW.u8 q4, q4, CData128
		VSHRN.u16 d10, q3, #8
		VSHRN.u16 d11, q4, #8
		VADDL.u8 q3,d10, CData16
		VADDL.u8 q4,d11, CData16
		
		/* vqmovn 16 to 8 [0~255] */
		VQMOVN.u16 d10, q3
		VQMOVN.u16 d11, q4

		VST1.u8 {d10} , [Dst_NV12_Ptr]!
		VST1.u8 {d11} , [Dst_NV12_Ptr]!

		TST YCount, #1
		BNE skipUV
		#------------------------------------------
		/* UV预处理，去奇数, 存入d0(R)，d1(G)，d2(B) */
		VUZP.u8 d0 , d3
		VUZP.u8 d1 , d4
		VUZP.u8 d2 , d5

		/*  计算UV
			U = ((-38 * R - 74 * G + 112 * B + 128) >> 8) + 128;
			V = ((112 * R - 94 * G -  18 * B + 128) >> 8) + 128;
			U:q3q5q7, V:q4q6q8
		*/
		VMULL.u8 q4, d0, CData112
		VMULL.u8 q3, d0, CData38
		VMULL.u8 q6, d1, CData18
		VMULL.u8 q5, d1, CData112
		VMLAL.u8 q3, d2, CData74
		VMLAL.u8 q6, d2, CData94

		VADDW.u8 q7, q5, CData128
		VADDW.u8 q8, q4, CData128
		VSUB.s16 q2, q7 , q3
		VSUB.s16 q3, q8 , q6

		VSHRN.s16 d0, q2, #8
		VSHRN.s16 d1, q3, #8
		VADDL.u8 q2, d0, CData128				
		VADDL.u8 q3, d1, CData128	

		VMAX.s16 q4, CData0,	q2
		VMAX.s16 q5, CData0,	q3 

		/* vqmovn 16 to 8 [0~255] */
		VQMOVN.u16 d0, q4
		VQMOVN.u16 d1, q5
		#------------------------------------------

		VST2.u8  {d0-d1}, [UV_Ptr]!

	skipUV:
		SUBS	XCount, #1
		BNE		beginRowNormal					/*		}while(--XCount);*/
	endRowNormal:
		SUBS	YCount, #1						/*	}while(--YCount);*/
		BNE		beginColNormal
	endColNormal:

		VPOP           {q8-q15}
		VPOP           {q0-q7}             
        POP            {r4-r12, pc}

附arm常用寄存器

• 
r0-r15 和 R0-R15
• 
a1-a4（自变量、结果或暂存寄存器，r0 到 r3 的同义词）
• 
v1-v8（变量寄存器，r4 到 r11）
• 
sb 和 SB（静态基址，r9）
• 
ip 和 IP（内部程序调用暂存寄存器，r12）
• 
sp 和 SP（堆栈指针，r13）
• 
lr 和 LR（链接寄存器，r14）
• 
pc 和 PC（程序计数器，r15）。

neon寄存器，注意：这玩意是重叠的，Q0 = D0 + D1 = S0 + S1 + S2 +S3
具体看上文CSDN里博客写的，就不赘述了。

• 
q0-q15 和 Q0-Q15（NEON™ 四字寄存器）
• 
d0-d31 和 D0-D31（NEON 双字寄存器，VFP 双精度寄存器）
• 
s0-s31 和 S0-S31（VFP 单精度寄存器）。

就酱

The text was updated successfully, but these errors were encountered:

Denislyl added the blog label Nov 8, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

运用NEON指令集加速RGB与YUV相互转换 #5

运用NEON指令集加速RGB与YUV相互转换 #5

Denislyl commented Nov 8, 2018 •

edited

运用NEON指令集加速RGB与YUV相互转换 #5

运用NEON指令集加速RGB与YUV相互转换 #5

Comments

Denislyl commented Nov 8, 2018 • edited

0、Neon指令

1、使用Intrinsic_C还是汇编

2、开启Neon之旅

3、CMake开启ASM

4、使用IntrinsicC库

5、Show me the code

Denislyl commented Nov 8, 2018 •

edited