upload libraries, demos on windows and macos

README.md

@@ -6,7 +6,7 @@ VNN是由[欢聚集团(Joyy Inc.)](http://www.huanju.cn/)推出的高性能、
 **特性**
 - 免费：无授权控制，无广告植入
 - 安全：零权限需求
-- 跨平台一致设计和体验：良好支持Andorid/iOS系统平台, 即将支持Windows/MacOS/Linux系统平台
+- 跨平台一致设计和体验：良好支持Andorid/iOS/Windows/MacOS(Intel & Apple Chip)系统平台, 即将支持Linux系统平台
 - 轻盈：全部功能SDK打包体积 Andorid 2.3M(arm64), iOS 2.0M(arm64)
 - 易用：API风格简洁、统一
 
@@ -50,21 +50,34 @@ VNN是由[欢聚集团(Joyy Inc.)](http://www.huanju.cn/)推出的高性能、
 
 [Android Demo APK](./demos/Android/vnn_demo.apk)  
 [Android Demo工程](./demos/Android)  
+[Windows Demo EXE](./demos/Windows/executable_demo/vnn_win_demo.exe)  
+[Windows Demo工程](./demos/Windows)  
 [iOS Demo工程](./demos/iOS)  
+[MacOS Demo工程](./demos/MacOS)  
 
 ---
 
 ## 集成指引与SDK文档
 
 [Android集成指引](./demos/Android/readme.md)  
 [iOS集成指引](./demos/iOS/readme.md)  
+[MacOS(Intel & Apple Chip)集成指引](./demos/MacOS/readme.md)   
+[Windows集成指引](./demos/Windows/readme.md)   
 [SDK文档索引](./doc/doc_index.md)   
 [常见问题](./doc/question_and_answer.md)
 
 ---
 
+## Acknowledgement
+参考了以下项目：
+- [ncnn](https://github.com/Tencent/ncnn)
+- [ppl](https://github.com/openppl-public/ppl.nn)
+- [MNN](https://github.com/alibaba/MNN)
+- [TNN](https://github.com/Tencent/TNN)
+
+---
+
 ## 更新计划
-- 2021.12.15 支持 Windows/MacOS 平台
 - 2021.12.30 支持 Linux 平台
 - 2022.01.15 新增人体关键点检测（Pose Detection）SDK
 - 发布模型转换工具链以及模型调用接口

demos/MacOS/readme.md

@@ -0,0 +1,69 @@
+- [1. MacOS Demo安装包](#1-macos-demo安装包)
+- [2. 运行MacOS Deom工程](#2-运行macos-deom工程)
+  - [1. 拷贝头文件与库文件](#1-拷贝头文件与库文件)
+  - [2. 拷贝模型文件](#2-拷贝模型文件)
+  - [3. 检查集成情况](#3-检查集成情况)
+  - [4. 运行](#4-运行)
+- [3. VNN SDK集成参考](#3-vnn-sdk集成参考)
+  - [1. 添加系统库依赖](#1-添加系统库依赖)
+  - [2. 添加VNN库](#2-添加vnn库)
+  - [3. 添加头文件搜索路径](#3-添加头文件搜索路径)
+  - [4. 设置安装时拷贝Framework](#4-设置安装时拷贝framework)
+  - [5. 放置模型文件、运行等](#5-放置模型文件运行等)
+
+# 1. MacOS Demo安装包
+因Apple签名机制的限制，暂不能提供dmg安装包体验，需要通过Demo工程生成安装包
+
+# 2. 运行MacOS Deom工程
+MacOS Demo工程已提前预设Framework路径、头文件搜索路径等，按如下操作即可运行      
+需要准备XCode（指引使用版本为版本为13.1），运行MacOS（版本10.10及以上）的计算机（Intel x64 CPU 和 Apple arm64 CPU均可）   
+## 1. 拷贝头文件与库文件
+拷贝根目录下的 ```libs``` 文件夹，放入 ```vnn_macos_demos``` 文件夹中  
+## 2. 拷贝模型文件
+拷贝根目录下的 ```models``` 文件夹中的内容放入 ```vnn_mac_demos/files``` 文件夹中  
+## 3. 检查集成情况
+用 XCode 打开 vnn_macos_demos 目录下的 ```vnndemo.xcodeproj```文件，如上述步骤正确，XCode将显示 ```Framework``` 集成情况如下   
+
+![MacOS_Guide_1_0](../../doc/resource/MacOS_Guide_1_0.png)    
+
+```models``` 集成情况如下   
+
+![iOS_Guide_1_1](../../doc/resource/iOS_Guide_1_1.png)
+
+## 4. 运行
+VNN MacOS 提供摄像头和相册两种演示模式，以下分别为手势识别SDK的摄像头Demo、人脸关键点检测SDK的相册Demo   
+![MacOS_Guide_1_1](../../doc/resource/MacOS_Guide_1_1.png)   
+
+![MacOS_Guide_1_2](../../doc/resource/MacOS_Guide_1_2.png)  
+
+点击Archive可生成ipa安装包用于分发测试，这里不展开说明   
+
+![iOS_Guide_1_5](../../doc/resource/iOS_Guide_1_5.png)   
+
+# 3. VNN SDK集成参考
+## 1. 添加系统库依赖
+VNN Demo 依赖9个系统库，按如下图添加这些依赖库   
+
+![iOS_Guide_1_3](../../doc/resource/MacOS_Guide_1_3.png)   
+
+## 2. 添加VNN库
+参考文档页面，根据需要的功能按需添加VNN库的framework。在Demo添加了全部的VNN framework   
+
+![iOS_Guide_2_1](../../doc/resource/iOS_Guide_2_1.png)   
+![MacOS_Guide_1_2](../../doc/resource/MacOS_Guide_1_4.png)   
+
+
+## 3. 添加头文件搜索路径
+![iOS_Guide_2_3](../../doc/resource/iOS_Guide_2_3.png)   
+
+## 4. 设置安装时拷贝Framework
+在 ```Build Settings```  选项卡下点击“+”，选择```“New Copy Flies Phase”```   
+
+![iOS_Guide_2_4](../../doc/resource/iOS_Guide_2_4.png)   
+
+将第2步添加的framework，添加到安装时拷贝的文件列表中   
+
+![MacOS_Guide_1_5](../../doc/resource/MacOS_Guide_1_5.png)   
+
+## 5. 放置模型文件、运行等
+参考上文 ```运行MacOS Deom工程```

demos/MacOS/vnn_macos_demo/osx/AppDelegate.h

@@ -0,0 +1,7 @@
+#import <Cocoa/Cocoa.h>
+
+@interface AppDelegate : NSObject <NSApplicationDelegate>
+
+
+@end
+

demos/MacOS/vnn_macos_demo/osx/AppDelegate.m

@@ -0,0 +1,21 @@
+#import "AppDelegate.h"
+
+@interface AppDelegate ()
+
+@end
+
+@implementation AppDelegate
+
+- (void)applicationDidFinishLaunching:(NSNotification *)aNotification
+{
+    // Insert code here to initialize your application
+}
+
+
+- (void)applicationWillTerminate:(NSNotification *)aNotification
+{
+    // Insert code here to tear down your application
+}
+
+
+@end

demos/MacOS/vnn_macos_demo/osx/CameraWindowCtrls/WindowCtrl_Camera_3DGameFaceStylizing.h

@@ -0,0 +1,9 @@
+#import "WindowCtrl_CameraMetalRender.h"
+
+NS_ASSUME_NONNULL_BEGIN
+
+@interface WindowCtrl_Camera_3DGameFaceStylizing : WindowCtrl_CameraMetalRender
+- (instancetype)initWithRootViewController:(NSViewController *)rootViewController;
+@end
+
+NS_ASSUME_NONNULL_END

demos/MacOS/vnn_macos_demo/osx/CameraWindowCtrls/WindowCtrl_Camera_3DGameFaceStylizing.mm

@@ -0,0 +1,209 @@
+#import "WindowCtrl_Camera_3DGameFaceStylizing.h"
+#import "vnn_kit.h"
+#import "vnn_common.h"
+#import "OSXDemoHelper.h"
+
+#if USE_STYLIZING && USE_FACE
+#   import "vnn_stylizing.h"
+#   import "vnn_face.h"
+#endif
+
+#define FACEMASK_WIDTH  (512)
+#define FACEMASK_HEIGHT (512)
+#define FACEMASK_CHANNEL (1)
+
+#define GAMEFACE_WIDTH  (512)
+#define GAMEFACE_HEIGHT (512)
+#define GAMEFACE_CHANNEL (3)
+
+@interface WindowCtrl_Camera_3DGameFaceStylizing ()
+@property (nonatomic, assign) VNNHandle                                 handle_face;
+@property (nonatomic, assign) VNNHandle                                 handle_gameface;
+@property (nonatomic, assign) VnnU8BufferPtr                            maskOutBuffer;
+@property (nonatomic, assign) VnnU8BufferPtr                            bgraMaskBuffer;
+@property (nonatomic, assign) VnnU8BufferPtr                            gameFaceOutBuffer;
+@property (nonatomic, assign) VnnU8BufferPtr                            bgraGameFaceBuffer;
+@property (nonatomic, strong) NSMutableArray*                           mtltexture_MaskArr;
+@property (nonatomic, strong) NSMutableArray*                           mtltexture_FaceArr;
+
+
+@end
+
+@implementation WindowCtrl_Camera_3DGameFaceStylizing
+
+- (instancetype)initWithRootViewController:(NSViewController *)rootViewController
+{
+    self = [super initWithRootViewController:rootViewController];
+    if(self){
+        [self initModel];
+    }
+    return self;
+}
+
+- (void)initModel{
+#if USE_STYLIZING && USE_FACE
+    {
+        const void *argv[] = {
+            [[[NSBundle mainBundle] bundlePath] stringByAppendingPathComponent:@"/Contents/Resources/files/models/vnn_face278_data/face_pc[1.0.0].vnnmodel"].UTF8String,
+        };
+        const int argc = sizeof(argv)/sizeof(argv[0]);
+        VNN_Create_Face(&_handle_face, argc, argv);
+    }
+
+    {
+        const void *argv[] = {
+            [[[NSBundle mainBundle] bundlePath] stringByAppendingPathComponent:@"/Contents/Resources/files/models/vnn_3dgame_data/face_3dgame[1.0.0].vnnmodel"].UTF8String,
+        };
+        const int argc = sizeof(argv)/sizeof(argv[0]);
+        VNN_Create_Stylizing(&_handle_gameface, argc, argv);
+    }
+
+    if (_handle_face > 0) {
+        const int use_278pts = 1;
+        VNN_Set_Face_Attr(_handle_face, "_use_278pts", &use_278pts);
+    }
+
+    size_t totalMaskDataSize = FACEMASK_CHANNEL *  FACEMASK_HEIGHT * FACEMASK_WIDTH * VNN_FRAMEDATAARR_MAX_FACES_NUM;
+    _maskOutBuffer = VnnU8BufferPtr(new VnnU8Buffer(totalMaskDataSize));
+
+    size_t totalDisneyDataSize = GAMEFACE_CHANNEL *  GAMEFACE_HEIGHT * GAMEFACE_WIDTH * VNN_FRAMEDATAARR_MAX_FACES_NUM;
+    _gameFaceOutBuffer = VnnU8BufferPtr(new VnnU8Buffer(totalDisneyDataSize));
+
+    size_t rgbaMaskDataSize = FACEMASK_HEIGHT * FACEMASK_WIDTH * 4;
+    _bgraMaskBuffer = VnnU8BufferPtr(new VnnU8Buffer(rgbaMaskDataSize));
+
+    size_t rgbaDisneyDataSize = GAMEFACE_HEIGHT * GAMEFACE_WIDTH * 4;
+    _bgraGameFaceBuffer = VnnU8BufferPtr(new VnnU8Buffer(rgbaDisneyDataSize));
+
+    _mtltexture_MaskArr = [[NSMutableArray alloc] init];
+    _mtltexture_FaceArr = [[NSMutableArray alloc] init];
+    for(int f = 0; f < VNN_FRAMEDATAARR_MAX_FACES_NUM; f++){
+        [_mtltexture_MaskArr addObject: [self.mtkView
+                                          generateOffScreenTextureWithFormat:MTLPixelFormatBGRA8Unorm
+                                          width:FACEMASK_WIDTH
+                                          height:FACEMASK_HEIGHT]];
+
+        [_mtltexture_FaceArr addObject: [self.mtkView
+                                          generateOffScreenTextureWithFormat:MTLPixelFormatBGRA8Unorm
+                                          width:GAMEFACE_WIDTH
+                                          height:GAMEFACE_HEIGHT]];
+    }
+
+#   endif
+}
+
+- (void)windowShouldClose:(NSNotification *)notification {
+    [super windowShouldClose:notification];
+#if USE_STYLIZING && USE_FACE
+    VNN_Destroy_Stylizing(&_handle_gameface);
+    VNN_Destroy_Face(&_handle_face);
+#   endif
+    [[NSApplication sharedApplication] stopModal];
+}
+
+- (void)processVideoFrameBuffer:(CVPixelBufferRef)pixelBuffer {
+#if USE_STYLIZING && USE_FACE
+
+    if (_handle_face && _handle_gameface){
+
+        VNN_Image input;
+        VNN_Create_VNNImage_From_PixelBuffer(pixelBuffer, &input, false);
+        input.mode_fmt = VNN_MODE_FMT_VIDEO;
+        const int img_h = input.height;
+        const int img_w = input.width;
+        if(self.mtkView.mtltexture_bluredMask == nil){
+            self.mtkView.mtltexture_bluredMask = [self.mtkView
+                                                  generateOffScreenTextureWithFormat:MTLPixelFormatBGRA8Unorm
+                                                  width:img_w
+                                                  height:img_h];
+
+            self.mtkView.mtltexture_offScreenImage = [self.mtkView
+                                                   generateOffScreenTextureWithFormat:MTLPixelFormatBGRA8Unorm
+                                                   width:img_w
+                                                   height:img_h];
+        }
+
+        VNN_FaceFrameDataArr faceArr;
+        memset(&faceArr, 0x00, sizeof(VNN_FaceFrameDataArr));
+
+        VNN_Apply_Face_CPU(_handle_face, &input, &faceArr);
+
+        VNN_ImageArr gameFaceDataArr;
+        gameFaceDataArr.imgsNum = faceArr.facesNum;
+        for (int f = 0; f < gameFaceDataArr.imgsNum; f++) {
+            gameFaceDataArr.imgsArr[f].width = GAMEFACE_WIDTH;
+            gameFaceDataArr.imgsArr[f].height = GAMEFACE_HEIGHT;
+            gameFaceDataArr.imgsArr[f].channels = GAMEFACE_CHANNEL;
+            gameFaceDataArr.imgsArr[f].pix_fmt = VNN_PIX_FMT_RGB888;
+            gameFaceDataArr.imgsArr[f].data = _gameFaceOutBuffer.get()->data + f * GAMEFACE_WIDTH * GAMEFACE_HEIGHT * GAMEFACE_CHANNEL;
+        }
+        VNN_Apply_Stylizing_CPU(_handle_gameface, &input, &faceArr, &gameFaceDataArr);
+
+
+        VNN_ImageArr faceMaskDataArr;
+        faceMaskDataArr.imgsNum = faceArr.facesNum;
+        for (int f = 0; f < faceMaskDataArr.imgsNum; f++) {
+            faceMaskDataArr.imgsArr[f].width = FACEMASK_WIDTH;
+            faceMaskDataArr.imgsArr[f].height = FACEMASK_HEIGHT;
+            faceMaskDataArr.imgsArr[f].channels = FACEMASK_CHANNEL;
+            faceMaskDataArr.imgsArr[f].pix_fmt = VNN_PIX_FMT_GRAY8;
+            faceMaskDataArr.imgsArr[f].data = _maskOutBuffer.get()->data + f * FACEMASK_WIDTH * FACEMASK_HEIGHT * FACEMASK_CHANNEL;
+        }
+        VNN_Get_Stylizing_Attr(_handle_gameface,  "_Mask", &faceMaskDataArr);
+
+        VNN_Free_VNNImage(pixelBuffer, &input, true);
+
+        id <MTLCommandBuffer> mtlCmdBuff = [self.mtkView.mtlCmdQueue commandBuffer];
+
+        for (int f = 0; f < faceMaskDataArr.imgsNum; f++) {
+
+            VNN_Rect2D vnn_rect = faceMaskDataArr.imgsArr[f].rect;
+            rectBox mtl_rect;
+            mtl_rect.x0 = vnn_rect.x0;
+            mtl_rect.y0 = vnn_rect.y0;
+            mtl_rect.x1 = vnn_rect.x1;
+            mtl_rect.y1 = vnn_rect.y1;
+
+            unsigned char* mask_bgra_ptr = _bgraMaskBuffer.get()->data;
+            unsigned char* mask_gray_ptr = _maskOutBuffer.get()->data + f * FACEMASK_WIDTH * FACEMASK_HEIGHT * FACEMASK_CHANNEL;
+            const size_t mask_n_pixel = FACEMASK_WIDTH * FACEMASK_HEIGHT;
+            tileGrayToBGRA(mask_gray_ptr, mask_bgra_ptr, mask_n_pixel);
+
+            [_mtltexture_MaskArr[f] replaceRegion:MTLRegionMake2D(0, 0, FACEMASK_WIDTH, FACEMASK_HEIGHT)
+                                              mipmapLevel:0
+                                                withBytes:_bgraMaskBuffer.get()->data
+                                              bytesPerRow:FACEMASK_WIDTH*4];
+
+            [self.mtkView renderRectTextureToBackground_With_MTLCommandBuffer:mtlCmdBuff
+                                                                  rectTexture:_mtltexture_MaskArr[f]
+                                                                      rectBox:mtl_rect
+                                                             offScreenTexture:self.mtkView.mtltexture_bluredMask
+                                                                  clearScreen:f == 0];
+
+
+            unsigned char* face_bgra_ptr = _bgraGameFaceBuffer.get()->data;
+            unsigned char* face_rgb_ptr = _gameFaceOutBuffer.get()->data + f * GAMEFACE_WIDTH * GAMEFACE_HEIGHT * GAMEFACE_CHANNEL;
+            const size_t face_n_pixel = GAMEFACE_WIDTH * GAMEFACE_HEIGHT;
+            convertRGBToBGRA(face_rgb_ptr, face_bgra_ptr, face_n_pixel);
+
+            [_mtltexture_FaceArr[f] replaceRegion:MTLRegionMake2D(0, 0, GAMEFACE_WIDTH, GAMEFACE_HEIGHT)
+                                              mipmapLevel:0
+                                                withBytes:_bgraGameFaceBuffer.get()->data
+                                              bytesPerRow:GAMEFACE_WIDTH*4];
+
+
+
+            [self.mtkView renderRectTextureToBackground_With_MTLCommandBuffer:mtlCmdBuff
+                                                                  rectTexture:_mtltexture_FaceArr[f]
+                                                                      rectBox:mtl_rect
+                                                             offScreenTexture:self.mtkView.mtltexture_offScreenImage
+                                                                  clearScreen:f == 0];
+        }
+        [mtlCmdBuff commit];
+        [mtlCmdBuff waitUntilScheduled];
+
+    }
+#endif
+}
+
+@end

demos/MacOS/vnn_macos_demo/osx/CameraWindowCtrls/WindowCtrl_Camera_CartoonStylizing_ComicStylizing.h

@@ -0,0 +1,9 @@
+#import "WindowCtrl_CameraMetalRender.h"
+
+NS_ASSUME_NONNULL_BEGIN
+
+@interface WindowCtrl_Camera_CartoonStylizing_ComicStylizing : WindowCtrl_CameraMetalRender
+- (instancetype)initWithRootViewController:(NSViewController *)rootViewController WithStyleType:(NSString *)type;
+@end
+
+NS_ASSUME_NONNULL_END