Releases: sdcb/OpenVINO.NET
0.6.5 - supports 2024.1.0
Full Changelog: 0.6.4...0.6.5
0.6.4 - supports 2024.0
Full Changelog: 0.6.3...0.6.4
0.6.3
0.6.1
OpenVINO.NET 0.6.1 版本发布说明
我们很高兴地宣布OpenVINO.NET的三个NuGet包现已更新至0.6.1版本。以下是本次更新的详细信息。
NuGet包更新概览
| 包名 | 版本 |
|---|---|
| Sdcb.OpenVINO |  |
| Sdcb.OpenVINO.PaddleOCR |  |
| Sdcb.OpenVINO.Extensions.OpenCvSharp4 |  |
Sdcb.OpenVINO 0.6.1
从0.6.0版本更新。这次更新主要是修复了在.NET Framework环境下，加载动态库失败的问题，提升了库的兼容性。
Sdcb.OpenVINO.Extensions.OpenCvSharp4 0.6.1
从0.2.1版本更新。这个版本的更新将会自动依赖更新后的Sdcb.OpenVINO核心库0.6.1版，替换了之前的0.2.0版本。另外，我们在这个包中增加了一个StackingVertically的重载方法，它使用了OpenCV内置的VConcat函数，通过此次优化可以提高约30~40%的性能。这项功能的加入要特别感谢 @AvenSun 的贡献：PR链接。
Sdcb.OpenVINO.PaddleOCR 0.6.1
从0.5.1版本更新。本次更新同样将自动依赖更新后的Sdcb.OpenVINO核心库至0.6.1版本，并且将依赖的Sdcb.OpenVINO.Extensions.OpenCvSharp4包更新至0.6.1版本。通过这次的链式更新，用户将能够享受到性能提升的优势。
总结
感谢所有用户的持续支持，我们期待你体验OpenVINO.NET新版本带来的改进。如有任何问题或建议，请通过项目的GitHub页面与我们联系。
OpenVINO.NET 0.6.1 Release Notes in English
We are delighted to announce that the three NuGet packages for OpenVINO.NET have now been updated to version 0.6.1. Here are the detailed updates.
Overview of NuGet Package Updates
| Package Name | Version |
|---|---|
| Sdcb.OpenVINO | |
| Sdcb.OpenVINO.PaddleOCR | |
| Sdcb.OpenVINO.Extensions.OpenCvSharp4 | |
Sdcb.OpenVINO 0.6.1
Updated from version 0.6.0. This update mainly fixes the issue of dynamic library loading failures in the .NET Framework environment, improving the library's compatibility.
Sdcb.OpenVINO.Extensions.OpenCvSharp4 0.6.1
Updated from version 0.2.1. This version of the update will automatically depend on the updated core library Sdcb.OpenVINO version 0.6.1, replacing the previous version 0.2.0. Additionally, we have added an overloaded method of StackingVertically to this package, which makes use of the built-in VConcat function in OpenCV, and this optimization can improve performance by about 30~40%. We would particularly like to thank @AvenSun for their contribution to this feature: PR link.
Sdcb.OpenVINO.PaddleOCR 0.6.1
Updated from version 0.5.1. This update will also automatically depend on the updated core library Sdcb.OpenVINO to version 0.6.1 and will update the dependent Sdcb.OpenVINO.Extensions.OpenCvSharp4 package to version 0.6.1. Through this chained update, users will be able to enjoy the benefits of performance improvements.
Summary
Thank you all for your continuous support. We look forward to you experiencing the improvements brought by the new version of OpenVINO.NET. If you have any questions or suggestions, please contact us through the project's GitHub page.
Contributors
Assets 2
0.6.0 - supports 2023.2
中文
新发布核心NuGet包：
Sdcb.OpenVINO
此为核心包，更新自0.5.4，以适配2023.2.0的版本，同时也向下兼容2023.1.0的版本。此包先尝试加载2023.2.0的动态库，如果加载不成功，则再加载2023.1.0的动态库。你可以通过OVCore.Version.GetAbbreviatedVersion() >= 2320来知道当前加载的动态库是2023.2.0版本以上的还是以下的。
加载顺序是先加载依赖，再加载主包，例如：openvino_c为主包，依赖于openvino，而openvino则依赖于libtbb。加载过程中会先加载libtbb，再加载openvino，最后加载openvino_c——这个功能之前必须要.NET 6或以上才启用，在这个发布中，该功能降到到.NET 5就能用了。
新发布的2023.2.0版本的运行库（从2023.1.0更新）：
| 包名 | 图标 | 备注 |
|---|---|---|
| Sdcb.OpenVINO.runtime.centos.7-x64 |  |
|
| Sdcb.OpenVINO.runtime.linux-arm |  |
替换此前的Sdcb.OpenVINO.runtime.debian.9-arm，因为它实际上不止debian 9可以使用 |
| Sdcb.OpenVINO.runtime.linux-arm64 |  |
替换此前的Sdcb.OpenVINO.runtime.debian.9-arm64，因为它实际上不止debian 9可以使用 |
| Sdcb.OpenVINO.runtime.rhel.8-x64 |  |
|
| Sdcb.OpenVINO.runtime.ubuntu.18.04-x64 |  |
|
| Sdcb.OpenVINO.runtime.ubuntu.20.04-x64 |  |
|
| Sdcb.OpenVINO.runtime.ubuntu.22.04-x64 |  |
|
| Sdcb.OpenVINO.runtime.android-arm64 |  |
非官方，预览 |
| Sdcb.OpenVINO.runtime.android-x64 |  |
非官方，预览 |
| Sdcb.OpenVINO.runtime.win-x64 |  |
使用2023.2.0版本的动态库的最重要改进是，遇到OpenVINO的C++异常时，Sdcb.OpenVINO核心库会自动收集报异常的具体原因（此功能是在我向OpenVINO官方提议后实现）。
以下我向openvino提交的建议已在2023.2版本中解决：
在Linux迷你运行时的OpenCvSharp4 NuGet包：
| 包名 | 图标 |
|---|---|
| Sdcb.OpenCvSharp4.mini.runtime.centos.7-arm64 |  |
| Sdcb.OpenCvSharp4.mini.runtime.centos.7-x64 |  |
| Sdcb.OpenCvSharp4.mini.runtime.debian.11-arm64 |  |
| Sdcb.OpenCvSharp4.mini.runtime.debian.11-x64 |  |
| Sdcb.OpenCvSharp4.mini.runtime.debian.12-arm64 |  |
| Sdcb.OpenCvSharp4.mini.runtime.debian.12-x64 |  |
| Sdcb.OpenCvSharp4.mini.runtime.ubuntu.22.04-arm64 |  |
| Sdcb.OpenCvSharp4.mini.runtime.ubuntu.22.04-x64 |  |
| Sdcb.OpenCvSharp4.mini.runtime.android-arm64 |  |
| Sdcb.OpenCvSharp4.mini.runtime.android-x64 |  |
注意
由于.NET 8采用了这个突破性变更，此前2023.1.0动态库的Linux的NuGet包可能在.NET 8中用不了，我在2023.2.0版本的动态库包中解决了这个问题。此外，所有的OpenCvSharp4都解决了这个问题，因此如果你遇到了，请更新到最新版。
另外，尽管Sdcb.OpenVINO.PaddleOCR并未更新，但它依赖于Sdcb.OpenVINO核心库，因此如果你未安装Sdcb.OpenVINO却想体验最新版功能，你需要手动再安装最新版的Sdcb.OpenVINO。
English
Newly released core NuGet package:
Sdcb.OpenVINO
This is a core package, updated from 0.5.4 to support version 2023.2.0 while also being backwards compatible with version 2023.1.0. This package first tries to load the 2023.2.0 dynamic library; if unsuccessful, it will then load the 2023.1.0 dynamic library. You can determine whether the currently loaded dynamic library is version 2023.2.0 or higher by using OVCore.Version.GetAbbreviatedVersion() >= 2320.
The loading order is to load dependencies first, followed by the main package; for example, openvino_c is the main package, which depends on openvino, and openvino depends on libtbb. The loading process first loads libtbb, followed by openvino, and finally openvino_c, previous this loading method is only enabled in .NET 6, now it also enabled in .NET 5 in this release.
Newly released runtime libraries for version 2023.2.0 (updated from 2023.1.0):
| Package Name | Icon | Remarks |
|---|---|---|
| Sdcb.OpenVINO.runtime.centos.7-x64 | |
|
| Sdcb.OpenVINO.runtime.linux-arm | |
Replaces the previous Sdcb.OpenVINO.runtime.debian.9-arm, as it was actually usable beyond just debian 9 |
| Sdcb.OpenVINO.runtime.linux-arm64 | |
Replaces the previous Sdcb.OpenVINO.runtime.debian.9-arm64, as it was actually usable beyond just debian 9 |
| Sdcb.OpenVINO.runtime.rhel.8-x64 | |
|
| Sdcb.OpenVINO.runtime.ubuntu.18.04-x64 | |
|
| Sdcb.OpenVINO.runtime.ubuntu.20.04-x64 | |
|
| Sdcb.OpenVINO.runtime.ubuntu.22.04-x64 | |
|
| Sdcb.OpenVINO.runtime.android-arm64 | |
Unofficial, Preview |
| Sdcb.OpenVINO.runtime.android-x64 | |
Unofficial, Preview |
| Sdcb.OpenVINO.runtime.win-x64 | |
The most significant improvement of using the 2023.2.0 dynamic libraries is that when an OpenVINO C++ exception occurs, the Sdcb.OpenVINO core library will automatically collect the specific reasons for the exceptions (this feature was implemented after my proposal to the OpenVINO official).
The following suggestions I submitted to openvino have been resolved in the 2023.2 version:
- Multithreading inference not supported on ARM64
- Asynchronous inference enhancement
- Retrieval of C++ exception information
- Model load from memory enhancement
OpenCvSharp4 NuGet packages on Linux mini runtime:
| Package Name | Icon |
|---|---|
| Sdcb.OpenCvSharp4.mini.runtime.centos.7-arm64 | |
| Sdcb.OpenCvSharp4.mini.runtime.centos.7-x64 | [
v0.5.3
大家好，我发布了核心库Sdcb.OpenVINO 0.5.3，这个版本解决了一个在Xamarin Android中运行报错的问题，启用了安卓版本支持：
https://www.nuget.org/packages/Sdcb.OpenVINO/0.5.3
Hello everyone, I have released the core library Sdcb.OpenVINO 0.5.3. This version solves an error problem when running in Xamarin Android and enables basic Android version support:
https://www.nuget.org/packages/Sdcb.OpenVINO/0.5.3
v0.5.2
中文
该版本放宽了核心库Sdcb.OpenVINO的最低版本要求，从.NET Framework 4.8放宽到了.NET Framework 4.6.1，但当前代码是这样写的：
https://github.com/sdcb/OpenVINO.NET/blob/cdbeab2b85055b5c6ac20658794a031a8b4fe8ea/src/Sdcb.OpenVINO/Natives/NativeMethods.cs#L21-L28
这导致在.NET Framework版本< 4.8时，会尝试链接到openvino_c而非dll\win-x64\openvino_c.dll，这会导致触发一个DllNotFoundException。
正常使用Sdcb.OpenVINO.PaddleOCR不需要手动安装Sdcb.OpenVINO，因为前者依赖了后者，会自动安装，如果您使用.NET Framework版本小于.NET 4.8，就可以手动安装一下Sdcb.OpenVINO 0.5.2版本，这样就能用了。
English
This version has relaxed the minimum version requirement of the core library, Sdcb.OpenVINO, from .NET Framework 4.8 to .NET Framework 4.6.1. However, the current code is written as follows:
https://github.com/sdcb/OpenVINO.NET/blob/cdbeab2b85055b5c6ac20658794a031a8b4fe8ea/src/Sdcb.OpenVINO/Natives/NativeMethods.cs#L21-L28
This results in an attempt to link to openvino_c instead of dll\win-x64\openvino_c.dll when the .NET Framework version is < 4.8, causing a DllNotFoundException.
Typically, there is no need to manually install Sdcb.OpenVINO to use Sdcb.OpenVINO.PaddleOCR as the former automatically installs due to its dependency on the latter. If you are using a .NET Framework version less than .NET 4.8, you can manually install Sdcb.OpenVINO version 0.5.2, which should then work excellently.
v0.5.1
中文
大家好，我发布了下面3个包的新版本：
- Sdcb.OpenVINO 0.5.1（从0.4.5）
- Sdcb.OpenVINO.Extensions.OpenCvSharp4 0.2.0（从0.1.0）
- Sdcb.OpenVINO.PaddleOCR 0.5.1（从0.3.1）
各有如下改进或新功能：
Sdcb.OpenVINO 0.5.1（从0.4.5）
- 给DeviceOptions加入了NumStreams, SchedulingCoreType, EnableHyperThreading三个属性，允许配置推理流数量（默认自动），在大小核CPU上指定大核或小核（默认全用），是否启用超线程（默认不启用）
- 新加了（internal的）TensorBuffer类，解决了Tensor类引用的内存所有权不明确的问题，今后Tensor将明确拥有或共享内存的所有权，如果使用数组，则会创建一个数组的引用，如果使用Mat，则创建一个Mat的引用（根据实际情况，使用ROI引用或者克隆）
- BaseModel的CreateInferRequest函数改为CreateCompiledModel，今后将鼓励CompiledModel共享（而非InferRequest共享）
Sdcb.OpenVINO.Extensions.OpenCvSharp4 0.2.0（从0.1.0）
- 增加了FastClone，并将WeakRef改为Obsolete（过时）
- 删除了内存不安全的AsByteSpan函数
- 实现了MatTensorBuffer（表示对Mat拥有或共享所有权的类）
Sdcb.OpenVINO.PaddleOCR 0.5.1（从0.3.1）
- 所有推理类实现线程安全（原理：将所有推理类从共享InferRequest改成共享CompiledModel）
- 修复了在不启用旋转识别时导致推理结果不正确的问题（感谢@ddeef提出）
- 将QueuedPaddleOcrAll类改为Obsolete（过时），建议全用PaddleOcrAll（因为线程安全）
- 180度图像分类器现在除了返回是否需要旋转，还返回了信心值
0.5.1新版本OCR性能测试结果
（使用i9-9880h，ChineseV3模型，静态图，启用180度旋转识别，静态图，其它全默认，每个数字都是100次平均，单位毫秒）
159 164 155 163 161 163 159 162 170 167
内存约396MB~1033MB，CPU使用率约55%~69%
使用SemaphoreSlim（2并发）
166 166 174 182 185 184 175 165 183 166
内存约328MB~344MB，CPU使用率约56%~58%
使用SemaphoreSlim（4并发）
153 155 159 158 162 159 166 181 162 160
内存约346MB~366MB，CPU使用率约60%~62%
0.3.1测试结果：
2消费者，低延时模式，100张平均耗时(ms)
158 153 154 156 157 160 161 157 159 163
内存约737MB~783MB
0.5.1可以在保持线程安全的同时，以较低的内存实现相似的性能指标，同时不会有多消费者崩溃的问题。
English
Hello everyone, I have released new versions of the following three packages:
- Sdcb.OpenVINO 0.5.1 (upgraded from 0.4.5)
- Sdcb.OpenVINO.Extensions.OpenCvSharp4 0.2.0 (upgraded from 0.1.0)
- Sdcb.OpenVINO.PaddleOCR 0.5.1 (upgraded from 0.3.1)
The improvements or new features are as follows:
Sdcb.OpenVINO 0.5.1 (upgraded from 0.4.5)
- Added the NumStreams, SchedulingCoreType, and EnableHyperThreading attributes to DeviceOptions, allowing configuration of the number of inference streams (automatic by default), specifying large or small cores on a large-small core CPU (all cores are used by default), and whether to enable hyper-threading (disabled by default)
- Added the (internal) TensorBuffer class, addressing the unclear memory ownership issue of the Tensor class, from now on Tensor will clearly own or share memory. If an array is used, a reference to an array will be created; if Mat is used, a reference to Mat will be created (choosing to use either ROI reference or clone, depending on the actual situation)
- Changed the CreateInferRequest function in BaseModel to CreateCompiledModel. Future operations will encourage sharing of CompiledModel (rather than sharing InferRequest)
Sdcb.OpenVINO.Extensions.OpenCvSharp4 0.2.0 (upgraded from 0.1.0)
- Added FastClone, and marked WeakRef as Obsolete
- Removed the memory-unsafe AsByteSpan function
- Implemented MatTensorBuffer (a class that owns or shares the ownership of Mat)
Sdcb.OpenVINO.PaddleOCR 0.5.1 (upgraded from 0.3.1)
- All inference classes have been made thread-safe (by changing all inference classes from sharing InferRequest to sharing CompiledModel)
- Fixed an issue that caused incorrect inference results when rotation recognition was not enabled (thanks to @ddeef for pointing it out)
- Marked the QueuedPaddleOcrAll class as Obsolete, it is suggested to fully use PaddleOcrAll (because it's thread-safe)
- In addition to determining whether rotation is needed, the 180-degree image classifier now also returns a confidence value
OCR Performance Results with Version 0.5.1
(Tested on i9-9880h, ChineseV3 model, static graph, with 180-degree rotation recognition enabled, other settings default, each number is the average of 100 times, in milliseconds)
159 164 155 163 161 163 159 162 170 167
Memory usage: About 396MB~1033MB, CPU utilization: About 55%~69%
Using SemaphoreSlim (2 concurrent)
166 166 174 182 185 184 175 165 183 166
Memory usage: About 328MB~344MB, CPU utilization: About 56%~58%
Using SemaphoreSlim (4 concurrent)
153 155 159 158 162 159 166 181 162 160
Memory usage: About 346MB~366MB, CPU utilization: About 60%~62%
Test results for version 0.3.1:
2 consumers, low latency mode, average time for 100 images in milliseconds:
158 153 154 156 157 160 161 157 159 163
Memory usage: About 737MB~783MB
Version 0.5.1 is able to maintain thread safety, achieve similar performance metrics with lower memory usage, and won't have crash issues with multiple consumers.
Contributors
Assets 2
0.4.5
中文
新发布的包列表
- Sdcb.OpenVINO 0.4.5
- Sdcb.OpenVINO.Extensions.OpenCvSharp4 0.1.0
- Sdcb.OpenVINO.PaddleOCR 0.3.1
Sdcb.OpenVINO 0.4.5有如下变化：
- 给BaseModel加入PrePostProcessor处理功能
- 删除CppObject类以节省文件，将它合并到CppPtrObject类
- 增加ov_get_and_reset_last_error函数，以为未来准备这个issue：https://github.com/openvinotoolkit/openvino/issues/20536（我提的）带来的便利
- 给Tensor类增加FromRaw从指针输入的方法
Sdcb.OpenVINO.Extensions.OpenCvSharp4
- 新加的一个包，主要增强OpenVINO和OpenCv的互操作便利性
- 提供了MatExtensions，用于一些常用的Mat操作，如Padding或normalize
- 提供了TensorExtensions，用于方便Tensor转Mat/Mat转Tensor之类，用户的代码不再需要指针了
Sdcb.OpenVINO.PaddleOCR 0.2.3 -> 0.3.1
- 大量代码增强，重磅推出
- 允许Cls阶段走batch以提高性能
- 允许检测阶段使用静态图以提高性能
- 允许识别阶段使用静态图以提高性能
- 为Cls阶段的返回值增加信心度
- 增加PaddleOcrOptions类以方便上面的静态图配置
- 将所有3个模型的预处理阶段都部分改成使用OpenVINO自带的PrePostProcessor以减少代码（性能差不多）
English Version
Newly released package list:
- Sdcb.OpenVINO 0.4.5
- Sdcb.OpenVINO.Extensions.OpenCvSharp4 0.1.0
- Sdcb.OpenVINO.PaddleOCR 0.3.1
Changes in Sdcb.OpenVINO 0.4.5:
- Added PrePostProcessor handling function to BaseModel
- Removed CppObject class to save files, merging it into the CppPtrObject class
- Added ov_get_and_reset_last_error function to prepare for this issue: openvinotoolkit/openvino#20536 (I raised it) for convenience in the future
- Added FromRaw method to Tensor class for input from pointer
Features of Sdcb.OpenVINO.Extensions.OpenCvSharp4 0.1.0:
- A new package mainly enhancing the interoperability of OpenVINO and OpenCv
- Provided MatExtensions for some common Mat operations, such as Padding or normalize
- Provided TensorExtensions for convenient Tensor to Mat/Mat to Tensor conversions, users' code no longer needs pointers
Improvements made from Sdcb.OpenVINO.PaddleOCR 0.2.3 -> 0.3.1:
- Significant code enhancements, heavily promoted
- Allowed Cls phase to go batch to improve performance
- Allowed detection phase to use static graph to improve performance
- Allowed recognition phase to use static graph to improve performance
- Added confidence level to the return value of Cls phase
- Added PaddleOcrOptions class to facilitate the above static graph configuration
- Partially changed the preprocessing stage of all three models to use OpenVINO's built-in PrePostProcessor to reduce code (performance is similar)