Opencv official nodejs binding

[piercus]

Descripe the feature and motivation

This issue is following a 2 years long conversation we are having with @UrielCh on the opencv slack (you can find us in the #opencv4nodejs channel)

A Nodejs binding for Opencv

The goal of this feature request is to get a Nodejs binding for opencv, auto-generated from C-Headers the same way Python, Java, Obj-C binding are autogenerated from C Headers

This will include :

• Conventions to transform C functions/names/parameters/Class in nodejs
• A python generation code from C headers similar to gen2.py
• Some C++ macros similar to opencv4nodejs's ones
• Code to integrate it as a C++ module included into the Opencv compilation process
• Unit tests inspired from opencv4nodejs's unit tests

And it should generate :

• C++ headers
• Typescript typing information
• Nodejs OpenCV Documentation
• npm packagable project

Milestones

Vocabulary :

C-Python binding : the C code that create the Python module
C-Nodejs binding : the C code that create the Nodejs module
Generator-Python : Existing Python code that generate C-Python binding code from opencv C headers
Generator-Nodejs : Python code to be written that generate C-Nodejs binding code from opencv C headers

Status is the following :

Step 0 : set up environment

• Create an outside project for experimenting -> https://github.com/UrielCh/opencv-sandbox
• Run the Generator-Python outside of the opencv project

Step 1: Make a Generator-Nodejs v1

• Translate the python-C helpers function into C nodejs helpers functions done in cc-common
• Translate the python C templates helpers function into python helpers functions done in cc-common
• Generate Nodejs Code and make it compile

Step 2: Discuss Generator-Nodejs design

• Generate the typescript signatures (generated in cc-generated/cv.d.ts)
• Review the C-Nodejs binding design
• [ ]

Step 3 : nodejs clean binding

• On a simple example class (AKAZE) write manually the C-Nodejs binding files .h and .cpp (done in https://github.com/UrielCh/opencv-sandbox/tree/class/cc-obj)
• Review the class .h and .cpp implementation against nodejs binding best practices
• Write a python unit test for the generator and compare the generated string with the expected one [IN PROGRESS @piercus]
• Create more generator unit test, some having function variants signatures, other with create/without create, with constructor/without constructor
• Take some nodejs unit test from https://github.com/justadudewhohacks/opencv4nodejs/tree/master/test/tests and put it in the project to test the final module
• Implement the generator and the helpers to make everything pass

Step 4 : integrate it back to opencv

• Make a point with the opencv team to present the status of the binding
• Integrate it into a larger build process, create the CMakeFile ...

Additional context

Opencv4nodejs

Opencv4nodejs is a native nodejs binding for nodejs made by @justadudewhohacks few years ago.

The original repo is not maintained since 2019, since 2021 we are trying to redirect the community to UrielCh's fork

Opencv4nodejs is widely used, but it has following problems :

• Main author has been lost
• It is not covering the whole lib
• Compilation is non-optimal

Main author has been lost

The author @justadudewhohacks has been lost by the community, leaving me and @UrielCh as mainmteners working on @UrielCh's fork, this create a lot of issues for the community state of npm package, Repo Searchability,

It is not covering the whole lib

The coverage of opencv4nodejs is quite good, (it can be seen in the doc )

But there is still no plan to provide a 100% full coverage because the C++ bindings is not generated from Python (as it is the case for official Python, Java and Obj-C bindings)

This non-generated situation creates other problems like :

• Arguments missing in some function
• Maintainability must be hand-made
• Wording convention from C to nodejs is not 100% coherent through the project

Compilation is non-optimal

Most of the people raising issues on opencv4nodejs are complaining about the unability to compile the project.

This compilation problems are related to both organisational and technical situation.

Technical situation :

• The current binding use nan, we think node-addon-api would be a better choice to allow pre-built retro-compatible binary
• The compilation of the binding is not made in 2 steps, first you need to compile opencv and then you need to compile the binding, it creates lot issues where you do not know exactly where the problem is coming from. The binding behing very dependant to the opencv version used, a lot of code has been written in order to check which version is installed, how to pass flags from the nodejs project to opencv4nodejs compilation process
• A specific project has been made to give the ability to compile opencv from the npm installer

Organisational situation, most of the people involved in the opencv4nodejs community are nodejs's people, typescript's devs, to put it simply, we don't like compiling things, CMake ...

Opencv.js

I'm less expert on Opencv.js, so i will let others give their feedbacks.

Opencv.js is made for the browser, this is not a binding to OpenCv but more a browser compatible generated code

I feel this is totally different for the following reasons :

• Opencv.js will only access browser-accessible ressources of your computer
• Opencv.js will never get full coverage of the OpenCv library

[Abhijrathod]

The feature being requested is the creation of a Node.js binding for OpenCV, which will be auto-generated from C-Headers. This is similar to how Python, Java, and Obj-C bindings are auto-generated from C-Headers. The goal is to have conventions in place to transform C functions/names/parameters/Class into Node.js, python generation code from C headers, C++ macros similar to opencv4nodejs's ones, the ability to integrate it as a C++ module included into the OpenCV compilation process, unit tests inspired by opencv4nodejs's unit tests, and the ability to generate C++ headers, TypeScript typing information, Node.js OpenCV Documentation, and an npm packagable project.

The first step involves generating code for a minimal perimeter binding, including imread/imwrite perimeter, autogenerated code, and unit tests. This will not be integrated into the OpenCV compilation code. The second step involves integrating the build into the OpenCV build process, checking how the compilation can be made together with the OpenCV compilation, and checking how packaging of the npm package can be done. The third step involves scaling the perimeter, adding some unit tests, and scaling the modules.

The motivation behind this feature request is that the current native Node.js binding for OpenCV (opencv4nodejs) has problems such as the main author being lost, not covering the whole library, and having non-optimal compilation. Additionally, most people involved in the OpenCV4NodeJS community are Node.js and TypeScript developers who dislike compiling things like CMake. The proposed Node.js binding will address these issues and will be auto-generated from C-Headers, providing full coverage of the OpenCV library and allowing for pre-built retro-compatible binaries.

[piercus]

@Abhijrathod are you interested to help ?

[vpisarev]

@piercus, @Abhijrathod, thank you, this is a great effort. Given how popular nodejs nowadays, it could definitely make many developers happy. By the way, we are participating, again, in GSoC 2023. The proposals shall be submitted between March 20th and April 4th. If you have some good students (actually, recent Google rules say it does not have to be real students) who can advance this project, and if we are given sufficient number of slots, and if you can also find some mentor (we can try as well, but nodejs is not among our areas of expertise) - with all those "if's" satisfied we can have a great project. What do you think?

[piercus]

@vpisarev GSoc was a good idea, but i feel we missed the deadline.

we have already started working on this with @UrielCh on https://github.com/UrielCh/opencv-sandbox

I have updated the initial post, with our current progress and milestones.