Skip to content

Home

Jump to bottom
Cocoa edited this page Aug 25, 2022 · 2 revisions

Welcome to the evision wiki!

How does this work?

  1. This project will first pull OpenCV source code from git (as git submodules).

  2. Inside the OpenCV project, there is an opencv-python module, 3rd_party/opencv/modules/python. If the opencv-python module is enabled,

    cmake ...
    -D PYTHON3_EXECUTABLE=$(PYTHON3_EXECUTABLE) \
    ...

    It will generate files for opencv-python bindings in cmake build dir, cmake_build_dir/modules/python_bindings_generator.

    We are interested in the headers.txt file as it tells us which headers should we parse (this header list changes based on the enabled modules).

    We also need to check another file, pyopencv_custom_headers.h. This file includes pyopencv compatible headers from modules that need special handlings to enable interactions with Python. We will talk about this later.

  3. Originally, the headers.txt file will be passed to 3rd_party/opencv/modules/python/src2/gen2.py and that Python script will then generate pyopencv_*.h in cmake_build_dir/modules/python_bindings_generator. Here we copy that Python script and modify it so that it outputs c_src/evision_*.h files which use c_src/erlcompat.hpp and c_src/nif_utils.hpp to make everything compatible with Erlang.

  4. c_src/opencv.cpp includes almost all specialised and generic evision_to and evision_from functions. They are used for making conversions between Erlang and C++. Some conversion functions are defined in module custom headers.

  5. This is where we need to make some changes to pyopencv_custom_headers.h. We first copy it to c_src/evision_custom_headers.h and copy every file it includes to c_src/evision_custom_headers/. Then we make corresponding changes to c_src/evision_custom_headers/*.hpp files so that these types can be converted from and to Erlang terms. The header include path in c_src/evision_custom_headers.h should be changed correspondingly.

  6. However, it is hard to do step 5 automatically. We can try to create a PR which puts these changed files to the original OpenCV repo's {module_name}/mics/erlang/ directory. Now we just manually save them in c_src/evision_custom_headers. Note that step 5 and 6 are done manually, calling py_src/gen2.py will not have effect on c_src/evision_custom_headers.h and *.hpp files in c_src/evision_custom_headers.

  7. Another catch is that, while function overloading is easy in C++ and optional arguments is simple in Python, they are not quite friendly to Erlang/Elixir. There is basically no function overloading in Erlang/Elixir. Although Erlang/Elixir support optional argument (default argument), it also affects the function's arity and that can be very tricky to deal with. For example,

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts \\ []), do: :nil
  def open(self, filename), do: :nil
  # ... other functions ...
end

    In this case, def open(self, camera_index, opts \\ []), do: :nil will define open/3 and open/2 at the same time. This will cause conflicts with def open(self, filename), do: :nil which defines open/2.

    So we cannot use default arguments. Now, say we have

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts), do: :nil
  def open(self, filename, opt), do: :nil
end

    Function overloading in C++ is relatively simple as compiler does that for us. In this project, we have to do this ourselves in the Erlang/Elixir way. For the example above, we can use guards.

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts) when is_integer(camera_index) do
    # do something
  end
  def open(self, filename, opt) when is_binary(filename) do
    # do something
  end
end

    But there are some cases we cannot distinguish the argument type in Erlang/Elixir becauase they are resources (instance of a certain C++ class).

    defmodule OpenCV.SomeModule do
  # @param mat: Mat
  def func_name(mat) do
    # do something
  end

  # @param mat: UMat
  def func_name(mat) do
    # do something
  end
end

    In such cases, we only keep one definition. The overloading will be done in c_src/opencv.cpp (by evision_to).

  8. Enum handling. Originally, PythonWrapperGenerator.add_const in py_src/gen2.py will be used to handle those enum constants. They will be saved to a map with the enum's string representation as the key, and, of course, enum's value as the value. In Python, when a user uses the enum, say cv2.COLOR_RGB2BGR, it will perform a dynamic lookup which ends up calling corresponding evision_[to|from]. evision_[to|from] will take the responsibility to convert between the enum's string representation and its value. Although in Erlang/Elixir we do have the ability to both create atoms and do the similar lookups dynamically, the problem is that, if an enum is used as one of the arguments in a C++ function, it may be written as void func(int enum) instead of void func(ENUM_TYPE_NAME enum). However, to distinguish between overloaded functions, some types (int, bool, string, char, vector) will be used for guards. For example, void func(int enum) will be translated to def func(enum) when is_integer(enum), do: :nil. Adding these guardians help us to make some differences amongst overloaded functions in step 7. However, that prevents us froming passing an atom to def func(enum) when is_integer(enum), do: :nil. Technically, we can add one more variant def func(enum) when is_atom(enum), do: :nil for this specific example, but there are tons of functions has one or more ints as their input arguments, which means the number of variants in Erlang will increase expoentially (for each int in a C++ function, it can be either a real int or an enum). Another way is just allow it to be either an integer or an atom:

    def func(enum) when is_integer(enum) or is_atom(enum) do
  :nil
end

    But in this way, atoms are created on-the-fly, users cannot get code completion feature for enums from their IDE. But, finally, we have a good news that, in Erlang/Elixir, when a function has zero arguments, you can write its name without explictly calling it, i.e.,

    defmodule M do
  def enum_name(), do: 1
end

1 = M.enum_name
1 = M.enum_name()

    So, in this project, every enum is actually transformed to a function that has zero input arguments.

How do I add support for other/custom OpenCV modules?

Because of the reason in step 6, when you enable more modules, if that module has a specialised custom header for python bindings, the custom headers will be added to cmake_build_dir/modules/python_bindings_generator/pyopencv_custom_headers.h.

Then it should be manually copied to the c_src/evision_custom_headers directory.

After that, you perhaps need to make corresponding changes in the copied file.

Clone this wiki locally