Build fails "late" if no internet connection

[GuillaumeDesforges]

System information (version)

• OpenCV => 4.5.5.64
• Operating System / Platform => x86_64-linux
• Compiler =>

Detailed description

If I clone the repository and run the build in an isolated environment without internet, the build first fails to download the GAPI component

https://github.com/opencv/opencv/blob/e0ffd3e8a5c25b634a853ce695c4d23d3057b361/modules/gapi/cmake/DownloadADE.cmake#L5-L14

and that failure does not stop the build.

However, at the end of the build, it fails to build the wheel

  Traceback (most recent call last):
    File "/nix/store/yfwbxnn1j32nyfhi2yzx859dij4xgl3s-python3.9-pip-21.3.1/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
      main()
    File "/nix/store/yfwbxnn1j32nyfhi2yzx859dij4xgl3s-python3.9-pip-21.3.1/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/nix/store/yfwbxnn1j32nyfhi2yzx859dij4xgl3s-python3.9-pip-21.3.1/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/nix/store/0xxs98jm9dpvybfgh7kghdq8ayvkp87m-python3.9-setuptools-57.2.0/lib/python3.9/site-packages/setuptools/build_meta.py", line 221, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "/nix/store/0xxs98jm9dpvybfgh7kghdq8ayvkp87m-python3.9-setuptools-57.2.0/lib/python3.9/site-packages/setuptools/build_meta.py", line 207, in _build_with_temp_dir
      self.run_setup()
    File "/nix/store/0xxs98jm9dpvybfgh7kghdq8ayvkp87m-python3.9-setuptools-57.2.0/lib/python3.9/site-packages/setuptools/build_meta.py", line 258, in run_setup
      super(_BuildMetaLegacyBackend,
    File "/nix/store/0xxs98jm9dpvybfgh7kghdq8ayvkp87m-python3.9-setuptools-57.2.0/lib/python3.9/site-packages/setuptools/build_meta.py", line 150, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 503, in <module>
      main()
    File "setup.py", line 240, in main
      skbuild.setup(
    File "/nix/store/aa1q4aw1baz4pcvcfqb992knp13azkas-python3.9-scikit-build-0.12.0/lib/python3.9/site-packages/skbuild/setuptools_wrap.py", line 625, in setup
      _classify_installed_files(cmake_manifest, package_data, package_prefixes,
    File "setup.py", line 416, in _classify_installed_files_override
      raise Exception("Not found: '%s'" % relpath_re)
  Exception: Not found: 'python/cv2/gapi/.*\.py'

Steps to reproduce

Cone then build without Internet

or build in an isolated environment using Nix

• Install Nix https://nixos.org/download.html
• Allow flake support https://nixos.wiki/wiki/Flakes#Non-NixOS
• Run nix build github:GuillaumeDesforges/bug-poetry2nix-opencv

Issue submission checklist

• I report the issue, it's not a question
• I checked the problem with documentation, FAQ, open issues,
  forum.opencv.org, Stack Overflow, etc and have not found any solution
• I updated to the latest OpenCV version and the issue is still there
• There is reproducer code and related data files: videos, images, onnx, etc

[alalek]

4.5.5.64

You need to report to opencv-python downstream repo.
Probably priority is low as such environment doesn't pass prerequisites.

---

OpenCV disables related modules (gapi) if dependencies can't be fetched. Check CMake logs. No issue with opencv itself.
But setup.py from opencv-python still want to use missing files.

---

If you drop Internet connection, then you should pre-download / provide somehow required dependencies.
See vcpkg as example.

---

Avoid building through pip/setup.py, use source code from GitHub (AFAIK there is already such NIX package).

---

Transferring issue...

[asmorkalov]

Fix: #639

[asenyaev]

Hi @GuillaumeDesforges!

I've fixed this skipping gapi if there is no files in #642.

However, there is another solution, you can:

1. Pre-download these files on another device with non-isolated internet. You can do it using wget or curl or using cmake and opencv repository. For example:

cd opencv && mkdir generate && cd generate && cmake -DOPENCV_DOWNLOAD_PATH=/path/to/folder .. 

*If you need opencv_contrib modules, also define -DOPENCV_EXTRA_MODULES_PATH=/path/to/opencv_contrib/modules flag.

2. Check files in a defined folder.
3. Move them on another device with isolated internet.
4. Define the folder to these files using CMAKE_ARGS variable and -DOPENCV_DOWNLOAD_PATH flag. For example:

CMAKE_ARGS=-DOPENCV_DOWNLOAD_PATH=/path/to/pre-downloaded/files pip wheel . --verbose

You can also do it using a source package on PyPi:

CMAKE_ARGS=-DOPENCV_DOWNLOAD_PATH=/path/to/pre-downloaded/files pip install -v --no-binary=opencv-python opencv-python

[asenyaev]

In addition, we wanted to add these pre-downloaded files in a source package, but there are different files what depend on OS / 64/32-bit builds. If to keep all of them in the source package, it can be about of 300-400Gb for an archive, what is prohibited to upload on PyPi in case of limits.

[GuillaumeDesforges]

Hi, #642 would be very helpful.

Most likely I can tinker around to download the GAPI dependency and provide it to the build if there is a way to substitute the download to a local file. Looking at your comment, it seems doable.

Thanks for your reply and work!

[asmorkalov]

No action items on our side.